Running infrastructure tests using TestInfra

  • Posted by: James Alford
  • Category: Ansible, DevOps, Python, TestInfra, Testing

TestInfra setup

TestInfra is an infrastructure testing framework that makes it easy to write unit tests to verify the state of a server. It is a Python library that uses the powerful pytest test engine. In order to run tests using TestInfra we need to have Python3 and the testinfra library. Firstly if you don’t already have it, install python3 and then install testinfra and ansible using pip. The code snippet below shows the steps to set up the environment.

$ python3 -m venv testinfra-venv
$ source testinfra-venv/bin/activate
(testinfra-venv) $ pip install testinfra
(testinfra-venv) $ pip install ansible

Once the dependencies have been installed, we can run a basic test to confirm that everything is installed correctly. To run this test, create a file called “test_basic.py” and using whichever editor you are comfortable with, add in the content from the code snippet below

import testinfra
def test_hosts_file(host):
  f = host.file('/tmp/testfolder')    
  assert f.exists

Once the file has been created, run the test to see if it runs. If you do not have a folder called testfolder in your /tmp directory then the test should fail, this is expected and the output can be seen below

(testinfra-venv) Jamess-MBP-2:~ jamesalford$ pytest test_testinfra.py
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jamesalford
plugins: testinfra-4.1.0
collected 1 item
test_testinfra.py F                                                                                                             [100%]
============================================================== FAILURES ===============================================================
_______________________________________________________ test_hosts_file[local] ________________________________________________________
host = <testinfra.host.Host object at 0x10aa00ed0>
    def test_hosts_file(host):
        f = host.file('/tmp/testfolder')
>       assert f.exists
E       assert False
E        +  where False = <file /tmp/testfolder>.exists
test_testinfra.py:6: AssertionError
========================================================== 1 failed in 0.10s ==========================================================

Now that we have seen TestInfra run, we can fix our infrastructure to have the required folder and test again to see a passed test.
Run the command `touch /tmp/testfolder` and re-run the test. The test should now pass as shown in the snippet below.

(testinfra-venv) Jamess-MBP-2:~ jamesalford$ pytest test_testinfra.py
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jamesalford
plugins: testinfra-4.1.0
collected 1 itemtest_testinfra.py .                                                                                                             [100%]========================================================== 1 passed in 0.06s ==========================================================

Running TestInfra against remote hosts using an ansible inventory

In a typical ansible playbook, there will be an inventory that has groups and hosts defined. The inventory could also be generated dynamically, for example an inventory for AWS infrastructure can be generated using a script called ec2.py. For the following explanation, we will use an example inventory that is shown below as if it is in a file named “inventory”.

[webservers]
web01
web02
[appservers]
app01
app02
[database]
db01

To run the tests using the file we created earlier (test_testinfra.py) then we need to run the same command but with some extra arguments.The extra arguments are --hosts=webservers where we define the host group in the inventory, --ansible-inventory=inventory where we define the inventory file name and --connection=ansible for the connection that we wish to use which will be ansible. Full command shown below

(testinfra-venv) $ py.test --hosts=webservers --ansible-inventory=inventory --connection=ansible test_testinfra.py

Using TestInfra to monitor infrastructure

As we have seen in the previous tests, the output that comes back from the test run is quite bloated and isn’t really going to be useful if we pass this to a monitoring tool. Luckily, TestInfra offers an integration with Nagios and allows you to run tests directly on the monitoring master node.To get a Testinfra output that is compatible with monitoring, we have to use the flags --nagios to enable the nagios output and -qq for quiet mode. The example code below shows how to run the test in this way locally.

testinfra-venv) Jamess-MBP-2:~ jamesalford$ py.test test_testinfra.py --nagios -qq
TESTINFRA OK - 1 passed, 0 failed, 0 skipped in 0.04 seconds

To run tests on remote hosts using an ansible connection and get an output in a monitoring friendly way, run the tests in the following way:

(testinfra-venv) $ py.test --hosts=webservers --ansible-inventory=inventory --connection=ansible -qq --nagios test_testinfra.py

Using variables in tests

When running test cases, you can use pytest’s parametrize to run the same test against multiple inputs.
In the example below we set a variable of data to be an array of ports which is then used in the parametrize function.
This means that the test will be run once for each item in the array and for each run, the variable “port” will be set to the value of the array item it is running on.

import pytest
data = [8080, 8090]

@pytest.mark.parametrize("port", data)
def test_listening_ports(host,port):
  local_ip = host.run("hostname -i|awk '{print $1}'")
  test_port = str(port)
  assert host.socket("tcp://" + local_ip.stdout.rstrip() + ":" + test_port).is_listening   # HTTP port

If you need to run multiple test cases but each case has multiple variables then you need to use tuples inside the array that you pass to parametrize. The below example is a health check that will curl the host on the defined port and endpoint, then check the response. The use of tuples means that we can define multiple tests as shown.

import pytest
data = [("80","health","200"),("8080","healthcheck","200"),("8080","info","200")]

@pytest.mark.parametrize("port,endpoint,response", data)
@pytest.mark.slow
def test_http_endpoints(host,port,endpoint,response):
  app_url = "http://$(hostname):" + port + "/" + endpoint
  resp = host.run("curl -m 1 -LI " + app_url)
  assert response in resp.stdout

If you would like to know more about how you could use TestInfra to run tests on an ansible playbook using Molecule then have a read of the blog post https://chorograph.com/ansible-playbook-testing-with-molecule-testinfra-docker

Author: James Alford