One of the biggest challenges facing an electrical, computer, or design engineer is bridging the gap between hardware and software. The task to write software to exercise hardware has always proved to be a challenging task. This is primarily due to the focus on learning language syntax and not enough time spent on learning debugging and software testing. The gap widens further when asked to prove that the software isn’t introducing problems when exercising hardware. This typically leads to hours of long full system tests to ensure repeatability which is not a scalable solution when hardware is involved due to slow responses. For Python developers, the solution is to write unit tests of the test code using pytest and the pytest-mock plugin to simulate hardware responses.
NOTE: While this post is written for the hardware engineer, this concept also extends to any external process or system such as a database.
At the communication layer, there is typically code to interact with hardware (or external system). This “driver” is usually coded as a class with methods. A driver will typically include low level functions such as initialize, send, read, and close. Higher level functions include methods such as set mode, configure output, capture data, program, and many others. In this interaction, there are a couple of items to check:
Are commands being properly constructed? The command itself may be dynamically generated and have variations depending on input parameters.
Are responses being properly consumed? The raw response may have some data post-processing that requires validation.
In these cases, a mock object can be created to simulate the hardware. The mock object can return the command that was sent and returns pre-programmed responses in a short amount of time. This is an important benefit as in many cases, hardware responses can be slow.
pytest and pytest-mock
While tests can be written in many different ways, pytest is the most popular Python test runner and can be extended through the use of plugins. pytest-mock is a plugin library that provides a pytest fixture that is a thin wrapper around mock that is part of the standard library. This post will discuss using the pytest-mock plugin to create mock objects to simulate responses.
from time import sleep
from fabric import Connection
def __init__(self, hostname):
self.connection = Connection(hostname)
def run(self, command):
return self.connection.run(command, hide=True).stdout.strip()
return self.run("df -h")
free_line = output.split("\n")
percent = free_line.split()
The driver above is a simple example. It uses a library called Fabric to establish an SSH connection. There are two methods:
run() – Allows any generic command to be issued on the target and returns the raw output. Before the output is returned, a five second delay is inserted to simulate a slow response.
disk_free() – Generates a command “df -h” and then calls the run() method with the generated command
extract_percent() – Parses the raw output and returns the disk free percent
Test Code in Integrated Style
from mock_tutorial.driver import Driver
d = Driver(socket.gethostname())
result = d.disk_free()
percent = d.extract_percent(result)
assert percent == "75%"
The above test code is typically written in a black box testing style to test this driver. First, the driver object is instantiated, the disk_free() function is called, the output is parsed, and then finally compared with an expected result.
In addition to the slow execution of 5.51 seconds, there is another problem. The output of “df -h” will most likely change over time and not stay at 75%. While this example is contrived, it illustrates the need for the output to checked in another way.
The above code was rewritten to use a mock object to patch (replace) the run() method. First, we need to import pytest (line 2) and call the mocker fixture from pytest-mock (line 5).
On lines 12-14, the run() method of the Driver class was patched with a pre-programmed response to simulate an actual response. This means that any call to run() will return the string value of output.
Line 18 will check the command that was sent to the run() method. When the disk_free() method is called, this will generate a command of “df -h” and call run() with this command.
Line 19 will check the parsing functions that extracts the percent from output. If the Use% in line 8 is changed, this will fail as this is the value that is being extracted.
After the mock objects have been added, the test time is reduced to 0.36 seconds. The slow run() method was patched to execute faster and also the code to parse the simulated output was checked.
Summary and Resources
This should provide a good starting point for developing fast performing unit tests in Python by patching slow response with mock objects. In addition to unit tests, integration tests should also be written however they can be executed less frequently.
This post is the first I’ve written on this topic in Python and I hope to delve into other pytest-mock methods in the future.