Eric Radman : a Journal

Using Test Doubles in Python

When I first start doing test-first programming I immediately started looking for libraries that would help me create test doubles. One such library I've used is fudge. Another approach is to build a fake class that knows how to keep state for the specific object that you'd like to replace.

A New Object

I rarely create a test double by subclassing the real thing. If we create a new class that implements only the method calls we use, the implementation can be kept simple and invocations of methods that we don't expect use will raise an exception. The following example can be used in place of smtplib.SMTP

import smtplib

class fake_SMTP:
    calls = []

    def __init__(self, _server):
        self.calls.append("__init__('%s')" % _server)

    def sendmail(self, _from, _to, _msg):
        self.calls.append("sendmail('%s', %s, <msg>" % (_from, _to))

    def quit(self):
        self.calls.append("quit()")

Now replace the class we're going to test in smtplib

setattr(smtplib, 'SMTP', fake_SMTP)

fake_SMTP is instantiated and called in the same way as the real

SMTP and it simply uses a list to record the signature of each method call, allowing me to test concretely and precisely.
# system-under-test: alert.py
import alert

def test_send_message():
    message = "hi"
    alert.send_message(['eradman@eradman.com'], message)
    assert_equals(fake_SMTP.calls,
        ["__init__('localhost')",
        "sendmail('user', ['eradman@eradman.com'], <msg>",
        "quit()"])

This method is powerful; I'm free to specify the order in which method calls are recorded, and with what details. In this case I'm not interested in what the message body is, so I only record a token <msg>. It would be just as easy to record the frist two lines or to track the method calls with a dictionary instead of a list.

Built-In Classes

Unfortunately Python does not allow you to modify some of it's core modules written in C. One way around this is to use a method the simply calls the real thing.

import datetime

def now():
    return datetime.now()

This method can be mocked using setattr() provided the rest of the codebase uses this method.

Test Discovery and Invocation

Python's unittest package includes a discovery method, but it's rarely useful because every test file is loaded into the same process, making namespace pollution nearly impossible to avoid. Instead prefer using a simple shell script to discover and load tests.

#!/bin/ksh
# Find tests starting with ut_ and run them. Each module must invoke unittest
# on it's own using
#
# if __name__ == '__main__':
#     unittest.main()
#

case "$1" in
    -h) shift; echo "usage: `basename $0` [search_pattern]"
        exit;;
esac

WD=`dirname $0`
PATTERN=$1

PYTHONPATH=$PYTHONPATH:$WD
find tests/ -name "${PATTERN:=ut_*.py}" | xargs -P 2 -n 1 python

Notice the use of xargs gave us a concurrent test runner at the cost of the additional RAM required for multiple intances of the Python interpreter. Experiment with the -P option to find out what's optimal.

Developer Friendly Test Fixutres in Python

Starting with 2.7, Python's unittest includes a very nice way to compare dictionaries. Instead of printing the two unequal values,

assertDictEqual print a human-readable diff

import unittest

a = {'key1': 5, 'key2': 7}
b = {'key1': 6, 'key2': 7, 'key3': 9}

class Test1(unittest.TestCase):
    def test_1(self):
        self.assertDictEqual(a, b)

unittest.main()

This makes comparing otherwise complex structures easy. This capability saves time, but more importantly it encourages concrete testing rather than sampling.

AssertionError: {'key2': 7, 'key1': 5} != {'key3': 9, 'key2': 7, 'key1': 6}
- {'key1': 5, 'key2': 7}
?          ^

+ {'key1': 6, 'key2': 7, 'key3': 9}
?          ^           +++++++++++

Comparing Blocks of Text

Sometimes subclassing unittest.TestCase is sensible, but if you inherit from more than one derived class this can become messy. In Python it may be better to create a new object to mix in that will use difflib to raise an exception with a messaged formatted as a unified diff

class CustomCompare(object):
    def assertTextEqual(self, s1, s2):
        diff = list(difflib.unified_diff(s1.splitlines(), s2.splitlines()))
        if diff != []:
            raise AssertionError("\n".join(diff))

Now add this to your test case

class Test1(unittest.TestCase, Comparison):
    def test_1(self):
        self.assertTextEqual(a, b)

The result look like this:

AssertionError:

@@ -1,4 +1,4 @@

 one
-two
+Two
 three

Last updated on November 26, 2016