Patrick's Software Blog

Learning Python Web Application Development with Flask

Python Unit Testing – Structuring Your Project

Overview

This blog post shows the basics of setting up the unit tests for your python project.  It starts with a basic example to show how unit tests execute and then moves on to a typical file/directory structure for a python project.

Very Basic Example

To start off with a basic example, we’ll use the following file/directory structure:

    test_project
        basicfunction.py
        basicfunction.ut.py

Here is the source code for basicfunction.py:

class BasicFunction(object): 
    def __init__(self): 
        self.state = 0 

    def increment_state(self): 
        self.state += 1 

    def clear_state(self): 
        self.state = 0

To start, let’s create a really basic unit test file that doesn’t even care about basicfunction.py yet:

import unittest

class TestBasicFunction(unittest.TestCase):
    def test(self):
        self.assertTrue(True)

if __name__ == '__main__':
    unittest.main()

The unit test file starts off by importing the built-in ‘unittest’ module from python.  Next, a basic class is created that we will eventually use to test out the basicfunction.py code.  For now, it just has a simple ‘assertTrue’ statement that will always pass.  At the end of the file, the following two lines are included:

if __name__ == '__main__':
    unittest.main()

What does this do?  These lines allow us to run the unit tests (defined in the TestBasicFunction class) from the command line.  Here’s how to run the unit tests from the command line:

$ cd test_project
$ python basicfunction.ut.py –v
test (__main__.TestBasicFunction) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

From the output above, you can see that be going into your project’s directory, you can simply execute the unit test.  I included the ‘-v’ flag to show the verbose output so you can see each unit test that is run (only one in this case).

To help illustrate the verbose output, update the unit test file (basicfunction.ut.py) to include a second unit test:

import unittest

class TestBasicFunction(unittest.TestCase):
    def test_1(self):
        self.assertTrue(True)

    def test_2(self):
        self.assertTrue(True)

if __name__ == '__main__':
    unittest.main()

Now the output from running the unit tests look like:

$ python basicfunction.ut.py -v
test_1 (__main__.TestBasicFunction) ... ok
test_2 (__main__.TestBasicFunction) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Basic Example

Not a bad start to our unit testing, but this isn’t a very complex example.  To increase the complexity somewhat, let’s actually test the source code in basicfunction.py.

Start by importing the class of interest (BasicFunction) from the basicfunction.py file, add a setUp() method to create a local instance of this class, and then write a third unit test to check the initialization of this class:

import unittest
from basicfunction import BasicFunction

class TestBasicFunction(unittest.TestCase):
    def setUp(self):
        self.func = BasicFunction()

    def test_1(self):
        self.assertTrue(True)

    def test_2(self):
        self.assertTrue(True)

    def test_3(self):
        self.assertEqual(self.func.state, 0)

if __name__ == '__main__':
    unittest.main()

Be careful with how you import the BasicFunction class, as simply using “import basicfunction” or “import BasicFunction” will give you an error, such as:

Traceback (most recent call last):
File "basicfunction.ut.py", line 2, in <module>
import BasicFunction
ImportError: No module named BasicFunction

Once the import is correct (“from basicfunction import BasicFunction”), here’s the output that you get from running the unit tests:

$ python basicfunction.ut.py -v
test_1 (__main__.TestBasicFunction) ... ok
test_2 (__main__.TestBasicFunction) ... ok
test_3 (__main__.TestBasicFunction) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.003s

OK

Here are a few addition test cases that show that we can fully test all of the methods within the BasicFunction class:

import unittest
from basicfunction import BasicFunction

class TestBasicFunction(unittest.TestCase):
    def setUp(self):
        self.func = BasicFunction()

    def test_1(self):
        self.assertTrue(True)

    def test_2(self):
        self.assertTrue(True)

    def test_3(self):
        self.assertEqual(self.func.state, 0)

    def test_4(self):
        self.func.increment_state()
        self.assertEqual(self.func.state, 1)

    def test_5(self):
        self.func.increment_state()
        self.func.increment_state()
        self.func.clear_state()
        self.assertEqual(self.func.state, 0)

if __name__ == '__main__':
    unittest.main()

Here’s the output of running the unit tests now that we have 5 test cases:


$ python basicfunction.ut.py -v
test_1 (__main__.TestBasicFunction) ... ok
test_2 (__main__.TestBasicFunction) ... ok
test_3 (__main__.TestBasicFunction) ... ok
test_4 (__main__.TestBasicFunction) ... ok
test_5 (__main__.TestBasicFunction) ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.003s

OK

Good progress!  We’ve got a basic unit test structure in place for testing our single class.

More Complex Example

Now it’s time to improve the structure of our project to more closely match how a python project should be structured:

    project2
        project2
            basicfunction.py
        test
            test_basicfunction.py

Copy over the latest versions of each of these files from the previous example into this new directory structure (there should be 5 test cases in your unit test file).  Let’s see if we can still run our unit tests by changing into the ‘unit_tests’ directory:

$ pwd
…/project2/test
$ python test_basicfunction.py -v
Traceback (most recent call last):
File "test_basicfunction.py", line 3, in &lt;module&gt;
from basicfunction import BasicFunction
ImportError: No module named basic function

With our new directory structure, python is not able to locate the basicfunction file.  This is slightly complicated, but based on the python documentation regarding modules, the python interpreter looks for an imported module in the following order:

  1. built-in module (such as unittest, os, …)
  2. directory containing the input script (or the current directory)
  3. directories specified by the PYTHONPATH environment variable
  4. installation-dependent default

Unfortunately, our basicfunction.py file is not located in any of these directories.  Remember in the previous example where we had the source code and unit test files in the same directory?  That made our life easy, as the python interpreter found the source code file in the current directory (#2 from the list above).

Starting with python 2.7, there is a very convenient way to run your unit tests: unit test discovery.  In order to use this nice feature, you need to have your directory structure set up similar to our current structure, with the __init__.py files added:

    project2
        project2
            __init__.py
            basicfunction.py
        test
            __init__.py
            test_basicfunction.py

Be very careful when naming “__init__.py”, as I ran into a very frustrating bug when I had only one underscore after init in my test directory (…/test/__init_.py instead of …/test/__init__.py).  What is the __init__.py file needed for?  This files indicates that the directory that it is in is a python package.  I’ll provide an example later to help illustrate this point…

There are some additional guidelines to follow when using unittest:

  • The directory containing your test cases should be named ‘test’.
  • The unit test files should be of the format test_*.py.

OK, let’s get these unit tests working with our current file/directory structure.  To re-cap, here is our directory structure:

    project2
        project2
            __init__.py
            basicfunction.py
        test
            __init__.py
            test_basicfunction.py

Both version of __init__.py are blank.

Here is the content of …/project2/project2/basicfunction.py:

class BasicFunction(object):
    def __init__(self):
        self.state = 0

    def increment_state(self):
        self.state += 1

    def clear_state(self):
        self.state = 0

Here is the content of …/project2/test/test_basicfunction.py:

import unittest
from basicfunction import BasicFunction

class TestBasicFunction(unittest.TestCase):
    def setUp(self):
        self.func = BasicFunction()

    def test_1(self):
        self.assertTrue(True)

    def test_2(self):
        self.assertTrue(True)

    def test_3(self):
        self.assertEqual(self.func.state, 0)

    def test_4(self):
        self.func.increment_state()
        self.assertEqual(self.func.state, 1)

    def test_5(self):
        self.func.increment_state()
        self.func.increment_state()
        self.func.clear_state()
        self.assertEqual(self.func.state, 0)

if __name__ == '__main__':
    unittest.main()

Change directories into your top-level directory (…/project2) and run the unit test discovery command:

$ python -m unittest discover -v
test.test_basicfunction (unittest.loader.ModuleImportFailure) ... ERROR

======================================================================
ERROR: test.test_basicfunction (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: test.test_basicfunction
Traceback (most recent call last):
File ".../python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File ".../python2.7/unittest/loader.py", line 232, in _get_module_from_name
__import__(name)
File ".../project2/test/test_basicfunction.py", line 3, in &lt;module&gt;
from basicfunction import BasicFunction
ImportError: No module named basicfunction

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Darn, it didn’t work!  Honestly, I thought this would work when I first wrote it, but I was not clear on how the module imports worked with the python interpreter.  Here’s the issue: we’re running out of the top-level directory and both project2 and test are directories below our current working directory.  Remember back to how python interpreter looks for an imported module in the following order:

  1. built-in module (such as unittest, os, …)
  2. directory containing the input script (or the current directory)
  3. directories specified by the PYTHONPATH environment variable
  4. installation-dependent default

We’re trying to work with #2, which is …/project2.  In order to find the basicfunction module, we need to change our unit test import statement:

Original: from basicfunction import BasicFunction
Updated:  from project2.basicfunction import BasicFunction

After updating …/project2/test/test_basicfunction.py with the updated import statement, you should now be able to run the unit tests:

$ pwd
…/project2
$ python -m unittest discover -v
test_1 (test.test_basicfunction.TestBasicFunction) ... ok
test_2 (test.test_basicfunction.TestBasicFunction) ... ok
test_3 (test.test_basicfunction.TestBasicFunction) ... ok
test_4 (test.test_basicfunction.TestBasicFunction) ... ok
test_5 (test.test_basicfunction.TestBasicFunction) ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.001s

OK

SUCCESS!!! I was really happy to get to this point, as having problems with importing modules in python is a headache.

Let’s re-investigate the need to __init__.py in our directories.  Try deleting all of the *.pyc and __init__.py files in your project:

$ pwd
…/project2
$ rm */*.pyc
$ rm */__init__.py

Now try re-running the unit tests:

$ python -m unittest discover -v

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

That’s odd, no unit tests were run even though we still have all of our unit tests in place.   Since we don’t have __init__.py in our project2 and test directories, the python interpreter does not recognize this directories as python packages.  Taking a baby step, let’s just add __init__.py to the test directory and re-run the unit tests:

$ touch test/__init__.py
$ python -m unittest discover -v
test.test_basicfunction (unittest.loader.ModuleImportFailure) ... ERROR

======================================================================
ERROR: test.test_basicfunction (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: test.test_basicfunction
Traceback (most recent call last):
File ".../python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File ".../python2.7/unittest/loader.py", line 232, in _get_module_from_name
__import__(name)
File ".../project2/test/test_basicfunction.py", line 2, in &lt;module&gt;
from project2.basicfunction import BasicFunction
ImportError: No module named project2.basicfunction

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Ohh no!  Now the python interpreter cannot find our basicfunction module again.  It must be because the basicfunction module is not being considered a python module without __init__.py.  By adding __init__.py to the project2 directory and re-running the unit test:

$ touch project2/__init__.py
$ python -m unittest discover -v
test_1 (test.test_basicfunction.TestBasicFunction) ... ok
test_2 (test.test_basicfunction.TestBasicFunction) ... ok
test_3 (test.test_basicfunction.TestBasicFunction) ... ok
test_4 (test.test_basicfunction.TestBasicFunction) ... ok
test_5 (test.test_basicfunction.TestBasicFunction) ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.000s

OK

Back to our successful state!

Conclusion

This blog post was intended to provide a walkthrough of how to set up a basic and then more complex unit test structure for your project.  There are a lot of nuances to this setup, especially when getting the module imports to work correctly.  Once this structure is in place, you should have a nice workflow for developing your unit tests!

Good References:

Blog Post providing an Introduction to Unit Test in Python
http://pythontesting.net/framework/unittest/unittest-introduction/

Unittest framework documentation
https://docs.python.org/2/library/unittest.html#module-unittest

Introductory Tutorial on Python Unit Testing (including a detailed list of all of the assert statements available)
https://cgoldberg.github.io/python-unittest-tutorial/

2 Comments

  1. Phillip Givens

    May 14, 2017 at 7:00 pm

    Thank you so much! This discovery mechanism is ridiculous!

  2. This is just what I was looking for and is explained precisely. Thanks!

Leave a Reply

Your email address will not be published.

*