Testing Django Applications (2)

Introduction

The previous blog post has introduced the Django TestCase, DjangoTestClient and pytest runner and its plugins (most notably pytest-django).

In this post, we'll continue with:

• Setting up testing content
• Testing form submissions
• Isolating code for testing
• Testing APIs
• Email, parametrization, counting queries, changing current time
• Marking tests
• Writing readable tests
• Rules of thumb

It looks there's still more to tell:

• Making tests part of the process (CI)
• Testing Javascript: Playwright, Selenium, ...
• Testing Django+HTMX
• ...

Maybe we'll get to that in another blog post.

Setting up testing content

Often your tests will require specific content. For example, let's consider a webshop. If you want to test an order, you will need a hierarchy of Django objects:

• An order
• An order line
• A product

That's a lot to set up.

As we mentioned before, you can do that with Django fixtures but that's clumsy. Your data will be in an incomprehensible format which is hard to maintain.

factoryboy

We recommend using factoryboy. Creating an order can be as simple as:

def test_order():
    order = OrderLineFactory().order

Where, in factories.py, we have:

class OrderFactory(factory.django.DjangoModelFactory):

    user = factory.SubFactory(UserFactory)

    class Meta:
        model = Order

class ProductFactory(factory.django.DjangoModelFactory):

    # generate "random" codes
    product_code = factory.Sequence(lambda n: f"upc-{n}")

    class Meta:
        model = Product

class OrderlineFactory(factory.django.DjangoModelFactory):

    order = factory.SubFactory(OrderFactory)
    product = factory.SubFactory(ProductFactory)
    quantity = 1

    class Meta:
        model = Orderline

As you see, we create an order with a quantity of one item per line.

But if you'd want to test an order for a different quantity, you can easily override this in your test:

orderline_qty_2 = OrderlineFactory(quantity=2)

Your test specifies exactly what it needs for the test. The result is a test that is very readable, because it exactly specifies its preconditions.

factoryboy or pytest-factoryboy?

In the previous example, we have been using factoryboy directly. It already has support for ORMs like Django's, so there's no need for an extra plugin.

But there's also pytest-factoryboy which adds some features.

Factory fixtures

pytest-factoryboy allows you to define an OrderFactory and register it in such a way that you get an order fixture.

We don't recommend this pattern, as both humans and IDEs will struggle to see where this is coming from.

Explicit is better than implicit. Just import your factory class, and create an object with it.

Testing form submission

Testing a form submission with Django's standard tools is as follows: You can test a POST request in the Django test client. You will need to specify exactly what to post. You will need to have that data match the fields on the Form class.

That's quite suboptimal. But we also have django-webtest.

django-webtest

Django-webtest gives you a different test client (webtest, from the Pylons project).

The webtest client has, among other things, a form attribute which makes it easy to interact with forms.

With pytest, the client is a fixture called django_app (but it also works with class-based tests):

def test_submit_name(django_app):
	page = django_app.get(reverse("add_name"))
	form = page.form
	form["name"] = "My name"
	result_page = form.submit().follow()
	assert Name.objects.last().name == "My name"

This shows we can interact with the form almost like a user would.

Some more django-webtest benefits

Django-webtest also makes other aspects of user interaction testing easier:

• You can click in a page
• You can do <string> in response to test whether the string is found in the response body.
• more powerful response body parsing

Isolating code for testing

When writing tests, it's always a good idea to test small chunks of code separately.

Mocking a method

Let's say you have a form which, after successful submit, should send an email.

You could (or should) split up the test in two parts:

1. Test that the form can be submitted, which calls a method that sends the e-mail
2. Test the e-mail sending method in isolation

In the test, below we do the first:

def test_submit_send_email(django_app, mocker):
    mocked_send = mocker.patch("myapp.utils.send_email")
	...
	result_page = form.submit().follow()
	mocked_send.assert_called_once()

The mocker fixture come from the pytest-mock plugin. Without pytest, you'd use Mocker directly.

The assert_called_once method here tests that our mail sending method was, well, called once. We can also check which arguments it was called with (user id, type of message, etc.).

Mocking an external API

Let's say you use an external API to get a list of available contact persons.

You wrote some simple code that reads the API, and we're going to call that your API client.

What you don't want is that when testing this client, a call to the real API is done. That is slow, might cost you tokens, or get your account suspended. It gets even more obvious when you want to test using the API to create things.

So we're going to act as if the API responded in a certain way:

def test_endpoint_list_success(mocked_responses):
	client = ApiClientFactory()
	mocked_responses.get(
		"https://some.api.com/Contact/GetList",
		json={"contacts": [
                    {"id": "walter"}, 
                    {"id": "donny"},
                ]},
	)
	assert client.contacts.get_id_list() == [
            "donny", "walter"
        ]

There are two packages used for mocking API responses. In the example above, we used Sentry's responses. To make it give us the mocked_responses fixture, we have in our conftest.py:

@pytest.fixture
def mocked_responses():
	with responses.RequestsMock() as rsps:
		yield rsps

There's also requests-mock, which is also works well but is subtly different.

The main difference is that it allows you to inspect the request payload in your test. In Sentry's responses, you can only do this by specifying "matchers" when setting up the mocked response (requests-mock also has this).

Testing your own API

Let's say you yourself have written an API, probably using some framework.

Best place to start is the framework's documentation:

• For example, if you used django-rest-framework, that comes with its own test client. It has some nice features for testing APIs specifically.
• fastapi also has its own test client.
• django-ninja recommends using Django's test client.
  (But I bet you could also use DRF's test client for this.)

Testing mail

We'll revisit our "mocking" example above. Here we test the e-mail sending method itself.

from myapp.utils import send_email

def test_send_email(mailoutbox):
    assert len(mailoutbox) == 0
    send_email()
    assert len(mailoutbox) == 1
    assert mailoutbox[0].subject == "My email subject"

mailoutbox is a pytest fixture. In plain Django tests you'd use django.core.mail.outbox.

Parametrization

To test the same thing with different value, you can parametrize tests. This executes exactly the same test, with different variables.

In the example below, we have a method is_prime that returns True if the input is a prime number, and False if not.

from myapp.utils import is_prime

@pytest.mark.parametrize(
    "number, expected_prime",
    [
        (1, True),
        (2, True),
        (3, True),
        (4, False),
        (5, True),
        (6, False),
    ],
)
def test_is_prime(number, expected_prime):
    assert is_prime(number) is expected_prime

This is a pytest mark.

To do the same thing without pytest, you might try parameterized.

Counting queries

Of course you always try to program robustly, and avoid N + 1 problems. You and your colleagues would never write code that suddenly makes the number of SQL queries explode, and render your app unusable, right? And next year, you'll still remember to prefetch an order's lines, and the lines' products, and all other necessary attributes, right?

Luckily, we don't have to be in doubt about that, because we can test it.

Here we test that the number of SQL queries done by the order list view does not depend on the number of orders a user has.

@pytest.mark.parametrize("number_of_orders", [1, 2, 3])
def test_order_list_queries(
    django_app, 
    django_assert_num_queries, 
    number_of_orders,
):
    owner = UserFactory()
    OrderFactory.create_batch(
        size=number_of_orders, user=owner
    )
    with django_assert_num_queries(37):
        django_app.get(reverse("order_list"), user=owner)

django_assert_num_queries is a pytest-django fixture. The Django runner equivalent is assertNumQueries.

Note that this does not keep you from creating extremely complicated and long-running queries. It's not an alternative to actual load testing. But it does keep you safe from the first pitfalls.

Travelling through time

Sometimes you need to go back to 1955:

@pytest.mark.freeze_time("1955-11-05")
def test_current_year():
	assert datetime.now().year == 1955

This marker is provided by pytest-freezegun. It builds on freezegun, which is what you can use if you don't use pytest.

Write readable tests

Tests are read when they fail. The person reading them doesn't know what the test is for, or has forgotten (you).

It's your task as a test author to explain to your colleague (or future self) what the idea was, so they can fix the test, fix their code, or both.

Give meaningful names

If you're doing this:

def test_user_permissions():
    user_1 = UserFactory(role=User.RoleChoices.EDITOR)
    user_2 = UserFactory(role=User.RoleChoices.AUTHOR)
    ...

Why not call them editor and author then?

And there's probably a better name for the test method than test_user_permissions. Maybe test_editor_can_edit ?

So, assuming this was implemented using a method is_editable_by on a Page model:

def test_editor_can_edit_any_page():
    editor = UserFactory(role=User.RoleChoices.EDITOR)
    author = UserFactory(role=User.RoleChoices.AUTHOR)
    page = PageFactory()
    assert page.is_editable_by(author) is False
    assert page.is_editable_by(editor) is True

Custom assertion message

Consider the following assertion:

assert Order.objects.for_user(joe).count() == 4

On failure, this gives:

E       AssertionError: assert 3 == 4
E        + where 0 = Order.objects.for_user(<Joe>).count()

Now compare with this assertion:

assert (
	Order.objects.for_user(joe).count() == 4
), "Joe should not see archived orders"

which gives:

E       AssertionError: Joe should not see archived orders
E       assert 3 == 4
E        +  where 3 = Order.objects.for_user(<joe>).count()

The second assertion, through its custom assertion message, gives extra information about what the test is trying to achieve.

Docstrings and comments

When the test name and the variables still don't tell the whole story, add comments and docstrings as needed.

In our code examples we removed docstrings for brevity. But they can help a lot to understand quickly what a test does.

Comments can quickly explain what a code block does, so the reader can quickly decide whether that's relevant to the test.

def test_view_documentation_success(
    django_app, tmp_path, settings
):
    """Docs view should serve HTML from build dir."""
    # Create a dummy file in a tmp dir, so this test
    # does not depend on generated HTML files.
    index_file = tmp_path / "index.html"
    index_file.write_text("Hoeba")

    # Allow django-sendfile2 to send it
    settings.OMS_DOCS_BUILD_DIR = tmp_path
    settings.SENDFILE_ROOT = tmp_path

More is not always better. Especially, repeating in English what the code is doing is not helpful. Assume that the reader can read the code, but wants to know why it should do what we test.

Don't be repetitive like in this example:

def test_user_is_active():
    """Test that user is active.

    In this test, we test that a user is active.
    """"
    # Use should be active
    assert (
        UserFactory().is_active() is True
    ), "User is not active"

Instead, be as brief as possible, and give the reason (here we just invented one):

def test_new_user_is_active():
    assert (
        UserFactory().is_active() is True
    ), "New user should be active to use password reset."

Rules of thumb

Keep it simple

Ideally, most of your tests should be short (max 30 lines), so they're easy to read. There may also be a place for a longer "complete happy flow" test, to show how it all fits together. But in general, shorter is better.

If you feel you have to write a lot of code to test something, it may not be the right direction. Taking a step back, and talking about it with someone else, can help.

Testing with real data, in a real database, should never occur. Use factories. If setting up this data takes long, re-use the setup.

Adapting code to test it?

Sometimes, it is only during testing that we find ways to simplify and clarify our code:

• Changing object methods to functions is a win. You're also opening up the door to using type annotations, which is recommended.
• Splitting up large functions into small ones is also good.
• Getting business logic out of view classes and moving it to separate files is also a good idea.

These are all good ideas, even before you consider testing. Ask yourself: "Would this be an improvement if it weren't for testing?"

When you find you are adding to the codebase just to test something, it's probably not a good idea.