Adam Johnson

Home | Blog | Projects | Colophon

Django's Test Case Classes and a Three Times Speed-Up

2019-07-15

Rearing Pony

This is a story about how I sped up a client’s Django test suite to be three times faster, through swapping the test case class in use.

Speeding up test runs is rarely a bad thing. Even small teams can repeat their test run hundreds of times per week, so time saved there is time won back. This keeps developers fast, productive, and happy.

Django’s Test Case Classes

A quick refresher on how the Django’s three basic test case classes affect the database:

The distinction between TransactionTestCase and TestCase can be confusing. Here’s my attempt to summarize it in one sentence:

TransactionTestCase that allows your code to use transactions, whilst TestCase uses transactions itself.

The Speed-Up Story

Recently I was helping my client ev.energy improve their Django project. A full test run took about six minutes on my laptop when using the test command’s --parallel option. This isn’t particularly long - I’ve worked on projects where it took up to 30 minutes! But it did give me a little time during runs to look for easy speed-ups.

Their project uses a custom test case class for all their tests, to add extra helper methods. It originally extended TransactionTestCase, with its slower but more complete database reset procedure. I wondered why this had been done.

I searched the Git history for the first use of TransactionTestCase with git log -S TransactionTestCase (a very useful Git option!). I found a developer had first used it in tests for their custom background task class called Task.

Task closed the database connection at the end of its process with connection.close(). This helped isolate the tasks. Since they’re run in a long running background process, using a fresh database connection for each task helped prevent a failure in one from affecting the others.

Unfortunately the call to connection.close() prevented use of TestCase when testing Task classes. Closing the database connection also ends any transactions. So when TestCase ran its teardown process, it errored when trying to roll back the transactions it started in its setup process.

Because of this, the developers used TransactionTestCase for their custom test case class. And they stuck with it as the project grew.

This was all fair, and the speed difference would not have been noticeable when there were fewer tests. Fixing it then allowed them to focus on feature development.

But as with test time things like this, the seconds added up over time. Much like the metaphorical frog in a slowly boiling pot of water.

Once I’d discovered this piece of history, I guessed most of the tests that didn’t run Task classes would work with TestCase. I swapped the base of the custom test class to TestCase, reran, and only the Task tests failed!

After changing only broken test classes back to TransactionTestCase, I reran the suite and everything passed. The run time went down from 375 seconds to 120 seconds. A three times speed-up!

Fin

I hope this post helps you find the right test case class in your Django project. If you want help with this, email me - I’m happy to answer any questions, and am available for contracts. See my front page for details.

Update (2019-07-05): Fellow Django core contributor Luke Plant shared a similar story on Twitter. He saw a test suite go from 20 minutes down to 7 - about three times faster again!

Thanks for reading,

—Adam


Subscribe via RSS, Twitter, or email:

Related posts:

Tags: django