The Fast Way to Test Django transaction.on_commit() Callbacks2020-05-20
transaction.on_commit() hook is useful for running tasks that rely on changes in the current database transaction.
The database connection enqueues callback functions passed to
on_commit, and executes the callbacks after the current transaction commits.
If the transaction is rolled back, the callbacks are discarded.
This means they act if-and-when the final version of the data is visible to other database connections.
It’s a best practice to use
on_commit for things like sending external emails or enqueueing Celery tasks.
(See my previous post Common Issues Using Celery (And Other Task Queues).)
Unfortunately, testing callbacks passed to
on_commit() is not the smoothest.
The Django documentation explains the problem:
TestCaseclass wraps each test in a transaction and rolls back that transaction after each test, in order to provide test isolation. This means that no transaction is ever actually committed, thus your
on_commit()callbacks will never be run. If you need to test the results of an
on_commit()callback, use a
TransactionTestCase is correct and works for such tests, but it’s much slower than
Its rollback behaviour flushes every table after every test, which takes time proportional to the number of models in your project.
So, as your project grows, all your tests using
TransactionTestCase get slower.
I cover this in my book Speed Up Your Django Tests in the section “
TestCase Transaction Blockers.”
on_commit() is one thing that can force you to use
TransactionTestCase, “blocking” you from the speed advantages of
Thankfully there’s a way to test them using
TestCase, with a little help from a targeted mock.
Django Ticket #30457 proposes adding a function for running
on_commit callbacks inside
I’ve used a snippet similar to those posted on the ticket in several client projects, so I figured it was time to pick up the ticket and add it to Django core.
I’ve thus made PR #12944 with
My PR is awaiting review and (hopefully) a merge, and it targets Django 3.2 which will be released nearly a year from now in April 2021. I’ve thus released it in a separate package django-capture-on-commit-callbacks, available now for Django 2.0+.
After you’ve installed it and added it to your project’s custom
TestCase class, you can use it like so:
These tests POST to a view at
/contact that uses an
on_commit() callback to send an email.
captureOnCommitCallbacks() causes it to execute the captured callbacks as its context exits.
The assertions are then able to check the HTTP response, the number of callbacks enqueued, and the sent email.
For more information see django-capture-on-commit-callbacks on PyPI.
I hope this helps speed up your tests,
Working on a Django project? Check out my book Speed Up Your Django Tests which covers loads of best practices so you can write faster, more accurate tests.
One summary email a week, no spam, I pinky promise.
- Common Issues Using Celery (And Other Task Queues)
- Django's Test Case Classes and a Three Times Speed-Up
Tags: django, python
© 2021 All rights reserved.