How to Use Django's Parallel Testing on macOS With Python 3.8+2020-07-21
test command takes the
--parallel flag to activate parallel testing.
Unfortunately, this has never worked on Windows, and with Python 3.8 it has stopped working on macOS.
Django uses the “fork” method in
multiprocessing to start the new test processes.
Windows does not support “fork”, and whilst macOS does, Python 3.8 changed its default to “spawn”.
If the start method is not “fork” when you run tests with the
--parallel flag, Django silently switches back to serial testing.
The “spawn” method works quite differently to “fork”, requiring a lot of initialization in the new test processes, so the current test framework is incompatible. This is a known issue, Ticket #31169, and it is being worked on by Ahmad A. Hussein as part of Django’s Google Summer of Code 2020. Hopefully we will merge the fix in Django 3.2.
For the time being, there’s a workaround for macOS. Alternatively, you can switch to pytest, since its parallelization plugin, pytest-xdist, works on all platforms.
On macOS on Python 3.8+, you can work around the lack of “spawn” support by reverting the multiprocessing start method to “fork”. This is only a temporary workaround, and has a small risk of introducing system library crashes to your test suite. Python 3.8 changed the macOS default because many system libraries crash when used after forking (Python Issue #33725).
The first step in the workaround is to disable the macOS system protection against forking. Do this by setting this environment variable in the terminal you run tests:
You can do this in your shell initialization file so it always applies - see this Stack Overflow answer.
The second step is to add a call to
multiprocessing.set_start_method() at the top of
main() in your
This is best accompanied with a check that
OBJC_DISABLE_INITIALIZE_FORK_SAFETY has indeed been set:
The check will stop
manage.py test from starting on macOS if
OBJC_DISABLE_INITIALIZE_FORK_SAFETY has not been set.
May your tests run fast in parallel,
Working on a Django project? Check out my book Speed Up Your Django Tests which covers loads of best practices so you can write faster, more accurate tests.
One summary email a week, no spam, I pinky promise.
- Make Django Tests Always Rebuild the Database if It Exists
- Getting a Django Application to 100% Test Coverage
- Avoid Hardcoding ID’s in Your Tests
© 2021 All rights reserved.