Efficient Reloading in Django’s Runserver With Watchman2021-01-20
If you start the development server on a Django project, it looks something like this:
When you make changes to a Python file, the server automatically reloads.
This is powered by a file watcher, and Django reports which one it’s using in the first line, defaulting to
StatReloader class is simple but reliable.
It works by running a loop that checks all your files for changes every second - this is pretty inefficient, especially as your project grows!
A lesser-known but better alternative is to use Django’s support for watchman. Support was added in Django 2.2, thanks to Tom Forbes in Ticket #27685.
Watchman is an efficient file watcher open sourced by Facebook. It works by receiving file change notifications from your operating system and bundling them together. When nothing is changing, it doesn’t need to do any work - saving processing power and consequently laptop battery life. And when something changes, Django gets notified about it in milliseconds. Watchman is also smart with changes, batching them together, and integrating with Git to wait for operations like switching branches to finish.
It’s just a few steps to get Django using watchman, covered in a couple paragraphs in the
Here I’ll cover the exact commands it needed me to run to set it up for DB Buddy on macOS, which is a pretty vanilla Django 3.1 project.
First, I installed watchman, following the installation instructions on its site.
On macOS, this meant just running
brew install watchman.
Second, I installed
pywatchman, the library Django uses to interface with watchman.
pip-compile from pip-tools to manage my requirements, so first I added
pywatchman to my
I then ran
pip-compile to pin the current version of
pywatchman in my
requirements.txt, resulting in this change:
I then ran
pip install -r requirements.txt to install the compiled requirements file.
Third, I set up a watchman configuration file in order to ignore my
The Django documentation hints that ignoring non-Python directories like this may be sensible to reduce load.
node_modules directory, all of which don’t need watching.
I followed the watchman configuration documentation and created a
.watchmanconfig file next to my
Fourth, I restarted
Django confirmed it’s using watchman by listing
WatchmanReloader as the file watcher on the first line:
I followed this up with a quick before/after comparison of CPU usage, whilst the server was completely idle and not serving any requests.
runserver took about ~1.6% of a CPU core.
WatchmanReloader, this dropped to 0%, with the
watchman process also showing 0%.
This project is also relatively small at the moment, with only 50 Python files and a few dependencies.
StatReloader’s work is proportional to the number of files, the difference will only be more stark on larger projects.
If you’re interested in learning more about the implementation of Django’s watchman integration, check out Tom Forbes’ EuroPython talk.
May your edit cycle be ever faster and more efficient,
Want better tests? Check out my book Speed Up Your Django Tests which teaches you to write faster, more accurate tests.
One summary email a week, no spam, I pinky promise.
- Introducing django-linear-migrations
- Prevent Unintended Data Modification With django-read-only
- Bonus Django Documentation Sites
© 2021 All rights reserved.