Disable Instrumentation Packages during Tests2020-07-14
Your Django project may use some packages that add instrumentation, for example:
- A development tool, such as django-debug-toolbar
- An error handling tool, such as Rollbar or Sentry.
- An Application performance Monitoring (APM) tool, such as New Relic or Scout APM.
Such packages often work by tracking every database query, cache operation, etc. The overhead for all this instrumentation is normally significant. For example, on a couple of project’s test suites, I’ve seen Sentry add about 8% to the total test run time.
Since you often don’t need their functionality during tests, you can disable these packages in your test settings file for a speed boost. Some packages provide an option to disable their instrumentation features. If that’s not available, you can change your settings to never load them during tests.
Here are some examples on how to disable such packages. If you are using a different instrumentation package, hopefully you can find similar configuration. (And please let me know the package and configuration so I can add it to this list.)
If you’ve followed the normal install instructions and haven’t customized
SHOW_TOOLBAR_CALLBACK, the toolbar is only active when
DEBUG = True.
Therefore you should only need to check that you don’t run tests in debug mode (more on this in the book).
Rollbar’s Django setup documentation lists its various settings, configured in Django through the
ROLLBAR setting dict.
If you set
False, the Rollbar middleware will disable itself and never install instrumentation:
If you use a single settings file powered by environment variables, you can do this by installing Sentry only when the environment variable is there:
If you use multiple settings files, only include the calls to
sentry_sdk.init() in the relevant files, such as staging and production.
If you’re using the default installation method, New Relic only adds its instrumentation when you run your application via
This means there’s no overhead during tests.
But if you’re using the manual integration, you’ll have a call to
newrelic.agent.initialize() in your
wsgi.py or similar.
You should make sure that this isn’t called during tests, perhaps by using an environment variable to prevent it.
Full disclosure: I’m the maintainer for the Scout APM Python agent.
Scout only installs its instrumentation if its
monitor setting is
This can be set via your Django settings or environment variables as
It defaults to
False, so ensure it’s either not set, or set explicitly to
False, during tests.
I hope this tip has left your test suite faster,
Working on a Django project? Check out my book Speed Up Your Django Tests which covers loads of best practices so you can write faster, more accurate tests.
One summary email a week, no spam, I pinky promise.
© 2020 All rights reserved.