Adam Johnson

Home | Blog | Training | Projects | Colophon | Contact

Disable Instrumentation Packages during Tests

2020-07-14 Don't be a sheep with your instrumentation

This post is an adapted extract from the Easy Wins chapter of my book Speed Up Your Django Tests, available now. This tip could take 8% off your tests' run time with a small configuration change!

Your Django project may use some packages that add instrumentation, for example:

Such packages often work by tracking every database query, cache operation, etc. The overhead for all this instrumentation is normally significant. For example, on a couple of project’s test suites, I’ve seen Sentry add about 8% to the total test run time.

Since you often don’t need their functionality during tests, you can disable these packages in your test settings file for a speed boost. Some packages provide an option to disable their instrumentation features. If that’s not available, you can change your settings to never load them during tests.

Here are some examples on how to disable such packages. If you are using a different instrumentation package, hopefully you can find similar configuration. (And please let me know the package and configuration so I can add it to this list.)

django-debug-toolbar

If you’ve followed the normal install instructions and haven’t customized SHOW_TOOLBAR_CALLBACK, the toolbar is only active when DEBUG = True. Therefore you should only need to check that you don’t run tests in debug mode (more on this in the book).

Rollbar

Rollbar’s Django setup documentation lists its various settings, configured in Django through the ROLLBAR setting dict. If you set enabled to False, the Rollbar middleware will disable itself and never install instrumentation:

ROLLBAR['enabled'] = False

Sentry

If you use a single settings file powered by environment variables, you can do this by installing Sentry only when the environment variable is there:

if "SENTRY_DSN" in os.environ:
    import sentry_sdk

    sentry_sdk.init(...)

If you use multiple settings files, only include the calls to sentry_sdk.init() in the relevant files, such as staging and production.

New Relic

If you’re using the default installation method, New Relic only adds its instrumentation when you run your application via newrelic-admin run-program. This means there’s no overhead during tests.

But if you’re using the manual integration, you’ll have a call to newrelic.agent.initialize() in your wsgi.py or similar. You should make sure that this isn’t called during tests, perhaps by using an environment variable to prevent it.

Scout APM

Full disclosure: I’m the maintainer for the Scout APM Python agent.

Scout only installs its instrumentation if its monitor setting is True. This can be set via your Django settings or environment variables as SCOUT_MONITOR. It defaults to False, so ensure it’s either not set, or set explicitly to False, during tests.

Fin

I hope this tip has left your test suite faster,

—Adam


Working on a Django project? Check out my book Speed Up Your Django Tests which covers loads of best practices so you can write faster, more accurate tests.


Subscribe via RSS, Twitter, or email:

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: django