How to search your virtualenv for mystery error messages

Now, look here… (generated by Stable Diffusion)

Sometimes error reporting loses details, and you don’t know where an error message comes from. In such cases, I have often found that the fastest way to track down the source is to search through the project and its virtualenv.

ripgrep is the best tool for such seaches. It’s a much-much faster version of classic grep, with bonus features.

In this post we’ll look at how to search your virtualenv with ripgrep, a real life example, and how to do the same with plain grep.

An Ansible eggsample

A few months ago, I was debugging an issue where Ansible would quit with a mysterious message, and no other details:

ERROR! A worker was found in a dead state

🤔🤔🤔

In order to find where in Ansible this message was triggered, I ripgrepped for it:

$ rg -uu -tpy 'ERROR! A worker was found in a dead state'

…but unfortunately this first search turned up zero results.

I figured the ERROR! part may be part of a generic error message display function, so I dropped that for a second search:

$ rg -uu -tpy 'A worker was found in a dead state'
venv/lib/python3.10/site-packages/ansible/plugins/strategy/__init__.py
837:                raise AnsibleError("A worker was found in a dead state")
863:                raise AnsibleError("A worker was found in a dead state")

Aha! From these results, I could dig in to the issue further. In this case I temporarily modified Ansible’s code to display more information.

In general, if your first searsch has no matches, try searching for just a part of the error message. Error logging often adds prefixes or suffixes, and message strings in code can be broken across multiple lines.

Alternatively with grep -r

If you can’t use ripgrep, plain old grep works as well. It doesn’t have any default filtering, so to search all files in the current directory, run it with -r for recursive:

$ grep -r 'Error message'

To filter to just .py files:

$ grep -r --include '*.py' 'Error message'

And if your virtualenv doesn’t live within your project, you can again specify multiple paths:

$ grep -r --include '*.py' 'Error message' . ~/.virtualenvs/example

Allllrighty then.

Beware that grep is much slower than ripgrep. On my machine, ripgrep is nearly ten times faster for an equivalent search.

Fin

For another example, see Anže Pečar’s blog post on searching through his virtualenv for vendored copies of six.

May your debugging adventures be fun and no longer than necessary,

—Adam


Make your development more pleasant with Boost Your Django DX.


Subscribe via RSS, Twitter, Mastodon, or email:

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: