Why does Python log a DeprecationWarning saying “invalid escape sequence”?

2022-11-04 Invalid essssscape sssssequence...

Take this code, which prints a shruggy man:

print("¯\_(ツ)_/¯")

If you run it with warnings enabled on Python 3.11, you’ll see:

$ python -W default example.py
/.../example.py:1: DeprecationWarning: invalid escape sequence '\_'
  print("¯\_(ツ)_/¯")
¯\_(ツ)_/¯

(On Python 3.12+, you’ll see SyntaxWarning instead, with no need to enable warnings.)

What does this mean?

Well, Python strings use backslash (\) and the next character as an escape sequence, that represents a special character Here are some common examples:

Escape Sequence	Meaning
`\n`	newline
`\'`	single quote
`\"`	double quote
`\\`	a single backslash

Python’s documentation on strings has a complete table.

The combination of \ plus a second character is called an escape sequence. The warning “invalid escape sequence” is telling you that you’ve written \ and then an unsupported character.

Python still interprets the string as \ plus the invalid character, hence the example shruggy emoticon displays correctly. But a future version of Python (maybe 3.13) will raise a SyntaxError for invalid escape sequences, stopping such code from running. Then, Python will be able to add more valid escape sequences in even later versions.

To fix the issue, you will normally want to add a second \, which is the valid escape sequecne for “a single backslash“:

print("¯\\_(ツ)_/¯")

No more warning:

$ python -W default example.py
¯\_(ツ)_/¯

Great.

You can also solve this by changing the string to a raw string, by adding a r prefix:

print(r"¯\_(ツ)_/¯")

Raw strings don’t use escape sequences at all, so every backslash will appear as-is. So, this change will break any valid escape sequences in the same string.

(Raw strings are particularly useful when writing regular expressions for the re module. Regular expression syntax uses \ as its own escape character, so using a raw string means you don’t need to double them up.)

Auto-fix this problem with pyupgrade

pyupgrade is a tool for upgrading Python code to the latest syntax. It has a fixer to rewrite invalid escape sequences automatically.

For example, running pyupgrade on the initial example:

$ pyupgrade example.py
Rewriting example.py

…it makes the change to a raw string:

-print("¯\_(ツ)_/¯")
+print(r"¯\_(ツ)_/¯")

Cool.

pyupgrade uses a raw string if the string only contains invalid escape sequences. If it contains any valid ones, it will instead add extra backslashes where necessary.

For example, if the string has a newline escape sequence:

print("¯\_(ツ)_/¯\n")

…then pyupgrade only doubles up the \:

-print("¯\_(ツ)_/¯\n")
+print("¯\\_(ツ)_/¯\n")

Very nice.

pyupgrade also has a nuber of other fixes to help you modernize your code.

Lint for this problem with flake8

If you aren’t ready to automatically fix your code, you can use the linter flake8 to detect invalid escape sequences early. The default configuration notes W605 for each instance:

$ flake8 example.py
example.py:1:9: W605 invalid escape sequence '\_'

Cool.

You can run flake8 in your text editor, to catch issues whilst you write your code.

Fin

—Adam

Learn how to make your tests run quickly in my book Speed Up Your Django Tests.

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: python