Python Type Hints - How to Upgrade Syntax with pyupgrade

2021-05-21 Ratcheting up the Python versions.

A couple of recent PEP’s have made writing type hints easier:

PEP 585 was released in Python 3.9, and PEP 604 will be released in Python 3.10, which is currently in beta. But we can use both syntaxes today, even from Python 3.7, with Mypy 0.800+ and postponed evaluations with from __future__ import annotations.

For example, take this code, which uses the old syntaxes:

from typing import Dict, List, Optional, Union

x: Dict[str, int] = {"a": 1}

y: Union[int, float] = 2.2

z: List[Optional[int]] = [1, None, 3]

We can rewrite it like so:

from __future__ import annotations

# PEP 585 builtin generics
x: dict[str, int] = {"a": 1}

# PEP 604 union operator
y: int | float = 2.2

# Combining them
z: list[int | None] = [1, None, 3]

But what about all our old code? Are we now burdened with two ways of doing things, until we do a lot of manual rewriting?

No!

Thanks to the pyupgrade tool by Anthony Sottile, we can automatically upgrade our syntax. pyupgrade knows how to rewrite many forms of old Python syntax into new equivalents.

We can specify our target Python version with a flag, e.g. --py39-plus for Python 3.9. pyupgrade will activate its PEP 585 and 604 type hint rewrites for appropriate target Python versions, or for files using from __future__ import annotations.

For example, we can perform the above rewrite like so:

$ python -m pip install pyupgrade
Collecting pyupgrade
  Downloading pyupgrade-2.18.1-py2.py3-none-any.whl (50 kB)
     |████████████████████████████████| 50 kB 1.4 MB/s
Collecting tokenize-rt>=3.2.0
  Using cached tokenize_rt-4.1.0-py2.py3-none-any.whl (6.1 kB)
Installing collected packages: tokenize-rt, pyupgrade
Successfully installed pyupgrade-2.18.1 tokenize-rt-4.1.0
$ pyupgrade example.py
Rewriting example.py

This rewrites the syntax, but leaves the imports in place:

from __future__ import annotations

from typing import Dict, List, Optional, Union

x: dict[str, int] = {"a": 1}

y: int | float = 2.2

z: list[int | None] = [1, None, 3]

Great! Now all we have to deal with is the old typing imports which are now unused. We could remove them manually, but there’s another* tool that can help with that: isort, by Timothy Crosley. isort’s --rm flag allows us to drop an import from target files.

(There’s a second tool, reorder_python_imports, also by Anthony Sottile. It has a similar --remove-import flag. I’m just used to isort.)

In this case we want to run it like so:

$ python -m pip install isort
Collecting isort
  Using cached isort-5.8.0-py3-none-any.whl (103 kB)
Installing collected packages: isort
Successfully installed isort-5.8.0
$ isort --rm 'from typing import Dict' --rm 'from typing import List' --rm 'from typing import Optional' --rm 'from typing import Union' example.py
Fixing /Users/chainz/tmp/hints/example.py

And now our code is cleanly upgraded to the shiny new syntax:

from __future__ import annotations

x: dict[str, int] = {"a": 1}

y: int | float = 2.2

z: list[int | None] = [1, None, 3]

Nice!

It’s even better if we run pyupgrade and isort across our codebase for every change. This will ensure we don’t add back old syntax accidentally, e.g. when copy-pasting from outdated blog posts. A great way of doing this is with the pre-commit framework, which both tools advertise integrations with (see: pyupgrade, isort).

Fin

I hope this helps you upgrade,

—Adam


🦄 Working on a Django project? Check out my book Speed Up Your Django Tests.


Subscribe via RSS, Twitter, or email:

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: mypy, python