Python Type Hints - How to Upgrade Syntax with pyupgrade
A couple of recent PEP’s have made writing type hints easier:
- PEP 585 added generic syntax to builtin types. This allows us to write e.g.
list[int]instead of usingtyping.List[int]. - PEP 604 added the
|operator as union syntax. This allows us to write e.g.int | strinstead oftyping.Union[int, str], andint | Noneinstead oftyping.Optional[int].
PEP 585 was released in Python 3.9, and PEP 604 will be released in Python 3.10, which is currently in beta. But we can use both syntaxes today, even from Python 3.7, with Mypy 0.800+ and postponed evaluations with from __future__ import annotations.
For example, take this code, which uses the old syntaxes:
from typing import Dict, List, Optional, Union
x: Dict[str, int] = {"a": 1}
y: Union[int, float] = 2.2
z: List[Optional[int]] = [1, None, 3]
We can rewrite it like so:
from __future__ import annotations
# PEP 585 builtin generics
x: dict[str, int] = {"a": 1}
# PEP 604 union operator
y: int | float = 2.2
# Combining them
z: list[int | None] = [1, None, 3]
But what about all our old code? Are we now burdened with two ways of doing things, until we do a lot of manual rewriting?
No!
Thanks to the pyupgrade tool by Anthony Sottile, we can automatically upgrade our syntax. pyupgrade knows how to rewrite many forms of old Python syntax into new equivalents.
We can specify our target Python version with a flag, e.g. --py39-plus for Python 3.9. pyupgrade will activate its PEP 585 and 604 type hint rewrites for appropriate target Python versions, or for files using from __future__ import annotations.
For example, we can perform the above rewrite like so:
$ python -m pip install pyupgrade
Collecting pyupgrade
Downloading pyupgrade-2.18.1-py2.py3-none-any.whl (50 kB)
|████████████████████████████████| 50 kB 1.4 MB/s
Collecting tokenize-rt>=3.2.0
Using cached tokenize_rt-4.1.0-py2.py3-none-any.whl (6.1 kB)
Installing collected packages: tokenize-rt, pyupgrade
Successfully installed pyupgrade-2.18.1 tokenize-rt-4.1.0
$ pyupgrade example.py
Rewriting example.py
This rewrites the syntax, but leaves the imports in place:
from __future__ import annotations
from typing import Dict, List, Optional, Union
x: dict[str, int] = {"a": 1}
y: int | float = 2.2
z: list[int | None] = [1, None, 3]
Great! Now all we have to deal with is the old typing imports which are now unused. We could remove them manually, but there’s another* tool that can help with that: isort, by Timothy Crosley. isort’s --rm flag allows us to drop an import from target files.
(There’s a second tool, reorder_python_imports, also by Anthony Sottile. It has a similar --remove-import flag. I’m just used to isort.)
In this case we want to run it like so:
$ python -m pip install isort
Collecting isort
Using cached isort-5.8.0-py3-none-any.whl (103 kB)
Installing collected packages: isort
Successfully installed isort-5.8.0
$ isort --rm 'from typing import Dict' --rm 'from typing import List' --rm 'from typing import Optional' --rm 'from typing import Union' example.py
Fixing /Users/chainz/tmp/hints/example.py
And now our code is cleanly upgraded to the shiny new syntax:
from __future__ import annotations
x: dict[str, int] = {"a": 1}
y: int | float = 2.2
z: list[int | None] = [1, None, 3]
Nice!
It’s even better if we run pyupgrade and isort across our codebase for every change. This will ensure we don’t add back old syntax accidentally, e.g. when copy-pasting from outdated blog posts. A great way of doing this is with the pre-commit framework, which both tools advertise integrations with (see: pyupgrade, isort).
If your Django project’s long test runs bore you, I wrote a book that can help.
One summary email a week, no spam, I pinky promise.
Related posts: