Python Type Hints - How to Enable Postponed Evaluation With __future__.annotations2021-05-15
Python 3.7 added a new “postponed evaluation” mode for type hints. This is opt-in for now, and may become the default in Python 3.11. In this post we’ll cover how it changes the behaviour of type hints, how to activate it, and the outstanding problems.
What is postponed evaluation?
When Python introduced type hints, the team decided type hint expressions should execute at runtime. This allowed flexibility and experimentation with the syntax.
Unfortunately it also meant that expressions have to follow Python’s rules about variable scope. This made forward references, to classes that aren’t yet defined, problematic.
For example, take this class that can copy itself:
-> Widget return type hint is a forward reference, as the
Widget class is not defined until the end of its
With the old behaviour, Python tries to evaluate this reference, causing a
Type checkers introduced a workaround: wrapping type hints in quotes, to turn them into strings. Any such strings are then evaluated at the end of the file, by the type checker only. Python’s runtime sees them only as strings. For example:
This code runs successfully and passes type checking.
PEP 563 defined the new postponed evaluation behaviour. This moves all type hints to follow this pattern, without requiring the string syntax.
We can opt into postponed evaluation in a file by adding
from __future__ import annotations at the top.
In our example:
Python no longer interprets the type hints at runtime, so it doesn’t hit the
And type checkers can still work with the type hints as they evaluate them after reading the whole file.
Postponed evaluation is opt-in from Python 3.7, and may become the default in Python 3.11. Originally the PEP planned to make this the default from Python 3.10. But, due to the compatibility problems, the Python Steering Council decided to postpone the change for one more release. Exactly what will change in Python 3.11 is still up for debate, but it be one step closer to turning on postponed evaluation by default.
The compatibility issues came from Python libraries that use type hints at runtime.
Python provides the
typing.get_type_hints() function to fetch the type hints for a function or class at runtime.
Unfortunately due to Python’s scoping rules it doesn’t always work with postponed evaluation.
For example, take this code:
Author class has a reference to
Book, which only exists inside the locals of the call to
When we run the code with the old behaviour, non-postponed evaluation, we see the dict of type hints for
But if we add
from __future__ import annotations to activate the new behaviour, the code crashes:
For now there is no fix in Python for this issue. Some libraries using runtime type hints have workarounds, such as Pydantic’s `update_forward_refs() function.
Thankfully, use of such local-scoped forward references is rare.
If it is stopping you adding
from __future__ import annotations to a whole module, consider splitting off the problematic code into its own module.
Adding to all files
Assuming you’re not on Python 3.11 (unreleased at time of writing), consider adding
from __future__ import annotations to all your files.
This can be done automatically with isort.
You can set up
isort in many ways, for example with the pre-commit framework.
Once you have
isort running, you can set its
add_imports option to add the import.
For example, here’s the configuration I’m using in my
(If you are on Python 3.11, you can swap to the
remove_imports option instead!)
After adding the import to all files, also check out
pyupgrade, which can upgrade your syntax.
For files with
from __future__ import annotations it automatically unquotes stringified type hints.
May you be
Want better tests? Check out my book Speed Up Your Django Tests which teaches you to write faster, more accurate tests.
One summary email a week, no spam, I pinky promise.
- Python Type Hints - How to Specify a Class Rather Than an Instance Thereof
- Python Type Hints - How to Fix Circular Imports
Tags: mypy, python
© 2021 All rights reserved.