Dropping Python 2 Support From Open Source Projects2019-03-13
I’ve recently dropped Python 2 support from most of the open source projects I maintain. Python 2 support ends 2020-01-01 (see pythonclock.org), and many major projects have signed the Python 3 Statement that declares that they will remove Python 2 support before, so this year is crunch time for migration.
I was originally thinking of maintaining Python 2 support until the end of the year. However, I changed my mind for a few reasons.
Firstly, Django dropped Python 2 support in version 2.0, at the start of 2018, setting a precedent.
Secondly, I have not done any professional work with Python 2 for some time. Whilst I’m using some great tools to make cross-compatibility easier, I still don’t trust myself to remember all the differences, and I especially don’t want to make a mistake and release something broken.
Thirdly, I created a Twitter poll which came out with 69% in favour of dropping today, and only 15% waiting until 2020-01-01.
Fourthly, fellow Django contributor Josh Smeaton pointed out to me that upstream packages dropping Python 2 support gives teams more ammunition to use to convince management that upgrading needs prioritization. Dropping Python 2 support can actually help such users.
Fifthly, most of the packages I’m maintaining are pretty stable. The versions that support Python 2 will remain on PyPI and in the case of a major bug or security issue, I can always make a bug fix release branched from them. It’s only new features that won’t be added with Python 2 support.
The following is the checklist I used on each project to move to Python 3 only, ordered by affected files. My packages supported Python 2 and 3 in a single codebase, using six for compatibility, and checking with modernize, isort, and tox to ensure cross-compatibility. There are other ways of achieving cross-compatibility, such as 2to3 and future, so if you’re trying to follow this list on a project, you may need to make adjustments.
Every Python File
- Remove the UTF-8 coding header (e.g.
# coding=utf-8), as Python 3 reads source code in UTF-8 by default.
- Remove the
__future__import header, which was used to backport Python 3 features into Python 2 (I used all the possible features,
from __future__ import absolute_import, division, print_function, unicode_literals)
- Remove usage of six, e.g.
- Remove any Python 2 only code paths - these are most clearly found when
if six.PY2:, but can also be done with comparison of
sys.version_infoor just comments mentioning “python 2” or “py2”.
- Remove usage of
codecs.openwhen reading the
long_description. This was used to support all UTF-8 emojis on Python 2, but Python 3 opens text files in UTF-8 mode by default so the normal
open()can be used.
sixand any other Python 2 compatibility dependencies from
- Remove any Python 2 only requirements based on markers from
python_requiresto declare Python 3.4+.
- Remove Python 2 trove classifiers from
[bdist_wheel]section which set up universal wheel building with
universal = 1. This is important, even if you have a package with
python_requiresset to 3.4+, if the wheel is uploaded as universal,
pip install <package>on Python 2 will fail by downloading it and then executing it. I only learnt this half-way, after an issue report from a kind user.
- Remove configuration for
isortto check for all
- Remove any Python 2 specific or compatibility packages from
requirements.in, as used by
pip-compile. This includes those mentioned in
install_requireswhich were pinned to ensure tests are repeatable, compatibility tools like
modernize, and backports like
mockwhich was merged into the standard library in Python 3.
pip-compileon Python 3 to generate a Python 3 only
requirements.txt. Previously I was using Python 2
pip-compileand then just trusting that was compatible with Python 3, which was a bit lazy.
- Remove Python 2 environments and configuration from
- Update README to declare “Python 3.4+ supported”.
- When releasing, bump major version, and add a changelog note that Python 2 is no longer supported.
Example Pull Request
For an example migration, see my pull request on django-perf-rec. It shows most of the above steps in action.
I hope this can help you with removing Python 2 support from a codebase, whether in an application or a package!
© 2019 All rights reserved.