Dropping Python 2 Support From Open Source Projects

Bye bye six

I’ve recently dropped Python 2 support from most of the open source projects I maintain. Python 2 support ends 2020-01-01 (see pythonclock.org), and many major projects have signed the Python 3 Statement that declares that they will remove Python 2 support before, so this year is crunch time for migration.

Reasoning

I was originally thinking of maintaining Python 2 support until the end of the year. However, I changed my mind for a few reasons.

Firstly, Django dropped Python 2 support in version 2.0, at the start of 2018, setting a precedent.

Secondly, I have not done any professional work with Python 2 for some time. While I’m using some great tools to make cross-compatibility easier, I still don’t trust myself to remember all the differences, and I especially don’t want to make a mistake and release something broken.

Thirdly, I created a Twitter poll which came out with 69% in favour of dropping today, and only 15% waiting until 2020-01-01.

Fourthly, fellow Django contributor Josh Smeaton pointed out to me that upstream packages dropping Python 2 support gives teams more ammunition to use to convince management that upgrading needs prioritization. Dropping Python 2 support can actually help such users.

Fifthly, most of the packages I’m maintaining are pretty stable. The versions that support Python 2 will remain on PyPI and in the case of a major bug or security issue, I can always make a bug fix release branched from them. It’s only new features that won’t be added with Python 2 support.

Migration Checklist

The following is the checklist I used on each project to move to Python 3 only, ordered by affected files. My packages supported Python 2 and 3 in a single codebase, using six for compatibility, and checking with modernize, isort, and tox to ensure cross-compatibility. There are other ways of achieving cross-compatibility, such as 2to3 and future, so if you’re trying to follow this list on a project, you may need to make adjustments.

Every Python File

  • Remove the UTF-8 coding header (e.g. # coding=utf-8), as Python 3 reads source code in UTF-8 by default.
  • Remove the __future__ import header, which was used to backport Python 3 features into Python 2 (I used all the possible features, from __future__ import absolute_import, division, print_function, unicode_literals)
  • Remove usage of six, e.g. replacing six.iteritems(a_dict) with a_dict.items().
  • Remove any Python 2 only code paths - these are most clearly found when branched with if six.PY2:, but can also be done with comparison of sys.version_info or just comments mentioning “python 2” or “py2”.

setup.py

  • Remove usage of codecs.open when reading the README.rst for long_description. This was used to support all UTF-8 emojis on Python 2, but Python 3 opens text files in UTF-8 mode by default so the normal open() can be used.
  • Remove six and any other Python 2 compatibility dependencies from install_requires.
  • Remove any Python 2 only requirements based on markers from extras_require.
  • Update python_requires to declare Python 3.4+.
  • Remove Python 2 trove classifiers from classifiers.

setup.cfg

  • Remove [bdist_wheel] section which set up universal wheel building with universal = 1. This is important, even if you have a package with python_requires set to 3.4+, if the wheel is uploaded as universal, pip install <package> on Python 2 will fail by downloading it and then executing it. I only learnt this half-way, after an issue report from a kind user.
  • Remove configuration for isort to check for all __future__ imports.

Testing

  • Remove any Python 2 specific or compatibility packages from requirements.in, as used by pip-compile. This includes those mentioned in install_requires which were pinned to ensure tests are repeatable, compatibility tools like modernize, and backports like mock which was merged into the standard library in Python 3.
  • Re-run pip-compile on Python 3 to generate a Python 3 only requirements.txt. Previously I was using Python 2 pip-compile and then just trusting that was compatible with Python 3, which was a bit lazy.
  • Remove Python 2 environments and configuration from tox.ini

Documentation

  • Update README to declare “Python 3.4+ supported”.
  • When releasing, bump major version, and add a changelog note that Python 2 is no longer supported.

Example Pull Request

For an example migration, see my pull request on django-perf-rec. It shows most of the above steps in action.

Fin

I hope this can help you with removing Python 2 support from a codebase, whether in an application or a package!

😸😸😸 Check out my new book on using GitHub effectively, Boost Your GitHub DX! 😸😸😸

Subscribe via RSS, Twitter, Mastodon, or email:

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: