‘pip install’ From a Git Repository
It’s quite common to want to
pip install a version of a package that hasn’t been released to PyPI, but is available on its Git repository host, such as GitHub. If the package is pure Python or has a relatively simple build process, you can normally install it directly via Git.
$ python -m pip install 'django @ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a'
You can put any kind of Git ref after the second
@, such as a branch or tag. In most cases, you’ll want to stick to immutable refs (tags or SHA’s), since they cannot change between installs.
Using quotes lets you put spaces around the first at sign, making things a bit clearer. Quotes also avoid misinterpreting some of the syntax we’ll cover in following sections.
python -m pip instead of just
pip? See my post on why it has fewer problems.)
You can use this same syntax inside requirements files, alongside your other requirement specifiers:
django @ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a psycopg2-binary==2.9.3
If you need to specify package extras, add
#egg=<pkg>[<extras>] on the end. For example, to install Django with its
$ python -m pip install 'django @ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a#django[argon2]'
Sometimes the project you want to install is not at the root of the repository, but in a subdirectory. In this case, add
#subdirectory=<path> at the end. For exmaple, to install this project from the repo subdirectory
$ python -m pip install 'django-stubs-ext @ git+https://github.com/typeddjango/django-stubs@9a41aa63ba22e599aa8f355eed6cb60c90f896c6#subdirectory=django_stubs_ext'
In this example, there’s a main project, django-stubs, and a sub-project, django-stubs-ext. This command installs django-stubs-ext from within its subdirectory.
If you need to combine extras and a subdirectory, list both options with a
$ python -m pip install 'django-stubs-ext @ git+https://github.com/typeddjango/django-stubs@9a41aa63ba22e599aa8f355eed6cb60c90f896c6#egg=django-stubs-ext[example]&subdirectory=django_stubs_ext'
(The 'example' extra doesn’t actually exist in django-stubs-ext. I just couldn’t find a real world example!)
On initial install, pip will:
- Clone the repository
- Fetch and check out the desired commit
- Build the package into a wheel (the
- Install that wheel (plus any dependencies)
pip’s output logs the steps in this process:
$ python -m pip install 'django @ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a' Collecting django@ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a Cloning https://github.com/django/django (to revision 80d38de52bb2721a7b44fce4057bcff571afc23a) to /private/var/folders/20/lzgtdyzs7wj5fc_90w1h3bhc0000gn/T/pip-install-sirxc970/django_b245ad848cac4a16a8639862854e9b94 Running command git clone --filter=blob:none --quiet https://github.com/django/django /private/var/folders/20/lzgtdyzs7wj5fc_90w1h3bhc0000gn/T/pip-install-sirxc970/django_b245ad848cac4a16a8639862854e9b94 Running command git rev-parse -q --verify 'sha^80d38de52bb2721a7b44fce4057bcff571afc23a' Running command git fetch -q https://github.com/django/django 80d38de52bb2721a7b44fce4057bcff571afc23a Resolved https://github.com/django/django to commit 80d38de52bb2721a7b44fce4057bcff571afc23a Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Collecting asgiref>=3.5.2 Using cached asgiref-3.5.2-py3-none-any.whl (22 kB) Collecting sqlparse>=0.2.2 Using cached sqlparse-0.4.3-py3-none-any.whl (42 kB) Building wheels for collected packages: django Building wheel for django (pyproject.toml) ... done Created wheel for django: filename=Django-4.2.dev20220928074807-py3-none-any.whl size=7843439 sha256=c7801e84dfa16f0f944fd875b2287936b09dff9914f41786d8733441442f63a2 Stored in directory: /Users/chainz/Library/Caches/pip/wheels/2c/fe/82/922c04ff998d31a0c66cbb08f691112b5e6a5a218eb4103011 Successfully built django Installing collected packages: sqlparse, asgiref, django Successfully installed asgiref-3.5.2 django-4.2.dev20220928074807 sqlparse-0.4.3
If you re-install the same repository at the same commit, pip will reuse the wheel it built previously:
$ python -m pip install 'django @ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a' Collecting django@ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a Using cached Django-4.2.dev20220928074807-py3-none-any.whl Requirement already satisfied: asgiref>=3.5.2 in ./venv/lib/python3.10/site-packages (from django@ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a) (3.5.2) Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.10/site-packages (from django@ git+https://github.com/django/django@80d38de52bb2721a7b44fce4057bcff571afc23a) (0.4.3)
This makes it nice and fast when you’re reinstalling a requirements file.
An alternative that avoids Git is to install from a tarball URL, that the major hosted Git solutions provide, for example:
$ python -m pip install https://github.com/django/django/archive/80d38de52bb2721a7b44fce4057bcff571afc23a.tar.gz
The original 2019 version of this article recommended using tarball URL’s over Git installs, because they used to be faster. Pip has since optimized its Git install mechanism, so that is normally faster now. Tarball URL’s may still be useful in niche situations though, for example if you need to install on a system without Git installed.
Improve your Django develompent experience with my new book.
One summary email a week, no spam, I pinky promise.
Tags: pip, python