If you’ve used Django migrations for a while, you may be familiar with this message:
$ python manage.py migrate CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0002_longer_titles, 0002_author_nicknames). To fix them run 'python manage.py makemigrations --merge'
This appears when the migration history for one of your apps branched to have two “leaf nodes”, that is, two final migrations. The simplest example has our first initial migration, then two conflicting second migrations:
+--> 0002_author_nicknames / 0001_initial +--| \ +--> 0002_longer_titles
This happens quite naturally when developing two features for same app, both with migrations.
The solution Django suggests is to create a merge migration with
makemigrations --merge. This creates another migration in our history that depends on the last two:
+--> 0002_author_nicknames +-+ / \ 0001_initial +--| |--> 0003_merge \ / +--> 0002_longer_titles +----+
This merge migration tells Django “it’s fine to run both branches of migrations and end up here”. This is a simple solution, and avoids modification of the exisiting migrations. But it has a number of drawbacks.
First, it’s a fix after the fact. You need to encounter the “Conflicting migrations detected” error before you step in and create the merge migration. This is normally after merging to your main branch, so requiring your CI to break and wait for a fix. In even a small team with a reasonable pace of change, you might find the CI on your main branch breaking with conflicting migrations several times a week.
Second, it doesn’t enforce the same execution order in all environments. The generated merge migration may define a different execution order of the migrations to the order that they were committed. When you use merge migrations, you allow two migrations to share the same number, so their order is dependent on the remainder of their names. For example:
- We merge
0002_longer_titlesand our CI automatically runs it it on our staging environment.
- We merge
0002_author_nicknames, and staging triggers the “Conflicting migrations detected” error.
- We create a merge migration,
makemigrations --merge, and this depends on
[0002_author_nicknames, 0002_longer_titles]- the opposite order to the merge order.
- We merge
0003_longer_titlesexecute on staging.
- We deploy to production, which runs the migrations based on the order in
0003_merge. This is different to the order they executed on staging.
This difference makes our staging environment a less useful simulation of production.
Third, it complicates rollbacks. It’s hard to reverse migrations when you can’t be sure of the order they ran in.
Enter my new package, django-linear-migrations. It ensures the migration history in your apps remains linear, such as:
0001_initial +--> 0002_author_nicknames +--> 0002_longer_titles
It does this by creating new files in your apps’ migration directories, called
max_migration.txt. These files contain the latest migration name for that app, and are automatically updated by
makemigrations. They mean that if a new migration is merged for an app, any other branches adding migrations for that app will have a merge conflict in your source control tool (e.g. Git).
This forces you to order migrations one after another. Since it only enforces from the current latest migration, it’s compatible with apps that have merge migrations in their past.
django-linear-migrations also provides a tool for handling the merge conflicts it creates, in its
rebase-migration command. Rather than repeat it here, I’ll point you to its documentation for an example.
If you too have struggled with merge migrations, please try out django-linear-migrations today!
Make your development more pleasant with Boost Your Django DX.
One summary email a week, no spam, I pinky promise.
- Django and the N+1 Queries Problem
- How to Unit Test a Django Form
- Introducing time-machine, a New Python Library for Mocking the Current Time