Django: model field choices that can change without a database migration

Some loverly flowers to choose from.

Adam Hill posted a question on Mastodon: he wants a model field that uses choices that doesn’t generate a database migration when the choices change. This post presents my answer. First, we’ll recap Django’s default behaviour with choice fields, a solution with callable choices, and the drawbacks.

The default behaviour

Take this model definition:

from django.db import models


class Colour(models.IntegerChoices):
    TEAL = 1, "Teal"


class Flower(models.Model):
    colour = models.IntegerField(choices=Colour.choices)

The enumeration type Colour is used for the choices of the model field Flower.colour. Colour.choices returns a list of tuples that contain values and labels.

Run makemigrations to create the initial migration for this model:

$ ./manage.py makemigrations example
Migrations for 'example':
  example/migrations/0001_initial.py
    + Create model Flower

And check the generated migration:

from django.db import migrations, models


class Migration(migrations.Migration):

    initial = True

    dependencies = []

    operations = [
        migrations.CreateModel(
            name="Flower",
            fields=[
                ("id", models.BigAutoField(...)),
                ("colour", models.IntegerField(choices=[(1, "Teal")])),
            ],
        ),
    ]

The colour field definition includes the serialized list with one choice: (1, "Teal").

Then, say we make a change to Colour:

 class Colour(models.IntegerChoices):
     TEAL = 1, "Teal"
+    PUCE = 2, "Puce"

makemigrations detects the change and generates a new migration:

$ ./manage.py makemigrations example
Migrations for 'example':
  example/migrations/0002_alter_flower_colour.py
    ~ Alter field colour on flower

With contents:

from django.db import migrations, models


class Migration(migrations.Migration):

    dependencies = [
        ("example", "0001_initial"),
    ]

    operations = [
        migrations.AlterField(
            model_name="flower",
            name="colour",
            field=models.IntegerField(
                choices=[(1, "Teal"), (2, "Puce")],
            ),
        ),
    ]

We can see the whole list of choices has been serialized again.

Run sqlmigrate on that migration:

$ ./manage.py sqlmigrate example 0002
BEGIN;
--
-- Alter field colour on flower
--
-- (no-op)
COMMIT;

The BEGIN and COMMIT statements are the start and end of a transaction. But in between, there’s nothing but comments, including “(no-op)”, meaning “no operation”. So this migration does not change the database at all. It exists only for Django’s internal state tracking.

Such no-op migrations can be a bit of a nuisance, especially when the list of choices changes often. They unnecessarily lengthen the migration history, which is what Adam Hill wanted to avoid.

A solution with callable choices

Django 5.0 extended choices to support callables: functions and classes with a __call__ method. Since the migrations framework will then serialize the callable instead of the choice list, we can use this to avoid the no-op migration. Let’s carry on the above example to see how this works.

First, introduce a function that returns the .choices list, and use it for choices in the model field:

from django.db import models


class Colour(models.IntegerChoices):
    TEAL = 1, "Teal"
    PUCE = 2, "Puce"


def get_colour_choices():
    return Colour.choices


class Flower(models.Model):
    colour = models.IntegerField(choices=get_colour_choices)

Then, run makemigrations again:

$ ./manage.py makemigrations example
Migrations for 'example':
  example/migrations/0003_alter_flower_colour.py
    ~ Alter field colour on flower

The migration uses the “serialized” function by importing it:

import example.models
from django.db import migrations, models


class Migration(migrations.Migration):

    dependencies = [
        ("example", "0002_alter_flower_colour"),
    ]

    operations = [
        migrations.AlterField(
            model_name="flower",
            name="colour",
            field=models.IntegerField(choices=example.models.get_colour_choices),
        ),
    ]

Then, we can change Colour again:

 class Colour(models.IntegerChoices):
     TEAL = 1, "Teal"
     PUCE = 2, "Puce"
+    SALMON = 3, "Salmon"

But now, makemigrations detects no changes:

$ ./manage.py makemigrations example
No changes detected in app 'example'

This is because get_colour_choices still exists with the same name, and the import path is all the history tracks.

Drawbacks

I can think of two drawbacks to this approach.

1. The no-op behaviour of choices is not guaranteed

The premise that choice-changing migrations are avoidable churn depends on them not affecting the database. But this fact is field-dependent.

Field classes may use choices to determine the column definition, which they should declare in Field.non_db_attrs. An example is the EnumField in django-mysql, which uses choices for the MySQL/MariaDB ENUM data type. In this case, migrations need to change the database to update which values are allowed there.

That said, none of Django’s built-in fields (currently) do this.

2. Ideally, choices should be enforced with a CheckConstraint

In 2020, I covered how choices aren’t enforced unless you add a CheckConstraint, a data integrity gotcha. I stand by the belief that nearly all fields using choices should come with a matching CheckConstraint, like:

from django.db import models


class Colour(models.IntegerChoices):
    TEAL = 1, "Teal"


class Flower(models.Model):
    colour = models.IntegerField(choices=Colour.choices)

    class Meta:
        constraints = [
            models.CheckConstraint(
                condition=models.Q(colour__in=Colour.values),
                name="%(app_label)s_%(class)s_colour_valid",
            )
        ]

Doing so prevents invalid data from being saved to the database:

In [1]: Flower.objects.create(colour=33)
...
IntegrityError: CHECK constraint failed: example_flower_colour_valid

In this case, any time you change the choices class, the check constraint will need a migration to propagate updates to the database, like:

$ ./manage.py makemigrations example
Migrations for 'example':
  example/migrations/0002_remove_flower_example_flower_colour_valid_and_more.py
    - Remove constraint example_flower_colour_valid from model flower
    ~ Alter field colour on flower
    + Create constraint example_flower_colour_valid on model flower

Since this would be needed, the “churn” for the field change is less of a problem.

Fin

In conclusion, using callable choices allows you to avoid some migrations when choices change. But it won’t help if you propagate choices to the database, which I think is generally a good idea. Still, it might be useful when the choice list is more dynamic, such as from a changing data source, or when choices are used for forms only, and arbitrary values should still be allowed in the database.

May you always choose wisely,

—Adam

😸😸😸 Check out my new book on using GitHub effectively, Boost Your GitHub DX! 😸😸😸

Subscribe via RSS, Twitter, Mastodon, or email:

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: