Django: introducing django-msgspec

Artist’s impression of a JSON factory.

It’s another day, another new package day here. Say hello to django-msgspec, a package of drop-in replacements for Django and Django REST Framework (DRF) components backed by msgspec.

msgspec is a C-based serialization library covering JSON, MessagePack, YAML, and TOML, with optional schema validation through typed Struct classes. Its JSON encoder and decoder are several times faster than the standard library’s, which makes django-msgspec a cheap performance win in the parts of Django that handle JSON.

Features

There’s a version of JsonResponse:

from django_msgspec.http import JsonResponse


def index(request):
    return JsonResponse({"title": "Hello, world!"})

…a test client with matching test case classes:

from django_msgspec.test import SimpleTestCase


class IndexTests(SimpleTestCase):
    def test_index(self):
        response = self.client.get("/", headers={"accept": "application/json"})
        assert response.status_code == 200
        # response.json() uses msgspec to parse the response body
        assert response.json() == {"title": "Hello, world!"}

…a version of Django’s json_script template tag, which is where this whole story began:

{% load django_msgspec %}
{{ sales_by_product_id|json_script:"chart-data" }}

…and a handful of components that you activate purely through settings, with no code changes at all:

SESSION_SERIALIZER = "django_msgspec.sessions.JSONSerializer"

SERIALIZATION_MODULES = {
    "json": "django_msgspec.serializers.json",
    "jsonl": "django_msgspec.serializers.jsonl",
}

REST_FRAMEWORK = {
    "DEFAULT_RENDERER_CLASSES": ["django_msgspec.rest_framework.JSONRenderer"],
    "DEFAULT_PARSER_CLASSES": ["django_msgspec.rest_framework.JSONParser"],
}

That covers session storage and signing, dumpdata / loaddata in both JSON and JSON Lines, and DRF request parsing and response rendering.

Everything encodes with an enc_hook that knows about Django’s lazy strings, so translated text passes through as you’d expect. It’s all tested against the currently supported versions of Python and Django, with 100% coverage.

Déjà vu?

Only three weeks ago, I introduced django-orjson, a near-identical package backed by the Rust-powered orjson. So yeah, you might be confused why I made another faster-alternative-JSON-package-wrapper package so soon after the first one.

After releasing django-orjson, several folks from the community reached out to me telling me about the issues with orjson and pointing to msgspec instead. Additionally, while trying to roll out django-orjson on a client project, I learned that certain documented behaviours and limitations in orjson were going to be road blockers.

Here’s a full list of what I learned:

msgspec’s advantages

msgspec answers the questions raised by the above points against orjson:

And most importantly, msgspec’s encoding and decoding are in the same performance ballpark as orjson’s!

msgspec has standard library incompatibilities too

By the way, msgspec isn’t fully compatible with json—here are the differences that I know about.

  1. Non-finite floats are encoded as null rather than the standard library’s NaN and Infinity:
>>> json.dumps(float("inf"))
'Infinity'

>>> msgspec.json.encode(float("inf"))
b'null'

I’d call this an improvement, since Infinity isn’t valid JSON and other parsers will reject it. But it is a change, so if you rely on round-tripping those values, take note.

  1. msgspec also only coerces keys that are string-like or number-like, so booleans and None are still rejected:
>>> json.dumps({None: "nothing"})
'{"null": "nothing"}'

>>> msgspec.json.encode({None: "nothing"})
Traceback (most recent call last):
  ...
TypeError: Only dicts with str-like or number-like keys are supported

Such keys should be rarer than numbers in practice.

What might come next in django-msgspec

django-msgspec covers the same ground as django-orjson today, but msgspec is a broader library than orjson, so there’s more potential for future development.

First, msgspec’s typed container class, Struct, lets you decode and validate in a single pass:

>>> import msgspec
>>> class Sale(msgspec.Struct):
...     product_id: int
...     count: int
...
>>> msgspec.json.decode(b'{"product_id": 1, "count": 2}', type=Sale)
Sale(product_id=1, count=2)

>>> msgspec.json.decode(b'{"product_id": "one", "count": 2}', type=Sale)
Traceback (most recent call last):
  ...
msgspec.ValidationError: Expected `int`, got `str` - at `$.product_id`

This could be useful for combining with Django views, DRF parsers, or even forms.

Second, msgspec can serialize and deserialize other data types, so they might be worth integrating.

Happy to take suggestions on the design here, on the issue tracker.

Fin

Please try out django-msgspec today and let me know how it goes.

May all your messages be to specification,

—Adam

😸😸😸 Check out my new book on using GitHub effectively, Boost Your GitHub DX! 😸😸😸

Subscribe via RSS, Twitter, Mastodon, or email:

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: