(All tags.)
Every time I pick up the Python job queue Celery after not using it for a while, I find I’ve forgotten exactly how RabbitMQ works. I find the Advanced Message Queuing Protocol (AMQP) concepts drop out of my head pretty quickly: Exchanges, Routing Keys, Bindings, Queues, Virtual Hosts…
If the title worried you about my mental state, you can relax. It’s about the Python code formatter Black!
My library apig-wsgi bridges between AWS API Gateway’s JSON format for HTTP requests and Python WSGI applications. Recently Théophile Chevalier opened an issue requesting the library add an extra WSGI environ variable. I closed it by pointing out that it’s not much code to add a custom WSGI middleware to do so (plus the exact key is a bit out of scope for the library).
I discovered GIFCities recently. It’s a project by the Internet Archive for browsing GIFs extracted from their Geocities archive.
DjangoCon Europe 2019 was great fun. I loved meeting old and new friends, giving my talk and workshop, and learning in the other talks.
Code coverage is a simple tool for checking which lines of your application code are run by your test suite. 100% coverage is a laudable goal, as it means every line is run at least once.
A discussion recently came up on django-developers mailing list about how to build the SQL query CREATE TABLE ... AS SELECT (CTAS) with Django’s ORM. This statement and its cousin INSERT ... SELECT are useful for re-shaping data inside your database, using SELECT queries.
This is a blog post version of the talk I gave at DjangoCon Europe 2019 on the 10th April.
A lot of people pick Flask over Django because they believe it is simpler to start with. Indeed, the Flask front page includes an 8 line “hello world” application, while the Django default project has 172 lines in 5 files, and still doesn’t say “hello world”! (It does show you a welcome rocket and have a full admin interface though, both are pretty fun).
I was at PyCon Namibia in Windhoek from the 19th to 21st of February, and had an amazing time! PyCon Namibia is one of the longest running PyCons in Africa, this being its fifth edition in as many years.
I quite often get asked for resources on where to learn Django. Here’s a list of the places I know!
I realized I’ve posted the 2017 and 2018 editions of the London Django Meetup December quizzes on my blog, but forgot to post the first one in 2016. So here it is reproduced below, if you’d like to play at home or scroll through to the answers to pick up on some more Django trivia.
Reading the Django 2.2 alpha announcement, the phrase “salmagundi of new features” stood out to me. I had to look up “salmagundi” in Wiktionary, where it is defined as:
On Monday evening I gave a quiz at the December London Django Meetup Group for the third year running - that makes it a tradition! Here it is so you can follow it at home - answers are at the bottom, no cheating.
This is about a small problem we faced with the models used for customers in YPlan, now Time Out Checkout.
My plugin pytest-randomly was recently moved into the pytest-dev organization on GitHub, making it a bit “more official” as a pytest plugin. Thanks to Bruno Oliveira for suggesting it, Florian Bruhin and Bruno for approving it on the pytest-dev mailing list, and Gordon Wrigley for helping with its development.
Yesterday evening I gave a quiz at the London Django Meetup Group for the second year running. Here it is so you can do it at home (no cheating!). Answers are at the bottom.
Django’s ORM is normally fast enough as-is, but if you’ve ever profiled a high traffic view with a fairly complicated query, you might have found that constructing QuerySet can take a noticeable portion of your request time. For example, I once found a query on the front page of the site I was working on that took 1ms to construct and 1ms for the database to answer. With a performance budget of 100ms, that was 1% gone on computing the exactly same SQL.
During PyCon UK I had the opportunity to work on open-sourcing our in-house Django performance testing tool, which has now been released as django-perf-rec. We created it over two years ago, and have been using and improving it since. It has been helping us to pre-emptively fix performance problems in our code, and now it can help you!
We recently upgraded our 160,000 lines of backend Python code from Python 2 to Python 3. We did with zero downtime and no major errors! Here’s how we did it, hopefully it will help anyone else still stuck on Python 2!
I recently released version 0.1.10 of my library django-mysql, for which the main new feature was a backend for Django’s cache framework called MySQLCache. This post covers some of the inspiration and improvements it has, as well as a basic benchmark against Django’s built-in DatabaseCache.
On Tuesday I gave another talk at the London Django Meetup Group, titled “Factory Boy Fun”, based upon my previous blog post of the same name. The blog post covers pretty much the same stuff, but if you want to flick through the slides quickly for an overview, they’re hosted on GitHub.
I’ve recently been working on improving the test suite at YPlan. The biggest change is moving towards dynamic fixtures for our Django models using “Factory Boy”. This library is essentially a tool that lets you define simple helper functions to generate random, sensible model instances quickly; by using them in tests you can avoid the static JSON fixture files that Django recommends you use in tests by default. Factories are also general purpose - they just generate data and use it to create a model - and so they can be re-used to fill your development database rather than dumping from production.
On Tuesday I gave a talk on Django at the London Django Meetup Group, titled “Django at Scale.” The slides are hosted on GitHub.
I was looking through the MySQL slow_log for YPlan and discovered that there were a lot of SELECT COUNT(*) queries going on, which take a long time because they require a full table scan. These were coming from the Django admin, which displays the total count on every page.
