Python: fix SyntaxWarning: '<type>' object is not callable
Take this code, which we want to return a list of shopping items with desired quantities:
SyntaxWarning: '<type>' object is not callableTake this code, which we want to return a list of shopping items with desired quantities:
Django’s test client is really useful for writing integration tests for your project. It’s great because it has a simple API for testing your application similarly to how a web browser would interact with it. Unfortunately it can be slow, because each call creates a request, passes it through all your middleware, view, maybe a template, then the response comes back through the same layers.
This is a test anti-pattern I’ve seen creep in on many Django test suites. I know of several large projects where it became a major undertaking to undo it. The good news is it’s easy to avoid adding it when you first write your tests. 🙂
I encountered this problem recently for a Django ticket for speeding up part of Django’s test runner. We have a list of tables to truncate. How can we figure out which are already empty, so we can skip issuing a somewhat costly TRUNCATE TABLE on them? You might think the database could skip TRUNCATE TABLE on empty tables, but it normally does work beyond removing the rows such as resetting auto-increment counters. In MySQL, TRUNCATE TABLE is even equivalent to DROP TABLE and CREATE TABLE, which requires removing and creating the table file on disk.
Whilst writing Speed Up Your Django Tests, I wanted to add a section about mocking the current time. I knew of two libraries for such mocking, but I found it hard to pick one to recommend due to the trade-offs in each. So I delayed adding that section and shaved a rather large yak by writing a third library.
Django’s transaction.on_commit() hook is useful for running tasks that rely on changes in the current database transaction. The database connection enqueues callback functions passed to on_commit, and executes the callbacks after the current transaction commits. If the transaction is rolled back, the callbacks are discarded. This means they act if-and-when the final version of the data is visible to other database connections.
My previously announced book “Speed Up Your Django Tests” is out now on Gumroad. I’ve been writing since the 3rd March, so it’s quite a relief to have launched it.
It’s occasionally useful to be able to tell which Django manage.py command is being run, in a code path that otherwise has no way of telling. For example, in Speed Up Your Django Tests, I describe how to modify manage.py to default use a test settings file when the test command is run.
At the start of March I started writing a blog post called “How to Speed Up Your Django Tests”. Before I knew it, the outline alone was 4,000 words! I realized writing it up would be a major undertaking. As lockdown arrived, I found the time to write it all up.
It’s been a year since I published How to Score A+ for Security Headers on Your Django Website, the blog post for my DjangoCon Europe 2019 talk. It’s seen some updates as both Django and web security have evolved, for example Feature-Policy is now required for an A+, and Django 3.0 includes built-in support for Referrer-Policy.
I maintain several open source Python projects, each in its own GitHub repository. I like to keep them all up to date according to a kind of template - similarity increases maintainability.
Imagine you have some Django views using the same two decorators:
I previously covered using Django’s CheckConstraint class to validate fields with choices and percentage fields that total 100%. Here’s another use case.
Python’s decimal module has concept of a “context”. This defines the default precision of new Decimals, how rounding works, and lots of other behaviour. Maths gets complicated!
I’ve mentioned CloudFront’s speed, or lack thereof, several times in my past posts.
Django’s default settings file has always included a BASE_DIR pseudo-setting. I call it a “pseudo-setting” since it’s not read by Django itself. But it’s useful for configuring path-based settings, it is mentioned in the documentation, and some third party packages use it.
I previously covered using Django’s CheckConstraint class to ensure a field with choices is constrained to only valid values. Here’s another use case, based on an application I worked on. It uses a check constraint to ensure a set of fields, representing percentages, always sum up to 100.
If you have some slow tests in your suite, you might want to run them only on CI. To do this, you can detect in your test runner if you’re running on a CI system.
At yesterday’s DJUGL meetup I heard a talk from Esau Rodríguez about a buggy new system deployment he survived. His team were deploying a new version of a system that changed the way it connected to its MySQL database, while simultaneously moving session keys from integers to UUIDs. When they discovered the database connection change was flaky, they rolled back to the previous version. This quickly lead to a user reporting being able to see another’s data!
If you’re hosting a website on a top level domain, you should set up both the bare domain (example.com) and the “www” subdomain (www.example.com). People expect to be able to type either version and see your site - no matter which version you advertise.
I’ve just moved all my open source repositories from using plain pip to python -m pip, in test scripts and documentation.
When you run Django’s manage.py makemigrations, it will try to generate a name for the migration based upon its contents. For example, if you are adding a single field, it will call the migration 0002_mymodel_myfield.py. However when your migration contains more than one step, it instead uses a simple ‘auto’ name with the current date + time, e.g. 0002_auto_20200113_1837.py. You can provide the -n/--name argument to makemigrations, but developers often forget this.
Django templates are often used to pass data to JavaScript code. Unfortunately, if implemented incorrectly, this opens up the possibility of HTML injection, and thus XSS (Cross-Site Scripting) attacks.
I previously blogged about how I configured my CloudFront hosted website to score A+ on securityheaders.com. I worked around CloudFront’s lack of an “add headers” feature by adding a Lambda@Edge function in JavaScript.
robots.txt is a standard file to communicate to “robot” crawlers, such as Google’s Googlebot, which pages they should not crawl. You serve it on your site at the root URL /robots.txt, for example https://example.com/robots.txt.