A Python Script Template, with and without Type Hints and Async

The Template

Voici:

#!/usr/bin/env python
from __future__ import annotations

import argparse
from collections.abc import Sequence


def main(argv: Sequence[str] | None = None) -> int:
    parser = argparse.ArgumentParser()
    # Add arguments here
    args = parser.parse_args(argv)

    # Implement behaviour here

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

This works with Python 3.7+.

Alternative versions below:

• without type hints
• async flavoured

Here’s what’s going on (in detail):

• We start with the shebang to run this script via python. This allows us to run our script with ./example.py, on Linux, macOS, and other Unixes.

  The script needs to have its executable bit set for this to work, which we can do with the chmod command, e.g. chmod +x example.py.

  Using the /usr/bin/env form is preferble to a hardcoded path like /usr/local/bin/python, because it allows our script to work with whatever Python interpreter is available.

• We use the from __future__ import annotations to enable postponed evaluation for type hints. This allows use of new type hint syntax. This future behaviour will become the default on Python 3.11.

• We define our main() function as optionally accepting a sequence of strings in argv, and returning an integer.

  argv represents the arguments passed to our script, from the command line or another program. When set to None, we use the actual arguments from sys.argv. Otherwise, any sequence of arguments can be passed in, making testing easy.

  (Note: sys.argv has type list[str], but we use Sequence[str]. This is more flexible, plus it’s the type that argparse’s parse_args() accepts.)

  The integer returned by main() is our script’s exit code. Having main() return the code, rather than calling sys.exit() itself, also improves testability. Tests can simply assert on return values, rather than handle SystemExit exceptions.

• We use argparse to parse the arguments. This is much easier than writing custom argument parsing code.

  The template has no arguments defined. We can add declare them the parser.add_argument().

  argparse has been Python standard library for a long time and has many neat features. There are alternative packages on PyPI, such as Click. But generally I avoid such packages, since argparse is “good enough” and having dependencies adds a maintenance burden.

  To learn about using argparse, check out its tutorial.

• Our main() returns the 0 exit code by default, signalling success. Any other (positive) number signals failure.

  For simple scripts we can stick to returning 1 for all kinds of error.

• At the bottom we use an if __name__ == "__main__" block. The code in this block will only run when we run our script, with one of:

  • ./example.py
  • python example.py
  • python -m example

  If we instead import from our script, with e.g. import example, then __name__ will contain our module’s name, "example". So, this block will not run.

  Although we can ‘get away’ with writing scripts without such a block, it’s best to add it as a matter of habit. The check ensures our script is importable without executing main(), a necessity for adding tests.

  The “__name__ == "__main__"” construct can be confusing. For a longer explanation see the Python docs explainer.

• For our main block, we call main(), then pass its exit code to a SystemExit exception which we raise. The SystemExit exception passes up to the Python interpreter, which then terminates with the given exit code.

  (Why not sys.exit() or exit()? See my post The Many Ways to Exit in Python.)

An Example Cat

Here’s an example script that partially replicates the cat utility. cat outputs the content of one or more files, with several options. Our example implements this core functionality, plus the -n flag to add line numbering:

#!/usr/bin/env python
from __future__ import annotations

import argparse
from contextlib import closing
from collections.abc import Sequence
from io import TextIOWrapper


def main(argv: Sequence[str] | None = None) -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("-n", action="store_true", dest="number")
    parser.add_argument("file", type=argparse.FileType("r"), nargs="+")
    args = parser.parse_args(argv)

    number: bool = args.number
    files: list[TextIOWrapper] = args.file

    for file in files:
        with closing(file):
            for i, line in enumerate(file, start=1):
                if number:
                    print(f"{i: >6} ", end="")
                print(line, end="")

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

We can see:

• We now declare two arguments with argparse: -n and file.

  We define -n as a flag that defaults to False, but will be set to True when passed on the command line. It’s saved in the number attribute.

  We define file at least one filename. argparse’s FileType conveniently checks the given files exist and opens them for us, and supports - to mean “stdin”.

• We extract the arguments from the args object into their own variables. This is so they have correct type hints.

  argparse’s dynamism means that type checkers sees parsed values as type Any, which essentially disables type checking. We fix this by copying the attributes into specifically typed variables. This does require us to match our variable types to argparse’s behaviour, but it reduces the overall chance for type-related bugs.

• We loop through all the files. We use closing on each file to ensure it is closed when we are done with it.

  For each file, we iterate over it line by line, with line numbers from enumerate(). We use print``()`` for output. Because split lines already contain the newline, we pass end="" to print() to prevent it adding an extra newline.

  When numbering is active, we match cat’s output by left-padding the line numbers to 6 characters with a string format spec.

Here are cat and our example in action:

$ cat hello1.txt
Hello:
    World!
    Adam!
$ ./example.py hello1.txt
Hello:
    World!
    Adam!

And with line numbering:

$ ./example.py -n hello1.txt hello2.txt
     1 Hello:
     2     World!
     3     Adam!
     1 Hello:
     2     Pythonista!
     3     Reader!

Sah-weet.

Testing Our Example Script

We can write tests for our script by importing main() and calling it with a list of arguments. Here are some example tests using pytest:

from example import main


def test_single_file(capsys, tmp_path):
    file = tmp_path / "test.txt"
    file.write_text("Hello world!\n")

    result = main([str(file)])

    assert result == 0
    out, err = capsys.readouterr()
    assert out == "Hello world!\n"
    assert err == ""


def test_single_file_with_numbering(capsys, tmp_path):
    file = tmp_path / "test.txt"
    file.write_text("Hello world!\n")

    result = main(["-n", str(file)])

    assert result == 0
    out, err = capsys.readouterr()
    assert out == "     1 Hello world!\n"
    assert err == ""

Note:

• We use pytest’s tmp_path fixture to obtain a temporary directory. We then create a test.txt file in that directory with the contents "Hello world!\n".
• We use pytest’s capsys fixture to capture our program’s output. This allows us to assert that it prints what we expect, and nothing on stderr.

Running the tests we see:

$ pytest -v test_example.py
========================= test session starts =========================
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
cachedir: .pytest_cache
rootdir: /Users/chainz/tmp/script-templates
collected 2 items

test_example.py::test_single_file PASSED                        [ 50%]
test_example.py::test_single_file_with_numbering PASSED         [100%]

========================== 2 passed in 0.01s ==========================

😎

The Template Sans Type Hints

If you’re not (yet) comfortable writing type hints, here’s a version without them:

#!/usr/bin/env python
import argparse


def main(argv=None):
    parser = argparse.ArgumentParser()
    # Add arguments here
    args = parser.parse_args(argv)

    # Implement behaviour here

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

This should work on Python 2.7, or even earlier (I didn’t check).

(Also, learn more about type hints with my Mypy posts!)

The Template But In Async Flavour

This async version uses async def for main(), and asyncio.run() to run it:

#!/usr/bin/env python
from __future__ import annotations

import argparse
import asyncio
from collections.abc import Sequence


async def main(argv: Sequence[str] | None = None) -> int:
    parser = argparse.ArgumentParser()
    # Add arguments here
    args = parser.parse_args(argv)

    # Implement behaviour here

    return 0


if __name__ == "__main__":
    raise SystemExit(asyncio.run(main()))

This works on Python 3.7+.

(We can also swap to the run() functions of one of the alternative async libraries AnyIO and Trio.)

Fin

May this template help you write some awesome scripts,

—Adam