Python: how time-machine is O(1) where freezegun is O(n)

The benchmark

I wrote a benchmark script to compare the two libraries, listed in full at the end of this section. The variable here is the number of attributes across all imported modules.

To roughly simulate production code, the benchmark generates modules as bare types.ModuleType instances and stuffs them straight into sys.modules. Each module object has 25 attributes, and one in ten of them has module-level references to the date and time functions, as if written with from datetime import date, datetime and from time import time.

import datetime as dt
import time


def make_modules(count):
    modules = {}
    for i in range(count):
        name = f"generated_module_{i}"
        module = types.ModuleType(name)
        for j in range(ATTRIBUTES_PER_MODULE):
            setattr(module, f"attribute_{j}", f"value_{j}")
        if i % 10 == 0:
            # As if: from datetime import date, datetime
            #        from time import time
            module.date = dt.date
            module.datetime = dt.datetime
            module.time = time.time
        modules[name] = module
    return modules

The benchmark then measures the time taken to mock and unmock time with each library, with these test skeletons:

TARGET = dt.datetime(2020, 1, 1)


def freezegun_test():
    with freezegun.freeze_time(TARGET, tick=True):
        pass


def time_machine_test():
    with time_machine.travel(TARGET):
        pass

A sweep then walks the module counts, installing the generated modules, timing both functions with timeit, and cleaning up:

for count in COUNTS:
    generated = make_modules(count)
    sys.modules.update(generated)
    try:
        total = len(sys.modules)
        freezegun_time = measure(freezegun_test)
        time_machine_time = measure(time_machine_test)
    finally:
        for name in generated:
            del sys.modules[name]
    results[count] = (total, freezegun_time, time_machine_time)

measure() uses timeit.Timer.autorange() to pick a loop count, repeats five times, and takes the fastest per-call time. The whole sweep then runs five times over, keeping the fastest time seen at each size. Keeping the fastest times removes noise from our results, as nearly all slowdowns are due to external reasons.

Here’s the whole thing, with inline script metadata so that uv run benchmark.py is all you need to run it:

"""Benchmark freezegun against time-machine as imported modules grow.

Generates module objects, installs them in sys.modules, and times a
mock-and-unmock cycle for each library. One in ten generated modules has
module-level references to the date and time functions, as if written with
``from datetime import date, datetime`` and ``from time import time``.
"""

# /// script
# requires-python = ">=3.11"
# dependencies = ["freezegun==1.5.5", "time-machine==3.3.0"]
# ///

from __future__ import annotations

import datetime as dt
import sys
import time
import timeit
import types
from collections.abc import Callable

import freezegun
import time_machine

TARGET = dt.datetime(2020, 1, 1)
COUNTS = [0, 500, 1000, 2000, 4000, 8000, 16000]
ATTRIBUTES_PER_MODULE = 25
ROUNDS = 5


def make_modules(count: int) -> dict[str, types.ModuleType]:
    modules = {}
    for i in range(count):
        name = f"generated_module_{i}"
        module = types.ModuleType(name)
        for j in range(ATTRIBUTES_PER_MODULE):
            setattr(module, f"attribute_{j}", f"value_{j}")
        if i % 10 == 0:
            # As if: from datetime import date, datetime
            #        from time import time
            module.date = dt.date
            module.datetime = dt.datetime
            module.time = time.time
        modules[name] = module
    return modules


def freezegun_test() -> None:
    with freezegun.freeze_time(TARGET, tick=True):
        pass


def time_machine_test() -> None:
    with time_machine.travel(TARGET):
        pass


def measure(func: Callable[[], None], repeat: int = 5) -> float:
    """Seconds per call, taking the fastest of several timed batches."""
    timer = timeit.Timer(func)
    number, _ = timer.autorange()
    return min(t / number for t in timer.repeat(repeat=repeat, number=number))


def sweep() -> dict[int, tuple[int, float, float]]:
    """Time both libraries at each module count, once each."""
    results = {}
    for count in COUNTS:
        generated = make_modules(count)
        sys.modules.update(generated)
        try:
            total = len(sys.modules)
            freezegun_time = measure(freezegun_test)
            time_machine_time = measure(time_machine_test)
        finally:
            for name in generated:
                del sys.modules[name]
        results[count] = (total, freezegun_time, time_machine_time)
    return results


def main() -> None:
    # Sweep several times over, keeping the fastest time seen for each size.
    # A single pass takes a couple of minutes, over which a machine can drift
    # in and out of being busy, which would otherwise bend the results.
    best = sweep()
    for _ in range(ROUNDS - 1):
        for count, (total, freezegun_time, time_machine_time) in sweep().items():
            _, best_freezegun, best_time_machine = best[count]
            best[count] = (
                total,
                min(best_freezegun, freezegun_time),
                min(best_time_machine, time_machine_time),
            )

    print(f"{'generated':>10} {'total':>10} {'freezegun':>13} {'time-machine':>13}")
    for count, (total, freezegun_time, time_machine_time) in best.items():
        print(
            f"{count:>10,} {total:>10,} "
            f"{freezegun_time * 1e6:>10,.1f} µs {time_machine_time * 1e6:>10,.1f} µs"
        )


if __name__ == "__main__":
    main()

Here are the results, recorded on Python 3.15 on my M1 MacBook with freezegun 1.5.5 and time-machine 3.3.0. Every total includes the 259 modules that are there before any are generated: those imported by the benchmark itself, plus the ones that freezegun imports, such as asyncio, the first time it freezes.

freezegun’s times climb in a straight line: about 1.4 ms of fixed cost plus 2.5 µs per module. time-machine stays at 1.5 µs whatever you throw at it, with no trend as the modules pile up.

So the ratio is not a fixed “N times faster”—it’s a function of your project’s size:

• 259 modules, a bare interpreter: 940 times faster.
• 1,259 modules, about the size of the small Django project from the 2021 post: 2,490 times faster.
• 16,259 modules, a big project with a lot of dependencies: 27,300 times faster.

Per start/stop cycle, freezegun at 16,000 generated modules costs 41 ms. A suite with 2,000 time-mocking tests spends 82 seconds doing nothing but find-and-replace. The same suite with time-machine spends 3 milliseconds.

Why freezegun is O(n): the sweep

freezegun replaces the date and time functions with fakes, then goes looking for every copy of the originals that other modules made when they imported them. That search is a loop over sys.modules, in freeze_time.start() (source):

for mod_name, module in list(sys.modules.items()):
    if mod_name is None or module is None or mod_name == __name__:
        continue
    elif mod_name.startswith(self.ignore) or mod_name.endswith(".six.moves"):
        continue
    elif not hasattr(module, "__name__") or module.__name__ in ("datetime", "time"):
        continue

    module_attrs = _get_cached_module_attributes(module)
    for attribute_name, attribute_value in module_attrs:
        fake = fakes.get(id(attribute_value))
        if fake:
            setattr(module, attribute_name, fake)
            add_change((module, attribute_name, attribute_value))

Every time you freeze time, every loaded module gets visited. fakes is a dict keyed by id() of the real objects, so a module that imported datetime.datetime has that name rebound to FakeDatetime.

_get_cached_module_attributes() gathers the attributes to check, with a little caching to speed things up: it remembers which of a module’s names held date and time objects last time, and reuses that list if the module’s contents hash the same. Even on a hit, though, computing that hash means listing and hashing every attribute name of every module, every time. Therefore, the work per freeze stays proportional to the number of module-level attributes across all loaded modules.

THus we can say freezegun has O(n) runtime complexity, where n is the number of module-level attributes across all loaded modules. The larger your project, the longer it takes, proportionally.

This slow process is also leaky for (at least) these two reasons:

1. It doesn’t discover some kinds of references, like class attributes, default arguments, closures, and C extensions that hold references to the original functions. These continue to yield the true time, even while most calls to the same function name are mocked.

2. The fake objects have observably different types:

   >>> import datetime as dt, time, freezegun
   >>> dt.datetime.__name__, time.time.__name__
   ('datetime', 'time')
   >>> with freezegun.freeze_time("2020-01-01"):
   ...     print(dt.datetime.__name__, time.time.__name__)
   ...
   FakeDatetime fake_time
   

   This difference can subtly break code that checks or records types.

Why time-machine is O(1): the swap

time-machine takes a different approach: it swaps what the date and time functions do, a bit like unittest.mock but for built-in functions.

In CPython, a built-in like time.time is a PyCFunctionObject. Its m_ml member points at a PyMethodDef struct, and that struct’s ml_meth member is the C function pointer that actually gets called.

That pointer is writable, so time-machine’s patch() overwrites it, saving the original first (source):

if (state->time_time->m_ml->ml_meth != _time_machine_time) {
    original_time = state->time_time->m_ml->ml_meth;
}
state->time_time->m_ml->ml_meth = _time_machine_time;

That’s the whole technique, repeated for the ten functions that read the clock, such as datetime.now() and time.time_ns(). Ten pointer writes, no matter how big your project is.

Thus we can say time-machine has O(1) runtime complexity: its runtime is constant, no matter how many modules are loaded or how many attributes they have.

This process also fixes the two sources of leakiness in freezegun discussed above:

1. The swap is done at the C layer, so every reference to the function object, no matter where it lives, sees the new behaviour.

2. The types of date and time functions are unchanged:

   >>> import datetime as dt, time, time_machine
   >>> dt.datetime.__name__, time.time.__name__
   ('datetime', 'time')
   >>> with time_machine.travel("2020-01-01"):
   ...     print(dt.datetime.__name__, time.time.__name__, time.time())
   ...
   datetime time 1577836800.0
   

Migrate today with time-machine’s migration CLI

If the numbers above look temptign to shave seconds or minutes from your test suite, time-machine makes it easier to migrate with its migration CLI that does some of the work for you. Run it with uvx, pointed at your test files:

$ uvx --from 'time-machine[cli]' python -m time_machine migrate tests/test_delorean.py
Rewriting tests/test_delorean.py

Given this file:

import datetime as dt

from freezegun import freeze_time


@freeze_time("1955-11-05 01:22")
def test_delorean():
    assert dt.date.today().isoformat() == "1955-11-05"


def test_clock_tower():
    with freeze_time("1955-11-12 22:04"):
        assert dt.datetime.now().hour == 22

…the tool rewrites the import, the decorator, and the context manager:

 import datetime as dt

-from freezegun import freeze_time
+import time_machine


-@freeze_time("1955-11-05 01:22")
+@time_machine.travel("1955-11-05 01:22", tick=False)
 def test_delorean():
     assert dt.date.today().isoformat() == "1955-11-05"


 def test_clock_tower():
-    with freeze_time("1955-11-12 22:04"):
+    with time_machine.travel("1955-11-12 22:04", tick=False):
         assert dt.datetime.now().hour == 22

Note tick=False appearing where freezegun didn’t pass tick. freezegun freezes time by default, whilst time-machine lets it tick, so the tool spells out the old behaviour to keep your tests passing. Once migrated, dropping tick=False where you can is worthwhile—time that advances is more realistic, although it means writing assertions against ranges rather than exact values.

The CLI does partial replacements, so it can leave a file in a broken state, such as an unused freezegun import next to a call it couldn’t rewrite. Run it from a clean commit and lean on your linters to find the leftovers. Ruff’s F401 and F821 rules catch most of them.

Fin

O(1) is where the heart is,

—Adam