Skip to content

deterministic-dice

To test a game of chance, hand it loaded dice: difficulty 0.0 always succeeds, 1.0 always fails, and suddenly a stochastic system has exact expected outputs.

The mechanism

The engine's roll (_engine/rng.py) is: success iff random() >= d, so P(success) = 1 − d. The two degenerate values are exact, not approximate:

  • d = 0.0 → random() >= 0 is always true → always succeeds
  • d = 1.0 → random() >= 1 is (effectively) never true → always fails

Every adjudicator test (tests/test_adjudicator.py) scripts a fake-provider response with one of these two difficulties, which turns "the LLM proposed and the engine rolled" into a deterministic assertion: valid deltas apply, failed rolls apply nothing but still cost the action point, malformed output is retried then rejected without costing the player.

Why this shape

The system under test has two sources of nondeterminism — the model and the dice — and the seam isolates both:

  1. The model is replaced by FakeProvider(responses=[...]), a scripted queue. Tests assert against exact proposals, including malformed ones (schema-retry paths are just another scripted sequence).
  2. The dice are engine-owned (see the ADR), so tests reach them through difficulty's degenerate values — no monkeypatching random.

Prediction: any "LLM proposes, engine disposes" system is fully unit-testable iff both seams exist. If the LLM rolled its own dice (narrating success), no fake provider could make outcomes assertable — one more reason the engine owns the dice beyond game fairness.

Where it bit us: nowhere in the tests — but the suite itself silently ran on the wrong interpreter for a while (system Python had playwright globally; uv pip without -p .venv had installed into it). Belief falsified 2026-07-08: "python -m uv pip install targets the project venv" — it targets the current interpreter. Always -p .venv, always run tests via .venv/Scripts/python.exe -m pytest.