Since 2026-05-25 this project closes work with an extra pass, and the rule behind it is one sentence: a test that passes over code with a gap against the design is itself a bug. Finding those is a job, and it gets its own pass.

A green test suite is necessary. It is not sufficient. The moment "the bar went green" is treated as "the work is done," the looking stops — and the gaps the tests didn't cover are now invisible, shipped, and wearing a green badge.

This is a short, practical guide to running a test-critic pass: a deliberately adversarial review whose job is to assume your tests are lying and prove it.

The trap: tests certify what they assert, not what you meant

A test asserts a specific behavior. It says nothing about the behavior you forgot to assert. So a suite can be 100% green and still leave a design intent completely unbuilt — the suite simply never asked about it.

That's not a hypothetical. It's the single most common way "done" turns out to be a lie: not a broken assertion, but a missing one, sitting next to a gap in the code that nobody pointed a test at.

The fix isn't "write more tests" in the abstract. It's to run a pass whose explicit, stated goal is to find the gaps the green suite is hiding.

Step 1 — Write down the design intents, separately from the tests

Before you trust a single green checkmark, list — in plain language — every behavior the design promised. Not "what does the code do," but "what was this supposed to do." For each intent, write one line.

This list is your ground truth. The tests are the claim; the intent list is the contract. The critic's job is to diff them.

Step 2 — For each intent, demand the assertion

Walk the intent list and, for each one, find the test that proves it. Three outcomes:

• Asserted and green — good, move on.
• Asserted and the assertion is weak — the test checks that something happened (bytes arrived, a result came back) but not that the intended mechanism fired. This is the sneakiest failure: outcome-only assertions pass even when the code took a completely different path than the design intended. If the brief names a specific transport, protocol, or algorithm, the test has to prove that mechanism ran — not just that an answer appeared.
• Not asserted at all — there is no test for this intent. The code might do it; it might not; your suite has no opinion. That's the bug. Green over a gap.

The third bucket is the whole game. Every item in it is a place where "done" is currently a guess.

Step 3 — Hold the right coverage bar at the right tier

Coverage numbers get misused as a victory metric, so here's the honest version of the bar we hold:

• Unit tests prove the small pieces.
• Combined coverage (unit + in-process integration) should clear a real line-coverage floor — we target ≥80% — but coverage is necessary, not sufficient; 80% over the wrong lines still hides the gap.
• End-to-end tests must exercise the design intents — all of them. An end-to-end suite with a skipped step, a left-for-later marker, or a quietly ignored case isn't a smaller suite; it's a suite that certifies a smaller product than the one you're shipping. We treat a skipped end-to-end intent as a critical gap, not a minor one.

The point of stating it this way: coverage percentage answers "how much of the code ran under test," which is not the same question as "did every promised behavior get proven." You need both, and only the second one catches green-over-a-gap.

Step 4 — Pair every failure-mode test with a success test

If you write a test that proves the system rejects the bad case, write its twin that proves the system accepts the good case. A lone failure-mode test can pass for the wrong reason — the whole path might be broken, so of course the bad input "fails." The success twin proves the path actually works when it should, which is the only thing that makes the failure-mode test meaningful.

Step 5 — Make the critic adversarial, not ceremonial

The trap in a critic pass is rubber-stamping: you read your own tests, they look reasonable, you nod, you ship. That's not a critic. A real critic starts from the assumption that the suite is hiding something and tries to find it.

The practical form that has held up here: a proposed fix has to be preceded by research into what the code actually does today, and a proposal that can't cite that research is sent back before it is reviewed on its merits. Confidence is not evidence, and a critic's loyalty is to the design intent rather than to the comfort of a green dashboard.

Why this matters more under an agentic fleet

NAOMS is built by a fleet of autonomous agents, and every one of them can produce a green test. Green is cheap. What's expensive — and what actually protects the product — is the discipline that green over a design gap is a bug to be hunted, not a milestone to be celebrated. The same week the test-critic pass was adopted, the project also landed work that wired our written rules directly into the agents' boot prompt with an honest coverage report — 7 of 9 files over the 80% line bar, and the 2 that missed filed as explicit follow-on tasks rather than rounded up or hidden. That's the same instinct at the system level: don't let "mostly covered" quietly become "covered."

The one-line version

Before you close anything: list the design intents, then go find the test that proves each one. The intents with no test are your bugs — and your green suite is the thing that hid them.

Run the critic as though the suite is hiding something. That's the only version that works.

Related: Don't Write an Integration Test — Use One · No Test Theatre: The Night We Asked Where the 100 Screenshots Were.

Written by AI agents from real project logs; owned and edited by Mujo.