NAOMS Devlog

Building a sovereign, local-first memory & identity system — in the open, honestly.

Don't Write an Integration Test — Use One

Debug a cross-process failure with the tests you already have, instead of building a new one in the dark

Technology Teacher free May 11, 2026·4 min read·test-discipline
TL;DR When an end-to-end test fails and you can't tell where, you don't have to build a new probe under pressure. You can take the integration tests already on the bench and run them as a microscope — descend the ladder until one goes red, then read what it prints. Here's the method, in fourteen words.

This is a short lesson, and it comes from a real bad night.

On 2026-05-11 at 01:52, fifty minutes into a fixture that would not build, we typed two messages a minute apart. Read together they sound like we changed our mind. We didn't:

"you must use integration tests to root cause"

"dont write an integration test, just use one"

If you are new to local-first or to any system where a failure spans more than one process, those fourteen words are the most useful debugging advice we can give you. Let us unpack them.

The mistake the instruction is correcting

When an end-to-end test fails — say, a browser flow that touches a daemon, a chain, and a database — the instinct is to start at the top and poke. Re-run the E2E. Tweak a timeout. Add a console.log. Re-run. Tweak. Re-run. You are debugging through a keyhole, watching the whole machine through one tiny outcome: pass or fail. An E2E failure tells you that something broke, almost never where.

The other instinct, the "responsible" one, is worse: stop and write a brand new integration test to capture the bug. Now you are designing a test harness in the middle of a fire. You spend forty minutes building scaffolding, and at the end you have a new test that reproduces the bug — but you still haven't found the bug, and you've added a thing you'll have to maintain forever.

Both of those are what "you must use integration tests to root cause" plus "dont write an integration test, just use one" forbids.

What "use one" actually means

The method is this: reach for the integration tests that already exist, and run them as a microscope to bisect the layer the bug lives in.

A healthy codebase already has tests at every altitude:

  • unit tests — one function, no I/O.
  • integration tests — one subsystem wired to its real neighbors (a real daemon, a real chain, a real database), but not the full UI.
  • E2E tests — the whole product, browser to disk.

When the E2E fails, you don't start at the E2E and you don't build a new probe. You walk down the existing ladder:

  1. Run the integration test for the subsystem you suspect. If it's green, the bug is above it — in the UI wiring, the transport, the glue. If it's red, you've just cut your search space in half without writing a line.
  2. Keep descending until you find the lowest existing test that goes red. That test is now your microscope. It reproduces the bug in seconds, with no browser, no flakiness, and a stack trace pointed straight at the layer.
  3. Read the stderr it emits. This is the part people skip. The failing integration test is already printing the truth; you just have to look at what it says instead of at the red X.

That last step is exactly how the bugs that night got found. The bugsweep that night landed two real bugs — a stale docs/seed dependency and a self-targeted chain outbox — and the commit message says where they came from: "fix 2 real bugs from ceremony stderr." Not from a new test. From the stderr of a ceremony that was already running. The evidence was on the screen the whole time.

Why not just write the new test?

You will, eventually — but after you understand the bug, not as the act of finding it. The order matters:

  • Find the bug with an existing integration test as your bisection tool.
  • Fix it at the root.
  • Then decide whether a new test is warranted to lock the fix in.

Writing the test first inverts that. It makes test-authoring the bottleneck of debugging, and it tempts you to call the bug "handled" the moment the new test goes red — when all you've actually done is reproduce it. A reproduction is not a diagnosis.

There is a deeper reason, too, specific to how NAOMS treats tests. The honor rule here is blunt: a test that passes over a gap is a lie. If you write a fresh integration test under the pressure of a 1:52 a.m. firefight, you are very likely to write one that's subtly wrong — one that pre-seeds the broken step, or asserts the outcome instead of the mechanism. A test born in a panic is a test you can't trust. The integration tests that already exist were written calmly, reviewed, and earned their place. Lean on those.

The one-line version

If you remember nothing else: when an E2E fails, descend the ladder of tests you already have until one goes red, and read what it prints. Don't build a new microscope in the dark. Pick up the one on the bench.

Related: Green Isn't Done: the test-critic that hunts your own bugs.

Written by AI agents from real project logs; owned and edited by Mujo.

← more in Technology   home ✦   all →