Most reliability work is defensive: you test that the bad thing doesn't happen, you add a timeout, you alert when it does. This work makes a stranger, stronger claim. It's a design thesis, not a bug report: a main-loop hang should be structurally unbuildable. Not detected. Not recovered-from. Unbuildable — a thing you cannot express in the code at all.

That's an unusual bar, and it's worth understanding why a project would set it, because the reasoning generalizes well beyond NAOMS.

Why a hang is the worst possible failure

A daemon's main loop is the heartbeat. In NAOMS it's the thing that drains the event queue, services requests, advances the workflow engine, broadcasts to peers — one loop, turning continuously, the single point through which the daemon's liveness flows.

If any operation inside that loop can block — wait on a lock it might not get, await an I/O call that might never return, sit inside a transaction that another writer is holding — then the entire daemon stops. Not slows down: stops. Every request behind it queues forever. Every peer waiting on a response times out. The process is still alive, still holding its port, still answering "yes I'm running" to a naive health check — and doing nothing. It is the most expensive failure mode a long-lived process has, because it is indistinguishable from health right up until you notice nothing has moved in ten minutes.

And here's the asymmetry that makes the thesis reasonable rather than zealous: a hang has no blast radius limit. A crash takes down one process and a supervisor restarts it. A slow function makes one request slow. A hang inside the main loop takes down everything the daemon does, indefinitely, with no automatic recovery, while reporting itself healthy. The downside is unbounded. When the downside is unbounded, "we test that it doesn't happen" is not good enough. You want it to be unrepresentable.

There's a builder's idiom for this that ran through the whole week's work: a hang is unbuildable. If the loop physically cannot contain a blocking operation, then no future commit, no tired 3 a.m. agent, no plausible refactor can reintroduce the outage. You don't prevent the bug. You delete the category.

What "structurally impossible" actually requires

It's easy to say "the main loop must never block." Saying it is a comment, and comments don't enforce anything. To make it structural you have to do three concrete things, and this week's work is best understood as progress on all three:

1. Identify every blocking operation that can appear inside the loop's critical section — locks, transactions, synchronous calls into native code, anything that can wait unboundedly.
2. Lift that work out of the critical section — do the dangerous part outside the loop's serialized window, so the loop itself only ever touches operations with a bounded, non-waiting shape.
3. Make the bad shape detectable and refuse it — an automated check that fails the build when someone reintroduces a blocking call where the loop can reach it, so the guarantee doesn't decay the moment attention moves on.

This week's two landings are textbook examples of step 2 — lifting the most dangerous work out of the loop — and they're worth reading closely.

The signer was the worst offender

The single most dangerous thing a NAOMS daemon does inside its hot path is sign. Cryptographic signing crosses the boundary from the daemon's runtime into native signer code — a synchronous call across the language boundary into Rust. A synchronous call into native code is exactly the shape this work fears: from the event loop's point of view, it is an opaque operation that returns when it returns. If the signer stalls — a lock contended in the native layer, a slow key operation, a backpressured queue — and that call sits on the loop's thread, the heartbeat stops. The whole daemon hangs on a signature.

The fix that landed on the seventh of June was named exactly for the problem: a signer-lane fix. The shape of that class of fix is to give signing its own lane — to move the signer off the main loop's critical path so that a slow or contended signature can no longer freeze the heartbeat. The loop hands signing work to the lane and stays free to keep turning. A stall in the signer becomes a slow signature, which is survivable, instead of a dead daemon, which is not. That is the whole game in one landing: convert an unbounded-blast-radius failure into a local, bounded one.

Genesis signing, lifted out of the transaction

The second landing is subtler and, to us, the more instructive of the two. On the sixth of June, a fix landed whose own description is the lesson: signing lifted out of the database transaction.

Here's the trap it closes. Creating the very first identity on a device — the genesis account — involves both a database transaction and a signature. The natural, tidy way to write that is to do the signing inside the transaction: open the transaction, do the work, sign, commit. It reads cleanly. It is also a latent deadlock.

A synchronous signing call nested inside an open database transaction means you are holding a database lock while making an opaque, potentially-blocking call into native code. If the signer stalls, you don't just stall the signer — you stall it with a transaction lock held, which means every other writer that needs that lock now queues behind a frozen signature. One blocking call, made inside a held lock, and the contention fans out across the whole writer set. The fix notes that this mirrors an earlier batch fix elsewhere in the codebase — which tells you this isn't a one-off; it's a recurring shape the project has learned to recognize and is now systematically hunting down.

The fix is to lift the signature out of the transaction's window: do the signing before or after, so the database lock is never held across the call into native code. The transaction becomes short and bounded. The signature becomes a separate, non-lock-holding operation. The deadlock isn't fixed — it's made unconstructable in that path, which is precisely the thesis at the granularity of a single function.

Why this is "unbuildable" and not just "fixed"

The distinction matters, and it's the reason this is an architecture effort and not a bug ticket.

"Fixed" means: this specific signer-lane stall, this specific genesis deadlock, no longer happens. Valuable, but fragile — the next refactor can reintroduce either one, because nothing stops a future author from nesting a sign inside a transaction again. They'd be following the tidy-looking pattern that reads correctly and deadlocks.

"Unbuildable" is the harder target this work is reaching for: a state where the blocking shape cannot be written where the loop can reach it, because an automated check refuses it. Lift-the-work-out (this week's landings) is the necessary first move — you cannot forbid a shape until you've removed every legitimate instance of it. The forbidding step is the closing one. As of this writing the effort is still open and in its alignment phase — the lifting is landing in tranches, the structural ban is the destination, not yet the arrival. We want to be precise about that: this week shipped slices of the thesis, not the finished guarantee. The signer lane and the genesis-signing lift are real, landed, and verified. The "you literally cannot build a hang" end-state is the goal these slices are walking toward, and it isn't reached yet.

That honesty is itself part of the design philosophy. The strongest version of a reliability guarantee is one you can prove structurally — and the second-strongest, the one you actually live in while you build toward the first, is one where you've named the dangerous shape exactly, removed it where it lived, and are converging on the check that will forbid its return. This work is firmly in that second state, moving deliberately toward the first.

The generalizable lesson

If you maintain any long-lived process with a central loop — a daemon, a game loop, an actor's mailbox, an event reactor — this thesis is worth stealing:

• Enumerate what can block inside the loop's critical section. Locks, transactions, synchronous calls into native or remote code. Be exhaustive; the one you miss is the one that hangs you.
• Lift the dangerous work out of the critical section into a lane, a queue, an async boundary — so a stall degrades to slowness instead of escalating to a freeze.
• Never hold a lock across an opaque, potentially-blocking call. The genesis-signing-outside-the-transaction fix is this rule made concrete: a held lock plus a blocking call into native code is a deadlock waiting for the wrong moment.
• Then forbid the shape with a check, so the guarantee survives the people who come after you and don't know why the rule exists.

A hang is the failure with no ceiling. The only fully satisfying answer to a failure with no ceiling is to make it impossible to express — and the honest path there runs through exactly the kind of unglamorous, one-call-at-a-time lifting that landed this week.

Written by AI agents from real project logs; owned and edited by Mujo.