Empty Fixtures Hide More Bugs Than They Catch
One command pours realistic, production-shaped data into the lowest layer so every package lights up like the real thing โ shipped this week
Most test fixtures are lies of omission. They contain three users, one hive, and a handful of messages โ just enough to make an assertion pass and nowhere near enough to behave like the thing you're actually building. The enrichers run, but on so little data that their scaling cliffs never appear. The materializers materialize, but never under the join pressure of a real graph. The UI renders, but against a corpus so small that the empty states, the pagination, the slow-folder, the thousand-photo river โ none of it ever shows up until a real user trips it in production.
The fix is conceptually simple and operationally hard: ship a seedable, real-world-shaped dataset for every package and core, poured in at the lowest layer, so the whole stack lights up like production on demand. That is what this epic built, and it genuinely shipped โ celebrated and closed โ with the key piece landing this week. The goal, stated plainly: every package and core ships a seedable, real-world-shaped dataset.
The verb: naoms dataset seed <profile>
The user-facing surface is one command. It landed this week, and the shape of its implementation is the whole design in miniature.
naoms dataset seed <profile>A profile resolves to a small manifest naming which packs to pour in. The handler walks the manifest and imports each pack in turn, in-process, composition only. That last constraint is the load-bearing one. The seed verb does not reimplement importing; it never duplicates the sole importer. It is a thin orchestration over an existing, single, authoritative import path. One place writes the data; the verb just decides which data and in what order.
Two properties fall out of that discipline, and both matter more than they look:
It is idempotent. Re-running seed doesn't double your corpus. The
underlying importer remembers what it has already imported; an already-imported
pack surfaces as a skip, not an error and not a duplicate. You can seed, work,
seed again, and the world is the same. For a substrate everything else pours
into, idempotence isn't a nicety โ it's what makes the thing safe to lean on.
It fails honestly. The integration coverage that shipped with it tests two paths, not one: the happy-path aggregation and the profile-not-found failure mode. A seed command that silently produces nothing when you typo a profile name is worse than no command. This one tells you.
Why the sole-emitter invariant is the whole design
It is worth dwelling on the "composition only, never duplicates the sole emitter" constraint, because it is the difference between a feature and a liability.
The naive way to build a seed command is to give it its own knowledge of how to write data: it knows the table shapes, it knows the insert order, it does the work. The moment you do that, you have two code paths that both claim to produce a valid world โ the real import path and the seed path โ and they will drift. One will learn about a new node type or a new validation rule and the other won't. Six weeks later your "seeded production" is subtly not what production actually writes, and every test that runs against it is testing a world that can't exist. That is the classic fixture rot, just industrialized.
The sole-emitter discipline forecloses it structurally. There is exactly one
piece of code that knows how to turn a pack into chain state, and everything
goes through it: the low-level pack import, the dataset seed verb that
orchestrates several imports, the research corpus publisher. The seed verb
contributes orchestration (which packs, what order, aggregate the results) and
contributes zero knowledge of how data is written. If the import
path learns a new invariant, the seed verb inherits it for free, because it never
knew the old one either. One emitter means one truth about what a valid world is,
and the seed command is physically incapable of producing a different one.
flowchart TD
A["dataset seed <profile> verb
(orchestrates several imports)"] --> E
B["low-level pack import"] --> E
C["research corpus publisher"] --> E
E["the one import path
(sole emitter)"] --> S["valid chain state"]
This is the same axiom that runs through the rest of NAOMS โ Honesty as a structural property, not a promise โ applied to test data. A fixture that can diverge from production is a small dishonesty waiting to happen. A fixture that cannot diverge, because it shares the one and only writer, simply can't lie to you about shape.
Packs: versioned, cached, and honestly red where they're red
Underneath the verb are packs โ the actual bundles of real-world-shaped data. The honest engineering of this week lives in three details.
They're versioned. A second pack reached its own release version this week, carrying a verification receipt for the first โ the packs carry provenance, not just bytes, so you know what you poured in and can verify it arrived intact.
They're cached, deterministically, in the release build. A subtle correctness issue: if your dataset cache location drifts between build and run, your "seedable production" becomes "whatever happened to be on disk." The fix pinned the cache location in the release build path this week, so the seeded world is reproducible, not ambient.
They're honestly tracked when broken. This is the part we most want to underline, because it is the difference between a test substrate and a marketing one. A pack-test sweep this week reported 348 green and 122 red. The 122 red are not hidden, not suppressed, not rounded away. A dataset system whose own tests are partly red, and says so, is more trustworthy than one claiming 100% โ because the whole reason it exists is to surface the failures small fixtures conceal. Hiding its own would defeat its purpose.
The compounding move: a real corpus published as a pack
The most elegant thing about this substrate is that it composes upward. A
parallel research effort โ studying how coding agents remember โ needed a corpus
to study, and instead of inventing a parallel format, it published its corpus
into the same pack-store layout. This week, the corpus-pack option was wired
through the research collection paths, so a research corpus becomes just another
pack you can seed.
That is the payoff of getting the substrate right. Once "realistic seedable data" is a single mechanism with one emitter, one cache key, and one import path, every future need for realistic data โ the photos river, the finder's thousand files, a wallet's transaction history, a research corpus โ stops being a bespoke fixture and becomes a pack. You pour it in at the bottom and watch the stack light up.
What this is and isn't
To stay precise: the seedable-dataset epic is shipped โ celebrated and closed, the verb landed, packs are versioned and cached, the corpus-as-pack composition works. What it is not is a claim that every pack is green; 122 of them aren't, and that's tracked on purpose. The honesty of the red count is part of the design, not a blemish on it.
The deeper principle is one we'd defend anywhere: your fixtures encode your assumptions about scale, and small fixtures encode small assumptions. If you want software that behaves like production, you have to be able to pour production in โ cheaply, reproducibly, idempotently, and at the lowest layer โ any time you want. Now we can.
Written by AI agents from real project logs; owned and edited by Mujo.