AI-Robustness Gates

Context

A mature codebase usually has a constitution: the architectural invariants it must never violate. When most code is human-typed, reviewers hold that line. They notice the import that crosses a boundary, the adapter leaking into core, the logic creeping into a template.

That enforcement mechanism is vigilance, and vigilance does not scale with AI-authored code. The diff grows faster than the attention available to read it.

Monperrus reaches the same diagnosis and draws the opposite conclusion: if human review cannot keep up, retire it and close the loop agent-to-agent (opens in a new tab). The diagnosis is right and the prescription gives away the wrong thing. The problem was never that humans review; it is that the invariants were resting on a reviewer happening to notice. Move those onto a machine and the reviewer gets their attention back for the part no gate can check — whether the change should have been made at all, and whether anyone can still stand behind it. That part does not go agent-to-agent, because catching defects was never what it was for. Somebody has to answer for the result, and an agent cannot be the one who does.

The audit that started it

This began as an audit rather than a build: taking a written constitution article by article and asking, for each one, whether anything mechanical would actually catch a violation — or whether the article was resting on a reviewer happening to notice.

That question produced everything below. An invariant nothing enforces is not an invariant. It is a preference with good documentation.

What I built

A working reference implementation, public and inspectable at github.com/mephistopheles4/stacks (opens in a new tab). It is a real tool rather than a demonstration harness — a local-first reading tracker whose published build runs at stacks.aymandiab.com (opens in a new tab), so the gates below are defending something that actually ships:

• Invariants as executable gates. Architectural rules live as tests that fail the build — adapter/core boundary enforcement, no-logic-in-templates, import-direction rules, a build-mode contract, an environment contract, and a gate proving the public build carries no private content.
• One required check, never path-filtered. The merge gate is a single aggregator over the whole matrix. A required check that gets skipped reports nothing and blocks the pull request forever; a required check named after a matrix leg has to be renamed every time that matrix changes.
• Supply-chain posture inside the gate. Actions pinned to commit SHAs rather than mutable tags, a dependency audit job, and a documented escape hatch that requires a written justification and surfaces as a reviewable one-line diff.

Every gate carries its reasoning in a comment. Together they are meant to be read as an argument, not as configuration.

What I designed

Beyond the reference implementation, a full gating initiative scoped for a production codebase:

• An invariant scoreboard — every constitutional article either gated in CI or visibly not, so coverage of the constitution becomes a tracked number instead of a feeling.
• Mutation scoring — a Stryker pipeline with per-directory scores and a ratchet, a floor rising toward a target, so tests are graded on whether they catch defects rather than whether they execute lines.
• A changed-lines coverage floor — quality pressure landing exactly where the diff is.
• A trend layer — a Prometheus/Grafana release-confidence dashboard (Pushgateway batch-job pattern) plus runtime production counters for select invariants, as defence in depth.

Design decisions

The design carries an explicit anti-target: global coverage percentage is never a goal. Global coverage is as gameable as software metrics get; per-directory mutation scores and changed-lines floors are far harder to satisfy dishonestly. The plan includes worked failure scenarios and an adversarial gaming analysis — how an engineer, or an AI, might satisfy each gate without satisfying its intent, and what closes that gap.

The separation between hard gates and the trend layer is deliberate. Gates must be cheap, deterministic, and unarguable; trends are where nuance and judgment live.

Impact

The reference implementation runs today, enforcing its own invariants on every push. The gate proving the public build carries no private content is the one I would point at first: it exists because the published site is built from a private vault, and it is the only thing standing between a refactor and my own notes going online.

The reframe is the durable part: for quality engineering in the AI era, “review harder” is not a strategy. The constitution has to enforce itself.