Building an Agentic CI/CD Pipeline: Four Agents, One Human Gate, and What Actually Breaks

Today it runs on four personal codebases — 174 automated agent executions, 47 tasks dispatched end-to-end, $173 in total AI compute across all of it. The architecture has two non-negotiables: every write is pre-checked against a blast-radius tier map that prevents touching high-risk paths without authorisation, and every agent run executes against a documented harness — defined inputs, automated validation before human review, and a clear escalation path. Trust expands on data. Not intuition.

Four agents, one human gate#

Every handoff between agents is a typed contract — not a raw file or text summary. Each agent's output is validated by the receiving stage before proceeding. A malformed or incomplete handoff is rejected immediately and goes back to the producing agent before the next stage sees it.

When CHECKER finds a semantic gap, it doesn't escalate immediately. It issues a targeted deficiency — what's wrong, which agent owns it, why it matters — and the owning agent retries. The pipeline does this up to twice. Only if the gap persists does it escalate to the MANAGER. The MANAGER sees escalations, not every run.

Each agent's output is hash-chained: every step signs what it received and what it produced. If a downstream agent's hash doesn't match the upstream output, the pipeline fails. That audit trail matters when someone eventually asks 'who changed what and on whose authority.' At team scale, that question has a compliance dimension, not just a process one.

Blast-radius tiers#

REVIEWER maps every file the BUILDER touches against a tier assignment. Here's what those tiers mean.

scroll to explore →

The constraint this enforces: the BUILDER cannot complete a write to a Tier-0 or Tier-1 path without an explicit tier authorisation in the original task brief. If the task brief says Tier-2 and the BUILDER attempts to touch a payment callback handler, the pre-write hook escalates to the MANAGER — not after the commit, before it. Across 52 Tier-0 and Tier-1 agent runs to date, zero incidents. That's the number that justifies the architecture.

An automated system that can touch anything is a liability. A system structurally prevented from reaching high-risk paths without a human checkpoint is a controlled system. The goal isn't to remove humans — it's to put humans in the loop at the right moment, with the right information.

The proof#

gh-118 is a real dispatch run — unedited. Issue #94: cost-route BUILDER and TESTER by tier without compromising the guardian quality floor. The file it needed to change is Tier-1. Red-zone. The pipeline classified it correctly before a single line was written.

What the numbers show#

All numbers are audit-verified across 4 repos — not projected. When CHECKER finds a gap — lint failure, coverage miss, or a task without sufficient context in the ticket — it refires or escalates. Refires self-correct without human involvement. Escalations reach the MANAGER as a structured packet. Across 174 runs, 12 escalated, 9 self-corrected via refire. None produced a silent failure.

The cost picture: $173 in AI compute across 47 tasks — ~$6 per completed task, including Opus on every review and check stage. The same tasks done manually: 2–3 hours each of fragmented engineering attention — context gathering, writing the fix, test coverage, PR scaffolding, waiting for review. At that rate, the economics aren't marginal. They're a different order of magnitude.

The forcing function I didn't anticipate: the pipeline only works on tickets with verifiable acceptance criteria. 'Fix the null pointer' → works. 'Improve reliability' → escalates immediately. That constraint pushes ticket quality up across all tickets, not just the ones the pipeline handles. The tool improves the process around it.

At team scale#

The endgame isn't faster engineers. It's engineers who stop reconstructing what a change does before deciding on it — they receive a structured packet and make a call. The BUILDER handles write-test-fix. The human handles architecture, escalation routing, and the decisions the machine isn't authorised to make.

01Architectural context

The BUILDER knows your code and your ticket. It doesn't know your architectural intent — why the contract looks the way it does, what was considered and rejected. Context injection from ADRs and decision records is the gap between 'handles bounded changes' and 'handles anything.'

02Trust policy

Right now the trust threshold is a personal call — I move it by hand. At team scale it becomes policy: a tier's gates loosen only after a clean track record — a set number of incident-free runs above a coverage floor, signed off by an owner — never adjusted ad hoc by whoever's blocked that day.

03Tier-map ownership

Who maintains the tier map as the codebase evolves? Who reviews trust decisions? Who gets escalation notifications? These are leadership decisions, not engineering decisions. The pipeline forces the conversation instead of letting it stay implicit. That's a feature, not a bug.

The next agent isn't another specialist — it's a TEAM LEAD that coordinates work across a team's pipelines: sequencing dependent tasks, surfacing cross-repo blast-radius conflicts, routing escalations to the right EM. Four agents on one repo becomes a network.

Pipeline prerequisites#

The pipeline is only as good as the repo it runs against. A pipeline-ready repo has three things: verifiable acceptance criteria on every ticket, a deterministic CI suite, and a calibrated blast-radius tier map. Remove any one and the pipeline degrades in a predictable way.

• Ambiguity breaks it first. A ticket without verifiable ACs either escalates immediately or produces a conservative no-op — CHECKER has no ground truth to verify against.
• Flaky tests break it second. If CI is non-deterministic, BUILDER can't converge — what should be a straight-through run becomes an escalation instead.
• Novel architecture breaks it third, for the reason covered above — BUILDER has bounded context, not architectural intent.

Connect#

If you're building an engineering organisation that wants to think seriously about agentic-at-team-scale — what governance looks like, what breaks first, how to expand trust incrementally — I'm actively exploring Senior Engineering Manager roles internationally. I'd rather talk about the hard problems than send a CV into a void.