# The machinery around the loop

> The small tools that keep a long agent loop honest: a linter for the charter and queue, Stop hooks, a watchdog, a reconcile step, a verifier, the pi extensions and a closeout audit.

- Canonical page: https://rob-knight.com/articles/the-machinery-around-the-loop/
- Kind: Article
- Published: 2026-09-26
- Updated: 2026-10-21
- Topics: agents

[The agent loop protocol](/articles/the-agent-loop-protocol/) is text, and text enforces nothing. A paragraph doesn't stop a root ending its turn on a status message, a root that dies at two in the morning doesn't report its own death, and a charter that breaks a rule reads the same as one that doesn't. So the loop has small tools around it, each there because another sentence wouldn't have fixed the failure it covers. The contract itself is at [/reference/loop-contract/](/reference/loop-contract/), with a page per harness for [pi](/reference/loop-harness-pi/), [Claude Code](/reference/loop-harness-claude/) and [Codex](/reference/loop-harness-codex/).

None of these tools chooses or accepts the work. Each one checks, records, re-prompts, recovers or reports, and the decisions about the work stay with the root or with me.

<figure class="diagram figure-breakout">
  <img class="on-light" src="/diagrams/44-tldr-the-machinery-around-the-loop-v2-light.svg" alt="At a glance: the protocol is text and text enforces nothing, so small tools sit around the loop, and each one checks, records, re-prompts or reports. Four columns. Checks: loop-lint charter, loop-lint queue, loop-lint loopmd, and the closeout audit on pi. Records: QUEUE, LANES and DECISIONS, the pi event log written by the harness, and the usage block in the closeout. Re-prompts: the Stop hook, up to three times, and pi continuation with its automatic recovery. Reports: the watchdog, which writes Grafana logs and sends three classes to the phone, and loop-verify, which reopens a task when its check fails. Highlighted along the foot: none of these decides anything about the work." width="1440" height="714" loading="lazy" decoding="async">
  <img class="on-dark" src="/diagrams/44-tldr-the-machinery-around-the-loop-v2-dark.svg" alt="" aria-hidden="true" width="1440" height="714" loading="lazy" decoding="async">
  <figcaption>Sorted by what each tool does, there's no column for choosing or accepting work, and that gap is deliberate.</figcaption>
</figure>

## loop-lint checks a charter, a queue and LOOP.md

`loop-lint` has three subcommands, one for each file a run depends on. It exits 1 on any error; warnings and info lines never fail it.

- `charter` checks a charter's shape: the `# Charter: <repo> loop<N>` header matching the file name, every required key (harness, root, aim, priorities, exclusions, lane cap, gate, gate route, standing file and the overnight mandate) present and filled, a `## First wave` table, at most 8 KB, and a `standing:` file that exists and still hashes to the sha256 the charter records.
- `queue` reads the task IDs from a `QUEUE.md` and fails any that the charter excludes, that's already Done or that carries an excluded label. `needs-owner` is always excluded. It reads each task through the Backlog CLI in the repository that owns the charter, so it sees the board's current state.
- `loopmd` checks a repository's `LOOP.md`: its size, its keys, dated facts and restated protocol. Online it also reads main's required checks from GitHub: it fails when `ci-required` doesn't match them, and warns when a `baseline-red` key looks stale or missing.

Preparation runs `charter` and `queue` until both are clean, and the root runs `queue` again before every wave.

## The run directory is the root's record

A root keeps its state in a handful of Markdown files in `codex/loop<N>/`: `QUEUE.md` for ready work, holds and the next landing batches, `LANES.md` with one line per live lane, an append-only `DECISIONS.md` and `QUESTIONS.md` for me. After a compaction or a provider error it re-reads those and the harness's own list of live lanes, reconciles the two and carries on.

On pi the harness keeps an event log as well. `loop-continuation` appends `open` when the run arms, with the charter's sha256, the root route and the first wave, and a `judgement` for each automatic recovery; `loop-state` appends `dispatch` and `return` for every lane, and `watch` and `heartbeat` for the root's own activity. `loop-state digest <log> --json` gives the live lanes after a compaction. The log records a lane only when its brief's first line parses as `Lane: <id> · Task: <ID> (<title>) · Tier: routine|guarded`, so the root's guard refuses a lane launch whose first line doesn't.

## Preparation is one skill

One skill, `agent-workflows:loop`, prepares every loop for every harness. It starts by running `codex-reconcile` (below), reads the last closeout's Parked and For ratification sections, builds the first wave from the board, puts every choice it needs from me in one prompt, writes the charter, `LANE.md` and the launch file, and lints, then hands me the launch file. I launch the loop. [The protocol article](/articles/the-agent-loop-protocol/#preparation) has the steps.

<figure class="diagram figure-breakout">
  <img class="on-light" src="/diagrams/17-the-tools-around-one-root-v4-light.svg" alt="A timeline of one loop run and the tools that act on it. Before launch, the loop skill prepares the run and loop-lint checks the charter and the queue. The launch says 'You are the root' and names the closeout path, and it starts the root session, drawn as a central bar. Above the bar, loop-lint queue runs before each wave, and lanes gate on the charter's gate route: the remote gate runner, with the workstation as the fallback. Below the bar, the Stop hook or pi continuation re-prompts the root, and the watchdog reads transcripts and never acts, sending logs and alerts to Grafana. At the end of the bar the root writes the closeout, then runs codex-reconcile, which reconciles the run directory with the other Mac. Separately, after the run, loop-verify runs daily on tasks labelled verify:&lt;date&gt;." width="1440" height="770" loading="lazy" decoding="async">
  <img class="on-dark" src="/diagrams/17-the-tools-around-one-root-v4-dark.svg" alt="" aria-hidden="true" width="1440" height="770" loading="lazy" decoding="async">
  <figcaption>Everything on the bar is the run's own sequence. The tools beside it either re-prompt the root or watch it, and loop-verify works on its own clock after the run has ended.</figcaption>
</figure>

## LOOP.md holds the repository's facts

Each repository that runs loops commits a `LOOP.md` of at most 6,144 bytes with nothing dated in it. It's mostly keys: `tier`, `gate`, `ci-required` and `grafana-stack` are required, and `ci-max-minutes`, `ci-status`, `guarded-paths` and `baseline-red` are optional. `ci-status` is a one-shot command with a `{sha}` placeholder that replaces the GitHub CI reads on another forge. Under the keys sit `## Credentials` (tree paths and roles, never values), `## Traps` and `## Mutexes`, and a `## Lane brief` section that preparation appends verbatim to every run's `LANE.md`, so repository rules reach lanes without anyone retyping them.

## The gate route keeps heavy gates off the workstation

Every charter names a gate route, and lanes run their gates only through it. In a repository that has one, the route is a remote gate runner: `gaterun -- just check` runs the worktree's `HEAD` plus its uncommitted changes in a fresh container on a small pool of arm64 gate hosts, with a persistent slot worktree and shared caches per host. The repository declares its image, user, environment and cached paths in a committed `.gaterunner.yaml`. The exit status is the command's own, except 70 when the runner couldn't prepare the run and 124 when no slot came free in time. A slot pacer spreads many agents on one Mac across the slots.

Gate containers hold no credentials and can't reach private networks, so a gate that needs either runs on the workstation instead. So does any gate when the runner can't prepare a run. There, a `- gate: <name> | <command>` row under `LOOP.md`'s `## Mutexes` declares the heavy gate, and a pi loop runs it through `loop-gate-lock <name>`, a kernel lock per gate name shared by every home, repository and worktree under one OS account. My [rule for the gate runner](/reference/rules/gate-runner/) has the details.

## The standing-authority registry

Authority that holds for every loop in a repository lives in a private registry, one file per repository with `## Grants` and `## Fences` sections. Tooling never creates an entry: the planner proposes one, I approve it and it's committed. The planner then copies the file's sha256 into the charter and the launch, and `loop-lint charter` fails a charter whose recorded hash doesn't match the file. The harness freezes the entry at launch, a mismatch grants nothing, and loop-pi's guard refuses root and lane writes to the registry. The public export never includes it.

That leaves one batch of questions per loop. Reads everywhere, and deploys to any target that isn't a customer's, production included, are standing in the contract. Never standing, and so asked every time they come up: destructive scope outside lab and dev, production capture windows, spend over the repository's cap, credential creation, policy or role changes, org-admin Grafana token writes outside a lab stack the entry names, making anything non-public public and a customer's deployment or tenant.

## Pinned subagents, because an effort in a brief does nothing

Claude Code runs lanes as five agents from my `agent-workflows` plugin: `worker`, `senior`, `senior-escalation`, `reviewer` and `scout`. Each agent's frontmatter fixes its model and effort, so "use high effort" in a brief does nothing. A `model` or `effort` passed at dispatch overrides the pin, so a root passes neither. The [Claude Code route table](/reference/loop-harness-claude/#routes) names each one.

A Claude Code lane can't clear a permission block the root could, so SSH, deploys, cloud mutations, secret-store writes and destructive git stay with the root on both variants.

Codex spawns the same five roles by `agent_type` and passes no model or effort, and a spawn with no matching installed agent is a harness fault, and the root stops and says so.

## A message with no tool call stops the run

Opus 5.5 keeps the operator updated as it works, and some updates end the turn with text and no tool call. Nothing in Claude Code resumes the run after that. Each harness file ends with a turn-endings block naming the endings that stop a run while work is owed. That block is text too, so each harness also has a backstop.

The Claude Stop hook arms when an operator message says "You are the root" and names exactly one `codex/closeout-<date>-loop<N>.md` path, or is a bare path to a launch file that does. It then lets a turn end only if:

- the closeout exists, was written after the launch, opens with `# Closeout: <repo> loop<N> · Charter: <sha256>` naming this loop, and has its four sections; or
- the message's last line is `PAUSED: <reason>`, or `WAITING: <what> until <UTC deadline>` with a deadline that parses.

Otherwise it blocks the stop and names what the closeout lacks, at most three times in one chain; the fourth stop goes through and writes an incident file for the watchdog. A closeout older than the launch means the session is quoting a finished run, so that writes no incident. A fault inside the hook always allows the stop, so a broken hook never traps a session.

The Codex hook arms only when the session's first operator message is a launch, never in a fork or a subagent. It releases on a counting closeout or `PAUSED:`, and blocks at most three times per turn. `WAITING:` releases nothing on Codex, because nothing wakes a Codex root that has ended its turn; the root holds its turn in `wait_agent` instead. Codex skips Stop hooks when a turn ends on an error or a failed compaction, so those stay with the watchdog.

## The watchdog reads transcripts and never acts

`loop-watchdog` is a launchd job on each Mac that runs every five minutes with no LLM. It reads Claude, Codex and pi transcripts and finds loop roots by their launch, using the same parser as the Stop hooks, so the two can't disagree about which session is a root or which closeout it owes. It never resumes, kills or signals anything.

Three classes reach my phone through Pushover: a Stop hook that gave up, a Stop hook that threw, and a pi root whose context overflowed while lanes were still live. Everything else is a Grafana log, and every phone alert is logged there too:

- a dead loop: the root's latest turn ended without a counting closeout or a `PAUSED:` or `WAITING:` marker, and no newer turn has started for 15 minutes;
- an open turn silent for more than 20 minutes, and an overrun `WAITING:` deadline;
- a Codex root making ten or more sub-minute waits within half an hour, which is what polling looks like from outside;
- gaps in the watchdog's own ticks.

A Codex root blocked in one long `wait_agent` writes nothing until the call returns, so the watchdog adds that call's effective timeout before calling it silent. On pi it reads the root's request checkpoint, so a long request that is still producing doesn't read as a stall. Alerts go through a durable queue that retries on the next tick, and a phone-tier alert reaches the phone once.

## codex-reconcile moves codex/ between the Macs

`codex/` is gitignored and machine-local, and a loop can be prepared on one Mac and run on the other. `codex-reconcile <repo>` runs at two points: in preparation before the last closeout is read, and once at closeout after the closeout is written. Nothing mirrors `codex/` in the background.

It's two-way and never deletes: a file on one side only is copied across. When a path differs on both sides, the Mac holding the newest run file wins, and the other side's copy is first moved to `codex/scratch/reconcile-superseded/<UTC>/`. It skips worktrees and build output, and an unreachable peer is one line and a zero exit. The root states the outcome in its reply and never retries it.

## The closeout's usage block

A closeout's `## Notes` carries what the run cost, and the root doesn't write that part from memory. It runs `agent-history loop --usage <closeout path>` against my [session catalogue](/articles/building-an-agent-transcript-indexer/), which prints a short Markdown block, and pastes it in: wall time, model calls and list-price cost, broken down by role and model, with an as-of timestamp because the catalogue lags the transcripts. It carries no prompt text. If the command fails, the root says why instead.

## loop-verify picks up evidence that can't exist yet

Some acceptance needs time: a natural event, a scheduled job, several days of telemetry. The contract's `evidence-later` park lands the code, marks the task Done with a `verify:<YYYY-MM-DD>` label and a `verify-cmd: just verify-<name>` note, and the lane adds that recipe.

`loop-verify` runs from a LaunchAgent every morning, finds Done tasks whose verify date has passed and runs exactly that recipe with a timeout, only in a clean checkout on main; anything else is logged instead of run. A pass removes the label. A failure reopens the task, labels it `verify-failed`, appends the exit summary and logs it. Either way it commits `backlog/` alone and pushes nothing.

## loop-pi does the same jobs on pi

The default for OpenAI-model loops is the pi coding agent through a dedicated home and launcher, `loop-pi`: one home per context (`~/.loop-pi-<context>`), plus a fast variant that sends every request at priority tier. Why is in [why I don't use Codex for long-running agent loops](/articles/why-i-dont-use-codex-for-long-running-agent-loops/); this is the machinery. The source is public at [rknightion/loop-pi](https://github.com/rknightion/loop-pi).

pi has no native subagents or approval prompts. Lanes come from a pinned `pi-subagents`, and the loop pieces are extensions of my own:

- `loop-guard` is an honest-mistake fence parsed from the command text, not a security boundary. It refuses force and delete pushes, any push from a lane, background launches, a spawn that overrides the pinned model or run deadline and any model outside the configured family. It runs my staging and Backlog guard scripts before shell and edit calls, keeps the root out of the charter, the launch file and the registry, and keeps lanes out of the run's state, charter, launch and closeout files. After every root push it writes a push log entry with the ref's old and new SHA.
- `loop-wait` gives waits that cost no model turns. A lane blocks on its own process with `watch_process`, and the root has `watch_start` for a process the extension owns and `wake_at` for timers. A fired timer, a finished watcher or a returning lane starts the next root turn.
- `loop-continuation` is pi's Stop hook. It arms on the same launch forms, refuses with an incident when the host or repository is wrong, the launch's `Standing:` line differs from the charter's or the run already opened under a different charter hash, and re-prompts a root that stops before the closeout. It also recovers a stopped root automatically: after a provider error, a stalled request or a request-ceiling abort it resumes the root with a backoff that starts at a minute and doubles to 15, with no attempt limit, and records each resume. Authentication, quota, policy and context-length failures stop the run with an incident instead, and an abort of mine is never resumed. Lanes aren't resumed automatically.
- `request-ceiling` bounds each model request, root and lanes alike, at 15 minutes without model progress and 75 minutes in all. [Bounding a model request on pi](/articles/bounding-a-model-request-on-pi/) covers it.
- `loop-state` writes the event log above and caps a lane return over 16 KB at its head, its tail and its `lane-return` block, keeping the full copy in loop-pi's own run directory.
- `lane-worktrees` runs each lane in its own worktree, copies every lane's exact candidate patch into loop-pi's run directory and lets the closeout remove only verified clean landed trees.
- `call-timing` records first-token timing and a request checkpoint for each model call, and `loop-status` is the root's status line: lanes against the slot cap, armed waits, the next deadline and the heartbeat age.

Lanes are the same five roles as on Claude Code, from pinned agent files that fix each one's model, thinking level, tools and a hard run deadline (four hours for a worker). Only `senior` may spawn children, at most two levels deep and only when its brief grants a count. Every agent also runs behind a launcher that refuses pushes, as a second fence. A missing agent is a harness fault, never a reason to substitute another. The [pi harness](/reference/loop-harness-pi/#routes) has the current routes.

## Preflight, audit and upgrades on pi

Each `loop-pi` start inside a git repository does two things before the model sees anything. It preflights the repository, because pi children load a target repository's `.pi/` settings, system prompt and extensions whatever its trust: it refuses a repository with any of those or with project agent files, one whose `.pi/` isn't gitignored, and a dirty checkout. Then it snapshots every remote ref, tag and release into loop-pi's run directory. A charter that reaches another repository runs `loop-pi-preflight` and `loop-pi-audit add` on it first.

At closeout `loop-pi-audit closeout` snapshots again and compares, against the charter frozen at arm. The default branch is judged commit by commit: each new commit has to lie in a range the push log recorded or touch only `backlog/`. A commit another Claude Code session made, which carries its session trailer and which the root then pushed on top of, is reported as foreign rather than failing the audit. A release-please release is granted only when the charter's `authority:` key grants it for that repository. Anything else, including a push the log never saw, fails the audit with a non-zero exit, and the root reports each one.

Each install builds into its own versioned directory and keeps the older ones, so a running root keeps the code it started on. pi transcripts sync to my home server like the other two harnesses' and load into the same catalogue with each child linked to its root.

## How the contract reaches repositories

The contract, the three harness files and the charter and lane templates live in a private repository and are never copied. A running root reads them in place from the local checkout, at the absolute paths its launch names. A repository's Backlog board carries a short pointer instead, saying where the files are, who reads what and that repository facts are in `LOOP.md`.

The pointer is build output:

- `registry.tsv` maps each consuming repository to its Backlog doc id. The ids can't be normalised, so nothing sweeps by filename.
- `bin/render` prints the expected content from committed blobs only, under a header naming the source commit.
- `just publish` runs by hand, in fresh clones under a per-host lock. It updates each document through the Backlog CLI, commits only the registered paths and pushes without force, so two Macs publishing at once can't overwrite each other.
- Before any clone, a guard refuses to publish a leak pattern, or a reference to a repository that isn't public, into a public repository.
- `bin/doctor` classifies every copy, and a scheduled job runs the same check against GitHub daily with read-only tokens. Both report drift and repair nothing; correction is manual on purpose.

The public copy is a separate export. `bin/export-public.py` builds a tree from an allowlist and rewrites installation-specific names such as private repositories, hosts and credential paths into neutral examples. This site's [reference pages](/reference/loop-contract/) publish that export, and they're the only public copy. The preparation steps and the registry are never published.

<figure class="diagram figure-breakout">
  <img class="on-light" src="/diagrams/27-how-the-contract-reaches-repositories-v3-light.svg" alt="A source column of seven protocol files: contract.md, harness-pi.md, harness-claude.md, harness-claude-openai.md, harness-codex.md, charter.md and lane.md. A bracket groups all seven and feeds three paths. Path A, read in place: the launch names absolute local paths and the root reads the files in place. Path B, repository pointer: a committed renderer writes a Backlog pointer in each repository. Path C, highlighted, public export: the export replaces private names, and this site's /reference/ pages publish it, the only public copy. Below the source column, under a rule labelled private only, the preparation steps and the standing-authority registry have no path out and are never published." width="1440" height="780" loading="lazy" decoding="async">
  <img class="on-dark" src="/diagrams/27-how-the-contract-reaches-repositories-v3-dark.svg" alt="" aria-hidden="true" width="1440" height="780" loading="lazy" decoding="async">
  <figcaption>Paths A and B point at the private files rather than copying them, so the export in path C is the only copy anyone else can read.</figcaption>
</figure>

## What is still unproven

The guard parses command text, so a push made from inside a script file or a task runner isn't logged and shows up only as ungranted at closeout. And the watchdog turns a silent loss into an alert. It doesn't get the lost hours back, and a dead loop only reaches Grafana, not my phone.
