Why I don't use Codex for long-running agent loops

A long-running loop needs its root to do nothing while lanes and CI run, and to wake on the event that matters. I can’t get that reliably from the Codex CLI, so pi is my default for OpenAI-model loops, with my own extensions published as rknightion/loop-pi. Codex runs one only if I name it, as I did for two loops in a private repository on 13 October. Claude Code has its own harness document for Claude models. The measurements below compare Codex and pi runs from late September and early October.

In plain terms: every time an agent thinks, the harness sends its whole conversation so far back to the model. The provider caches most of it, so each resend is cheaper and faster, but it is still a resend of the whole thing. An agent that checks on its helpers every few seconds pays that resend every few seconds. On one of my public repositories a Codex root did that for 18 hours and re-read 323 million cached tokens. Checking once every 19 minutes, which is what the loop asks of it, would have come to about 9 million.

Scope: the root of a multi-hour loop under the loop contract, on Codex CLI 0.156 to 0.159 with the GPT-6 models. I haven’t measured interactive Codex sessions and I’m not making any claim about them.

What a loop root needs

A few terms first, because the rest leans on them.

• The root is the agent running the loop. It reads the plan, hands work out and decides what happens next.
• A lane is a helper agent the root starts for one piece of work: implement this, review that, watch that CI run.
• A model call is one turn of the model. The harness sends the whole conversation so far and the model answers.
• The context is that conversation. A root a few hours in carries around 150,000 tokens on Codex in my loops, and around 230,000 on pi.
• The cache is the provider remembering a conversation it has already seen, so a resend costs less. Less, not nothing: every call still reads the whole context back out of the cache.

Most of the time a root has nothing to do while lanes implement, gates run and CI finishes, and in that time it should:

• sit idle without spending model turns;
• wake on the event that matters, a lane returning, a process exiting or a deadline;
• never generate inference just to stay alive.

Waiting costs by the wake. A Codex root blocked in a wait makes no model call, but each return is one call over its whole context, about 150,000 tokens, nearly all of them cached. A root that wakes every 30 seconds re-reads its whole context 120 times an hour; one that wakes every 19 minutes does it about three times.

What goes wrong on Codex

Five separate problems. The Codex harness works around each one; none of the workarounds removes the cause.

The wrong wait tool

Codex has two tools that wait. collaboration.wait_agent waits for lanes. Code mode’s functions.wait collects an exec cell the root started this turn. Roots call the second while lanes are in flight, with cell ids that don’t exist. The call fails at once and costs a full model turn. In one overnight loop in a private repository, loop 149, the root made 5,070 failed wait calls in 16 hours, against 37 wait_agent calls, out of 7,101 root model calls in all. Put simply, the root kept pressing a button that did nothing, and every press cost a full re-read of its context. The roots described this as the harness misrouting their call; a synthkit root paused with “collaboration calls misrouted repeatedly” while both lanes were still running.

No hook can see it

The Codex 0.157.1 wait handler returns no PreToolUse or PostToolUse hook payload for code-mode wait. Its comment calls this “runtime control for an existing code cell, not a standalone user action”. That is the release responsible for most invalid waits in the sample below. The Stop hook I use elsewhere doesn’t help either, because a root spinning on failed waits never ends its turn. Neither of those hook mechanisms can interrupt that wait loop.

A prose rule doesn’t either. A rule naming the bad ids, and then a rewrite naming the moments a root slips and the recovery, still left synthkit loop 44 making 1,391 invalid waits in 2,049 root calls.

The 19-minute floor

Roots pass short timeouts with deadlines hours away, so I set the minimum and default wait to 1,140,000 ms. That works, at a price: a lane’s deadline is noticed up to 19 minutes late, and a root that calls wait_agent after a child’s return has already landed in its context sits out the whole 19 minutes reporting nothing.

Completion doesn’t wake an idle parent

A Codex root has to hold its turn open inside a wait to hear from its lanes at all. A step gated on a clock time is held the same way, by calling wait_agent until the gate passes, because ending the turn wakes nothing.

Collaboration tools don’t exist inside a cell

With the default non_code_mode_only, spawn and wait are direct calls only, and a cell’s tool listing doesn’t show them. So a root with lanes in flight can’t wait on a process inside a cell without going deaf to its lanes, and each process wait, main’s CI or a release, goes to a scout, an LLM lane that watches one process and returns the raw readback. One root looked for the tools inside a cell, found none and drained with three ready lanes and about 11 hours of its run left.

The Codex settings

The Codex harness runs the root on gpt-6.1-sol at high, with these settings in the loop home:

• [features.multi_agent_v2] sets min_wait_timeout_ms and default_wait_timeout_ms both to 1140000, so every root wait_agent lasts 19 minutes unless a child messages or finishes.
• non_code_mode_only stays at its default, true. The model catalogue decides whether code mode is on, so the harness answers it with a direct-dispatch rule.
• [agents] max_concurrent_threads_per_session = 19 allows nineteen child threads; a spawn refused at the limit waits for a return and retries.
• model_context_window is 872000 and model_auto_compact_token_limit is 741200.
• Pinned custom agents cover the worker, senior, senior-escalation, reviewer and scout roles. The root spawns by agent_type and passes no model or effort, and a spawn with no matching installed agent is a harness fault.
• There’s deliberately no tool hook on sleeping or the cell wait. Codex exempts the cell wait from tool hooks, so a hook could only police half the rule.

The roots measured below ran Sol at medium with a 700,000-token context window.

How I found it

None of this shows from inside the loop. A root doesn’t count its own wakeups, its closeout doesn’t either, and here the root’s own explanation was that the harness had misrouted the call. The catalogue is what showed it. The hot and cold conversation archive covers how every transcript lands on my homelab server, and building an agent-transcript indexer covers the catalogue, which classifies each root model call by what it reacted to. A view in it counts the fake-cell waits per root, and counting them over transcripts is what turned “the root seems stuck” into 5,070 failed waits in one night.

Why pi

pi is earendil-works/pi, MIT. It has no native subagents and no approval prompts. It does ship MCP, codemode and tool search as built-in extensions, and the loop home switches all of them off; loop work is shell, git, Backlog and CI, and doing without MCP costs it little. The three properties that mattered came from reading its source:

• a custom tool may block indefinitely;
• its tool_call event covers every tool, with no exempt one;
• agent_before_settle lets an extension intercept a turn that is about to stop.

The pi v1.0.4 extension reference documents tool_call blocking and agent_before_settle continuation. It is a later public description of those mechanisms, not proof of the earlier evaluation’s installed version or of every internal tool path.

Spawning comes from pi-subagents: per-agent model and thinking level, fresh context, status, steer, stop, worktree isolation, concurrency and depth caps, and an async completion that starts a new root turn on its own. Its blocking bg_wait tool is switched off, so the root has no wait tool to misuse.

I rejected two alternatives.

• oh-my-pi: a separate Bun and Rust fork of pi, not a set of extensions, and in the version I read some of its tool paths emitted no tool_call at all, so a guard couldn’t see them. Its Rust core can’t practically be extracted from.
• A from-scratch orchestrator: pi-subagents already covers spawn, wake on completion, status, steer, stop, worktree isolation and the caps, so writing my own would rebuild what I can pin.

pi is not stable ground. It releases often, and its extension API still changes between minor releases. So loop-pi pins pi and pi-subagents exactly; a bump is its own change, with the full test run, a real loop and a note of what the extension API changed. The installer smoke-loads the root and lane entries before it swaps a build in and keeps older builds for rollback, and CI checks the npm provenance of both pinned packages.

What loop-pi adds

Eight extensions, pinned agent files and an installer that writes a launcher per pi home, ~/.loop-pi-<context>, built for loops. The machinery around the loop covers the internals; the pi harness page has the current routes a root reads. Three of the extensions are why the waiting problem goes away.

loop-wait gives the root watch_start, watch_stop, wake_at and wake_cancel. watch_start runs a quiet command from the extension and, when it exits or hits its deadline, pushes a message that starts a root turn. wake_at does the same for a timer. A lane launched with a deadline gets a timer for it, cancelled when the lane returns, and timers and watchers survive a restart. A scripted-provider test shows an idle root woken with zero assistant messages between arming and the wake. Lanes use watch_process, which blocks inside the tool, to wait on their own gates and processes; the guard stops the root using it while any lane is in flight, because a blocking call would delay pushed completions.

loop-continuation is the Stop-hook contract from the Claude plugin, ported. A root that ends a turn with no tool call is re-prompted, at most three times per chain, unless a counting closeout exists, its last line is PAUSED:, or its last line is a WAITING: backed by a timer. If no timer is armed at the stated deadline, the extension arms one, so the root wakes by then even if a child hangs silently. The fourth stop goes through and writes an incident file. It also checks the launch before it arms, and refuses a wrong host or repository, or a changed charter or Standing: hash.

loop-guard runs on every tool call, with no exempt tool. It blocks background launches, force and destructive pushes, git add -A and git commit -a, and runs the same Backlog and staging guard scripts Codex and Claude Code use. Lanes never push: the guard refuses every lane push, and only the root’s own shell runs git push. Every push the guard sees lands in a push log that the closeout audit checks commit by commit. Lanes are also blocked from secret-store writes, releases, merges, workflow dispatch and mutating gh api. The agent files name two models, gpt-6.1-sol for every role but escalation, which runs gpt-6-astra, and the guard blocks any model outside the configured family and any model, thinking or deadline override on a spawn. It registers its own lane entry and loop-wait’s as required child extensions, so no lane can opt out, and if that registration fails the root’s subagent calls are blocked.

The other five don’t touch the waiting. loop-state writes the run’s event log, lane-worktrees gives each lane its own worktree, and request-ceiling ends a model request after 15 minutes with no model progress or 75 minutes in all, then re-prompts the session. call-timing records first-token timing and response headers, and loop-status puts the root’s status line on screen.

Started inside a repository, the launcher refuses to run if the repository carries its own pi settings, extensions or agent files, isn’t clean, or doesn’t gitignore .pi/, because a repository’s agent files would outrank the home’s and pi-subagents writes into .pi/ in the checkout. It snapshots remotes, tags and releases before the run, and at closeout it reports any change that no charter grant and no logged push covers.

The launcher also has a fast variant, loop-pi-fast, which sends every request in the run at OpenAI’s priority tier. I choose it at launch; it changes speed, not authority.

Results

The figures come from the agent-history catalogue, not from loop reports, queried on the evening of 5 October. The comparison is every Codex loop from 25 September onward against every pi loop launched from 28 September to midnight on 4 October, London time, that has a completion receipt: 83 of the 89 launched in that window. Both groups count only loops that ran at least an hour and started at least one lane, and every root ran Sol at medium.

• Codex: 20 loops, 184 hours and 368 lanes, across cf2otel, synthkit and four private repositories.
• pi: 83 loops, 486 hours and 2,028 lanes, across twelve repositories.

A lane is any helper the root started, pollers, reviewers and mappers included. “While lanes run” means at least one helper was between its first and last event. Input tokens are what the harnesses report, cached plus uncached. Priced cost is the catalogue’s estimate at list price; the loops run on my ChatGPT subscription, so it’s a measure of size, not money spent.

Measure	Codex	pi
Lanes started per hour	2.0	4.2
Root model calls per lane	88.2	17.7
Root model calls per hour while lanes run	310	83
Minutes with no root call while lanes run	16%	50%
Invalid waits (share of root calls)	7,359 (22.7%)	none to make
Root context per call (median)	150k	233k
Cached tokens the root re-read per hour, lanes running	42.9M	20.8M
Cached tokens the root re-read per lane	12.8M	4.6M
Total input tokens per lane	19.2M	7.2M
Total input tokens per hour	38.4M	30.1M
Cached share of root input	98.9%	98.7%
Priced cost per lane	$4.60	$1.42
Priced cost per hour	$9.20	$5.92

Per unit of work pi is much cheaper: its root makes a fifth of the calls per lane, re-reads about a third of the cached context per lane, and total input per lane is 62% lower. It’s cheaper per hour too, even though it starts twice as many lanes an hour: total input per hour is 22% lower and priced cost per hour 36% lower.

The cache line is the one I care about. While lanes were running, a Codex root made a call about every 12 seconds and re-read 43 million cached tokens an hour. One wake every 19 minutes works out at about half a million. pi is nowhere near that floor either: its root calls about every 43 seconds while lanes run, reading returns, checking git and Backlog and dispatching the next lane, and re-reads 21 million an hour, half what Codex does. The difference is that a pi root goes quiet between bursts, with no call in half the lane-running minutes against one in six, and it never spins on a wait.

The spinning is the worst case, and synthkit loop 44 shows it. It ran for 18 hours with seven lanes. The root made 2,049 calls and re-read 323 million cached tokens, and the catalogue classes 1,614 of those calls as reactions to a timed-out or failed wait. At one wake every 19 minutes the whole run would have re-read about 9 million.

The catalogue classifies each root call by what it reacted to. The narrow idle share is the calls that followed a timed-out or failed wait or a no-op. On Codex it was 73.9% for loop 149, 82.4% for synthkit loop 44 and 72.0% for cf2otel loop 11. Across the twelve longest pi roots, 12 to 23 hours each, one call in 12,069 followed a wait and none was a no-op; status checks, which include ordinary git, Backlog and lane-state reads, were 9% to 26%. pi roots make no wait calls at all; across the 83 pi roots they armed 3,772 wake_at timers and 1,685 watch_start watchers.

The cleanest pair is one private repository that ran both: loop 149 on Codex overnight, and loop 152 on pi two days later. Over the first 831 minutes of each, which is all of loop 152, loop 149 used 1,366M input tokens and loop 152 used 810M, 41% less, while producing half as much output again, 1.77M tokens against 1.18M. Root calls were 6,185 against 1,444. By their reports loop 152 landed ten items with green CI on the exact head, and loop 149 landed one whose push CI failed. The scopes differed, so treat that as colour.

What cuts the other way:

• The spin is mostly one release: roots on Codex CLI 0.157.1 made 7,127 of the 7,359 invalid waits. Earlier releases made 218 in 12,883 root calls, under 2%, though the one loop on 0.159.2 still made 14 in 75. A Codex root that doesn’t spin is a lot cheaper. When one does, nothing in the harness can stop it. Later releases spin far less, though not never. Queried on 19 October, the 19 Codex root sessions on 0.162.1 that started at least one sub-agent, between 13 and 16 October, made 102 invalid waits in 13,673 root calls, 7.5 per 1,000, and 96 of those came from one root. Counted the same way, roots on 0.157.1 made 356 per 1,000.
• Two loops hold most of it: loop 149 and synthkit loop 44 hold 6,461 of the 7,359 invalid waits, 88%. Without them the Codex roots still made 75 calls per lane, against 18 on pi.
• Bigger pi contexts: a pi root carries about 1.6 times the context of a Codex root, so a cut to about a quarter of the calls per hour is only a halving of cached re-reads.
• pi has its own failure: on one afternoon some pi root requests on Sol at medium reasoned for 20 to 72 minutes after a quick tool result. The three longest ran out their output budget after more than an hour with nothing to show for it; the rest ended in errors or finished late. request-ceiling ends a request after 15 minutes without model progress and re-prompts the root.
• Quiet is not productive: Codex lanes made 0.80 commits each and pi lanes 0.71. Fewer root calls say nothing about how much work got done.

How to get it

The extensions are at rknightion/loop-pi, with the pinned versions, the agent files and the tests. Read these before you use them.

• Versions: pi and pi-subagents are pinned exactly in package.json. pi’s extension API still changes between minor releases, so expect to re-test on every bump.
• The guard is not a sandbox: it stops plainly typed mistakes. A command assembled at runtime inside interpreter code isn’t caught. Lanes can’t push at all, since the guard refuses it and each one runs behind a no-push launcher, and the push log records the root’s pushes. The closeout audit of remote refs, tags and releases is still the actual check.
• It is built for my protocol: the continuation extension arms on the launch message the pi harness describes, and the agent set and turn-endings text are the ones it names. Without that shape of launch, expect to change code.
• Models: my loops reach the GPT-6 models through codex-lb, an OpenAI-compatible proxy that I run on my ChatGPT subscription, using pi’s openai provider and the proxy’s own API key. pi’s openai-codex provider needs a ChatGPT token, which codex-lb doesn’t accept, so I don’t use it. The agent files name two models and the guard allows only the configured family, so a different provider or model set means editing the home settings and the agent files, and the installer refuses a build where they disagree.

Limits

This compares groups of loops on different repositories and different work, and is nowhere near a controlled test. Both groups include overnight time: 35% of the Codex wall time and 37% of pi’s fell between 22:00 and 07:00 London time. The numbers describe where calls and tokens went and say nothing about whether the work was better. “While lanes run” counts a lane from its first event to its last, so a blocked lane still counts, and for Codex only about a third of loop minutes can be tied to a running lane, so its lane-running rows rest on 64 hours. Shell-based polling isn’t classed as a wait, so the invalid-wait counts are a floor. The comparison stops in early October and covers neither side’s later changes. The October Codex runs appear only in the invalid-wait count above.