I retired this setup on 21 October 2026, and its loop harness with it. For OpenAI models in a loop I use pi with loop-pi; the gateway traps below still apply to anyone running Claude Code against LiteLLM.
Claude Code can run with OpenAI’s GPT models behind it. Getting a tool call back was the easy part. What took work was settings silently ignored, conversation history missing the cache and parallel agents queueing behind a shared upstream bridge.
I use a separate claude-openai home, a self-hosted LiteLLM proxy and codex-lb. Claude Code supplies the terminal harness, tools, hooks and subagent interface, and the backend runs OpenAI models. It is a single-user setup backed by ChatGPT subscriptions, unsupported by Anthropic for non-Claude models, and not a way around a provider’s terms.
The experiments below were recorded on 10 October. I checked the configuration on 11 October and the live service, its request logs and Grafana Cloud on 12 October; where a number comes from production logs rather than an experiment, I say so. A source file shows the intended setting, not that the running service has it or that every request obeys it. The machinery article covers the shared tooling.
The request path
The generated home sets CLAUDE_CONFIG_DIR through its own launcher. It clears inherited provider authentication and otel environment variables, refuses Chrome integration and authenticates through an apiKeyHelper that reads a LiteLLM virtual key from the credentials tree. It never copies native OAuth credentials into the OpenAI-model home.
A mod called gpt-lane-router chooses model and effort before Claude Code sends an Anthropic-format request to LiteLLM’s /v1/messages. LiteLLM translates that request into OpenAI Responses, then sends it to codex-lb’s /v1/responses. codex-lb handles account affinity and HTTP bridges over subscription-backed Codex access.
gpt-lane-router runs inside the Claude Code client, so a lane is resolved to a model and effort before LiteLLM sees the request.The gateway runs LiteLLM v1.105.0, pinned by image digest, with Postgres, one worker and no Redis. They keep deployments as openai/<model> on the Responses path, because the chat-completions alternative had dropped reasoning effort when tools were present. The LiteLLM configuration reference also describes a switch that routes Messages through chat completions instead of Responses. It changes the translation path, so it is not interchangeable with the Responses setup.
I evaluated Claude Code Router first. The recorded reasons for choosing LiteLLM were a shared gateway, per-client virtual keys, observability and an admin UI. It adds a service to operate and does not make the adapter transparent.
A model tier also needs an effort
Claude Code names tiers such as opus, sonnet, haiku and fable, but these OpenAI routes also need a reasoning effort. Mapping a tier to a model in LiteLLM alone does not give each agent role its intended effort.
The configured lane scheme uses pseudo-model IDs, such as lane-main or lane-scout. The mod’s agent.spawn hook assigns the role’s lane to supported built-in and agent-workflows agents. Explicit models, forks, workflow agents, other plugins and missing lane rows can bypass that assignment. Its turn.step hook then replaces the lane with an OpenAI model and effort on each request. One JSON table owns the mapping.
On 19 October the table gives lane-main, lane-opus, lane-senior and lane-reviewer Sol/high; lane-sonnet and lane-worker Sol/medium; lane-scout Sol/low; Haiku and Explore Luna/high; and fable and lane-senior-escalation Astra/high. Sol, Luna and Astra mean gpt-6.1-sol, gpt-6-luna and gpt-6-astra. That was the last table before the setup was retired.
Main is fixed high. The hook can preserve session effort for a lane with no fixed effort, but no row in the table does that. Changing /effort does not change these routes.
An explicit Agent(model: ...) wins over the role lane. An Explore dispatch with model: sonnet therefore uses the sonnet tier, not Explore’s Luna route. Generic spawns name a tier deliberately; pinned agent calls omit the override. Classifier calls and WebFetch summaries can bypass the mod, so LiteLLM also needs aliases for the lane IDs. Those aliases choose a model, but the mod does not enforce their effort.
The published harness and the generated lane table have to agree. When I wrote the first version of this, they did not: the harness named one effort for a lane while the generated table still said another. Regenerating the table fixed it, and nothing checks the two against each other except a person reading both.
Settings that failed before the first useful request
1. A fallback string invalidated the settings
The recorded setup incident used a string for fallbackModel. Claude Code expected a list, and the settings file stopped applying without the expected error. The installer now emits "fallbackModel": ["lane-opus"], and its regression test checks that shape. That was one observed invalid value, so it says nothing about how other settings errors fail.
2. Plugin installs rewrote the JSON
Claude Code reformatted settings.json after plugin installation. Comparing bytes then reported drift even when the settings meant the same thing. The installer compares parsed JSON for that file. Its test reformats an unchanged file, then changes a model value to tell formatting from semantic drift.
3. The gateway does not supply Anthropic’s search service
The home denies WebSearch and uses the Firecrawl MCP instead. WebFetch remains available. ENABLE_TOOL_SEARCH=false avoids the recorded adapter problem with tool-reference blocks. These are limits of this path, not of every gateway.
4. Context needs an explicit client policy
The generated settings set a maximum context and a compaction point. The listing below is a selection and contains no credential values. The helper path is an example absolute path to replace with the actual home path. I am not claiming Claude Code expands a tilde.
{
"model": "lane-main",
"fallbackModel": ["lane-opus"],
"effortLevel": "high",
"autoCompactWindow": 774000,
"apiKeyHelper": "/home/example/.claude-openai/api-key-helper.sh",
"permissions": { "deny": ["WebSearch", "EnterWorktree"] },
"env": {
"ANTHROPIC_BASE_URL": "https://litellm.example",
"ANTHROPIC_DEFAULT_OPUS_MODEL": "lane-opus",
"ANTHROPIC_DEFAULT_SONNET_MODEL": "lane-sonnet",
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "lane-haiku",
"ANTHROPIC_DEFAULT_FABLE_MODEL": "lane-fable",
"CLAUDE_CODE_MAX_CONTEXT_TOKENS": "872000",
"CLAUDE_CODE_GATEWAY_HINT_HEADERS": "1",
"CLAUDE_CODE_AUTO_MODE_SERVER": "0",
"CLAUDE_CODE_MODEL_CAPABILITIES": "gpt-*=-mid_conv_system;lane-*=-mid_conv_system",
"CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY": "1",
"CLAUDE_CODE_RETRY_WATCHDOG": "1",
"CLAUDE_CODE_RETRY_WATCHDOG_MAX_WAIT_MS": "1800000",
"ENABLE_TOOL_SEARCH": "false"
}
}Several of these switches are undocumented or temporary, so this is a version-specific account of the generated home, not a supported installation recipe.
5. A print-mode flag consumed the prompt
The setup notes record a print-mode invocation in which --allowedTools consumed the trailing prompt as another tool value. Sending the prompt on stdin avoided it. It still happens on 2.1.296: claude -p --allowedTools Read "reply ok" fails with “Input must be provided either through stdin or as a prompt argument”.
6. An unknown-model warning did not stop the run
The recorded acceptance run emitted [claude-code:unrecognized_model] for gpt-6.1-sol and still completed tool round trips, so the warning alone did not mean a failed request. That is historical evidence. It does not show that every current release prints the warning or that every unknown model is compatible.
Exporters can fail without stopping inference
7. The otel auth header needed a literal space
The LiteLLM operating notes record an exporter 401 with Authorization=Basic%20<token>. LiteLLM did not URL-decode the value. Authorization=Basic <token>, with a literal space, fixed that incident. The token belongs in the secret store, never in an article or a copied example.
8. One metric attribute disabled V2 metrics
LiteLLM’s V2 documentation says the filter resolves “lazily on the first metric a request records rather than at boot”. An invalid entry, gen_ai.response.model, in callback_settings.otel.attributes.include_list disabled metrics in the recorded setup, with no error at startup. The gateway now runs with no include_list at all and relies on LiteLLM’s own attribute ceiling to bound metric labels.
The configuration also uses a dedicated Prometheus listener and Alloy scraping /metrics/; the slashless path redirects. Traces, metrics and Pyroscope profiles go to Grafana Cloud. A successful model response says nothing about whether any of those exports works, so I checked each signal separately on 12 October. At 13:00 on 11 October the gateway had about 2,500 active Prometheus series under job="litellm", 360 of them OpenTelemetry gen_ai metrics. Tempo held LiteLLM’s server and database spans for the same window, and Pyroscope had profiles for the service.
The advertised window can differ from the effective one
9. Catalogue, upstream maximum and compaction are different values
The retained backend notes distinguish a 272,000-token catalogue window from an 872,000-token upstream maximum for the GPT-6 family. codex-lb can override its advertised window, but advertising more than upstream accepts still produces rejected requests.
The inspected client policy uses 872,000 with compaction at 85%, or 741,200. Claude Code compacts at autoCompactWindow minus a buffer of about 33,000, so the setting is 774,000 to land near that figure. pi expresses the same arithmetic as a reserve of 130,800 tokens. Those are configured thresholds, not measured usable context.
Claude Code’s own messages are not a reliable guide to the effective window. On 11 October a local /autocompact 800000 command reported that it capped the window to a 200,000-token model limit. A likely explanation, not a proven one, is that Claude Code treats any claude-* model id as first-party. A /model pick is saved as a claude-router-<hex> id, and for such an id Claude Code ignores CLAUDE_CODE_MAX_CONTEXT_TOKENS and clamps the window to 200,000. The installer now flags that id in the settings and says to pick lane-main instead, but I have not confirmed it was what that session was running. The spend logs disagree with the reported limit: 2,216 requests through the gateway carried more than 200,000 prompt tokens, and the largest carried 568,495. The configured maximum, the harness’s reported limit and the upstream boundary are three different numbers, and only the request logs show which one applied.
10. Auto mode uses the client classifier
The installer sets CLAUDE_CODE_AUTO_MODE_SERVER=0 because this OpenAI-model translation path does not supply Anthropic’s server checks. Anthropic documents the setting: “Classifier requests are then always Claude Code’s own, billed the same way, and the notice doesn’t appear.” The recorded acceptance run used that client classifier and cleared a Bash action. That covers the one recorded action, not equivalent safeguards across providers.
LiteLLM’s server-side auto-mode support forwards safeguards to Anthropic provider routes. It does not establish server review for these OpenAI deployments. That post attributes a 23 October transition date to Anthropic, while the Anthropic reference calls the client setting temporary without stating that deadline. The date is a secondary-source claim, and I am not promising continued auto-mode support.
Two cache failures with different causes
11. Moving reminders to the front destroyed the stable prefix
OpenAI’s prompt-caching guide states: “Cache reuse requires the entire rendered prefix to match.” The local codex-lb Responses normaliser hoists ordinary system and developer messages into top-level instructions. The setup investigation found Claude Code sending per-turn reminders as mid-conversation system messages. Through that translation path, a reminder added late in the history changed text near the front of the next prompt.
The tools could still cache while the conversation items missed, so an aggregate cache ratio could not diagnose it. The 10 October investigation used per-item usage.attribution retained in LiteLLM’s spend logs. It recorded 210,765 conversation-item tokens with zero cached. A classifier control without those reminders cached its items normally.
The capability override in the settings listing makes Claude Code put reminders inline instead. Production logs from 11 October, after both cache fixes, bear that out. Across 2,385 warm main-thread turns over 20,000 prompt tokens, the median request had 99.5% of its prompt cached and the aggregate was 92.3%. The gap between median and aggregate is a minority of turns that miss badly. Worker lanes cache worse: lane-worker had a 95.2% median and a 73.0% aggregate. Those numbers come from one day of my own traffic and are not a benchmark.
The normaliser has exceptions for Responses Lite tool bundles and preserved non-text content. The merged codex-lb repair says it will “only hoist system/developer items whose type is omitted or "message"” and preserve other typed items. That public source supports the distinction. It does not verify the private cache experiment or identify the deployed build.
12. Shared affinity made parallel agents wait on one bridge
LiteLLM’s adapter derives a prompt-cache key from the session ID inside Claude Code’s JSON metadata.user_id. The recorded problem was that the main thread, subagents and side calls all shared it. codex-lb then put parallel traffic through one HTTP bridge’s creation gate. Some requests waited, timed out and rerouted to accounts with cold caches.
The callback below splits that affinity by agent ID and, for selected side calls, request class. It uses LiteLLM’s async_pre_call_hook contract at v1.105.0, which allows returning a modified request dictionary. Enable gateway hint headers so those fields reach the proxy. The main request keeps its bare session ID when it has no splitting hint, and compaction keeps the same affinity only under that same condition.
import hashlib
import json
from litellm.integrations.custom_logger import CustomLogger
SPLIT_CLASSES = {"auxiliary", "side_reply"}
class ClaudeCodeCacheKey(CustomLogger):
async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
if call_type not in ("anthropic_messages", "aanthropic_messages"):
return data
metadata = data.get("metadata")
if not isinstance(metadata, dict) or not isinstance(metadata.get("user_id"), str):
return data
try:
user_id = json.loads(metadata["user_id"])
except ValueError:
return data
if not isinstance(user_id, dict) or not isinstance(user_id.get("session_id"), str):
return data
headers = (data.get("proxy_server_request") or {}).get("headers") or {}
headers = {str(k).lower(): v for k, v in headers.items()}
parts = []
if headers.get("x-claude-code-agent-id"):
parts.append(str(headers["x-claude-code-agent-id"]))
if headers.get("x-claude-code-request-class") in SPLIT_CLASSES:
parts.append(headers["x-claude-code-request-class"])
if parts:
digest = hashlib.sha256("|".join(parts).encode()).hexdigest()[:12]
user_id["session_id"] = f"{user_id['session_id'][:36]}:{digest}"
metadata["user_id"] = json.dumps(user_id, separators=(",", ":"))
return data
proxy_handler_instance = ClaudeCodeCacheKey()Save the module as claude_cache_key.py in the gateway’s module path and register claude_cache_key.proxy_handler_instance as a LiteLLM callback. It changes cache affinity. It provides neither access control nor anonymisation, and the short digest can theoretically collide. Requests without recognised JSON metadata or splitting hints pass through.
The hook limits the session portion to 36 characters and adds a colon plus 12 digest characters, because the setup notes record adapter truncation at 64. In the controlled test, three parallel subagents produced five separate cache lanes after the split, and the worst bridge wait fell from 9.5 seconds to 0.66 seconds. A cache lane can represent side traffic as well as an agent, so five lanes does not mean five agents.
Production did not reproduce the 0.66 seconds. codex-lb’s request logs for 11 October show 108 of 4,583 gateway requests waiting more than a second at the bridge creation gate, 28 waiting more than five and a worst case of 9.9 seconds. Every one of those slow requests was partway through an existing cache key, not the first request on it, so they are not the shared-key queueing the split was written to fix. I have not found what causes them yet. Both cache fixes need checking after every LiteLLM upgrade.
The UI and the home still need operating
13. Native Microsoft SSO removed the second login
Traefik forward-auth stays in front of the admin UI. LiteLLM’s own Microsoft SSO replaces the second username/password page but does not remove the perimeter layer. The configuration enables automatic redirection and assigns an Entra app role valued proxy_admin, with delegated User.Read for the Graph lookup.
The LiteLLM SSO documentation states: “If you need to access the UI via username/password when SSO is on navigate to /fallback/login.” Keep that recovery route protected. API paths need bearer-key authentication without an interactive SSO redirect. A provisioning script with an assignment or consent step does not prove the live assignment exists. Check licensing and role handling against the release you deploy.
14. A changelog option can exist and still do nothing
The changelog review left idleCompaction unset. The recorded finding was that it was default-on but server-controlled, with cache requirements the gateway key did not satisfy. A schema entry alone does not prove a feature ran.
The generated home enables the retry watchdog with a 30-minute maximum wait. Safety guards for Backlog, staging and paths use onFailure: "block" with a timeout; the style validator is not one of those guards. Local transcript garbage collection was deliberately left out because the archive is append-only. Installed hook registrations and compaction behaviour still need runtime checks.
15. An extra proxy hop needs a reason
I decided to keep native Claude subscription traffic out of this gateway and to leave Codex and pi talking to codex-lb directly. The decisions avoid another failure point and an unverified change to thread affinity. They do not show that those clients cannot use LiteLLM. Version-specific OAuth support and provider terms need their own check before any such migration.
LiteLLM’s Skills Gateway catalogue did not replace the existing git-owned skill sources. Its MCP gateway remains at most a bounded pilot for central credential handling. Full-response caching was not selected for these growing conversations: reusing a prompt prefix and replaying a complete response are different operations. Repeated requests could benefit from response caching, and this workload’s assessment does not establish that agent requests never repeat.
One worker and no Redis remain the chosen deployment. Add workers only when measurements put the delay inside LiteLLM rather than in the upstream bridge or model; otherwise they add coordination without fixing the observed wait.
The installer owns settings, rules, hooks, the mod and launcher. It compares semantic drift and skips a busy home. The OpenAI-model home has its own transcript namespace and a private configuration backup that contains no transcripts. Virtual-key spend logs showing zero model cost reflect subscription accounting, not free inference. Those logs also retain full request and response content, so their access and retention are part of the deployment’s privacy boundary.
Gotchas found after the first version
16. Worktree entry asked the owner on every lane
EnterWorktree on a loop worktree outside Claude Code’s own worktree directory asks the owner every time, and every lane hits it. The installer now denies the tool, as the settings listing shows, and lanes work in their worktree by absolute path.
17. Parallel GPT tool calls could arrive with empty arguments
When GPT makes parallel tool calls, codex-lb’s backend can send each call’s arguments only in the final response.function_call_arguments.done event, with no deltas. LiteLLM’s adapter forwards {}, and Claude Code fails with a missing required parameter. Between 10 and 12 October that was 59% of the tool errors on this home. The upstream report is BerriAI/litellm#45348 and the fix is pull request #45356; both were still open on 19 October. A local callback emits the full arguments once for any tool block that streamed none, and its state check after each upgrade says whether the adapter now handles it. Upstream’s fix does not cover arguments carried only on the finished item.
18. A restart read as a model that did not exist
While the LiteLLM container restarts, the reverse proxy drops its routes and answers 404. Claude Code reads a 404 as model_not_found, does not retry and switches to fallbackModel, which is also a 404, so every running lane ends its turn. A low-priority fallback route that answers 502 for /v1 fixes it, because clients retry a 502. On one recreate with lanes live, that was about 22 seconds of 502 and no failed lane. Restarting between loops is still the safer choice.
19. The auto-mode classifier stays on Sol
I tried routing the classifier calls to Luna. A replay of 300 captured calls from one loop showed no gain at medium or high, more false blocks and the same latency, so the classifier stays on Sol. It arrives under the lane-sonnet alias and sends no effort, so it runs at the server’s default Sol effort, not the lane table’s medium. That fits the lane section’s point that an alias picks a model and enforces no effort.