A coding agent session that runs for an hour does not cost twelve times what a five minute one does. It costs a lot more than that, and the reason is not the model getting chattier. It is the shape of the loop: every turn sends the whole conversation back to the model, and you pay for all of it again.

I spend most of my working day inside agent loops, either driving Claude Code or writing the loop myself for an LLM feature. So a paper that measures where the money in those loops goes caught my attention. [System Optimizations for Cost-Efficient Agent Harnesses](https://arxiv.org/abs/2611.12122) comes from researchers at the University of California, Berkeley, Northwestern University and the University of Illinois at Urbana-Champaign. Its headline is blunt: changing the harness (the system prompt, the tools, the loop around the model) barely moves accuracy on current models, but it can move cost by up to 2.6x on the same model and the same task.

## How an agent call is billed

LLM calls are stateless. The harness keeps the conversation and sends the full thing on every step: system prompt, tool schemas, the task, every earlier tool result and every earlier model message. Prompt caching means you do not pay full price for the part the provider has seen before, but you still pay for it. The paper writes the cost of one call as four terms:

`Equation 1 in the paper`

```
C = p_r * R  +  p_w * W  +  p_u * U  +  p_o * O

R  cache-read tokens     (context the provider already cached)
W  cache-write tokens    (new input, written to cache for later calls)
U  uncached input tokens
O  output tokens
```

Here are the prices the authors used for Sonnet 5.5, per million tokens. These are AWS Bedrock US in-region prices from the paper's Table 1, which include a 10% surcharge over the standard rate:

`Sonnet 5.5, paper Table 1 (USD per 1M tokens)`

```
uncached input   2.20
cache write      2.75
cache read       0.11
output          11.00
```

A cache read is 1% of the output price and 4% of a cache write. That looks like a rounding error, and that is the trap. The cache read applies to the *entire* context on *every* call. The paper's own example: a single grep at 100K tokens of context costs about the same as 10K tokens of new input plus 1K tokens of output, even if the grep itself returns three lines.

## What the paper measured

The authors looked at two kinds of traces. Public ones from end users of Claude Code and Codex (the TraceLab and FreeInference datasets, averaging 99K and 140K tokens and 84 and 87 steps), and their own benchmark runs on SWE-bench Verified, SWE-bench Pro, Terminal-Bench 2 and OfficeQA Pro. The benchmark runs used GPT-6-Luna, GPT-6.1-Sol, Sonnet 5.5, Opus 5.5 and a locally served Qwen3.8-27B, all with input caching on and medium thinking effort.

### The harness changes cost, not accuracy

They ran 100 tasks (40 SWE-bench, 40 SWE-bench Pro, 20 Terminal-Bench 2) through four harnesses. Two are Claude Code and Codex, unmodified. Two are minimal Python ReAct loops the authors wrote. Each sends this much fixed prefix on every call:

`Paper Table 2: fixed tokens per call`

```
harness        system prompt   tool schemas (count)
Claude Code    4.1K            8.7K (18)
Codex          6.1K            5.3K (21)
Basic-full     3.4K            0.82K (6)
Basic-bash     0               0.24K (1)
```

Accuracy was close across all four. Even Basic-bash, with no system prompt and a single bash tool, matched Claude Code and Codex, and did better on some models. Current models just chain shell commands and write their own scripts. Cost was another story: Claude Code and Codex easily reached 2x the cost of the basic harnesses. When they inspected the first call, a lot of the extra prompt was about user experience, such as how to format output for a terminal. Useful for a person watching, but paid for on every turn.

### Five places the money goes

- **Re-reading context dominates.** Even with caching, cache reads grow from 13% and 30% of cost on short traces to 53% and 73% on long ones (benchmark and public traces respectively, Section 3.2).
- **Tool calls cost as much as context.** At similar context sizes, runs with more tool calls cost 1.6x to 26x more, because every tool call is a new turn and every turn re-reads the context. Since each call also grows the context, cost grows roughly quadratically with tool calls (Section 3.3).
- **Tool output is mostly unused.** The agent later refers to only 27% of tool-output tokens. File reads, the biggest source, are at 29%. 36% of tool outputs are never used at all, and they stay in context until the run ends (Section 3.4).
- **Search eats turns.** On benchmark traces, listing and grepping take about 43% of turns and 28% of cost. In their OfficeQA Pro layout experiments it is 57% to 89% of turns. And 42% of directory listings repeat one that another session of the same model and harness already made (Section 3.5).
- **Unrelated subtasks share one context.** Longer sessions interleave more independent threads of work. Replaying traces with each subtask only seeing what it depends on saves 8% of input tokens on the shortest tasks and 33% on the longest, and 2% to 30% of cost (Section 3.6).

> **Note:** The authors are careful that Section 3 is hindsight analysis. You cannot know at runtime which 27% of a tool output the agent will need. Treat those numbers as a ceiling, and the Section 5 results as what a first working system actually gets.

## A toy model you can run

I wanted to feel the shape of this, so I wrote a small cost model. It is my own code, not the paper's, and it is deliberately simple: a fixed harness prefix, every turn the model writes 300 tokens and a tool returns 2,000, and the provider caches everything. New tokens are billed as cache writes, the rest of the context as cache reads.

`agent_cost.py`

```python
# agent_cost.py: a toy cost model of an agent loop (standard library only).
# Prices are Sonnet 5.5 per 1M tokens as listed in Table 1 of the paper
# (AWS Bedrock US in-region, 10% surcharge included). Swap in your own.
P_READ, P_WRITE, P_OUT = 0.11, 2.75, 11.00
M = 1_000_000

def run(turns, prefix=12_800, tool_out=2_000, out=300,
        compact=0.0, gc_every=0, gc_drop=0.0):
    """Each call re-sends the whole context. Tokens the provider already
    cached bill as cache reads, tokens added since the last call bill as
    cache writes, and generated tokens bill as output."""
    cached, context, stale = 0, prefix, 0
    cost = reads = 0.0
    for t in range(1, turns + 1):
        new = context - cached
        reads += P_READ * cached / M
        cost += (P_READ * cached + P_WRITE * new + P_OUT * out) / M
        cached = context
        result = int(tool_out * (1 - compact))
        context += out + result
        stale += result
        if gc_every and t % gc_every == 0:
            drop = int(stale * gc_drop)
            context -= drop
            stale -= drop
            cached = prefix  # the edit breaks the cache after the prefix
    return cost, reads, context

def row(name, turns, **kw):
    cost, reads, ctx = run(turns, **kw)
    print(f"  {name:<32} ${cost:6.3f}  reads {reads/cost:4.0%}  end ctx {ctx/1000:5.1f}K")

for turns in (40, 150):
    print(f"{turns}-turn session")
    row("baseline, 12.8K harness prefix", turns)
    row("bare harness, 0.24K prefix", turns, prefix=240)
    row("tool output compaction -14%", turns, compact=0.14)
    row("batching: 30% fewer turns", int(turns * 0.7), tool_out=int(2_000 / 0.7), out=int(300 / 0.7))
    row("context GC every 25 turns", turns, gc_every=25, gc_drop=0.47)
    print()

def net_saving(delta, n_left, left, llm_cost=0.0, extra_turns_cost=0.0):
    """Equation 2 in the paper:
    S = p_r * N * delta - (p_w * L + C_llm) - sum of extra turn costs."""
    return (P_READ * n_left * delta - P_WRITE * left) / M - llm_cost - extra_turns_cost

print("drop 20K stale tokens, 40K left behind:")
for n in (5, 20, 50, 100):
    print(f"  {n:>3} turns to go -> net ${net_saving(20_000, n, 40_000):+.3f}")
```

And the real output from running it:

`python3 agent_cost.py`

```
40-turn session
  baseline, 12.8K harness prefix   $ 0.656  reads  37%  end ctx 104.8K
  bare harness, 0.24K prefix       $ 0.568  reads  33%  end ctx  92.2K
  tool output compaction -14%      $ 0.603  reads  36%  end ctx  93.6K
  batching: 30% fewer turns        $ 0.576  reads  29%  end ctx 104.8K
  context GC every 25 turns        $ 0.701  reads  29%  end ctx  81.3K

150-turn session
  baseline, 12.8K harness prefix   $ 4.472  reads  67%  end ctx 357.8K
  bare harness, 0.24K prefix       $ 4.232  reads  66%  end ctx 345.2K
  tool output compaction -14%      $ 4.018  reads  66%  end ctx 315.8K
  batching: 30% fewer turns        $ 3.551  reads  59%  end ctx 357.7K
  context GC every 25 turns        $ 3.867  reads  39%  end ctx 112.9K

drop 20K stale tokens, 40K left behind:
    5 turns to go -> net $-0.099
   20 turns to go -> net $-0.066
   50 turns to go -> net $+0.000
  100 turns to go -> net $+0.110
```

![Bars for single calls in the toy model. The new-token part stays the same on every call, while the part for re-reading old context grows until it is most of the bill by call 150.](https://mahmadmanzoor.com/shots/post-agent-harness-cost/raster/per-call-bill.webp)

*Per-call cost in the toy model. The new tokens cost the same every time. The growing part is everything already said, read again.*

A few things fall out of this that match the paper's direction, even with made-up workloads:

- Cache reads go from 37% of the bill in a 40-turn session to 67% in a 150-turn one. The paper sees 53% to 73% on long real traces. Same mechanism.
- Fewer turns for the same work is the biggest win in the long session, because a turn you skip is a full context read you never pay for.
- Garbage collection *loses* money in the short session ($0.701 against $0.656) and saves about 14% in the long one. Removing tokens from the middle of the context breaks the cache after the edit point, so everything after it is billed as a cache write again, at 25x the read price.
- The harness prefix matters less here than in the paper. My toy only charges for re-reading the extra 12.5K tokens. It does not model behaviour changes from a different prompt, and the paper's benchmark runs were short, where a fixed prefix is a bigger share.

That third point is the most useful thing in the paper for anyone building their own loop, and it has an equation.

## When compressing context actually saves money

Compression only saves cache reads. It costs a rewrite of the context you keep, plus any model output used to summarize, plus extra turns if the agent later needs something you removed. Equation 2 in the paper puts that together. If you remove Δ tokens with N turns left, leaving L tokens behind, using C_llm of model output, and causing E extra turns:

`Equation 2 in the paper`

```
S = p_r * N * Δ   -   (p_w * L + C_llm)   -   Σ cost of extra turns
    cache-read        compression cost         re-fetching what
    savings                                    you threw away
```

The last four lines of my script's output are this equation with no LLM cost and no extra turns. Dropping 20K tokens out of a 60K context loses money with 5 or 20 turns to go, breaks even at 50, and only pays off past that. With no extra turns, the break-even point is N = (p_w / p_r) * L / Δ. Sonnet 5.5's write price is 25x its read price, so with half as many tokens removed as kept you need 50 turns left before the edit pays. On GPT-6-Luna in the same table the ratio is 12.5x, so the same compaction pays off sooner on that model. That ratio is worth checking for whichever model you run.

So the rules for compression are: remove a lot, do it early so many future turns benefit, make it cheap, and do not remove what the agent will need. Every technique in the paper is a different trade on those four.

## Agency: an interposition layer, not a new harness

The authors did not build another coding agent. They built Agency, which they describe as a harness interposition platform inspired by Valgrind. It runs an unmodified Claude Code or Codex binary and sits between it and everything it touches.

![Agency as four steps left to right: the unmodified harness, a sandbox engine using ptrace and cgroups, a host engine that logs and runs hooks, then the LLM or tool.](https://mahmadmanzoor.com/shots/post-agent-harness-cost/raster/agency.webp)

*Optimizations are hooks in the host engine, so one written once works across harnesses and models.*

- A sandbox engine inside the container launches the harness under `ptrace` and `cgroups`, so it sees every LLM call, child process and file access.
- Over a Unix domain socket, a host engine translates provider and harness specific formats (each LLM API's dialect) into one shared Agency format, logs everything to SQLite, and runs user hooks that can edit the context or tool arguments before dispatching, and edit results on the way back.
- Context edits work through a hash list. Each message block is hashed, and a lookup table maps hashes to either the original block, an empty block, or a compressed one. Every time the harness re-sends its message list, Agency swaps blocks through the table, so an edit stays applied across calls even though the harness never knew it happened.

They checked that the layer itself is not adding cost: Claude Code with Haiku 5.5 on five SWE-bench Verified tasks, 100 runs each, cost $0.0746 per run with Agency and $0.0766 without. The per-task change was -1.50%, with a 95% interval of -4.69% to +2.04%.

## The five techniques, and why each one saves money

### 1. Tool output compaction

This runs on new tool output before it enters the context, so it never invalidates the cache and every future turn benefits. It is deliberately conservative: whitespace and header stripping, format-based compaction, deduplication, and something the paper calls cross-output line pointers. A hash table finds lines in the new output that exactly match lines in older outputs and replaces them with pointers. In a read, edit, run loop, the second test run then mostly shows what changed. Average compaction is only 14%, but the compaction cost is zero. In isolation it cut code-task cost to 0.95 of baseline.

### 2. Context garbage collection

Old tool output gets less useful as the run moves on. Instead of paying an LLM to decide what to delete, they use gradient-boosted decision trees. For each line since the last compaction, the trees take features like the line's age, how many unique identifiers it has, and how often it was referenced, and predict how likely it is to be reused and how many turns remain. Lines under a threshold (0.3 in the paper) are candidates. Then the harness evaluates Equation 2 with those predictions and only fires if the predicted saving is positive. A compaction removes 47% of tokens on average, but irrelevant context has to pile up first, which usually takes 6 to 8 turns. Alone: 0.88 of baseline cost on code tasks.

### 3. Subtask stack machine

This one treats context as a stack and pops a subtask's history when it is done. Each turn is classified by its tool calls into a phase: setup, search, read, edit, test or execute. Neighbouring turns in the same phase form a segment, and every T segments (T = 6) are replaced with one summary block. The summary keeps the tool calls but drops outputs for setup, search, read and execute turns. For edit turns it keeps up to 300 characters of output, and for test turns the last five lines. That is a 91% token cut per compaction, but it only fires about once every 20 turns, so it mainly helps long runs. Alone: 0.94 of baseline.

Being aggressive has a price: new input rose 24% (summaries are new tokens) and tool calls rose 9% (the agent re-fetched what the summary dropped). Cache reads fell 25%, which still came out ahead.

### 4. Metadata-assisted search

Instead of making the agent walk a directory tree with `ls`, `find` and `grep`, they build a read-only SQLite index offline and describe it in the user prompt. For code it has two tables: `files(path, language, n_lines)` and `entries(path, name, kind, parent, line_start, line_end)`, filled by a rule-based extractor that parses the syntax tree for classes and functions. One SQL join replaces a chain of search turns. Here is a stdlib version of the same shape for Python files:

`build_index.py`

```python
# build_index.py: a small files/entries index in the shape the paper describes.
import ast, sqlite3, sys
from pathlib import Path

root = Path(sys.argv[1])
db = sqlite3.connect(sys.argv[2])
db.executescript("""
DROP TABLE IF EXISTS files; DROP TABLE IF EXISTS entries;
CREATE TABLE files (path TEXT PRIMARY KEY, language TEXT, n_lines INT);
CREATE TABLE entries (path TEXT, name TEXT, kind TEXT, parent TEXT,
                      line_start INT, line_end INT);
""")
for f in root.rglob("*.py"):
    src = f.read_text(errors="ignore")
    rel = str(f.relative_to(root))
    db.execute("INSERT INTO files VALUES (?, 'python', ?)", (rel, src.count("\n") + 1))
    try:
        tree = ast.parse(src)
    except SyntaxError:
        continue
    def walk(node, parent=None):
        for child in ast.iter_child_nodes(node):
            if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
                kind = "class" if isinstance(child, ast.ClassDef) else "function"
                db.execute("INSERT INTO entries VALUES (?, ?, ?, ?, ?, ?)",
                           (rel, child.name, kind, parent, child.lineno, child.end_lineno))
                walk(child, child.name)
    walk(tree)
db.commit()
print(db.execute("SELECT count(*) FROM files").fetchone()[0], "files,",
      db.execute("SELECT count(*) FROM entries").fetchone()[0], "entries")
```

```bash
$ python3 build_index.py . index.db
3 files, 8 entries
$ sqlite3 -readonly index.db \
    "SELECT path, name, kind, line_start, line_end FROM entries
     WHERE name LIKE '%saving%' OR name = 'run';"
agent_cost.py|run|function|7|27
agent_cost.py|net_saving|function|42|45
```

On OfficeQA Pro, a document retrieval benchmark, this was the biggest single effect in the paper: up to 80% fewer search calls, the target file found in 5.6 turns instead of 8.1, and cost down to 0.68 of baseline. On coding tasks the agent barely used it (0.2 lookups per run against 4.8 on OfficeQA), because the models already know the layout of popular repositories from training and go straight to the right file.

### 5. Tool call batching

The simplest one is a prompt. They tried forcing batching at the harness level and suggesting batches after each tool call, but those were unreliable on smaller models and added output. What worked best overall was a short instruction: make every call whose arguments you can already write in the same turn, and only wait when a call needs an earlier one's output. That gave up to 30% fewer turns, with 1.8 tool calls per turn on average.

In the GPT-6-Luna ablation, though, it *raised* cost by 2%. Steps fell from 32.0 to 26.0, but Luna issued 11% more tool calls when told to batch, adding calls it did not need. My toy model assumes the same work in fewer turns, and that assumption is exactly what a weaker model breaks.

![Bar chart of each technique's cost ratio in isolation, from the paper's Table 4: tool output compaction 0.95, context garbage collection 0.88, subtask stack machine 0.94, metadata-assisted search 0.68 on OfficeQA, tool call batching 1.02.](https://mahmadmanzoor.com/shots/post-agent-harness-cost/raster/techniques.webp)

*Each technique alone, from the paper's ablation. Only one is a big win on its own, and one loses money on a small model.*

## What it adds up to

All five combined, across GPT-6-Luna, GPT-6.1-Sol, Sonnet 5.5 and Opus 5.5 on Claude Code, Codex and Basic-full:

- Cost ratio between 0.65 and 0.85 in every model and harness pair, 0.77 on average, with an accuracy ratio of 0.97.
- Accuracy held at 0.97 to 0.99 for Sol, Sonnet and Opus. Luna dropped to 0.93, mostly on OfficeQA tasks where it misused the SQLite tables.
- OfficeQA with random UUID file names in a deep tree fell to 0.51 to 0.66. Coding benchmarks saved less: 0.80 to 0.92 on SWE-bench, 0.82 to 0.95 on SWE-bench Pro, 0.90 to 0.96 on Terminal-Bench 2.
- Longer runs saved more: under 100K total tokens ended at 0.93, 100K to 1M at 0.76, above 1M at 0.67.
- Not free everywhere. 17% to 29% of individual tasks cost more than baseline, likely short tasks where an index lookup or re-reading removed content outweighs the saving.

One concrete trace from the paper: a SWE-bench Pro task with Luna on Basic-full. The baseline context grew steadily to 61.3K tokens over 92 calls. The optimized run, with three stack machine compactions on top of per-call output compaction, ended at 16.3K, and sent 2.6M prompt tokens in total against 4.0M. Both runs solved the task.

I could not find a public release of Agency's code, so for now it is a design to learn from. The authors also note that real-world traces are 2 to 3x longer than the benchmarks, so the benchmark numbers likely understate the savings on real sessions, and that the whole analysis depends on the current pricing formula, which could change.

## What I would change in an agent today

You do not need an interposition layer to use most of this. If you write your own loop, or you choose how to drive Claude Code or Codex, here is where I would start.

1. **Log the four token counts on every call**

   You cannot fix what you do not see. Anthropic's Messages API returns `input_tokens`, `cache_creation_input_tokens`, `cache_read_input_tokens` and `output_tokens` in `usage`. Write them to a file per call and compute Equation 1. Then look at the read share by turn number. If it climbs past half, your sessions are long enough that the techniques above pay off.

   ```python
   import json
   P = {"read": 0.11, "write": 2.75, "input": 2.20, "output": 11.00}  # per 1M, use your model
   def call_cost(u):
       return (P["read"] * u["cache_read_input_tokens"]
             + P["write"] * u["cache_creation_input_tokens"]
             + P["input"] * u["input_tokens"]
             + P["output"] * u["output_tokens"]) / 1e6
   
   with open("usage.jsonl") as f:
       for turn, line in enumerate(f, 1):
           u = json.loads(line)
           print(turn, f"${call_cost(u):.4f}", u["cache_read_input_tokens"])
   ```

2. **Count your fixed prefix**

   Every tool schema and every line of system prompt is re-read on every turn. Disable MCP servers and tools you are not using for the task at hand. The paper's Basic-bash result says frontier models do fine with less than you think.

3. **Compact tool output before it enters the context**

   This is the only technique with zero compaction cost. Strip ANSI codes and blank runs, keep the tail of test output instead of the full log, collapse repeated lines. I already run [RTK](https://github.com/rtk-ai/rtk), an open-source CLI proxy the paper cites, which does this for shell output in my own Claude Code setup.

4. **Never edit the middle of the context on a whim**

   Anything that rewrites earlier messages breaks the cache from that point. Run Equation 2 first, with an honest guess for turns remaining. If the session is nearly done, leave it alone.

5. **Give the agent an index when names are unhelpful**

   Generated directories, logs, UUID-named files and document dumps are where metadata search won big. For a well-known repo the model already knows the layout, so skip it.

6. **Split unrelated work into separate contexts**

   Start a new session for a new task, or hand an independent subtask to a subagent that returns only its result. The subtask analysis shows why: every call in one thread otherwise re-reads the history of all the others.

7. **Test a batching prompt on your model before trusting it**

   It cut turns on strong models and raised cost on a weak one. Measure turns and tool calls per task with and without it.

The part of this paper that stays with me is the reframing. Harness work has mostly been about getting the model to succeed. For current frontier models it mostly succeeds anyway, and the harness has become the place where the bill is decided. The fix is not a smarter model. It is fewer re-reads of things the model no longer needs.

---

*Originally published on [my site](https://mahmadmanzoor.com/blog/agent-harness-cost-not-accuracy). More posts and projects at [mahmadmanzoor.com](https://mahmadmanzoor.com).*
