Agent loops for product features: budgets, errors, stops and checkpoints

The tutorial version of an agent is ten lines: call the model, run whatever tools it asks for, append the results, repeat until it stops. That loop is fine in a notebook. Inside a product it is a while (true) that spends money on every pass, can take actions on a customer's data, and has no idea that a person is waiting for an answer.

Most of the code I write for an agent feature is not the loop. It is the fence around it. This post walks through that fence using a teaching example: an assistant inside a roofing company's scheduling app that can find a job, look up open inspection slots and move an inspection when a customer asks.

The code uses the Anthropic TypeScript SDK with a hand-written loop. The SDK also ships a tool runner that drives the loop for you and has hooks for approvals. I write the loop by hand when I want the checkpoints and budgets to be plain code I can read in one screen.

Tools shaped for the model, not for the database

The tools decide most of what an agent can do well. My rules for them are short.

Name them after product actions, not infrastructure. find_jobs, list_slots and propose_reschedule are better than run_sql or http_request, because each one carries its own permissions and validation. Keep them few: every extra tool is another choice the model can get wrong. Return compact text, capped in size, rather than raw JSON rows, because everything a tool returns is paid for again on every later step. And put the order of use in the description, since the model reads it.

export const listSlots = {
  name: "list_slots",
  description:
    "List open inspection slots for one job's crew, soonest first. Returns at most 8 slots. " +
    "Call find_jobs first to get the job id. Call propose_reschedule with one of these slots.",
  strict: true,
  input_schema: {
    type: "object",
    properties: {
      job_id: { type: "string", description: "Job id from find_jobs, like JOB-4471" },
      from_date: { type: "string", description: "Earliest date to consider, YYYY-MM-DD" },
    },
    required: ["job_id", "from_date"],
    additionalProperties: false,
  },
} as const;

strict: true guarantees the input matches the schema. I validate it again with Zod in the handler anyway, because the handler is also called from tests and from the approval path, and it should not trust any caller.

Error results the model can recover from

When a tool fails, the worst things to return are nothing, a stack trace, or a generic "error". The model will either guess, retry the same call, or tell the user something false. A tool result with is_error: true and a sentence that says what went wrong and what to do next turns most failures into one extra step.

I sort failures into three groups. Bad input gets the validation issues and "fix these fields and call again". Expected business failures, like a slot that was just taken, get a specific next step. Everything else gets logged with the full detail on my side and returns a short message that tells the model not to retry and to tell the user the step could not be done. That last instruction matters. Without it, a model that hits a broken dependency will try again and again with small variations, and each try is a paid step.

export async function execute(call: Anthropic.ToolUseBlock, ctx: Ctx): Promise<Anthropic.ToolResultBlockParam> {
  const fail = (content: string): Anthropic.ToolResultBlockParam =>
    ({ type: "tool_result", tool_use_id: call.id, is_error: true, content });

  const tool = TOOLS[call.name];
  if (!tool) return fail(`There is no tool called ${call.name}.`);

  const input = tool.schema.safeParse(call.input);
  if (!input.success) {
    const issues = input.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; ");
    return fail(`Invalid input (${issues}). Fix these fields and call again.`);
  }
  try {
    return { type: "tool_result", tool_use_id: call.id, content: await tool.run(input.data, ctx) };
  } catch (e) {
    if (e instanceof NotFoundError) return fail(`${e.message}. Call find_jobs with the customer's name or street to get the job id.`);
    if (e instanceof SlotTakenError) return fail("That slot was just taken. Call list_slots again and offer another.");
    ctx.log.error(e, { runId: ctx.runId, tool: call.name });
    return fail("This step failed for an internal reason. Do not retry it. Tell the user it could not be completed.");
  }
}

Budgets and stopping rules

A loop needs to know when to stop, and "when the model says it is done" is only one of the answers. These are the exits I put in every agent loop.

The step cap and the token budget are the hard limits. I set the step cap from the task: a reschedule needs about five steps, so a cap of twelve leaves room for a couple of recoveries without letting a confused run wander for fifty. The token budget is a running sum of output tokens across steps. Some newer Claude models also accept a task budget in the request, which tells the model how much room it has so it can wrap up on its own. I use that when it is available, and I keep my own cap as well, because a limit the model is told about and a limit the code enforces are different things.

Repetition is the exit I see people skip. If the model makes the same call with the same input twice and it did not help, a third time will not either. I count calls by name plus input and stop on the third repeat. Truncation and refusal are not reasons to keep going either; they get their own status so the product can say something sensible.

const LIMITS = { maxSteps: 12, maxOutputTokens: 60_000, maxRepeats: 2 };

export async function runAgent(state: RunState, ctx: Ctx): Promise<Outcome> {
  const repeats = new Map<string, number>();

  while (state.step < LIMITS.maxSteps) {
    state.step++;
    const started = Date.now();
    const res = await client.messages.create({
      model: "claude-sonnet-5-5",
      max_tokens: 16000,
      cache_control: { type: "ephemeral" },
      system: SYSTEM,
      tools: TOOL_DEFS,
      messages: state.messages,
    });
    state.outputTokens += res.usage.output_tokens;
    state.messages.push({ role: "assistant", content: res.content });
    await ctx.trace.step(state, res, Date.now() - started);

    if (res.stop_reason === "end_turn") return { status: "done", text: textOf(res) };
    if (res.stop_reason === "refusal") return { status: "stopped", reason: "refusal" };
    if (res.stop_reason === "max_tokens") return { status: "stopped", reason: "truncated" };

    const calls = res.content.filter((b): b is Anthropic.ToolUseBlock => b.type === "tool_use");
    const results: Anthropic.ToolResultBlockParam[] = [];
    const pending: Anthropic.ToolUseBlock[] = [];

    for (const call of calls) {
      const key = call.name + JSON.stringify(call.input);
      repeats.set(key, (repeats.get(key) ?? 0) + 1);
      if (repeats.get(key)! > LIMITS.maxRepeats) return { status: "stopped", reason: "repeating" };
      if (NEEDS_APPROVAL.has(call.name)) pending.push(call);
      else results.push(await execute(call, ctx));
    }

    if (pending.length > 0) {
      await ctx.checkpoints.save(state.runId, { state, pending, partial: results });
      return { status: "awaiting_approval", pending };
    }
    state.messages.push({ role: "user", content: results });
    if (state.outputTokens > LIMITS.maxOutputTokens) return { status: "stopped", reason: "token_budget" };
  }
  return { status: "stopped", reason: "step_limit" };
}

The cache_control line at the top level caches the growing conversation, so each step reads the earlier steps from the prompt cache instead of paying full price for them again. In a loop that resends its whole history on every pass, that is the single biggest cost lever. Note also that all tool results for one assistant turn go back in a single user message. Splitting them across messages works, but it teaches the model to stop making parallel calls.

Human checkpoints

Moving a customer's inspection is a write, so the loop does not do it. When the model calls propose_reschedule, the loop saves its state and returns awaiting_approval. The product shows the office user the exact change: job, old slot, new slot. The loop is now idle and costs nothing while it waits.

Resuming is just appending the missing tool results and calling the same function. An approved call runs through execute like any other. A rejected one returns a plain result that says so, without is_error, because a person saying no is information, not a failure.

export async function resume(runId: string, decisions: Record<string, "approve" | "reject">, ctx: Ctx) {
  const { state, pending, partial } = await ctx.checkpoints.load(runId);
  const results: Anthropic.ToolResultBlockParam[] = [...partial];
  for (const call of pending) {
    results.push(
      decisions[call.id] === "approve"
        ? await execute(call, ctx)
        : { type: "tool_result", tool_use_id: call.id, content: "The user rejected this change. Do not propose it again; ask what they would prefer." },
    );
  }
  state.messages.push({ role: "user", content: results });
  return runAgent(state, ctx);
}

The checkpoint stores the message history exactly as it came back, thinking blocks included. Current models expect earlier turns to be replayed unchanged, so I never edit or trim a saved history before resuming.

Trace every step

When an agent gives a strange answer, the question is always "what did it see and what did it do". I write one record per step: run id, step number, model, stop reason, the tools called with their inputs, token counts including cache reads, and the time it took. Tool calls get their own records with the result size and any error. If you already run OpenTelemetry, these map neatly onto spans; if not, JSON lines in your normal logs are enough to start.

The trace has three uses. It answers support questions without guessing. It shows where time goes, which is usually the model calls, not the tools. And a run that went wrong becomes a new eval case with its input already captured, which is how the loop gets better instead of just more guarded.

What I put around every agent loop

1. Shape tools for the model

   Product verbs, few of them, strict schemas, small text results, and the order of use in the description.

2. Return errors the model can act on

   Say what failed and what to do next, and say "do not retry" when retrying cannot help.

3. Enforce budgets in code

   A step cap from the task, a running token budget, and a stop on repeated identical calls.

4. Treat every stop reason as a product state

   Done, truncated, refused, over budget, repeating and awaiting approval each get their own handling.

5. Pause for people at writes

   Save the run, show the exact change, and resume by appending the results.

6. Trace every step

   Model, stop reason, tools, tokens and time, one record per step.

An agent feature fails in ordinary ways: slow, expensive, stuck or wrong. Every item above exists to make one of those failures visible and bounded, so that the clever part, the model choosing what to do next, happens inside limits I chose.