Token Counting

Token Budget Planning for Multi-Agent Workflows

Jul 23, 2026TokenTest Team
English中文

A single LLM call is easy to budget. A multi-agent workflow is not.

The planner writes a brief. Retrievers fan out. Workers consume the retrieved context. A reviewer reads their outputs. One failed tool call triggers a retry. By the time the final answer appears, the same information may have crossed several model boundaries—and each boundary can create new input and output usage.

That is why token budget planning for multi-agent workflows must happen at the workflow level, not just in each prompt.

This guide gives you a practical allocation formula, a worked planner–retriever–worker–reviewer example, and a paste-ready test you can run through TokenTest to verify live usage behavior.

The short answer

Use one hard workflow cap, then allocate it across agent calls, tool payloads, retries, and a shared reserve:

workflow_hard_cap
= sum(expected_calls × per_call_token_cap)
+ tool_result_allowance
+ retry_reserve
+ shared_emergency_reserve

For every model call, define:

per_call_token_cap
= maximum_input_tokens
+ maximum_output_tokens

Track cached input and reasoning tokens separately when the provider reports them, but do not pretend they disappear from the workflow. Cached input is still input usage, and reasoning tokens can consume output allowance even when they are not visible in the final answer.

Why per-agent limits are not enough

Suppose every agent has a reasonable-looking output limit. The workflow can still exceed its budget because of four multipliers.

1. Fan-out multiplies calls

One planner call may launch two retrievers and three workers. A 2,000-token worker budget is not 2,000 workflow tokens when three workers run—it is up to 6,000.

2. Handoffs are counted again

The planner's output is output usage on the planner call. When that plan is inserted into a worker prompt, it becomes input usage on the worker call. Summing usage at every model boundary is not double-counting; it reflects how multi-step inference is actually consumed and reported.

3. Tool results expand downstream prompts

Search snippets, database rows, code output, and file extracts may be larger than the agent instruction that requested them. If tool payloads have no size limit, downstream input can dominate the entire run.

4. Retries hide in orchestration logic

A timeout, invalid JSON response, failed tool call, or reviewer rejection can repeat an expensive step. A production budget needs a retry allowance before the retry happens.

A workflow token budget formula that survives production

Budget each role with the same structure:

role_budget
= maximum_call_count
× (maximum_input_tokens + maximum_output_tokens)

Then calculate the workflow cap:

workflow_hard_cap
= planner_budget
+ retriever_budget
+ worker_budget
+ reviewer_budget
+ tool_result_allowance
+ retry_reserve
+ shared_emergency_reserve

This is deliberately conservative. It is a hard-cap model, not a forecast. For day-to-day planning, keep a second number based on median observed usage. The hard cap protects the system; the observed budget helps you optimize it.

Worked example: a 16,636-token research workflow

Imagine a workflow that researches a question, drafts an answer, and checks it before delivery.

Role or reserve Maximum calls Input cap per call Output cap per call Allocated tokens
Planner 1 900 220 1,120
Retrievers 2 650 80 1,460
Workers 3 1,800 450 6,750
Reviewer 1 2,400 300 2,700
Tool-result allowance — — — 1,200
Retry reserve — — — 2,406
Shared emergency reserve — — — 1,000
Workflow hard cap 16,636

The model-call subtotal is 12,030 tokens. The retry reserve is 20% of that subtotal, or 2,406 tokens. Tool results and the emergency reserve are added separately.

This table creates decisions the orchestrator can enforce:

Allocate by role, not by equal percentages

Equal slices look simple, but agent roles do different work.

Planner: small output, high leverage

The planner should usually produce a compact task graph, not a long essay. Give it enough input to understand constraints and enough output to define tasks, dependencies, and acceptance criteria. If the plan is verbose, every downstream agent pays for that verbosity again.

Retriever: strict output structure

Retrievers often need little generative output. Their budget risk is the returned evidence. Limit result count, excerpt length, and total tool payload. Ask for identifiers and concise snippets rather than full documents when possible.

Worker: largest productive allocation

Workers usually need the largest role budget because they combine instructions, evidence, and intermediate state. Allocate by task difficulty. Do not give every parallel worker the same maximum if only one branch performs the hard synthesis.

Reviewer: protected, not optional

Reserve the reviewer budget before workers begin. Otherwise, early agents can spend the whole workflow cap and leave no room to validate the answer. The reviewer should check task completion, contradictions, citation coverage, and formatting—not rewrite the entire response by default.

Use soft limits and hard caps together

A hard cap answers, “When must this run stop?” A soft limit answers, “When should the orchestrator change behavior?”

Set both.

Guardrail Example behavior
50% consumed Continue normally
70% consumed Stop launching optional branches
85% consumed Summarize state and route directly to review
Reviewer reserve reached Block new worker calls
Hard cap reached Stop the workflow with a budget-exhausted status

Soft limits make the workflow degrade gracefully. A hard cap without soft limits often ends with an abrupt failure after expensive work has already happened.

Track the token fields that explain overruns

At minimum, record these fields for every model call:

{
  "workflow_id": "research-042",
  "agent_role": "worker",
  "attempt": 1,
  "requested_model": "your-model-alias",
  "returned_model": "reported-model-version",
  "input_tokens": 0,
  "cached_input_tokens": null,
  "output_tokens": 0,
  "reasoning_tokens": null,
  "total_tokens": 0,
  "tool_result_tokens": 0,
  "remaining_workflow_budget": 0,
  "finish_reason": "stop",
  "task_valid": true
}

Use null for “not reported.” Do not turn a missing cached-token or reasoning-token field into zero. Zero means the provider explicitly reported none; null means you do not have the evidence.

OpenAI's Agents SDK documentation shows run-level usage aggregation for requests, input tokens, output tokens, and total tokens, with detailed fields for cached input and reasoning output when available. Its tracing documentation also describes traces and spans across agent runs, model generations, function calls, handoffs, and guardrails. That combination is the right mental model: trace the workflow structure, then attach usage to every billable model boundary.

Count before, verify after

Use three layers instead of trusting one number.

  1. Local estimate: useful while editing prompts, but only when the tokenizer and request template match the target model closely enough.
  2. Provider-native preflight: use a provider's counting endpoint when available. OpenAI documents POST /v1/responses/input_tokens; Anthropic documents POST /v1/messages/count_tokens; Google documents countTokens for Gemini models.
  3. Live usage verification: run the real endpoint and inspect returned usage, stop behavior, cache evidence, and reasoning-token evidence.

Preflight counting estimates the request. Live usage reveals the full workflow behavior, including generated output and retries.

Multilingual workflows need separate budgets

Do not apply an English token ratio to every language.

Tokenization varies by tokenizer, model, punctuation, script, code mixing, and chat template. Tool schemas and repeated English field names can also change the ratio in a multilingual request.

For each production language:

  1. Keep the task meaning and output schema equivalent.
  2. Count the complete request, not just the translated user sentence.
  3. Run the same workflow shape and call limits.
  4. Compare median input, output, retry rate, and task validity.
  5. Set the production cap from the higher observed percentile plus headroom.

If one language consistently consumes more input, adjust retrieval size or prompt structure for that language rather than silently shrinking output quality.

Paste-ready test for TokenTest

Use this prompt as a controlled worker task. Keep the model, temperature, output limit, and endpoint fixed across runs.

You are the worker agent in a multi-agent research workflow.

Budget constraints:
- Return valid JSON only.
- Use no more than 180 visible words.
- Include exactly three evidence items.
- If evidence is insufficient, set "status" to "needs_more_evidence" instead of guessing.

Task:
Explain why workflow-level token budgets must include fan-out, retries, tool results, and downstream handoffs.

Output schema:
{
  "status": "complete | needs_more_evidence",
  "summary": "string",
  "evidence": [
    {"factor": "string", "budget_effect": "string"}
  ],
  "next_action": "string"
}

Run it in TokenTest, then inspect whether the endpoint reports plausible input, output, and total usage; whether the output stops near the configured limit; and whether cache or reasoning details appear when the route claims to support them. Repeat the test at least three times before using the median as a planning input.

TokenTest's current evaluation console is designed for live endpoint checks across token usage, capability, route protocol, safety boundaries, and channel reliability. Its usage-integrity checks include usage presence, total consistency, input monotonicity, output reasonableness, stop-limit linkage, stream usage, cache evidence, and thinking or reasoning evidence. TokenTest states that API keys are not stored, but you should still follow your organization's credential and data-handling policies.

A practical orchestration policy

The following policy works as a starting point:

before launching a call:
  projected = current_usage + call_input_cap + call_output_cap

  if projected > workflow_hard_cap:
    stop with budget_exhausted

  if remaining_budget <= reviewer_reserve + emergency_reserve:
    block new optional workers
    route current evidence to reviewer

after every call:
  record reported usage
  update remaining budget
  validate the output
  release unused reserved output only if no retry is pending

The important detail is that the orchestrator checks the budget before each call, not only after the provider returns usage.

Common token budgeting mistakes

Budgeting only the final answer

The user may see 300 tokens while the workflow consumed thousands across planning, retrieval, drafting, and review.

Subtracting cached tokens from context planning

Caching may change billing or latency behavior, depending on the provider, but cached tokens are still part of the input sequence. Track them separately; do not use them as free context.

Letting the reviewer rewrite by default

A full rewrite creates another large generation. Prefer a reviewer that returns a pass/fail decision and a bounded patch list. Rewrite only when the answer fails a defined gate.

Using the maximum context window as the operating budget

A model's context window is a technical ceiling, not a safe workflow target. Your operating budget also needs output room, retry room, tool variability, and downstream review capacity.

Comparing efficiency before validating the task

A shorter invalid answer is not more efficient. Record task_valid and compare token efficiency only among outputs that satisfy the task.

Final checklist

Before deploying a multi-agent workflow, confirm that you have:

Then run the workflow against the real route in TokenTest, save the raw usage evidence, and replace planning assumptions with observed medians and high-percentile limits.

FAQ

How much retry reserve should a multi-agent workflow have?

Start from observed retry rates by step. Before you have production data, a bounded reserve such as one retry for the most failure-prone critical step is more defensible than unlimited retries. The 20% reserve in this guide is an example, not a universal rule.

Should every agent receive the same token budget?

No. Allocate by role, call count, task complexity, and downstream reuse. Planners and retrievers often need concise outputs; synthesis workers usually need more; reviewers need a protected reserve.

Do reasoning tokens count against output limits?

For OpenAI reasoning models, official documentation says reasoning tokens are part of output-token usage and are controlled by max_output_tokens. Other providers may report or limit them differently, so verify the route you actually use.

Can one token counter estimate every model in the workflow?

Not reliably. Different model families and serving stacks can use different tokenizers, chat templates, special tokens, and tool encodings. Prefer model-native counting and live usage evidence.

Sources