Token Counting for Structured Outputs and Function Calling Prompts

Token counting for structured outputs and function calling is easy to underestimate. The visible user prompt may be short, but the actual model request can also include system instructions, JSON schemas, tool descriptions, prior assistant tool calls, tool results, and reserved output space.

The practical rule is simple: budget the serialized request the model receives, not only the text a user typed.

This guide shows how to count the full request, why local estimates can drift, and how to test structured-output and tool-calling workloads with reproducible examples.

Quick answer

For a structured-output or function-calling workflow, plan for these token groups:

1. Base messages: system, developer, user, and relevant conversation history.
2. Tool definitions: function names, descriptions, parameter names, JSON Schema keywords, enums, and nested objects.
3. Structured-output schema: the format contract used to constrain the final response.
4. Assistant tool calls: generated function names and argument JSON.
5. Tool results: database rows, search results, API payloads, errors, and other returned context.
6. Final answer: the structured JSON or natural-language response.
7. Safety margin: headroom for production inputs that are longer than your test sample.

There is no reliable universal rule such as “every function adds N tokens.” The total changes with the model, API, schema, message format, and actual tool results. Count the complete request for the endpoint you will use.

Why schemas and tools change the token budget

In a plain chat request, most developers remember to count the prompt and expected answer. Function calling adds another layer: the model must receive enough information to decide whether to call a tool and how to construct valid arguments.

OpenAI’s function-calling documentation explicitly notes that function definitions are injected into the system message and count against the context window as input tokens. That makes every field in a tool definition part of the practical prompt budget, even when users never see it in the chat transcript.

A schema like this is useful, but not free context:

{
  "type": "function",
  "name": "lookup_order",
  "description": "Find an order by its public order ID.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Public order ID, for example ORD-10482."
      },
      "include_events": {
        "type": "boolean",
        "description": "Whether to include shipment event history."
      }
    },
    "required": ["order_id"],
    "additionalProperties": false
  },
  "strict": true
}

Long descriptions, repeated wording, large enums, and deeply nested objects can all increase input size. The correct optimization target is not the shortest possible schema; it is the smallest schema that still produces reliable tool selection and valid arguments.

Structured outputs and function calling are different budgets

These two features are related, but they solve different problems.

Feature	Primary job	Token impact to measure
Structured output	Constrain the final response to a schema	Request format/schema overhead plus the generated JSON output
Function calling	Let the model choose and parameterize an application tool	Tool definitions, assistant tool-call arguments, tool results, and later turns
Both together	Use tools, then return validated final JSON	All of the above in one workflow

Structured output does not eliminate output tokens. If the model returns a JSON object, the keys, values, braces, punctuation, and any explanations allowed by the schema still occupy output tokens.

Function calling can create multiple metered stages. A typical flow looks like this:

system + user + tool definitions
            ↓
assistant tool call with JSON arguments
            ↓
tool result added to the conversation
            ↓
assistant final response

If the model calls three tools, your application may add three argument payloads and three result payloads before the final answer. A small initial prompt can therefore become a large multi-turn context.

A practical token-budget formula

For reliable token counting for structured outputs and function calling, use a formula that includes both fixed request components and the data accumulated during the tool loop.

Use this planning equation:

planned context = fixed instructions
                + user input at a realistic percentile
                + conversation history
                + tool definitions
                + expected tool-call arguments
                + expected tool results
                + structured-output schema
                + reserved final output
                + safety margin

Do not calculate only the first turn. For an agentic workflow, calculate the largest request you expect to send during the entire loop. The request immediately before the final answer is often larger than the initial request because it includes tool calls and tool results.

Exact counting example for a structured response

OpenAI documents an input-token counting endpoint for the Responses API. It accepts the same request shape and returns the exact input count without generating a response. That makes it preferable to a rough character or word estimate when the selected API and model support it.

Paste this request into your API client and replace the model with the exact model used in production:

curl https://api.openai.com/v1/responses/input_tokens \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "input": [
      {
        "role": "system",
        "content": "Extract the support request into the required schema."
      },
      {
        "role": "user",
        "content": "Order ORD-10482 arrived damaged. Refund to the original payment method."
      }
    ],
    "text": {
      "format": {
        "type": "json_schema",
        "name": "support_request",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "order_id": {"type": "string"},
            "issue": {"type": "string"},
            "requested_action": {"type": "string"}
          },
          "required": ["order_id", "issue", "requested_action"],
          "additionalProperties": false
        }
      }
    }
  }'

Record the returned count, then run controlled variants:

• Remove nonessential field descriptions.
• Replace a large enum with a shorter stable code list.
• Split one broad schema into route-specific schemas.
• Compare the English and Chinese production inputs separately.
• Add the longest realistic user message, not only a happy-path sample.

Change one variable per run so the token difference has a clear cause.

Exact counting example with a function tool

Now measure the request with a tool definition included:

curl https://api.openai.com/v1/responses/input_tokens \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "input": "Where is order ORD-10482?",
    "tools": [
      {
        "type": "function",
        "name": "lookup_order",
        "description": "Find an order by its public order ID.",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {"type": "string"},
            "include_events": {"type": "boolean"}
          },
          "required": ["order_id"],
          "additionalProperties": false
        },
        "strict": true
      }
    ]
  }'

Run the same request without tools. The difference is the request-side overhead for that specific tool definition and request shape. Do not reuse that difference as a permanent constant across models or APIs.

Count the tool loop, not only the first request

The most common budgeting mistake is measuring the initial request and ignoring the follow-up turn.

Suppose the model emits this tool call:

{
  "name": "lookup_order",
  "arguments": {
    "order_id": "ORD-10482",
    "include_events": true
  }
}

Your application then adds a tool result:

{
  "order_id": "ORD-10482",
  "status": "in_transit",
  "carrier": "Example Express",
  "estimated_delivery": "2026-07-30",
  "events": [
    {"time": "2026-07-27T08:10:00Z", "status": "departed_sort_center"}
  ]
}

The next model request now contains the earlier messages, the assistant tool call, and this result. In production, tool payloads can be far larger than the tool schema itself. Search snippets, database records, logs, and error traces should be trimmed to the fields the model needs.

Useful controls include:

• Return compact objects instead of entire upstream API responses.
• Limit result counts before inserting data into the model context.
• Remove duplicated labels, metadata, and null fields.
• Summarize older tool results when the application can do so safely.
• Keep stable IDs so the model can request detail only when needed.

Why a local tokenizer estimate can differ

This is where token counting for structured outputs and function calling often diverges from a simple local text count.

Local tokenizers remain useful for fast comparisons, but they are not always the final billing authority for structured requests.

Differences can appear because:

1. Chat messages have formatting overhead. Roles and message boundaries are not represented by the visible text alone.
2. Tool definitions may be transformed internally. Providers can serialize schemas into model-specific instruction formats.
3. Models use different tokenizers. A count for one model should not be treated as exact for another.
4. Conversation state can add context. A continuation request may include prior items that are absent from your local string.
5. Images and files use separate accounting rules. A text-only tokenizer cannot fully reproduce multimodal usage.
6. APIs evolve. Constants copied from an old cookbook or SDK example can become stale.

Use local counting for fast diffs and CI guardrails. Use the provider’s exact counting endpoint or the live API-reported usage object as the production reference.

Use TokenTest to verify the endpoint’s reported usage

TokenTest is useful after local planning because it tests the behavior of the actual endpoint and model route you intend to use.

A practical verification loop is:

1. Enter the OpenAI-compatible or Anthropic-compatible endpoint.
2. Use a test-only API key and select the real model ID.
3. Run the Token Usage Audit and Tool Channel checks.
4. Inspect whether input and output usage fields are present and internally consistent.
5. Compare short and longer inputs to confirm input-token monotonicity.
6. Check whether tool calls, stop signals, streaming usage, cache evidence, or reasoning-token fields appear as expected for that route.

TokenTest also provides an offline response analyzer. You can paste a raw response JSON object into the “Analyze response JSON” field without providing an API key. For example:

{
  "model": "your-resolved-model-id",
  "usage": {
    "input_tokens": 842,
    "output_tokens": 96,
    "total_tokens": 938
  }
}

This sample is only for checking response-shape handling; replace it with an actual response from your endpoint when validating production behavior.

A repeatable optimization workflow

Use this sequence for token counting for structured outputs and function calling:

1. Capture the complete request. Include instructions, messages, tools, schemas, and continuation state.
2. Measure the baseline. Use an exact count endpoint when available.
3. Measure the largest loop turn. Add representative tool calls and realistic tool results.
4. Reserve output before trimming input. Protect enough room for the final JSON or answer.
5. Change one component. Shorten a description, reduce an enum, or trim a tool result.
6. Recount and record the delta. Keep changes that reduce tokens without harming behavior.
7. Run behavioral tests. Confirm the smaller schema still selects the correct tool and produces valid arguments.
8. Validate live usage. Compare local or count-endpoint results with the response’s reported usage.
9. Test every production language. English and Chinese prompts should have independent budgets.

For a broader release workflow, use the token-aware prompt review process. If your application has a long tool loop, also review the production context-window planning guide.

Token-counting checklist

Use this checklist whenever token counting for structured outputs and function calling becomes part of a release review:

Before shipping, confirm that you have counted:

• [ ] System and developer instructions
• [ ] Realistic user input
• [ ] Relevant conversation history
• [ ] Every enabled tool definition
• [ ] Function-call argument JSON
• [ ] Tool result payloads
• [ ] Structured-output schema
• [ ] Final JSON or text output reserve
• [ ] Retry or repair turns
• [ ] English, Chinese, and other production-language variants
• [ ] Safety margin for unusually large inputs
• [ ] Live endpoint usage after deployment

Final takeaway

Token counting for structured outputs and function calling should be treated as request engineering, not a word-count exercise. Measure the full payload, model the largest tool-loop turn, preserve final-output headroom, and validate the endpoint’s own usage reporting.

Start with an exact request count where the provider supports it. Then use TokenTest to verify that the production route exposes consistent token usage and tool-channel behavior before you rely on it for cost planning or context-window limits.

Sources

• OpenAI: Counting tokens
• OpenAI: Function calling
• OpenAI: Structured model outputs
• OpenAI Cookbook: How to count tokens with tiktoken