How to Size Few-Shot Prompt Examples Without Wasting Context
Few-shot prompting can improve consistency, formatting, and edge-case handling. It can also quietly consume the context you need for the real user input and the model’s answer.
The practical question is not “What is the perfect number of examples?” There is no universal count. The better question is:
What is the smallest set of few-shot prompt examples that passes your quality tests while preserving enough room for production input and output?
A reliable workflow starts with a small, deliberately varied example set, measures the complete request, and adds another example only when it fixes a documented failure. This guide gives you a sizing formula, a 0/1/3/5-example test ladder, compression rules, and fixtures you can run against your target endpoint with TokenTest.
Quick answer: start small and earn every example
Use this default testing sequence:
- Run the task with zero examples.
- Add one clean example that demonstrates the expected format.
- Test three examples that cover the normal case, an edge case, and a confusing case.
- Test five examples only if the three-example version still has repeatable failures.
- Keep the smallest version that meets your quality, latency, and token-budget requirements.
This is a test ladder, not a claim that every model needs three or five examples. Anthropic’s current multishot guidance says that three to five examples can improve results for Claude and recommends examples that are relevant, diverse, clear, and consistently structured. Google’s few-shot guidance likewise emphasizes showing representative examples and notes that the optimal number varies. OpenAI’s prompt engineering guide recommends using diverse input-output examples to help a model learn a task from the prompt.
Your production decision still needs to come from evaluation on your model, endpoint, language, and real request shape.
Why few-shot prompt examples become expensive
Each example usually repeats more than the example text itself. A typical demonstration contains:
- a role or delimiter,
- an input,
- an output,
- formatting syntax,
- repeated field names,
- explanatory labels,
- and sometimes the same policy text copied into every example.
If one example costs 220 input tokens, five examples do not merely “add five examples.” They can add roughly 1,100 input tokens before message wrappers, tools, retrieval, the live user request, and the output reserve are considered.
Visible text is also not the entire request. OpenAI’s token-counting guidance separates plain input counting from request-shaped counting that can include messages, tools, files, and other input types. Anthropic provides a token-counting endpoint for Messages requests, and Gemini exposes token-counting methods for model-specific request content. The exact accounting method differs by provider, which is why a generic character count is not a sufficient production budget.
Use a full-request budget, not an example-count rule
Start with this equation:
available_example_tokens =
context_limit
- fixed_request_tokens
- production_input_tokens
- retrieval_and_tool_tokens
- output_reserve
- safety_margin
Where:
| Budget component | What to include |
|---|---|
| Context limit | The limit that applies to the exact model and endpoint you will call |
| Fixed request tokens | System instructions, policies, response schema, message wrappers, stable tool definitions |
| Production input tokens | A realistic high-percentile user input, not the shortest staging sample |
| Retrieval and tool tokens | Retrieved passages, function definitions, tool results, image/file metadata, conversation history |
| Output reserve | Enough space for a complete useful answer, including structured output or reasoning behavior where applicable |
| Safety margin | Room for multilingual growth, longer edge cases, tokenizer changes, and request-shape variation |
Only the remainder belongs to few-shot prompt examples.
Example sizing worksheet
Suppose a team approves an 8,000-token working budget for one request shape. This is an internal engineering budget, not a model specification.
| Component | Tokens |
|---|---|
| System instructions and schema | 900 |
| Production user input | 1,700 |
| Retrieval and tools | 1,200 |
| Output reserve | 2,400 |
| Safety margin | 600 |
| Available for examples | 1,200 |
If each full input-output example costs 260 tokens, four examples would cost about 1,040 tokens and fit the example allowance. Five would cost about 1,300 tokens and exceed it. The decision is not “four is best.” The decision is “four is the most this budget can hold, and we should still keep fewer if one or three pass the evaluation.”
Measure the incremental cost of each example
Do not estimate examples in isolation. Measure the same complete request repeatedly:
T0 = full request with 0 examples
T1 = full request with 1 example
T3 = full request with 3 examples
T5 = full request with 5 examples
increment_1 = T1 - T0
increment_3 = T3 - T1
increment_5 = T5 - T3
Record quality next to token usage:
| Variant | Input tokens | Task pass rate | Format pass rate | Edge-case failures | Keep? |
|---|---|---|---|---|---|
| Zero-shot | Measure | Measure | Measure | List failures | Baseline |
| One example | Measure | Measure | Measure | List failures | Only if useful |
| Three examples | Measure | Measure | Measure | List failures | Only if better |
| Five examples | Measure | Measure | Measure | List failures | Only if needed |
The winning variant is the smallest one that passes your release criteria. If five examples use more context but do not fix a new failure, remove them.
This comparison turns few-shot prompt examples into a measurable release decision instead of a stylistic preference.
Choose examples by coverage, not quantity
Three near-duplicate examples usually provide less value than three examples with distinct teaching roles.
A compact set should cover:
- The canonical case: the most common request and the exact output structure.
- A boundary case: empty values, ambiguous language, unusually long input, or a borderline classification.
- A contrast case: two inputs that look similar but require different outputs.
Add a fourth or fifth example only when it represents a recurring production error that the existing set does not teach.
Before adding an example, write its job in one sentence:
This example exists to teach the model how to ________.
If the blank duplicates another example, merge or delete it.
Compress few-shot prompt examples without removing the lesson
The goal is not to make examples cryptic. The goal is to remove tokens that do not teach behavior.
1. Remove repeated instructions
Put shared rules once before the example block. Do not repeat “Return valid JSON” or the full taxonomy inside every demonstration.
2. Shorten inputs while preserving the decision boundary
Keep the phrase, field, or context that changes the answer. Delete decorative background information.
3. Use the real output format
If production requires JSON, show JSON. Do not spend tokens showing a prose explanation that the application will never accept.
4. Keep labels consistent
Stable delimiters make examples easier to scan and reduce accidental variation. Anthropic explicitly recommends consistent structure and uses XML-style tags as one option for separating examples.
5. Remove explanations the model does not need
An example often needs the correct output, not a paragraph explaining why it is correct. Keep rationales only when the reasoning pattern is itself part of the required behavior.
Before and after compression
Verbose:
Example 1
The following is an example of a customer support message. Please analyze the message and determine the correct category from the complete list of categories. You must return a JSON object.
Customer message: I was charged twice for the same monthly subscription and would like the duplicate charge refunded.
Correct answer: The correct category is billing_duplicate_charge because the customer explicitly reports two charges for one subscription.
Compressed:
<example>
Input: Charged twice for one monthly subscription. Refund the duplicate.
Output: {"category":"billing_duplicate_charge"}
</example>
The compressed version preserves the decision boundary and production output shape while removing repeated policy and explanation.
Repeat this compression pass across all few-shot prompt examples, then rerun the same evaluation before accepting the smaller version.
Paste-ready few-shot prompt fixture
Use this fixture to compare example counts. Run the same request with only the <examples> block changed.
Classify the support message into exactly one category:
billing_duplicate_charge, billing_refund_pending, account_login, product_bug.
Return JSON only:
{"category":"category_name"}
<examples>
<example>
Input: Charged twice for one monthly subscription. Refund the duplicate.
Output: {"category":"billing_duplicate_charge"}
</example>
<example>
Input: My refund was approved last week but has not reached my card.
Output: {"category":"billing_refund_pending"}
</example>
<example>
Input: Password reset works, but the new password is rejected at sign-in.
Output: {"category":"account_login"}
</example>
</examples>
Input: The export button creates an empty CSV even though the dashboard shows data.
Output:
Create four variants:
- zero examples,
- only the first example,
- all three examples,
- all three plus two additional examples for real recurring failures.
Run each variant through the exact endpoint and model you plan to use. On TokenTest, compare the endpoint’s usage evidence rather than trusting only a local text estimate. The live product checks include input-token monotonicity, total-token consistency, output reasonableness, cache-token evidence, and thinking/reasoning-token evidence where the endpoint exposes them. TokenTest also states on its homepage that the API key entered for a test is not stored.
For a broader workflow, use the token-aware prompt review process and the production context-window planning guide.
Protect output headroom before adding examples
Few-shot prompt examples compete with the answer for shared capacity. Do not let a larger demonstration block silently reduce the output below what the task requires.
Define the output reserve first:
- maximum expected fields for structured output,
- longest acceptable answer,
- tool-call arguments or intermediate results,
- retry or repair overhead,
- and any provider-specific reasoning or thinking behavior you need to account for.
Then fit examples into what remains. If a new example would consume protected output space, you have four choices:
Treat few-shot prompt examples as a variable budget component, never as capacity that is automatically entitled to the remaining context.
- compress an existing example,
- replace a weaker example,
- retrieve an example dynamically for the current case,
- or increase the approved request budget if the endpoint and economics allow it.
Test languages separately
Do not assume an English example set and its Chinese translation consume the same number of tokens. Tokenization depends on the model and tokenizer, and punctuation, scripts, JSON escaping, and mixed-language content can change the total.
Count and evaluate each localized request separately. The related guide, Why Chinese Prompts Can Use Different Token Budgets Than English Prompts, explains why one shared multiplier is unsafe.
Localization also changes example quality. A literal translation may lose the ambiguity, tone, or category boundary the original example was designed to teach. Revalidate both tokens and behavior.
When to use dynamic examples instead
A static few-shot block is simple, but it sends the same examples for every request. Dynamic selection may be a better fit when:
- the example library is large,
- user requests fall into distinct domains,
- only a few examples are relevant to each request,
- the long tail changes frequently,
- or static examples consume too much of the prompt token budget.
Retrieve a small number of relevant examples, but include retrieval overhead in the same full-request measurement. Dynamic selection can reduce repeated context, yet poor retrieval can introduce irrelevant or contradictory demonstrations.
A practical release checklist
Before shipping a few-shot prompt, confirm:
- [ ] The zero-shot version was measured.
- [ ] Every example has a distinct teaching purpose.
- [ ] The complete request was counted, including tools, schemas, retrieval, and history.
- [ ] Output reserve and safety margin were protected before examples were added.
- [ ] The 0/1/3/5 variants were evaluated on the same test set.
- [ ] The smallest passing variant was selected.
- [ ] English and Chinese versions were counted and evaluated separately.
- [ ] Local estimates were compared with live endpoint-reported usage.
- [ ] Token usage is checked again when the model, endpoint, schema, or examples change.
Final rule: examples must pay rent
Few-shot prompt examples are valuable when they prevent specific failures. They are waste when they repeat the same lesson, preserve unnecessary prose, or take space from real input and useful output.
Start with zero. Add one example to teach the format. Add diverse examples to cover documented failure modes. Measure the entire request every time. Keep the smallest set that passes.
Start a TokenTest endpoint evaluation and compare your zero-, one-, three-, and five-example request shapes before the next prompt change reaches production.