Let us draw the boundary first. This article does not compare which model is "smarter" or "has a longer context window". The site already has a reasoning image-model comparison and an open-vs-closed image cost account, both following the same "count the money, not the capability" discipline. This piece migrates that method into the Agent setting.
Here is the exact scope.
What we count: the cumulative token fees an Agent task pays to the inference service from start to finish, plus the engineering cost invested to bring that fee down. We care about "total cost = direct inference fee + amortized rework + extra spend on failed retries".
What we do not count: model accuracy, tool-call success rate, end-to-end latency, except where latency directly changes the cost structure. These are treated as externally given. We are stubborn on exactly one dimension: how the context gets billed, and how it gets billed repeatedly.
Data and assumptions: this article fabricates no unit price. Wherever a concrete price is involved, we use symbols, for example input unit price as P_in, output as P_out, cache-hit price as P_cache, or we say "refer to the official pricing page". Every magnitude claim is labeled "engineering estimate / illustrative basis", not a measured result.
Why this strictness? An Agent context bill is easy to underestimate. A chat request that looks like 8K context can swell into tens of K of backfilled trajectory in an Agent. We have seen teams size their inference budget from a single chat call and then watch it triple once tool loops are active. Underestimate it and the budget collapses.
Problem Definition: Agent and Chat Have Different Cost Structures
Chat context grows linearly. The user says one line, the model replies, history appends, each round only looks at the delta:
C_chat(t) ≈ P_in · (H_base + t·Δ_in) + P_out · (t·Δ_out)
where H_base is the system prompt and Δ the single-round delta. Simple and predictable.
Agent context is compound accumulation, stacking at least four layers:
- Resident system prompt: re-carried and paid full price every round (unless cached).
- Tool-call result backfill: each tool call returns content stuffed back into context, length diverging with the task.
- Multi-round trajectory accumulation: history plus tool I/O plus reasoning, monotonically increasing.
- Failure-retry replay: one tool error often forces resending the preceding N rounds to the model.
Single-round cost formula:
C_agent(turn_i) ≈ P_in · (S_sys + Σ_{k≤i} T_k + O_i) · (1 − h·c) + P_out · R_i
where S_sys is the resident prefix, T_k round k's tool output, O_i this round's input, R_i this round's output, h the cache hit rate, c the discount after a hit (0<c<1).
This exposes two sore spots: S_sys is paid again every round (multiplied by rounds), and ΣT_k grows linearly with no upper bound. Chat's "linear" is gentle; Agent's "compound" carries a multiplier. That multiplier is where our five levers pry money out. We evaluate each on benefit, cost, risk, and where it pays.
The Five Levers
1. Prefix Caching / Prompt Caching
The highest-ROI "soft lever". The system prompt, few-shot examples, even early trajectory, if unchanged, get their KV cached; later requests pay the cheap cache-read price instead of full input.
Price gap (illustrative, official page for specifics): a miss bills input at P_in; a hit at P_cache, usually far smaller. But writing the cache may incur a one-time write or storage fee, and the cache invalidates if the prefix changes by even one character or the gap exceeds TTL.
Engineering estimate: for Agents with a resident prompt and repeated trajectory (fixed-flow bots), hit rate can be high and caching drives S_sys repetition near zero. For divergent, every-round-different Agents (open coding assistants), hit rate is low and benefit limited. Applicability: the larger S_sys and the more repetition, the more valuable; otherwise shorten S_sys first.
2. KV Cache Compression and Sparse Attention
The hard lever of the technical route, and the star of this batch. DeepSeek V4.1 Flash's stated direction is precisely "significantly compress the KV Cache to lower Agent-scenario cost". The only facts we cite: 552B-parameter MoE, asymmetric Causal-Encoder-Decoder, 8B input / 16B output activation, native multimodal, KV Cache significantly compressed, API name changed to deepseek-flash, Tencent WorkBuddy/CodeBuddy and OpenCode integrated (source: DeepSeek official account, via ai-bot.cn 9/10).
"Compress the KV Cache" is an official qualitative statement; we write no fabricated ratio. The principle: KV Cache consumes VRAM and indirectly sets the marginal cost of long context; compressing it fits a longer trajectory on the same hardware, or lowers unit cost at the same length.
The operator base for sparse attention connects to the DSA TopK kernel in this batch (see /en/posts/deepseek-deepselect-resource). But sparse is not free: sparsified positions may lose signal, a risk for tasks needing to "look back" at long trajectories. Applicable to long-trajectory, high-concurrency Agents tolerating minor loss; for short tasks the gain may not cover implementation cost. For integration and cost behavior see /en/posts/deepseek-v4-1-flash-open-source-hotspot and /en/posts/deepseek-v4-1-flash-integration-sop.
3. Context Compression / Summary Folding
The "lossy" lever. Past a threshold, a model summarizes preceding rounds into one paragraph, replacing the original long text. Benefit is explicit: ΣT_k is truncated, input tokens drop.
Cost is explicit too: information loss (summaries swallow details, troubleshooting finds "that key error got folded away"); debugging difficulty (hard to tell model failure from summary corruption); extra cost (the summary itself calls a model once). Engineering estimate: threshold too high saves nothing, too low loses too much. Use "reproducible regression tests pass" as the safety criterion.
4. Tool Output Trimming and Structured Return
The highest-ROI engineering lever, yet most ignored. Whatever the tool returns, the model must read. If a tool returns 5KB of JSON while the model needs two fields, the rest is pure waste.
Practice: return only fields the model needs (structured, minimal); truncate, paginate, or summarize over-long returns and pull again on demand; use a schema to constrain format. It barely loses information (deleting redundancy), costs only interface changes, yet chops a large chunk off O_i and T_k. For many teams its ROI beats any fancy caching or compression.
5. Trajectory Replay vs Incremental Commit
The "retry cost multiplier" lever. Many frameworks, on error, resend the entire trajectory (including prior tool outputs) to retry. A 30-round trajectory means one retry pays 30 rounds again.
Incremental commit solidifies confirmed results into external state (file, database); on retry only "this step + necessary context" is resent, no full replay. Cost: the framework must support state externalization and breakpoints, a heavier change. Engineering estimate: the higher the retry rate (jitter, unstable tools), the larger the multiplier benefit. A 20%-retry, 30-round Agent paying full replay can see retries form a significant cost share; incremental commit removes that duplication.
Comparison Tables
Table 1: five levers on benefit magnitude, implementation cost, risk, scenario (prices on official page; engineering estimate / illustrative):
| Lever | Benefit (illustrative) | Impl cost | Main risk | Applicable scenario |
|---|---|---|---|---|
| Prefix caching | High (prefix cost to a fraction) | Low (toggle + TTL) | Hit rate fragile; one char invalidates | Large prompt, high repetition |
| KV Cache compression / sparse | High (lower long-trajectory cost) | High (model/framework) | Sparse may lose signal | Long trajectory, high concurrency |
| Context compression / folding | Medium-high (past threshold) | Medium (policy + tests) | Loss, hard debug | Very long, fault-tolerant tasks |
| Tool output trimming | High (near lossless cut) | Low (interface/schema) | Map needed fields | Almost every Agent |
| Replay to incremental commit | Medium-high (retry multiplier) | Medium-high (state) | Heavy change, bugs | High retry, long unstable |
Table 2: three Agent scenarios under strategies, illustrative cost (P_in, P_out, P_cache; official page for specifics):
| Scenario | No optimization | Caching only | Caching + trim + fold | All levers |
|---|---|---|---|---|
| 10-round tool task | High (S_sys + full per round) | Medium (saves S_sys) | Lower (cuts redundancy) | Low (no full replay) |
| Long coding Agent | Very high (unbounded) | Medium-high | Medium (fold controls) | Medium-low |
| Batch offline (10k/day) | Extreme (volume x price) | Low (hit amortizes) | Low (scale effect) | Lowest (stacked) |
"High / medium / low" are illustrative magnitudes, not measured amounts; plug your own P_in / P_out / P_cache and hit rate into the formula to recompute.
Actionable Conclusions by Scale
Individual developer (a few to dozens of runs/day):
- Tool-output trimming first: zero cost, near lossless, immediate.
- Then prefix caching: config change, fast payback if prompt is large.
- Context compression last: during debugging, losing info is annoying.
Small team (hundreds/day):
- Prefix caching plus trimming together, highest ROI.
- Add compression with regression tests for quality.
- If retries dominate cost, consider incremental commit.
Batch offline (tens of thousands/day):
- Stack all levers; volume turns savings into real money.
- Watch cache hit rate and tool-output length, the decisive variables.
- Prefer KV-Cache-compression models (integrated deepseek-flash; see hotspot and SOP posts), whose advantage amplifies under long-trajectory concurrency.
Short version: small trims first, medium leans on caching, large goes full stack.
Cold Thinking: Reading a Vendor's "Cost Down X%"
When a vendor says "cost reduced X%", assume it is the optimal value under a specific workload, not a guarantee for yours.
Three variables decide your bill, and they are yours alone: cache hit rate (how repetitive, does the prompt change, gaps exceed TTL?); context distribution (how long, steady or hundreds of rounds?); tool-output length (lean schema or whole-page JSON?). No vendor measures these for you.
Build your own metrics: instrument the framework to record each round's input/output tokens, hit flag, retry count, tool-output volume, then aggregate a per-task context cost distribution. Only with it do you know which lever to pry first, instead of trusting launch numbers.
Do not mythologize any single technique. "Compress the KV Cache" is a direction, but sparse loses signal; caching is sweet but fragile; compression saves money but traps debugging. Engineering is a combination punch, picked by your scale.
A fixed cadence makes it manageable: replay one representative task set per week under a fixed budget, record the p50 and p95 context cost per task, and diff them after any model, framework or prompt change. Treat a p95 regression as a bug rather than noise, because it usually traces back to a single runaway tool output or a silently broken cache. That turns cost from a monthly surprise into a number you can actually steer.
For long-context capability boundaries and earlier cost talks, see /en/posts/deepseek-v4-flash-hotspot and /en/posts/deepseek-v4-flash-codex-benchmark-review; the image "count the money" treatment is in this batch's /en/posts/open-vs-closed-image-model-review.
FAQ
Q1: What is the most common reason a prefix-cache hit fails?
A1: The system prompt or resident prefix changed between requests (even an extra space or version string), or the gap exceeded the cache TTL. Troubleshoot by confirming byte-level identical prefix, then checking TTL.
Q2: Does KV Cache compression make the Agent dumber?
A2: Officially only the qualitative "significantly compress the KV Cache to lower cost" is given, no ratio. Sparsification may lose signal, a risk for look-back tasks, usually tolerable. Whether it hurts yours must be judged by your own regression tests, not generalized.
Q3: What threshold for context compression is good?
A3: No universal value. Use "reproducible regression tests pass" as the criterion: start conservative, tighten until quality regresses, then step back one notch. Never guess a number.
Q4: Tool-output trimming sounds crude, is it worth it?
A4: Yes, often the highest-ROI step. Near lossless (deleting redundancy), only interface or schema changes, yet it chops a large chunk off every round's input. Many teams add caching and compression but forget this, bailing from a leaking bucket.
Q5: Should a small team take on incremental commit?
A5: Look at retry rate. Stable tools and low retries make full replay cheap, not worth state externalization; high retries and long trajectories make repeated billing glaring, and only then does incremental commit pay off. Instrument first, then decide.