Context Compression: How to Stop Agent Context Bloat

If you have ever watched an agent handle a long task, you have probably seen this movie: the first twenty minutes are fast and sharp, then somewhere around the one-hour mark it starts answering the wrong question. A single code search dumps tens of thousands of tokens of raw log into the context, the API bill climbs by the hour, and by the end the agent has forgotten what it was originally asked to do. The root cause is usually not a dumber model. It is context bloat: conversation history, tool outputs, and the wreckage of failed retries pile up layer after layer inside the window, and the model drags the entire burden through every single reasoning step. Context compression is the fix: move out what you no longer need, thin out what you must keep, and let the agent work in a clean context every step.

This SOP walks the full pipeline in six steps: decide whether you actually need compression, wire context-mode into your harness, configure thresholds and routing, implement layered compression, and verify the result with a before-and-after token ledger. The tool at the center is mksglu/context-mode (npm package name context-mode, TypeScript, 27,378 stars per the GitHub API snapshot on Oct 13). Repo: github.com/mksglu/context-mode

One boundary up front, because it matters: context-mode is licensed under the Elastic License 2.0 (ELv2), which is not a traditional open-source license. The core restriction in the LICENSE file is a single clause: you may not provide the software to third parties as a hosted or managed service. Personal and internal team use is unrestricted. Running it to speed up your own team's agents is completely fine; wrapping it into a SaaS you sell to others is explicitly off the table.

Step 1: Does your agent actually need compression?

Not every agent needs this. A three-to-five-step task that never fills a third of the window does not justify the machinery. Run through this symptom checklist first:

1. Slowing down. Print the input token count for every step of a representative long run. If the curve grows linearly with step count — step 30 carrying ten times the input of step 5 — the model is paying inference time on redundant history at every turn.
2. Getting expensive. Input tokens dominate the bill. In agent workloads, cumulative input tokens routinely exceed output tokens by an order of magnitude, because the entire history gets re-read on every call.
3. Getting fuzzy. Late in the run it starts ignoring early constraints (you asked for TypeScript, it rewrites in JS), repeats explorations that already failed, or drifts away from the original goal.
4. Hitting the ceiling. The run dies with a context-length error halfway through, or the harness silently truncates early turns — truncating exactly your critical constraints.

The diagnostic is cheap: turn on logging for prompt length and token usage per step (every major API returns a usage field), then check two things — the shape of the token curve, and what fraction of step N's input is raw tool output. If raw tool output is over half the input and the curve is climbing steeply, compression is clearly indicated. If tasks finish in under ten steps and the window never fills past thirty percent, leave it alone.

Step 2: Install context-mode and wire it into your harness

context-mode ships as an npm package and connects through two channels: MCP and hooks. The project was created in February 2026 with the latest push on October 12, so it is actively maintained. On traction: the README badge claims a Hacker News number-one spot with 570+ points, plus headline figures of 98% reduction for sandboxed tool outputs and enforced routing across 17 platforms via MCP plus hooks. These are the vendor's own claims, not independently verified — treat them as directional.

Install via npm:

npm install context-mode

Channel one is MCP: register context-mode as an MCP server so the agent can call its compression and retrieval capabilities over the standard protocol (check the repo README for exact command arguments):

{
  "mcpServers": {
    "context-mode": {
      "command": "npx",
      "args": ["context-mode", "serve"]
    }
  }
}

Channel two is hooks: attach context-mode to your harness's tool-call hooks so that bulky tool outputs get intercepted before they ever enter the context — written to disk and replaced with a file reference plus a summary. This is where the payoff lives: the model no longer swallows tens of thousands of tokens of raw log, and instead receives a summary with an on-demand reference.

Checkpoint: after registration, run a task with a large output (read a big file, run a full test suite) and verify the hook fires — the tool output is replaced, a file appears on disk, and the agent can still pull back what it needs through the reference. If any link in that chain breaks, fix the pipe before configuring anything else.

Step 3: Configure thresholds and routing — when to compress, and what

Compression should not be a cliff where everything gets wiped at 100% capacity; it should degrade in stages. Academia offers a ready reference: the OPENDEV paper's five-stage progressive compression scheme (Bui 2026, as described in the paper) — warn at 70% of the budget, apply observation masking at 80%, fast pruning at 85%, aggressive masking at 90%, and full compression at 99%. The point is to give the agent a corresponding action at each pressure level instead of one brutal cut near the limit.

Vendors are productizing the same idea. According to a cloudpro article dated October 12, 2026, OpenAI's Responses API now offers compaction, with example configs showing a compact_threshold field set to 200000; Microsoft Agent Framework ships summarization, removal, and folding strategies. The direction is the same everywhere: thresholds plus strategies, becoming standard agent infrastructure.

Beyond thresholds, routing matters more: different content types should flow through different compression pipelines.

• Bulky tool outputs go through offload (write to disk, replace with reference plus summary);
• Conversation history goes through summarize;
• Dead exploratory turns (failed attempts, already-resolved errors) go through prune — just delete them;
• Entities never enter the compression pipeline: order numbers, amounts, file paths, hard constraints — anything that breaks the task if garbled — stay verbatim or get stored independently. This is the entity-solidification rule from a set of seven engineering heuristics compiled in a 51cto blog post (Chinese engineering community source); several others come up below.

One easily missed cost item: compression interacts with prompt caching. Anthropic's context engineering guide lists compaction as a core strategy for long-horizon tasks, and notes that you should use cache_control breakpoints to preserve prefix caching across compactions (as cited by agentpatterns) — the moment your context prefix changes, every cached prefix token is invalidated, and you save on tokens only to pay for cache rebuilds. Design cadence and cache breakpoints together.

Step 4: Implement layered compression — offload, summarize, prune

For method, borrow the layered scheme from LangChain Deep Agents (as analyzed on agentpatterns.ai): Tier 1 offloads large tool outputs to disk, replaced by a file reference plus a summary; Tier 2 summarizes conversation history, keeping four elements — goal, state, constraints, next step — and dropping exploratory turns and resolved errors. Translate that into three executable actions:

Offload (Tier 1). When a tool output exceeds your size threshold, write it to disk and leave only "summary plus reference" in context. This fits self-contained large blobs: test logs, code search results, scraped pages. When the agent needs detail later, it reads back through the reference — only the part it needs.

Summarize (Tier 2). When history crosses your threshold, compress it into a summary that must contain the four elements: the task goal, the current state, the key constraints, and the next planned step. What you drop matters as much as what you keep. Exploratory turns (tried three paths, two dead ends) compress to one conclusion sentence. Resolved errors keep the conclusion, not the stack trace. All entity content follows the Step 3 rule: verbatim or stored independently.

Prune (maps to OPENDEV's 85% fast-pruning stage). Deterministic deletion that requires no semantic understanding: stale observation masks, old outputs superseded by newer versions, pure duplicates. Cheapest to execute, so do it first.

Is the layering worth it? A widely cited empirical figure (research cited by agentpatterns, secondhand): trimming context down to the last five tool-call pairs plus summaries produced a 91.6% task completion rate, versus 71% for the agent keeping full context. Compression did not just cut cost — it improved completion, because the model stopped hunting for signal in noise. Secondhand figures, directional only — do not assume copying the recipe reproduces them.

Two engineering details. First, the summarization and compression passes themselves can run on a small model — the 51cto engineering writeup argues 1.5B to 3B models suffice, so there is no reason to pay your main agent's frontier-model rates for summaries. Second, every layer needs its own checkpoint: did the offloaded file actually land on disk? Does the summary contain all four elements? Did anything you pruned get referenced again by a later step? Walk all three questions every time.

Step 5: Measure — the before-and-after token ledger

Without measurement, compression is superstition. Before changing anything, record a baseline: pick three to five representative long tasks and log, for each, total input tokens, total output tokens, end-to-end latency, and completion quality (human score or an eval script). Wire in context-mode, configure thresholds and routing, rerun the same set, and reconcile.

Token usage comes straight from the usage field on API responses; aggregate it in your harness:

ledger = {}

def record(task_id: str, step: int, usage: dict):
    entry = ledger.setdefault(task_id, {"input": 0, "output": 0, "steps": 0})
    entry["input"] += usage["input_tokens"]     # track cached_tokens separately
    entry["output"] += usage["output_tokens"]
    entry["steps"] += 1

def report(ledger: dict):
    for task_id, e in ledger.items():
        print(f"{task_id}: input={e['input']} output={e['output']} steps={e['steps']}")

Reconcile on four ratios. How much did total input tokens drop (the direct gain)? How much did wall-clock time drop? Did output tokens grow abnormally (if so, the model may be re-reading offloaded files too often — improve summary quality)? And did the completion rate fall? That last one is the floor: if quality drops, every other gain is worthless.

The README's advertised 98% reduction is a marketing figure. Your actual savings depend on how much of your workload is tool output and where you set thresholds. Trust your own ledger; other people's numbers are for expectation management only.

Step 6: Three pitfalls to avoid

Pitfall one: silent loss. Aggressive summarization discards a constraint that looked unimportant at the time and turns out to be critical forty steps later; the task drifts off course with no error anywhere in the logs — swallowed by a summary, not deleted by a bug. Countermeasures: prefer offload over summarize — offloaded content stays readable forever; solidify entities so key numbers and constraints bypass the pipeline; log every compression so you can trace back after a failure. All three come from the seven-heuristics list cited earlier (51cto source).

Pitfall two: compressing too early. If thresholds are set too low, the agent gets compressed while it is still building its understanding of the task, and the goal drifts — it navigates the whole run on a distorted map. Remember Anthropic's guidance (cited by agentpatterns): start from maximum recall and converge downward, never the reverse. Configure your first version with deliberately generous thresholds, confirm quality is intact, then step down one tier at a time, rerunning the regression set each time.

Pitfall three: summary drift. A summary gets summarized, which gets summarized again, and the error compounds until the agent is faithfully executing a mission briefing that has drifted far from reality. Countermeasures: keep the task goal verbatim and never summarize it; store key decisions independently rather than inside the summary; cap the number of summarization layers and rebuild from the source beyond that; run the Step 5 regression set regularly, and when completion rates slip, check summary quality first.

Context compression is not exotic technology. It is housekeeping for the agent's memory: big items go to storage, the running log gets summarized, garbage gets pruned, and the numbers that matter get copied onto a sticky note next to the monitor. The tool builds the pipeline; thresholds and routing are your judgment calls; the ledger is the only referee. Start with the long task that has been getting slower every week — print the per-step token counts and look at them.

Related reading

• Agent long-term memory SOP: the other half of the problem
• Agentic caching costs compared: counting the token bill
• Agent observability SOP: measure what compression does

FAQ

Q1: Models already have million-token windows. Is compression still necessary? A: Yes. A big window is not a free window — input tokens are billed in full on every call, and long contexts dilute the model's attention, which is exactly where the fuzziness comes from. Compression addresses both, regardless of window size.

Q2: Can I use context-mode commercially? A: Yes, for internal use. The ELv2 license permits personal and internal team use; the single hard restriction is that you may not provide it to third parties as a hosted or managed service — so do not wrap it into a SaaS. Read the LICENSE file yourself before going live.

Q3: Does compression lose information? What if it does? A: It can, and that is the biggest risk, especially in the summarize layer. Three safeguards cover most of it: offload to disk before summarizing so content stays retrievable, keep entities out of the pipeline, and log every compression so you can trace back after a failure.

Q4: What thresholds should I set? A: Use the OPENDEV five-stage ladder as a template (70% warning, 80% masking, 85% pruning, 90% aggressive masking, 99% full compression), start with everything set high to protect quality, and step down gradually. Anthropic's guidance is the directional rule: begin from maximum recall and converge, never open with an aggressive cut.

Q5: How do I verify compression did not hurt task quality? A: Keep a fixed regression set of representative long tasks and rerun it after every threshold or routing change, comparing completion rates and output quality. Token savings without quality measurement is self-deception — a 90% token reduction paired with broken tasks is worse than no compression.