中/EN
Frontline Hotspot
Frontline Hotspot

Context Windows Hit the Million-Token Mark: Real Need or Arms Race

Gemini 1.5 pushed context to 1M tokens in 2024; by mid-2026 "million-level" is standard for top models, and the Chinese camp is pushing tens of millions. What long context solves, what it doesn't (lost in the middle), cost, and "needle in a haystack"-plus three tips for choosing.

Published July 27, 20264 min read

A Million Tokens Becomes Standard

When Gemini 1.5 pushed the context window to 1 million tokens in 2024, the industry treated it as news. By mid-2026, "million-level" is already standard for top-tier models: Gemini, GPT, and Claude, plus China's Qwen, GLM, and Kimi all sell on long context, and the Chinese camp has even begun exploring the "tens of millions of tokens" range. The arms race is visibly accelerating.

What Long Context Solves-and Doesn't

Solves: you can feed in a whole book, an entire codebase, or dozens of contracts at once-no more manual chunking, retrieval, and context-stitching. Many scenarios that used to need RAG are directly covered.

Doesn't solve: "fits in the window" doesn't mean "finds it and uses it correctly." Long context widely suffers from "lost in the middle"-information placed in the middle is more likely to be ignored; the more tokens, the more scattered the attention, and key details still slip through.

Cost and "Finding a Needle in a Haystack"

Long context isn't cheap. Stuffing 1 million tokens into a single request costs dozens to hundreds of times more than a short conversation. And the industry benchmark NeedleInAHaystack shows that recall for most models declines as context length grows; the newer RULER benchmark further proves that "nominal length" and "effective length" are often far apart.

"Can fit it in" and "can use it" are two different things.

How Regular People Should Choose

  1. Use long context on demand: don't enable a million tokens for daily chat-it's expensive and often less stable; reserve it for cases that genuinely need it-whole codebases, long documents, batch contracts.
  2. RAG isn't dead: for very large-scale scenarios needing precise recall and traceability, retrieval-augmented generation is still more accurate and cheaper than stuffing the full context.
  3. Watch "effective context," not "nominal context": for the numbers vendors tout, check independent benchmarks (NIAH, RULER)-don't just trust the marketing page.

In one line: context length is like memory-the bigger the better, but what matters is "effective addressing." Piling on numbers is marketing; competing on recall is real skill.

References

This article is AI-assisted and human-edited. Last updated: 2026-07-27

FAQ

Is a bigger context window always better?
Not necessarily. "Fits in the window" doesn't mean "finds and uses it accurately." Long context suffers from "lost in the middle"-the more tokens, the more scattered the attention, and key details still slip through. Watch "effective context," not "nominal context."
With a million tokens, do I still need RAG?
Yes. For very large-scale scenarios needing precise recall and traceability, RAG is more accurate and cheaper than stuffing the full context. Long context suits feeding in a whole book / codebase at once; RAG suits precise retrieval.
Is long context expensive?
Yes. Stuffing 1M tokens into a single request costs dozens to hundreds of times more than a short conversation. Don't enable a million tokens for daily chat; reserve it for cases that genuinely need it-whole codebases, long docs, batch contracts.

Related