Pinecone Bets RAG Is Obsolete. The Data Disagrees

The company that made retrieval-augmented generation a household term just told its own 800,000 developers to stop doing it. Here is what that means if you are choosing between RAG and a 2 million token context window in 2026.

Your engineering team spent 2024 building a retrieval pipeline. Chunk the docs, embed them, store them in a vector database, retrieve the top matches, stuff them into a prompt. It worked, mostly. Then Gemini shipped a 2 million token context window, Claude and GPT-5.4 hit 1 million, and someone on Slack asked the question everyone is now asking: why not just paste the whole knowledge base in and skip the plumbing?

That question has a real answer now, and it is not the one either side of the debate wants. A 2 million token context window does not replace retrieval-augmented generation. It changes what retrieval is for. And the company that spent four years teaching the industry how to build RAG vs long context pipelines just told the market, in public, that the pattern it popularized is already the bottleneck.

The context window race just hit a new ceiling

By April 2026, five frontier labs had all crossed the same line. Claude Opus 4.6, GPT-5.4, Gemini 3.1 Pro, Qwen 3.6 Plus, and Llama 4 Maverick each shipped a 1 million token context window. Meta pushed further with Llama 4 Scout, advertising 10 million tokens, though independent testers found its usable recall breaks down well short of that number. Google’s Gemini line has sat at the 2 million token mark since early 2026, which is why “2 million token context window” is now the phrase enterprise buyers type into Google before they type anything else.

By June 9, at least 13 models had crossed the 1 million token line, according to a pricing comparison from Morph. What that comparison also revealed is that “1 million tokens” is not one product. It is thirteen different products with wildly different economics.

Source: Morph, June 9, 2026. A 71x spread across the field.

That 71x spread is the first sign that “just use a bigger window” is not a strategy. It is a pricing decision you have not made yet.

Context rot: why bigger windows are not always better

In July 2025, three researchers at the vector database company Chroma published a report that has become the most-cited technical pushback on long-context marketing copy. Kelly Hong, Anton Troynikov, and Jeff Huber tested 18 frontier models, including the GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 families, on tasks specifically designed to hold difficulty constant while varying only input length.

The finding that should worry anyone planning to dump a full knowledge base into a prompt: every single model got less reliable as the input got longer, even on tasks a human would call trivial. And in a twist that inverts a common assumption among RAG engineers, models performed worse on well-organized, logically coherent source documents than on the same content shuffled into random order.

“Models do not use their context uniformly.” Kelly Hong, Anton Troynikov, and Jeff Huber, Chroma Research, “Context Rot: How Increasing Input Tokens Impacts LLM Performance,” July 2025

Chroma is a retrieval infrastructure vendor, so this finding is also commercially convenient for the company publishing it. That is worth disclosing. It does not make the methodology wrong. The 18-model benchmark is open source and independently reproducible, and it lines up with a separate, older finding known as “lost in the middle”: accuracy drops 20 to 30 percentage points when the answer sits in the middle of a long document instead of at the start or end, a pattern first documented by Liu et al. and replicated across model families since.

Put together, these results point to a rule NVIDIA’s own RULER benchmark backs up: the effective, reliable portion of a context window typically runs at 50 to 65% of the number on the marketing page. Some of Chroma’s own findings suggest the real, safe margin for production workloads is tighter still, closer to a quarter or a third of the advertised maximum.

The real cost of going long

Even where accuracy holds up, long-context prompting is not cheap next to modern retrieval. A 2026 arXiv study titled “Long Context vs. RAG for LLMs” ran a direct cost comparison across GPT-5.4-mini and nano on document-grounded question answering. The result: long-context prompting averaged roughly $0.1181 per query, against $0.0045 to $0.0046 for keyword or semantic retrieval. That is a 10x-plus cost gap, and it is the more conservative of the figures floating around; some blog posts cite gaps as high as 1,250x, but those appear to compare different cost baselines and should be treated with skepticism.

Anthropic’s prompt caching cuts input costs by up to 90% and latency by up to 85% on repeated long prompts, which matters more than raw context size for most production bills. The lesson is not “context is expensive.” It is that caching, batching, and retrieval scope are the real levers, and a bigger window without any of those disciplines is the most expensive way to solve the problem.

Pinecone just bet against the category it built

On May 4, Pinecone, the vector database that made RAG a standard pattern for roughly 800,000 developers and 9,000 paying customers, launched Nexus, which it calls a “knowledge engine for agents,” alongside KnowQL, a query language built around six primitives: intent, filter, provenance, output shape, confidence, and latency budget.

Pinecone’s own framing is blunt. It describes retrieval-at-inference, the classic chunk-and-embed pattern the company spent four years teaching the market, as the “ten blue links era of agentic retrieval.” Its argument: agents stuck in retrieve-read-retrieve loops complete only 50 to 60% of tasks and burn 85% of their effort just fetching context, before any actual reasoning happens.

Instead of retrieving raw chunks at query time, Nexus precompiles source data into structured, cited, task-specific artifacts ahead of time, so an agent queries a compiled answer rather than a pile of documents. Harrison Chase, the CEO of LangChain and the person widely credited with popularizing the term “context engineering,” backed the framing on Pinecone’s own launch post.

“Building reliable, long-horizon agents is fundamentally a context engineering problem.” Harrison Chase, CEO, LangChain, on Pinecone’s Nexus launch post, May 4, 2026

Janakiram MSV, the cloud and AI analyst who covers infrastructure shifts for The New Stack, called out just how unusual this is. Most vendors keep selling into a category long after the market has moved past it. Pinecone named the shift itself.

“Pinecone just declared the RAG era over.” Janakiram MSV, The New Stack, “The company that made RAG mainstream is now betting against it,” May 6, 2026

Our read: MSV’s framing is closer to right than Pinecone’s own marketing copy. This is not “RAG is dead.” It is RAG’s naive, retrieve-then-hope form getting replaced by something more deliberate, the same shift Anthropic’s Skills and Cursor’s project rules are pushing at the editor and agent-framework layer. The pattern is not new. The vendor saying it out loud is.

The Subquadratic wildcard: 12 million tokens, unverified

One day after Pinecone’s launch, Miami-based startup Subquadratic emerged from stealth with $29 million in seed funding and a model called SubQ, built on what it calls a Subquadratic Selective Attention architecture. Founded by CEO Justin Dangel and CTO Alexander Whedon, both veterans of Meta, the company claims SubQ’s research version supports a 12 million token context window, roughly 120 books, while scaling compute linearly rather than quadratically with input length.

The headline number, as reported by SiliconANGLE: SubQ scored 95% on the RULER 128K benchmark at about $8 in compute, against 94% accuracy and roughly $2,600 for Claude Opus on the same test, a claimed 300x cost reduction. Backers reportedly include Tinder co-founder Justin Mateen and early investors in Anthropic, OpenAI, Stripe, and Brex.

Treat every one of those numbers as “reported by Subquadratic” until someone outside the company replicates them. As of this writing, no independent benchmarking team has confirmed the 52x attention speedup, the 92.1% needle-in-haystack recall at 12 million tokens, or the roughly 1,000x compute reduction the company claims at full context length. If verified, it would be the largest single jump in usable context the field has seen. If not, it joins a long list of long-context claims that looked revolutionary on launch day and ordinary six months later.

So is RAG dead? The growth data says no

Here is the part the “RAG is dead” headlines tend to skip: RAG-adjacent infrastructure spending is still growing fast, and growth data does not lie the way marketing copy can. Market-sizing firms disagree sharply on the exact dollar figures. Grand View Research puts the market at $1.2 billion in 2024, growing to $11 billion by 2030 at a 49.1% compound annual growth rate. Precedence Research estimates $2.76 billion in 2026 climbing to $67.42 billion by 2034. MarketsandMarkets lands in between, at $1.94 billion in 2025 growing to $9.86 billion by 2030. Cite one firm at a time, since the numbers do not reconcile with each other, but the direction across all three is the same: a technology genuinely on its way out does not post 38 to 49% annual growth.

Production engineers writing on DEV Community made the practical case bluntly: no context window, however large, holds an enterprise knowledge base running to millions of documents. A single 1 million token Claude Sonnet-class prompt runs roughly $3 at list pricing, and that does not scale to production query volumes the way retrieval does. Their position is that RAG’s continued growth is itself the strongest evidence against the “dead technology” framing, not despite the long-context hype but because of what enterprises are actually shipping underneath it.

What this means for your stack

Stop treating this as RAG versus long context. Treat it as a context budget you have to manage regardless of which technique you use.

• Cap your assumptions at 25 to 30% of the advertised window. That is roughly what Chroma’s own findings suggest is the safe, reliable slice of any long-context claim, sticker number aside.
• Pair retrieval with compaction. For long agent sessions, summarization and compaction loops matter more than raw window size, because irrelevant content is what causes context rot, not length alone.
• Do not rip out retrieval infrastructure on the assumption long context replaces it. Teams that did this in 2024 and 2025 are the ones now eating the 10x-plus cost premium documented above.
• Watch where vendor R&D is actually pointed, not where the marketing copy points. Pinecone’s own pivot from raw retrieval toward precompiled, agent-queryable artifacts is a better signal than any single benchmark chart.
• Evaluate new entrants before migrating production workloads. Subquadratic’s numbers are compelling on paper and unverified in practice. Run your own evals on your own data first.

One more thing regulated industries should not skip: RAG’s retrieval logs double as an audit trail. Raw long-context prompting does not produce one by default. In finance, healthcare, or legal workflows, that gap is not academic. It is a compliance requirement waiting to surface during an audit, usually at the worst possible time.

Frequently asked questions

Does a bigger context window replace RAG?

Rarely. Long context reduces the need for aggressive retrieval on smaller, bounded corpora, but no window, even 12 million tokens, holds an enterprise knowledge base with millions of documents. Long-context prompting also runs roughly 10x or more expensive per query than modern retrieval in controlled 2026 benchmarks.

What is “context rot”?

Context rot is measurable performance degradation as an LLM’s input length grows, even on simple tasks. Chroma Research tested 18 frontier models in 2025 and found every one degraded with length, with logically coherent documents sometimes hurting performance more than shuffled ones.

What causes the “lost in the middle” problem?

Models attend most reliably to information at the very start and end of their context window. Liu et al.’s benchmark found accuracy drops 20 to 30 percentage points when the answer sits mid-context, a pattern replicated across GPT, Claude, and other model families since.

How much does a 1 million token prompt cost?

It depends heavily on the model. As of June 2026, filling a 1 million token window ranges from about $0.14 on DeepSeek V4 Flash to $10.00 on Claude Fable 5, a 71x spread, before caching discounts are factored in.

Is RAG still worth building in 2026?

Yes, for most production systems with large, dynamic, or compliance-sensitive corpora. RAG-related infrastructure spend kept growing at 38 to 49% CAGR across multiple market estimates even as long-context windows expanded, and 2026 is shaping up to be a hybrid-architecture year rather than a winner-take-all contest.

The bottom line

Nothing here says long context is a bad bet or that RAG is finished. What the evidence actually supports is narrower and more useful: raw context length is not the same thing as usable context, cost scales against you faster than accuracy does, and the vendor that built the RAG category is now telling the market to build the next layer up, not to abandon retrieval altogether.

Watch three things over the next 6 to 18 months. First, whether independent labs confirm any of Subquadratic’s numbers, since that would be the first real architectural break from quadratic attention costs. Second, whether Pinecone’s Nexus and KnowQL numbers hold up in production the way they did in Pinecone’s own benchmarks. Third, whether “context engineering,” the discipline of deliberately curating what enters a model’s window regardless of technique, becomes a formal job function the way “prompt engineering” did in 2023.

The teams that win this cycle will not be the ones who pick a side in the RAG-versus-context debate. They will be the ones who stopped treating context size as a proxy for context quality months before everyone else did.

Want the next infrastructure shift in your inbox before it hits the front page? Subscribe to The Neural Loop at neuralwired.com/newsletter.