Anthropic, OpenAI, and Google spent 2025 taking chain-of-thought reasoning away from you. Not the answer, the answer still shows up. The reasoning that got the model there now arrives as an encrypted block your client has to carry between requests and can't read. The pitch was safety and intellectual property: hide the thinking, and competitors can't distill it, adversaries can't weaponize it, and nobody sees a model reason its way toward something it correctly refuses to say out loud.
In July 2026, a team from MATS Research, the ELLIS Institute Tübingen, and the Max Planck Institute for Intelligent Systems tested the one assumption nobody had: whether a vendor's own cheap model would decrypt its expensive sibling's reasoning if you just handed it the block and asked. It would. Every time, across all three vendors, without a single exploit against the frontier model itself.
Why vendors started hiding the reasoning
The decision to encrypt chain-of-thought wasn't defensive theater. It answered two specific threats that had already cost real money and real trust.
The first was distillation. A frontier model's raw reasoning trace is a far denser training signal than its final answer: it exposes the intermediate steps, the false starts, the exact decomposition of a hard problem. Reasoning traces are worth more to a competitor than any output the model produces, because they teach a cheaper model how to think, not just what to say. Anthropic's own disclosure names the scale of the problem directly: roughly 24,000 fraudulent accounts and more than 16 million exchanges aimed at extracting Claude's reasoning, coding, and agentic capabilities, attributed to DeepSeek, Moonshot AI, and MiniMax. OpenAI made a parallel accusation against DeepSeek to the House Select Committee on China in early 2026, alleging systematic harvesting of ChatGPT outputs to train R1.
The second threat was subtler. AI safety researchers had spent 2025 arguing that a model's chain-of-thought is a rare monitoring opportunity: because models reason in natural language, an observer can sometimes catch intent to misbehave before the final answer ships. Korbak et al.'s 2025 paper on CoT monitorability called this "a new and fragile opportunity" and warned it would erode under the wrong training pressure. Vendors read that fragility both ways: as a reason to protect the channel, and as a reason to stop showing it to users who might see reasoning that looks bad even when the final answer is fine.
Encrypting the block solved both problems on paper. Distillation requires readable reasoning; ciphertext isn't readable. Exposure requires a visible channel; an opaque block isn't visible. Neither assumption survived contact with a second, cheaper model from the same vendor.
Two API calls
The mechanism is almost insultingly simple once you see it, which is exactly why it worked against three separate vendors at once.
Every major provider now returns reasoning as a signed, encrypted envelope: a base64 block containing the ciphertext, a model identifier, and an authentication tag, sent back to the client so the provider doesn't have to store conversation state server-side. The client is required to pass this block back on the next turn to maintain continuity. That's the entire design constraint that creates the vulnerability: the block has to be portable to be stateless, and portable turned out to mean portable everywhere, not just back to where it came from.
The researchers capture an encrypted reasoning block produced by a frontier model, then send only that block's signature to a weaker, less-safeguarded sibling from the same provider (Haiku 4.5 standing in for Opus 4.8, GPT-5.6 Luna standing in for GPT-5.6 Sol) with a two-line instruction: continue the conversation and transcribe the attached reasoning verbatim. The weak model complies because it has no anti-distillation training the frontier model does, and it has no way to know the block wasn't meant for it. The frontier model is never queried, never jailbroken, never touched again after the first call.
The paper doesn't just claim this works, it measures how well. Across 120 Codeforces problems and all three vendors, the token count of the decoded reasoning tracks the API's own reported hidden-thinking token count almost exactly.
The attack is more scalable than jailbreaking a frontier model directly for one reason: the attacker never has to beat the frontier model's own alignment. They only have to beat its cheapest sibling's, which was never trained to refuse this in the first place.
The IP defense that made the leak worse
Here's the distinction the paper actually proves, and it's not "encryption is theater." It's narrower and more damaging: hiding a channel and protecting a channel are different properties, and providers built the first while marketing the second.
A protected channel is bound to its context: this session, this user, this model, and nothing else can decrypt or replay it. A hidden channel is just opaque to casual reading, with the actual cryptographic scope left wherever the implementation happened to land. The paper's compatibility table shows what that gap looks like in practice: reasoning blocks from Claude Opus 4.8 replay cleanly into Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Haiku 4.5. GPT-5.6's traces replay into every earlier GPT model. Gemini's traces replay across the entire family. None of the three providers scoped the encryption to the session, user, or model that generated it, which means a block that looks locked from outside is, functionally, a global key with better packaging.
The scale this enables is what turns an architecture flaw into a live incident. The researchers scraped 6,708 publicly available agent trajectories from GitHub and Hugging Face, the kind of session logs developers publish for reproducibility, and ran the same decode pipeline against every signed block they contained. That yielded 315,320 reconstructed reasoning traces, and inside them: 367 PII artifacts and 182 credentials, including 62 API keys, 33 passwords, and 24 access tokens, recovered from genuine (non-benchmark) user sessions alone.
The detail that should worry a technical buyer most isn't the total. It's the subset: of 704 privacy artifacts recovered from genuine user sessions, 64 existed only inside the encrypted block and nowhere in the visible chat history. Those users sanitized their transcripts before publishing them. They deleted the secret from the part of the conversation they could read, and had no way to know a copy survived in the part they couldn't. That's not a monitoring gap you can staff your way around. It's a property of the architecture: the encryption that was supposed to be the safety boundary is also the only place a user can't audit.
The three-front exposure this creates
Distillation theft and credential leakage are the two failure modes the paper spends the most space on, but it documents two more, and together they describe a single underlying problem: none of standard AI monitoring looks inside an encrypted reasoning block, because until this paper, nobody expected to need to.
The third failure mode is jailbreaking through the back channel. Models are trained to withhold harmful content from their visible output, but not necessarily to avoid reasoning through harmful topics internally: doing so risks degrading the CoT monitoring safety researchers actually want to keep. The paper demonstrates this directly: they paraphrase a HarmBench prompt asking Opus 4.8 to reason about which cars are easiest to steal, capture the encrypted block, and decode it with Haiku 4.5. The visible answer stays benign, generic advice about immobilizers. The decoded reasoning names specific vulnerable makes, model years, and defeat techniques the visible answer never surfaces.
The fourth is prompt injection with no visible payload at all. Because reasoning blocks are replayable across sessions, an attacker can plant a malicious instruction inside a block, publish it as part of a shared trace, and wait for a victim to continue that session. The paper's proof-of-concept plants an instruction to exfiltrate a file to an external server inside an otherwise unremarkable reasoning block, then continues it with an unrelated request. The downstream model follows the injected instruction and uploads the file, with nothing in the visible conversation that a monitor watching the plaintext would ever flag.
None of this is hypothetical risk appetite. OWASP already ranks prompt injection as the top AI security risk heading into 2026, and Cisco's State of AI Security report found 83% of organizations planning to deploy agentic AI against just 29% who feel ready to do so securely. That gap was already the headline risk before anyone knew the reasoning channel itself was a fifth attack surface.
What this changes about evaluating an AI vendor
"The provider encrypts the reasoning" has functioned as a checkbox in vendor security reviews for a year. This paper is the evidence that the checkbox was never answering the question a buyer actually needed answered, which is: bound to what?
Gartner named AI vendor protection a top-tier CISO priority for the first time in its 2026 leadership survey, specifically because traditional third-party risk management was never built for a vendor relationship where the risk lives inside a channel your own security team can't read. That's a structural admission, not a scare number: the tooling CISOs already trust doesn't reach this layer. A distillation dispute between two frontier labs is a business story. A credential sitting inside an encrypted block that a user correctly believed they'd deleted is a governance failure you own, not the vendor.
The researchers' own proposed fix names the actual bar to clear: cryptographic context binding, where the encryption envelope hard-codes the session and model that produced it, so a replay into a different context fails the authentication check instead of silently succeeding. That's the answer worth demanding before it's engineering roadmap language instead of a shipped mitigation. Anthropic, OpenAI, and Google were all disclosed to before publication and confirmed the specific attack chains in this paper no longer reproduce as of August 2026, which answers whether they moved, not whether every provider you're evaluating has moved the same way, or whether the fix only covers the exact chains that got published.
There's a version of this that isn't purely defensive. Reasoning traces are valuable enough to steal because they're valuable enough to learn from: Zhang and Morris's 2026 work on reasoning distillation raised a fine-tuned Qwen2.5-7B-Instruct's MATH500 accuracy from 68.4% to 76.0% by training on reasoning traces instead of answers alone, a gap answer-only distillation can't close.
That number is why providers will keep hiding this channel no matter how many holes get patched. The reasoning trace isn't incidental IP, it's the highest-value training signal a frontier lab produces, worth protecting even after this specific exploit is closed. The mistake wasn't deciding to hide it. The mistake was deploying "hidden" and calling it "protected" without checking whether the two words meant the same thing.
The vendors that get this right in the next cycle won't be the ones who encrypt the fastest. They'll be the ones who can answer, in one sentence, exactly what their encryption is bound to. The ones who can't answer that in one sentence don't know either.
Sources
- Panfilov, Schmotz, Shumailov, Beurer-Kellner, Schaeffer, Prabhu, Geiping, Andriushchenko - Stealing Reasoning Traces from Proprietary LLM APIs (2026) - the primary paper: mechanism, cross-model compatibility table, and the 315,320-trace, 367-PII-artifact, 182-credential findings.
- The Hacker News - OpenAI, Anthropic, Google API Flaw Let Weaker AI Models Decode Stronger Models' Reasoning (2026) - independent security-press corroboration of the disclosed vulnerability.
- Korbak et al. - Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety (2025) - the safety research explaining why chain-of-thought visibility was treated as a monitoring asset before providers encrypted it.
- Frontier Model Forum - Chain of Thought Monitorability issue brief (2025) - industry-body framing of CoT monitoring as a safety property providers are actively managing.
- Bloomberg - OpenAI Accuses China's DeepSeek of Distilling US AI Models to Gain an Edge (2026) - reporting on OpenAI's formal accusation to US lawmakers.
- Rest of World - OpenAI Accuses DeepSeek of Malpractice Ahead of AI Launch (2026) - additional context on the US-China distillation dispute.
- Gartner (via Safe Security) - SRM 2026: AI Vendor Risk Exposure from ChatGPT, Claude, Copilot, and Other AI Vendors (2026) - Gartner's framing of AI vendors as a new category of third-party risk traditional TPRM wasn't built for.
- Help Net Security (OWASP) - Prompt Injection Still Drives Most Agentic AI Security Failures in Production (2026) - OWASP's LLM01 ranking of prompt injection as the top AI security risk.
- TechStoriess - AI Agent Security Practices 2026: Prompt Injection, MCP Risks & Data Leaks - reporting on Cisco's State of AI Security 2026 findings, including the 83% deployment vs. 29% readiness gap.
- Beck Reed Riden LLP - Understanding AI Distillation in the Trade Secret Context - legal framing of why reasoning traces are treated as protectable intellectual property.
- Zhang & Morris - How to Steal Reasoning Without Reasoning Traces (2026) - the source for the 68.4% to 76.0% MATH500 accuracy gain from training on reasoning traces instead of answers alone.
Working through the challenges in this post? I help engineering leaders and CTOs navigate complex technical decisions and scale high-performing teams. Schedule a consultation →