Cloudflare Returned Other Customers' Prompts in 155 Responses. Your AI Feature's Tenant Isolation Is a Vendor Property (2026)

For thirteen days this past September, Cloudflare's AI model handed some customers a piece of someone else's conversation, and nobody noticed until a security researcher proved it on purpose.

Quick answer: Cloudflare's 26 October 2026 disclosure says 155 Workers AI responses, served between 27 August and 9 September 2026 by the GLM-5.3 model, contained other customers' prompts and generations, caused by a GPU memory bug in SGLang (the open-source LLM inference engine), in disaggregated prefill/decode serving. Every response returned HTTP 200 with a coherent, correctly-counted answer, and no error or alert.

That silence is the real story for anyone shipping an AI feature on someone else's inference stack.

You're reading this on a directory's blog, so let's get the disclosure out of the way first. SaaSCity runs a free SaaS directory, and like every company putting AI into a product this year, we'd be lying if we said tenant isolation in inference wasn't a thing we now check before we trust a vendor's "secure by design" slide. This post isn't a pitch dressed as news. It's the incident, the mechanics, and the checklist we'd actually run.

What happened, in one paragraph

On 26 October 2026, Cloudflare engineers Kevin Flansburg and Michelle Chen published a coordinated disclosure titled "Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang." Between 27 August 2026, the day GLM-5.3 from Z.ai launched on Workers AI as @cf/zai-org/glm-5.3, and 9 September 2026, 155 responses leaked fragments of other customers' sessions. Every single affected request exceeded 681,280 tokens. GLM-5.3 was the only model hit. Every other disaggregated Workers AI deployment ran a KV pool bigger than its context length, which turned out to be the only thing standing between "normal operation" and "someone else's prompt in your output."

The spec-sheet number versus the real number

Cloudflare advertised GLM-5.3 on Workers AI with a 1,048,576-token context window, but the KV cache pool actually serving it, per Cloudflare's 26 October 2026 report, held only 681,280 tokens, about 65% of what the spec sheet promised. That gap is the part that should bother any founder who has ever written a feature bullet before the backend fully supported it. Cloudflare knew about the gap before launch. The assumption was that the inference engine would cleanly reject anything over the pool size with an error. Pre-launch benchmarks showed successful HTTP status codes and a streaming response body with no error frames, so the team moved on.

That's the pattern behind a lot of AI feature launches: the context window on the pricing page is a model capability, not a deployment fact. The pool behind it is the real number, and nobody checks it because the happy path looked fine in testing.

What was advertised	What actually served it	Gap
1,048,576-token context window	681,280-token KV pool	~35% short
Expected behavior over limit	HTTP 400 rejection	Request silently admitted instead
Post-fix pool size	~2.04 million tokens	~2x the advertised context

How a rejected request still got served

A rejected request still got served because SGLang's rejection signal reached one admission queue but not the other, and the failure sequence is worth tracing step by step.

SGLang's scheduler is supposed to reject an over-length prompt through a function called Req.set_finish_with_abort. On a single node, that works as designed: the abort gets recorded, the prompt is swapped for a one-token placeholder so the request can drain cleanly, and the client gets an error. The bug lived in disaggregated prefill/decode (PD) serving, where prefill and decode run on separate GPU pools connected over RDMA via Mooncake. In that mode, the admission queues on both sides check req.finished() (whether the request completed), not whether it was specifically rejected. A request that set_finish_with_abort had just killed still reported itself as "finished" in a way that read as legitimately live, because the rejection landed in a field called to_finish instead of the finished_reason field admission actually inspected.

So the router told the decode node the real prompt length (eight KV pages, in Cloudflare's own diagram), and decode reserved all eight. Prefill admission let the rejected request through anyway. Prefill ran its one-token placeholder forward pass, wrote exactly one page of KV cache (call it P0), shipped that single page over RDMA, and signaled "transfer complete." Decode's admission queue accepted the handoff and its attention mechanism read all eight reserved pages, but only P0 had been written by this request. Pages P1 through P7 still held whatever the previous occupants had left there, because KV cache pages are never zeroed between requests. They just get overwritten by the next tenant that claims them, which is normal, necessary for throughput, and makes page ownership entirely the inference engine's job to track correctly. Here, it didn't.

The output of all that: HTTP 200, a coherent answer, a prompt_tokens count matching the long original prompt, and a response that arrived faster than a real eight-page prefill should take, which is the detail that should have raised a flag. Nothing in the API response said "this is wrong."

Who found it, and how fast Cloudflare moved once they knew

Researcher Gadi Evron of Knostic reported the bug through Cloudflare's HackerOne program on 8 September 2026 at 23:11 UTC, with a script that reproduced it on demand. Cloudflare's response, in UTC on 9 September 2026, is the part of this story that actually looks good:

Time (UTC)	Action
08:00	Verification begins
13:22	Security incident declared; two independent accounts reproduce the leak within 25 minutes
14:00	Model set to private; requests start returning HTTP 400
14:35	Root cause confirmed: pool smaller than context; canary node with larger pool confirmed clean
14:52	All prefill nodes rolled to corrected configuration
19:04	Full deployment rebuilt with corrected config plus a Cloudflare-authored engine patch, validated against the researcher's script
19:40	Model restored

The model was offline within 40 minutes of the incident being formally declared, and the fix shipped the same day it was confirmed. That's a genuinely fast incident response. It just happened 13 days after the exposure window opened, which is the gap that matters more than the response time once it was found.

The fix, and why SGLang's own history makes this worse than one bug

Cloudflare's remediation had two parts. First, the obvious one: the pool now holds roughly 2.04 million tokens, about double the 1,048,576-token context, so a valid prompt always fits and the rejection path is simply never reached. Second, an engine patch (Cloudflare's draft is sglang PR #38704) retires any request carrying a pending rejection at the PD admission queue, checking its original error before bootstrap, allocation, or KV transfer even start. Decode nodes now also verify that the transferred token count matches what they reserved.

Cloudflare's own admission is worth quoting directly: "This was one critical gap in our testing: we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence." They now check KV pool size against context length pulled from the running engine before any disaggregated deployment change ships, a check that arguably should have existed before launch.

SGLang's security update on lmsys.org, published the same day as Cloudflare's post as part of a coordinated disclosure with RadixArk (SGLang's maintainers), confirms Cloudflare notified them on 9 September 2026 and states plainly that this entire class of aborted-transfer race condition was present in default PD configurations from the v0.4.x line in early 2025 through v0.5.21, a year and a half of exposure across every operator running disaggregated SGLang, not just Cloudflare.

The upstream paper trail is long enough that it's a case study in how slowly these things get fixed even once reported. SGLang issue #30233 flagged this exact bug on 6 July 2026, three months before Cloudflare's incident. Fixes landed in stages: #36988, merged 6 September and shipped in v0.5.20, retires aborted prefill requests before their result reaches decode. #38935, titled "Do not admit intake-rejected requests to a PD handoff" and merged 16 September 2026 in v0.5.21, closes the admission-queue gap directly: its motivation section states that Req.set_finish_with_abort stored the rejection in to_finish rather than finished_reason, which is the exact root cause described above. #39500, also in v0.5.21, adds a per-transfer KV checksum you enable with --disaggregation-enable-kv-checksum. Deferred KV release became the default after #41023. SGLang v0.5.21 shipped 2 October 2026, and v0.5.22, which Cloudflare now recommends as the floor for every disaggregated operator, shipped 19 October 2026, a week before the disclosure went public.

Cloudflare went further than patching the known bug. A page-tagging prototype, called "integrity by construction," changes a tag on every KV page reallocation and checks ownership before decode is allowed to read it. Tested against GLM-5.2 in mid-August 2026, it measured under 1% throughput and tail-latency cost, but got shelved for reliability and architecture-specific limits. Cloudflare is also pursuing formal verification of the RDMA KV cache transfer protocol itself, following the same approach the company used on its Topaz DNS work and Meerkat/QuePaxa research. The HN thread on this, sitting around 149 points and 47 comments within a day of publication, is worth skimming for how many infra engineers immediately recognized the pattern from their own disaggregated deployments.

What this actually proves about tenant isolation

This incident proves tenant isolation in an AI feature is a property of the inference path, not of the application code calling it. None of the application code was wrong. There was no SQL injection, no missing row-level security check, no auth bypass, the kind of thing founders audit religiously because it's the layer they wrote. Compare it to Supabase's 16,326 exposed databases from misconfigured row-level security, where the bug was literally in the access-control layer teams own. This bug sat somewhere else entirely. Two GPUs disagreed about who owned a page of memory, several layers below any API call your team ever wrote, and that disagreement returned someone else's prompt as a perfectly formed, perfectly coherent answer.

You can write flawless access control and still leak data, because the hosting layer underneath your fetch() call made a wrong assumption about a rejection code. The failure mode is silent instead of loud: a convincing HTTP 200 does more damage than a crash would, because nobody investigates a 200.

The threshold is also the warning. All 155 leaked responses sat above 681,280 tokens. Nobody sends a million-token prompt to chat about the weather. The people pushing context that far are the ones pasting in whole codebases, whole contracts, whole customer datasets. That means the narrow edge case where this bug triggers is exactly the edge case where the most sensitive payloads live. The same logic shows up in LLMjacking against exposed Ollama endpoints and the LiteLLM 1.107.0 backdoored credential stealer: the AI supply chain keeps finding new ways to leak at the layer under the one you're watching.

A short checklist, since you're going to ship AI features anyway

None of this requires a security team, just a few questions to your vendor and a few lines of logging.

• Ask your inference provider what their KV pool size is relative to the advertised context window, and whether output is validated for coherence at max context, not just whether requests return HTTP 200.
• Drop a per-tenant canary string into long-context requests and assert it never shows up in another tenant's completion.
• Log response provenance on every AI call: model ID, version, endpoint, request ID.
• Pin model versions. Floating aliases mean you can't even tell which build served a bad response.
• Put disclosure SLAs and incident-notification clauses into vendor contracts and DPAs now, before you need them.
• If you self-host SGLang in disaggregated mode, verify pool-versus-context on every prefill and decode node after any change to weights, quantization, parallelism, or memory fraction. Run v0.5.22 or later with deferred release and KV checksums turned on.

If 155 responses contained someone else's data, somebody has to decide whether that trips GDPR Article 33's 72-hour breach-notification clock. Cloudflare disclosed on its own schedule, 26 October for an incident that closed 9 September. Your notification obligations come from your contract with the vendor, not from when their blog post goes up.

The part you can actually control

You can't audit a stranger's KV pool, and you definitely can't patch one. What you can own is the layer above it: the relationship with the people who find your product, and what you tell them when something breaks. That's the case for distribution you control instead of distribution you rent. SaaSCity is a gamified startup directory with a live city map and human editorial review: a free listing page and a building on the map for every founder, a badge that gets you a dofollow backlink and a Monday launch slot, a Quick Pass at $19.99 that goes live within 24 hours, and a Premium plan at $39.99 that adds a written launch post with three dofollow links. Our Domain Rating sat around 47 to 56 at the last Ahrefs refresh.

Audit your vendor's isolation claims before you ship the feature that depends on them, not after a researcher on HackerOne finds the 681,280-token line nobody checked. The next KV cache bug won't send a press release 13 days early.

If you're building with GLM-5.3, SGLang, or any disaggregated inference stack and want a place to show real users what you shipped, list it on SaaSCity: free, indexed, and the backlink is dofollow the moment you add the badge.