Remediating Cross-Request KV Cache Exposure in SGLang and GLM-5.3

Understanding the GLM-5.3 KV Cache Exposure

Between August 27 and September 9, 2026, a tenant isolation issue affected the GLM-5.3 Workers AI model on Cloudflare, causing a small number of customers with prompts longer than 681,280 tokens to receive responses containing data from previous requests. Cloudflare has confirmed that no other model hosted on Workers AI was affected and that ordinary long-context use patterns matched the affected requests rather than malicious exploitation.

The vulnerability stemmed from two overlapping factors. First, a configuration mismatch occurred where the GLM-5.3 context window was set to 1,048,576 tokens, but the prefill GPU pool could only hold 681,280 tokens. Second, a bug in the SGLang inference engine handled over-length requests by substituting a one-token placeholder instead of issuing an immediate client error, allowing stale KV cache pages to be read by the decode node.

Immediate Configuration and Engine Fixes

Upon verifying the security report, Cloudflare temporarily took the model offline to implement critical fixes. Operators running disaggregated SGLang setups must adjust their configurations to address these risks immediately.

• Match Pool Size to Context Length: Ensure the KV pool size can fully accommodate the model context window to prevent triggering rejection paths on valid requests.
• Engine Upgrades: Update SGLang dependencies to version v0.5.22 or later. This incorporates patches like #36988 and #38935, which retire or reject over-length requests at the admission queue before allocation or KV transfers occur.
• Deferred Release & Batch Bounds: Run with deferred KV release enabled on both roles and bound Mooncake transfer batches to prevent timed-out calls from leaving dangling writes.

Practical Recommendations for Self-Hosters

Practitioners managing self-hosted LLM infrastructure using disaggregated prefill and decode architectures should adopt strict validation routines. Always measure and verify the KV pool size against the advertised context length directly from startup logs following changes to quantization, tensor parallelism, or memory fractions.

Additionally, enable runtime corruption detection flags such as --disaggregation-enable-kv-checksum and --kv-canary where supported by your model architecture, keeping in mind that coverage varies across different model families.

Source

Official announcement or documentation