SGLang and Google speed up Meta's Muse Spark Open on a single TPU v7 host

The SGLang project says Meta's Muse Spark Open can run on a single TPU v7 host with SGLang-TorchTPU, and that it has worked with RadixArk and Google to speed it up under SemiAnalysis's AgentX agent workload. These are the project's own figures:

• 3x faster decode at small batch size, with a custom megakernel and DFlash2 speculative decoding.
• 18% faster decode on 500K-token contexts.
• Faster prefill for large agent prompts, using a fused MoE kernel and sequence parallelism.
• Twice the context per chip, with a compact fp8 KV cache.
• Up to 77% more agent throughput, with HiCache and session-aware eviction.

A PyTorch post from November 10, which SGLang quotes, reports 433 tokens per second per user and up to 25.9K tokens per second per chip on AgentX at 1M context. It says all 62 layers of the 816B-parameter MoE model run as one Pallas program. Code and recipes are in the SGLang cookbook, and the team says it is working on prefill/decode disaggregation and further attention and MoE optimizations.