Ling and NVIDIA speed up Ling-3.0-flash on a single DGX Spark by 42.5%

The Ling team says that, working with NVIDIA, it optimized inference for the open-weight Ling-3.0-flash (124B total parameters, 5.1B active per token) on one NVIDIA DGX Spark. NVIDIA published a technical blog post on the work. The figures are the team's own.

Three techniques made the difference:

• Humming MXFP4 MoE kernels: +9.3% decode throughput.
• Online FP8 LM-head quantization: +21.7% cumulatively.
• NEXTN-based multi-token prediction (MTP): drafts and verifies several tokens per round, which reduces the number of steps.

Across 1,319 GSM8K questions, decode throughput rose 42.5%, from 37.81 to 53.89 tokens per second, and mean latency fell 24.5%, from 15.21 s to 11.49 s, with no statistically detectable change in accuracy. On a synthetic workload of 32K input and 2K output tokens, the combined stack reached 102.88 tokens per second, 185.1% above the baseline. The team cautions that this is workload-specific: MTP accepted 2.68 tokens per round on GSM8K against 3.90 on the synthetic test, so real workloads should be benchmarked independently.

The model is on Hugging Face and ModelScope, and the team links a deployment cookbook on GitHub.