Sarvam AI Releases Open-Weight Sarvam Prism, a 105B MoE Decision Model Running in 44 ms
India’s Sarvam AI open-sourced Sarvam Prism under Apache 2.0, combining a 105B-10B MoE base, Brier loss calibration, and FP8 CUDA Graphs to hit 44 ms median latency.
Indian AI lab Sarvam AI entered the open System One ecosystem on November 3 with the release of Sarvam Prism, an Apache 2.0-licensed decision model built as a LoRA fine-tune of its Sarvam 105B-10B mixture-of-experts architecture.
15x inference speedup to 44 milliseconds
Sarvam reported that Prism scored 71.8 averaged across three decision indices—the highest score among 25 models it tested under 100 ms. Reaching that speed required aggressive systems optimization: an unoptimized 16-bit build across two NVIDIA B200 GPUs originally took 664 ms per median request. By quantizing weights to FP8 to fit on a single 120 GB+ GPU, merging the LoRA adapter, executing all 128 experts inside one grouped kernel, and capturing the forward pass as a CUDA Graph, Sarvam cut median latency 15-fold to 44 ms with no reported accuracy loss.
Calibration by design
To address enterprise requirements around confidence reliability and option-order bias, Sarvam trained Prism’s single-pass scoring head with two equally weighted objectives: cross-entropy with 5% label smoothing to discourage overconfidence, and a Brier loss term that heavily penalizes confident errors. Sarvam said Prism’s errors remain within its confidence bounds and are invariant to option ordering.
Weights are available on Hugging Face under Apache 2.0 for self-hosting, alongside hosted access on Sarvam Indus, demonstrating that large sparse MoE backbones can compete directly on latency with much smaller dense decision models.