Nota optimizes Kimi K3.1 Pro to cut GPU costs by up to 20% - Chosunbiz

Kimi K3.1 Pro, with its 2.8 trillion parameters, is the largest among publicly released open-weight AI models by total parameters. As models grow, so do the memory and GPU resources required to run them, increasing infrastructure costs for companies. Nota's optimization allows the model to run on eight of the relatively cheaper B200 GPUs instead of eight of Nvidia's latest B300 models.

Cost Reduction

By Nota's estimate, using B200s instead of B300s can reduce GPU purchase costs by up to about 20%. In cloud environments, under the same model-run conditions, B200 GPU rental rates were up to about 22% lower.

Technical Innovation

Nota applied pruning, selectively removing expert modules that perform specific tasks inside the model. This cut expert modules by up to 37%, reducing memory usage while maintaining key inference performance. The company scored at least 99% of the original model on GPQA Diamond, which assesses advanced scientific reasoning.

Performance Enhancement

While lowering memory use, the optimization increased the amount of information the model can process at once. In an eight-B200 GPU setup, the model with expert modules reduced by 30% can process up to 512,000 tokens of context, and the model reduced by 37% can handle up to 1 million tokens.

The freed-up memory is used to run a small helper model, called a draft model, that assists with answer generation. Speculative decoding—where the draft model proposes what comes next and the main model verifies it in one step—helps boost answer generation speed even on eight B200 GPUs.