Homelabber Runs Massive 2.8-Trillion-Parameter AI Model On Discontinued Intel Optane Memory, Without A GPU - Wccftech

A hardware enthusiast has managed to run one of the world's largest open-weight AI models on a server powered entirely by a CPU, with no GPU acceleration. The secret? Discontinued Intel Optane persistent memory modules, which provide enormous memory capacity at a fraction of the cost of conventional server DRAM.

The experiment involves running Moonshot AI's Kimi K3, a massive 2.8-trillion-parameter Mixture-of-Experts model, on a third-generation Intel Xeon Scalable processor featuring 40 cores. Kimi K3 activates approximately 104 billion parameters per token, with a theoretical context window of one million tokens.

Optane Makes It Possible

Rather than spending on high-capacity DDR4 or expensive professional GPUs, the enthusiast opted for second-generation Intel Optane Persistent Memory 200 Series (Barlow Pass). These modules provide persistent, byte-addressable memory in capacities reaching 512 GB per stick. Although Intel abandoned future Optane development in 2022, secondhand modules remain an interesting alternative for memory-intensive workloads.

Performance and Thermal Challenges

Using llama.cpp with Optane configured in App Direct mode, the builder successfully loaded the model without requiring GPU acceleration. Initial performance was modest, reaching just 0.85 tokens per second with 1-bit quantization. However, disabling Direct Access (DAX) increased throughput to 1.50 tokens per second, an approximately 76% improvement.

A separate 3-bit quantization test achieved approximately 0.75 tokens per second. At that speed, generating 50,000 tokens would require roughly 18.5 hours, highlighting the significant performance compromises. Furthermore, aggressive 1-bit quantization can adversely affect model accuracy.

Perhaps the most important discovery concerns thermal behavior. Under sustained inference workloads, the Optane DIMMs generate substantial heat. Individual modules can operate at power levels reaching 18W, with thermal throttling beginning at 83°C and protective shutdown occurring at 85°C. The builder installed two dedicated 12W blower fans to manage temperatures.

Future Plans

Despite the limited performance, the experiment demonstrates how aging enterprise hardware can still find unconventional applications in modern AI workloads. The builder plans to experiment with GPU acceleration and hybrid DRAM/Optane configurations to improve inference speeds.