I ran the full 2.8T-parameter Kimi-K3 on a 64GB Mac Mini

The complete 2.8-trillion-parameter Kimi-K3 AI model can now run locally on a 64GB M4 Pro Mac mini using external SSD storage, according to a new GitHub project. The implementation streams all 93 model layers from SSD, processing expert by expert through Apple's MLX and Metal frameworks. While performance is slow at approximately 32.4 seconds per token, the approach enables processing of highly confidential documents without cloud transmission.

The project uses the full MXFP4 checkpoint (~1.42 TiB) without pruning experts or reducing model depth. During operation, the system peaks at about 15GB of MLX allocator memory, though prefill operations can reach 22-24GB. The slow processing speed makes interactive chat impractical, but the creator notes that for certain professional workflows, days of processing may be faster than waiting for human specialists.

Confidential Applications

The developer tested three applications where local processing provides advantages over cloud services. For medical records analysis, the system processed a synthetic patient case in 3.3 days, identifying inconsistencies in clinical data without exposing private information to external services. In a semiconductor manufacturing case, it traced equipment issues in 3.0 days, maintaining confidentiality about production processes. For R&D strategy, it analyzed patent materials in 4.7 days, suggesting potential inventions and trade secret considerations.

Performance Comparison

Local processing took 3-5 days per case compared to about six minutes for cloud inference through OpenRouter at a cost of approximately $0.327. The project emphasizes that the comparison isn't about raw speed but about value: three days of local processing versus a three-month wait for a specialist or two weeks for attorney review. The system uses a USB4 NVMe drive on a rear port, with each decode token reading roughly 120GB of logical data, making storage bandwidth the limiting factor.

Technical Implementation

The project extends prior work on SSD and expert streaming, specifically implementing the K3 architecture for Apple Silicon with a fused Metal MXFP4 kernel and latent MLA cache. Unlike the reference implementation which uses CUDA/FlashAttention/Triton paths, this version uses MLX operations and Metal kernels. The developer notes that read() into buffers performed better than mmap for cold data, measuring 3,551 MB/s versus 389 MB/s due to page faults defeating readahead.

The project includes validation tools and sample outputs, with a real 1,383-token answer generated in about 13.7 hours of wall time. The creator emphasizes that local inference doesn't automatically ensure security or compliance, as the machine, its disks, backups, and access controls still matter. The obvious next experiment, they note, would be testing whether smaller models could achieve similar results on the same cases.