A new implementation for LLMs through Nvidia GPUs natively on Apple Silicon
Hawking Systems has released Hawking Runtime, a project enabling local model inference and NVIDIA GPU computing on Apple Silicon Macs.
The project currently has two main components: an extension for local model inference and an eGPU solution. The extension uses llama.cpp to provide local model inference with an OpenAI-compatible API and model management capabilities, requiring an Apple Silicon Mac and Python 3.12 to operate.
The eGPU component is a native macOS DriverKit driver that enables NVIDIA GPUs connected over Thunderbolt to work with Apple Silicon. The reference setup uses an NVIDIA RTX 4000 SFF Ada connected to an M3 Ultra Mac Studio, with demonstrated driver initialization, GPU kernel execution, and numerical verification.
Technical Implementation
For the extension, users can install it via pip and initialize it with 'hawking init', which detects the Mac's resources and sets up the runtime. The server can then be started with 'hawking serve', providing a default endpoint at http://127.0.0.1:8277/v1 that supports streaming, tool calls, and the Responses API.
The eGPU implementation includes the DriverKit driver, native execution code, CUDA kernel catalogue, diagnostics, and build tooling. The project has demonstrated PCIe Gen2 ×4 achieving approximately twice the measured Gen1 transfer rate, with Gen3 and Gen4 still in development.
Developer Approach
The purpose of publishing the source is to let developers inspect the implementation, reproduce the work, adapt it to their hardware, and build on it.
Hawking Systems encourages using coding agents to work through the eGPU setup, as hardware configurations differ. The source code is published to allow developers to inspect, reproduce, adapt, and build upon the implementation for their specific hardware needs.