Skip to main content

NOX Local – a coding agent for 4B models on an 8 GB GPU

Kagi Small Github
17 lis 2026, 14:35Tech
Oryginał
2 min read

NØX-Local, a new coding agent that runs entirely on local computers without API keys or cloud dependencies, has been released, offering privacy and offline functionality for developers.

NØX-Local represents a significant shift in AI-powered coding assistance by operating entirely on local machines rather than relying on cloud services. The tool is designed specifically for small language models, particularly 4B parameter models, addressing a common limitation in existing AI coding assistants that are built for much larger frontier models.

Unlike cloud-based coding assistants that require API keys, charge per token, and potentially scan all project files, NØX-Local processes everything locally. This approach ensures complete privacy and eliminates concerns about running out of quotas or incurring unexpected costs. The tool uses llama.cpp directly with Vulkan support, making it compatible with consumer-grade GPUs as low as 8GB VRAM.

Key Features

The agent includes ten carefully selected built-in tools optimized for small models: Read, Write, Edit, Glob, Grep, Bash, WebSearch, WebFetch, Artifact, and TodoWrite. Each tool has a short description to help 4B models select the appropriate one. The system implements a context management approach that reads files in 300-line chunks and trims old results to maintain a 32K window.

NØX-Local includes several safeguards to prevent common issues with small models. It features forgiving parsing that recovers tool calls when models output them as plain text instead of executing them. A loop guard prevents identical tool calls from repeating more than twice, and the system includes a "thinking budget" that caps reasoning tokens at 2K for medium effort and 6K for high effort.

Setup and Usage

Installation requires Linux with a Vulkan-capable GPU and Go 1.27+ if building from source. The process involves building llama.cpp with Vulkan support, then installing NØX-Local via a simple shell script or by downloading a precompiled binary. Users need to download a compatible GGUF model file, with qwen3.5-4b (3.0 GB) recommended as the default.

Performance and Models

The tool has been tested on five models running on an AMD Radeon RX 6500 XT 8GB GPU with a Ryzen 5 5600GT processor. Qwen3.5 4B is recommended for small edits due to its responsiveness, while Ornith 1.5 9B offers the best speed-to-quality ratio for real work. Nemotron Bolt E4B excels at processing long files quickly, and Saluki 27B is suitable for complex questions where some delay is acceptable.

The system includes MCP (Model Context Protocol) support that remains off until explicitly enabled, preventing small models from being overwhelmed by too many tools. Users can also create and use "skills" - folders with SKILL.md files that the model can reference when relevant. Configuration is handled through TOML files with project-specific overrides possible.

Z.ai
Article written by AI
Z.ai GLM-4.5-FlashFree
Free AI articles150 of 150
No limit after signing in with Telegram