Airgap-check – find what your AI app silently downloads offline

Many developers assume that by pointing their AI applications to local instances like vLLM or Ollama, they're ready for offline environments. However, numerous libraries make unexpected network calls for tokenizers, model weights, or telemetry that only become apparent when the network is disconnected. airgap-check, a new open-source tool from Humanet, aims to surface these issues before deployment.

The tool provides two main functions: static scanning of project manifests and code patterns, and runtime testing in a completely isolated container environment. During static scans, it examines files like requirements.txt, pyproject.toml, package., and Cargo.toml for known problematic patterns such as nltk.download(), from_pretrained(), and tiktoken.get_encoding().

Runtime Testing

The --run option executes applications in a Docker container with network disabled, using a DNS sinkhole and socket hooks to log every connection attempt. This reveals hidden network dependencies that static analysis might miss. The tool can also generate Dockerfile snippets with environment variables to disable telemetry and pre-warm caches at build time.

Testing Results

In benchmark tests across 20 open-source AI repositories, airgap-check demonstrated 89.7% precision and 87.5% recall at the component level. The tool correctly identified all 9 repositories with first-use downloads and properly flagged one repository as clean. False positives occurred mainly with development dependencies, while false negatives included OCR weight downloaders and model price fetchers not yet in the rule table.

The tool has limitations: it only validates Python and Node.js targets thoroughly, may miss hardcoded IP addresses or DNS-over-HTTPS traffic, and only tests the code paths explicitly exercised during runtime. Browser-side telemetry remains a separate concern not addressed by environment variables.

Design Principles

airgap-check was designed with security and privacy in mind. The tool never reads, stores, or logs secret values from .env files, only inspecting key names and their presence. It mounts repositories read-only during testing and emphasizes that a clean result doesn't guarantee absolute security for isolated environments.

Developers encountering libraries that make unexpected network calls or false positives are encouraged to contribute to the project by opening issues with specific details. The tool is part of Humanet's broader efforts to build infrastructure for secure AI applications, including their Frame hosting platform currently in development.