Weir - Catch credential leaks in the AI agents you build and use

Weir, a new open-source security tool, helps developers detect when their AI agents accidentally expose API keys during operation. The tool works by planting fake keys where real ones would live and monitoring whether these values leak out through traces, logs, console output, or other channels. It also provides real-time monitoring for Claude Code sessions, showing which credentials enter the model's context or are sent in tool calls.

Two detection methods

The tool offers two primary approaches to detecting leaks. First, developers can test their own agents by planting unique fake keys and running the agent against a local fake model. Weir then searches all outputs for these exact values to determine if a leak occurred. This method can be integrated into CI pipelines or run locally on a laptop.

Second, Weir can monitor Claude Code sessions through a status line that tracks how many real credentials enter the session's context or are sent in tool calls. An optional redaction hook can strip known keys from tool results before the model sees them, replacing them with [weir: NAME removed]. The tool never displays actual key values, only their presence or absence.

Unique approach

Unlike traditional secret scanning tools that look for strings that resemble secrets in code, Weir proves whether a specific value actually left through an agent's operation. It checks every possible exit point including spans, logs, metrics, console output, prompts sent to the model, and requests to other services. On Linux systems with unprivileged user namespaces, Weir can run agents in complete network isolation to catch direct connections.

The tool is designed to be offline and deterministic, with no AI model making decisions and no data sent anywhere. It has already identified CVE-2026-116691 (GHSA-j7qq-vr8c-xvw3), an MLflow vulnerability fixed in version 3.17.0. Weir provides clear verdicts: 'leak', 'clean', or 'incomplete' if a required key wasn't exercised during testing.

Limitations

Weir has several limitations to consider. It only finds whole values, not hashed, split, or truncated ones. The network isolation feature works only on Linux with unprivileged user namespaces. The status line only detects keys in the inventory as whole values, and the inventory covers one project at a time. The redaction hook can't undo commands that already ran or catch keys pasted into chat.

Despite these limitations, Weir offers a valuable addition to security tooling for AI development. The tool is released under Apache-2.0 license with no restrictions, and can be installed via pip with 'pip install weir-scan weir canary judge --sample'. The name 'Weir' comes from the engineering term for a low dam that measures water flow without stopping it.