Doc-foundry – AI writes Security detection rules, then proves they work
A new GitHub project called DOC-foundry creates an adversarial multi-agent workflow that turns cybersecurity threats into validated detection rules through a Blue-vs-Red testing process.
The DOC-foundry project, published on GitHub, implements a novel approach to cybersecurity detection engineering using adversarial multi-agent workflows. The system employs LangGraph, Pydantic, FastMCP, and Claude technologies to transform threats like CVEs, CISA KEV entries, or ATT&CK techniques into validated detection rules that have survived rigorous testing.
What sets this tool apart is its ability to produce three critical artifacts together: a Sigma detection rule that has survived a Blue-vs-Red adversarial loop, a threat report documenting the convergence history, and a proof bundle containing positive corpus, benign baseline, adversarial mutations, and coverage labels with FP/FN numbers.
Adversarial Testing Process
The system uses four role-specialized LLM agents (Threat Analyst, Blue Detection Engineer, Telemetry, and Red-Team Adversary) to create and test detection rules. A deterministic referee handles all truth-deciding tasks including FP/FN evaluation, adversarial evasion testing, and coverage verdicts. This ensures reproducible results with the same input always producing the same verdict and reproducibility hash.
The thing nobody else produces is #3. A model can write you a Sigma rule; it cannot hand you a proof that the rule works, an adversary that provably evades the weak version, and an honest record of how the rule was hardened until it held.
Review Chain Pipeline
DOC-foundry includes a second pipeline called the Review Chain that asks different questions than the proof-focused first pipeline. This pipeline simulates stakeholder reviews through a draft → Red Team → revision → SOC Lead → final rule progression, creating a complete audit trail of how the rule was shaped through feedback.
The adversarial nature of the process creates productive tension between reviewers. Red Team feedback pushes Blue to broaden rules to cover more evasion techniques, while SOC Lead feedback pushes to narrow rules to reduce alert volume. The final rule represents the balance between these competing pressures.
Practical Implementation
The tool can be run with various threat inputs, from live CVEs to canned test scenarios. Output bundles are written to Outputs// containing rule.yml, report.md, and proof.. The system also includes a demo UI built with FastAPI that allows users to watch the agents work in real-time as they author detection rules.
A worked example using CVE-2026-48939 (Joomla iCagenda arbitrary file upload RCE) demonstrates the process in action. The initial draft was refined through three versions based on eight findings from Red Team and SOC Lead reviewers, with the final rule incorporating parent lineage requirements, exclusion of known-good interpreter chains, and expanded triage fields.
Related
