Open Source AI Agents for Cybersecurity: Inspect, Then Trust - TechSpective
Security teams increasingly adopt open-source AI agents without proper review, creating new risks as these systems can actively interact with internal data and systems.
A developer finds an AI agent that does almost exactly what the security team needs. Someone else has already written the skill, connected the tools, and shared the project publicly. Reusing that work can save days, but the security team still has to decide whether it trusts what somebody else built.
Adoption is outpacing review. OWASP covers it in its AI Agent Security Cheat Sheet, and NIST addresses it in its AI Risk Management Framework (AI RMF). Both treat it as a new category of risk for boards and practitioners. Teams across an organization are building and borrowing agents, sometimes without a central review process.
Hidden Risks
These systems can also act rather than just generate an answer. An agent may be able to query internal data, call a tool, or make a change using whatever access somebody has given it. Community-built AI agents, skills, and Model Context Protocol (MCP) servers make that problem harder to track.
What makes an AI agent risky before it ever runs? An AI agent is risky when its instructions, tools, and permissions reach further than its stated job. Source code is just one place to look. Reviewers also need to know what the large language model (LLM) has been instructed to do and which tools it can call.
A skill may consist largely of instructions telling a model how to perform a task. Those instructions aren't always obvious from the rendered file.
Security Review Essentials
Open-source agents change. A review of one version does not cover code added later, so any approval needs to identify the commit or release that was inspected. The instructions sent to the model need their own review.
That includes looking for hidden or encoded instructions and checking whether the directives match what the component claims to do. Dependencies and install hooks still need the supply-chain checks applied to other software. The permissions need to match what the agent is there to do.
Runtime testing can uncover behavior that is difficult to establish from the repository alone. The review needs to cover what the component contains and what it does when installed. The recently announced CyberAgents Exchange AI Inspector provides a current example of formal agentic AI review.