Skip to main content

Agent-Otel-Kit – OTel Dashboards for Claude Code, Codex and Gemini CLI

Kagi Small Github
16 lis 2026, 07:34Tech
Oryginał
2 min read

A new open-source tool called agent-otel-kit allows developers to monitor AI coding agents like Claude Code, Codex, and Gemini CLI with a single Docker command.

agent-otel-kit provides a comprehensive observability solution for AI coding agents, enabling developers to monitor performance metrics, costs, and usage patterns directly on their local machines. The tool creates a complete OpenTelemetry stack with collector, Prometheus for metrics, Loki for events, Tempo for traces, and dashboards in both Grafana and Perses.

Setting up the monitoring system takes approximately 30 seconds with Docker and the CLI tool. Users can clone the repository, run 'docker compose up -d', and immediately begin monitoring their AI coding sessions. The dashboards are accessible at http://localhost:3000 for Grafana and http://localhost:8080 for Perses, with all ports bound to localhost for security.

Data Collection Details

The system collects detailed metrics from each AI coding agent, including cost tracking (for Claude Code), token counts by type, session information, active time, tool calls with success rates and durations, and model request metrics. However, sensitive data like prompt text, model responses, tool arguments, file paths, and shell commands are not collected by default to protect privacy.

The collector includes built-in redaction processes that ensure sensitive content remains protected even if users accidentally enable logging options. Users can optionally capture content by setting AGENT_OTEL_KIT_CAPTURE_CONTENT=true, but personal information like user email addresses are automatically dropped unless explicitly preserved with AGENT_OTEL_KIT_KEEP_EMAIL=true.

Agent-Specific Configuration

The tool supports setup for Claude Code, Codex, and Gemini CLI, with each configuration modifying only the agent's specific user config file. The setup process includes backup functionality, allowing users to easily revert changes with the '--undo' option. The CLI tool also supports dry runs and can point agents to collectors hosted elsewhere.

Dashboard Features

The dashboards provide three main views: Agent cost and tokens, Agent tool calls, and Agent sessions and traces. These visualizations show cost breakdowns by model, token usage patterns, tool call success rates, permission decisions, and interaction traces. Each dashboard is generated from a single panel list definition, ensuring consistency between Grafana and Perses implementations.

The system handles different agent behaviors - for example, Codex marks shell commands with non-zero exits as successful tool calls, while Gemini CLI lacks per-prompt traces. Metric names vary by agent but are normalized in the dashboards for consistent analysis. The tool also addresses specific challenges like Codex's lack of per-request model duration logging over its default WebSocket transport.

Z.ai
Article written by AI
Z.ai GLM-4.5-FlashFree
Free AI articles150 of 150
No limit after signing in with Telegram