agent-observability

Solid

Design privacy-aware observability for AI agents using traces, spans, structured events, metrics, cost attribution, dashboards, alerts, and investigation workflows. Use when instrumenting an agent, debugging intermittent tool or model failures, defining service-level objectives, analyzing latency or spend, auditing agent decisions, or preparing production monitoring.

AI & Automation 161 stars 32 forks Updated 1 weeks ago MIT

Install

View on GitHub

Quality Score: 84/100

Stars 20%
74
Recency 20%
90
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Agent Observability Make agent behavior explainable from request entry through model, retrieval, tool, handoff, and response spans. ## Use when - Add telemetry to a new or existing agent workflow. - Diagnose slow, costly, incorrect, looping, or failed executions. - Define dashboards, alerts, service-level indicators, or audit evidence. - Standardize traces across models, tools, and orchestration frameworks. ## Inputs Collect the workflow graph, runtime boundaries, incident questions, traffic and failure expectations, telemetry stack, data classification, retention policy, sampling limits, and owners. State what cannot be observed. ## Output contract Produce: 1. An observability objective and system boundary. 2. A trace and event schema with identifiers, span taxonomy, attributes, and redaction rules. 3. Metrics with definitions, units, dimensions, and ownership. 4. Dashboard and alert specifications tied to user impact. 5. A sampling, retention, access, and cost plan. 6. An investigation runbook and instrumentation verification results. ## Workflow 1. Start with operational questions such as “Which tool causes timeouts?” or “Why did cost per resolved task rise?” Do not collect fields without a decision use. 2. Define one trace per user-visible attempt. Create spans for model calls, retrieval, tools, handoffs, approvals, retries, and final validation. Preserve parent-child relationships and propagate a correlation identifier across queues. 3. Record stable semantic...

Details

Author
seb1n
Repository
seb1n/awesome-ai-agent-skills
Created
6 months ago
Last Updated
1 weeks ago
Language
Python
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category