dt-obs-genai

Solid

Analyzes observability signals from customer GenAI applications with DQL. Reads OpenTelemetry GenAI spans and LLM evaluation bizevents. Use for: golden signals (traffic, errors, latency, saturation); LLM signals (model, provider, tokens); cost/token analytics, usage attribution, and prompt caching; agent signals (tool calls, steps, failures, loop detection, Smartscape topology); conversation/session analytics; guardrails (blocked/truncated responses); and evaluation signals (quality, pass/fail). Trigger: "LLM latency", "token usage by model", "cost by model and provider", "cost per conversation", "who is driving token spend", "do I have prompt caching", "failing agent tool calls", "find runaway agents", "responses truncated or blocked", "failed evaluations", "am I hitting rate limits", "token throughput / TPM", "provider throttling or 429s". Do NOT use for: Davis CoPilot/MCP telemetry (dt-platform), generic service metrics (dt-obs-services), logs (dt-obs-logs), or non-GenAI tracing (dt-obs-tracing).

AI & Automation 117 stars 24 forks Updated 3 days ago Apache-2.0

Install

View on GitHub

Quality Score: 88/100

Stars 20%
69
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# AI Observability (GenAI) Skill Analyze AI Observability signals from customer GenAI applications using DQL — golden signals, LLM signals, token and cost analytics (with usage attribution and prompt-caching economics), agent signals (including loop/runaway detection and Smartscape topology), conversation/session-level analytics, guardrails, and evaluation quality. --- ## Core Capabilities ### Golden Signals The four classic observability signals — traffic, errors, latency, and saturation — apply directly to GenAI applications. Traffic is request throughput over time; errors are spans where `span.status_code == "error"`; latency is the `duration` field (a Grail duration value — divide by the `1ms` literal, `duration / 1ms`, for a numeric millisecond value); saturation is proxied by total token throughput per minute (input + output tokens combined). ```dql fetch spans, from: now()-24h | filter isNotNull(gen_ai.request.model) | summarize total = count(), errors = countIf(span.status_code == "error"), by: {gen_ai.request.model} | fieldsAdd error_rate_pct = if(total > 0, errors * 100.0 / total, else: 0.0) | sort error_rate_pct desc ``` → **Full traffic, latency, and saturation queries:** See [references/golden-signals.md](references/golden-signals.md) ### LLM Signals LLM signals describe which model and provider served each request, what operation type was invoked (`chat`, `execute_tool`, `invoke_agent`, `create_agent`), and how tokens were consumed. Use these to benchma...

Details

Author
Dynatrace
Repository
Dynatrace/dynatrace-for-ai
Created
3 months ago
Last Updated
3 days ago
Language
Shell
License
Apache-2.0

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

instrument-llm-observability

Use this when adding tracing/observability to an LLM or AI-agent application - capturing prompts, tool calls, token usage, latency, and cost per step. Trigger whenever someone wants to "add tracing", "instrument", "monitor", "see what my agent is doing", or debug an LLM app in production. Prefer vendor-neutral OpenTelemetry unless a specific platform is already in use.

26 Updated today
ContextJet-ai
DevOps & Infrastructure Solid

dt-obs-services

Service performance monitoring with RED metrics (Rate, Errors, Duration) and runtime-specific telemetry for Java, .NET, Node.js, Python, PHP, and Go. Use when analyzing service health, SLA compliance, or runtime issues. Trigger: "service response time", "error rate", "throughput", "SLA compliance", "service mesh overhead", "JVM GC", "Java heap", "Node.js event loop", ".NET CLR", "Python threads", "PHP OPcache", "Go goroutines", "service performance", "p95 latency", "request failures", "database response time by name". Do NOT use for explaining existing queries, product documentation questions, infrastructure metrics (use dt-obs-hosts), log analysis (use dt-obs-logs), or distributed tracing workflows (use dt-obs-tracing).

117 Updated 3 days ago
Dynatrace
AI & Automation Featured

agent-observability-spec

Specify the tracing, metrics, and alerting for an AI agent or LLM feature in production. Use when asked what to log for an LLM app, design agent tracing or spans, define quality and cost monitors, or answer 'how do we know if the agent is misbehaving?'. Produces an observability spec with a trace schema, metric definitions with owners and alert thresholds, sampling and retention policy, and a privacy note for logged content.

1,228 Updated today
mohitagw15856