ai-observabilitylisted

# AI Observability ## Objective Make AI behavior diagnosable and measurable without turning telemetry into a sensitive-data store. ## Workflow 1. Map AI components, providers, prompts, datasets, tools, users, trust boundaries, objectives, and material failure modes. 2. Define trace and span structure for requests, model calls, retrieval, reranking, tools, guardrails, and final output. 3. Capture versions, latency, tokens, cost, status, structured errors, quality signals, and correlation IDs. 4. Apply field-level allowlists, redaction, sampling, retention, access control, and regional requirements before collecting content. 5. Build dashboards and alerts around user impact, quality regressions, provider failure, cost anomalies, and unsafe tool behavior. 6. Connect production signals to versioned evaluation cases and incident workflows without silently training on user data. 7. Test telemetry failure, provider outage, partial traces, and redaction effectiveness, then document evidence and residual blind spots. ## Safety and authorization - Do not collect prompts, completions, retrieval context, or user identifiers by default. Justify every content field. - Review hosted observability data flows, subprocessors, retention, and access before enabling export. - Observability must degrade safely and must not block the user journey unless the security policy requires it. ## Evidence and completion - Record scope, versions, datasets, environments, assumptions, exclusions, and