ai-agent-observability-evaluation

# AI Agent Observability Evaluation ## Operating contract ## Inputs | Input | Required | Purpose | |---|---|---| | Domain evidence | yes | task contract, trace schema, evaluation dataset, success thresholds, privacy rules, and sampling window | ## Outputs - Produce: trace and outcome scorecard, failure taxonomy, replay evidence, regressions, and release recommendation. ## Capability and permission boundaries Default to read-only analysis. Read only scoped records; redact secrets and regulated data. Writes, execution, network calls, production configuration, customer communication, billing changes, and delegation require explicit authority and an identified owner. Never widen tenant, time-window, or system scope implicitly. ## Degraded mode When required telemetry, evidence, execution, network access, or write authority is unavailable, return a partial result with each unassessed item labelled, preserve the safest existing state, and state the evidence or approval needed to continue. Never convert missing evidence into a pass. ## Decision rules | Condition | Action | |---|---| | Scope, owner, or threshold is missing | Stop the affected decision and request it | | Evidence is incomplete but read-only analysis is safe | Produce a qualified partial result and gap list | | A mutation exceeds authority or tenant boundary | Block it and route for approval | | Evidence meets the stated threshold | Issue the output with provenance and owner | ## Anti-Patterns - Treating a...