evalslisted

## Customization **Before executing, check for user customizations at:** `~/.claude/LIFEOS/USER/CUSTOMIZATIONS/SKILLS/Evals/` If this directory exists, load and apply any PREFERENCES.md, configurations, or resources found there. These override default behavior. If the directory does not exist, proceed with skill defaults. ## 🚨 MANDATORY: Voice Notification (REQUIRED BEFORE ANY ACTION) **You MUST send this notification BEFORE doing anything else when this skill is invoked.** 1. **Send voice notification**: ```bash curl -s -X POST http://localhost:31337/notify \ -H "Content-Type: application/json" \ -d '{"message": "Running the WORKFLOWNAME workflow in the Evals skill to ACTION"}' \ > /dev/null 2>&1 & ``` 2. **Output text notification**: ``` Running the **WorkflowName** workflow in the **Evals** skill to ACTION... ``` **This is not optional. Execute this curl command immediately upon skill invocation.** # Evals - AI Agent Evaluation Framework ## What It Does Evaluates AI agents — their transcripts, tool-call sequences, and multi-turn conversations, not just single outputs. Three grader types cover it: code-based for deterministic checks, model-based for nuanced quality rubrics, and human for the gold standard. Scores with pass@k (capability) and pass^k (consistency). Splits evals into capability suites (~70% target) and regression suites (~99% target), and plugs into Algorithm ISC rows as a verification method. ## The Problem You c