skill-forge-benchmarklisted

# Skill Benchmarking & Performance Tracking Measure and compare skill performance across iterations with statistical rigor using multiple trials, variance analysis, and trend tracking. ## Process ### Step 1: Define Benchmark Configuration Accept configuration as: - **Existing eval set**: Path to `evals/evals.json` (from `/skill-forge eval`) - **Benchmark config**: Custom config with trial count and thresholds **Benchmark config schema:** ```json { "skill_name": "my-skill", "skill_path": "./my-skill", "eval_set_path": "./evals/evals.json", "trials_per_eval": 3, "baseline_type": "no_skill", "previous_benchmark": null, "thresholds": { "min_pass_rate": 0.8, "max_avg_tokens": 100000, "max_avg_duration_seconds": 120, "min_improvement_ratio": 1.0 } } ``` ### Step 2: Execute Benchmark Runs For each eval, run `trials_per_eval` times (default: 3) to get reliable metrics: 1. Execute with-skill runs (3x per eval) 2. Execute baseline runs (3x per eval) 3. Capture per-run: pass/fail, token count, duration 4. Save each run's `timing.json` and `grading.json` Use `agents/skill-forge-executor.md` for parallel execution where possible. ### Step 3: Aggregate Results Run `python scripts/aggregate_benchmark.py <workspace>/iteration-<N> --skill-name <name>`: **Output `benchmark.json` schema:** ```json { "skill_name": "my-skill", "iteration": 1, "timestamp": "2026-03-06T12:00:00Z", "summary": { "total_evals": 10, "with_skill": { "pass