agent-evaluation

Featured

Run and score exactly one Benchmark Case run with CLI execution, Trace provenance checks, and private Rubric isolation.

AI & Automation 199 stars 23 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 93/100

Stars 20%
77
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Agent Evaluation Act as an internal leaf worker. For one valid request, run and score exactly one Benchmark Case once, then return minimal protocol metadata. Do not design or refine the Benchmark, modify the Test Agent State, or write `scoreboard.yaml`. Do not use `run_subagent` or `input_subagent`. ## Before you start This Skill is invoked by `benchmark-design` or Benchmark mode in `agent-optimization`. Require one unambiguous protocol request containing every identity field below. If the request is missing, duplicated, or conflicting, return `invalid_request` without creating a Workspace or launching the Test Agent. Do not ask an interactive clarification from this leaf worker. ## Privacy boundary Before the final protocol YAML, emit no assistant text; use private reasoning and tool calls only. Never serialize Statement or artifact contents, Rubric items, expected values, correct outcomes, per-item scoring, diagnostics, secret configuration, Workspace paths, or Trace paths into an assistant message. The final assistant message is the protocol YAML only. It may echo the public identity fields supplied by the caller. A valid request contains exactly one value for each field: ```text protocol_version: 1 case_id: <case_id> run: <1_based_run_index> expected_version: <tested_agent_state_version> test_agent_id: <test_agent_id> benchmark_dir: <absolute_benchmark_dir> provider: <provider> model_id: <upstream_model_id> ``` ## Validate and prepare Resolve the Project, Test ...

Details

Author
Prism-Shadow
Repository
Prism-Shadow/penguin-harness
Created
1 weeks ago
Last Updated
today
Language
TypeScript
License
Apache-2.0

Similar Skills

Semantically similar based on skill content — not just same category