weak-agent-test

Featured

Run the weak-agent adversarial test harness against docx-cli. Spawns weak exercise agents (Haiku by default, Sonnet to probe, or a local agent harness's pre-produced runs) to perform real document tasks over six scenarios — five editing (MNDA form-fill + font fidelity, invoice table-edit/restructure + logo replace, résumé styling, contract redlining + commenting, contract finalize via accept/reject + comment reply/resolve) and one authoring (T. S. Eliot poetry journal: multi-column, verse, footnotes, links, figure) — renders every result with Word, has opus judge them against ground-truth rubrics, measures each exercise's tool economy, token cost, wall-clock, and correctness (from transcripts for Claude, the exercise.json ledger for the local harness), and synthesizes a prioritized ergonomics report. Use when the user says 'adversarial review', 'test docx-cli with weak agents', 'run the haiku harness', 'weak agent test', or wants to re-run yesterday's adversarial process.

Data & Documents 204 stars 9 forks Updated 6 days ago MIT

Install

View on GitHub

Quality Score: 91/100

Stars 20%
77
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Adversarial review — weak-agent harness for docx-cli This harness answers one question: **can weak agents actually use docx-cli to get real work done, and what should we fix first?** It runs the `weak-agent-test` workflow (`.claude/workflows/weak-agent-test.js`), which fans out one weak exercise agent per scenario (Haiku by default — swappable to Sonnet via `args.model`), renders every output with Microsoft Word, grades each against ground-truth criteria with an **opus** judge, and has **opus** synthesize a prioritized improvement report. Exercise agents do NOT self-report tool counts — every tool-economy and token number is **measured** after the run (agents under-count their own calls ~2×, so self-reports were dropped): from the agent transcripts for the Claude arms, from each scenario's `exercise.json` ledger for the local arm. Both roll up into the same Run-metrics table (tokens, wall-clock, tool split, correctness) via `exercise-metrics.ts`. The test corpus is **bundled with this skill** under `scenarios/`, one folder per scenario, named after its key (`scenarios/mnda/`, `scenarios/invoice/`, …). Each scenario folder is self-describing and holds everything that scenario needs: - `task.md` — the AGENT-FACING request, written as a human delegating the work: the goal, the data, the intent — and **no tool vocabulary** (no `docx` commands, locators, or OOXML terms), because discovering which features deliver the outcome is part of what's measured, - `criteria.md` —...

Details

Author
kklimuk
Repository
kklimuk/docx-cli
Created
4 months ago
Last Updated
6 days ago
Language
TypeScript
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category

Data & Documents Listed

docx

Use when the user asks to create, edit, convert, validate, preview, or password-protect Microsoft Word .docx documents. Triggers include "markdown to docx", "docx to markdown", "fill Word template", "accept tracked changes", "validate docx", "preview docx as image", "encrypt/decrypt docx", and related .docx round-trip or template-fill tasks.

1 Updated today
MatrixFounder
Code & Development Listed

agent-benchmark

Self-benchmark: YOU write the code, adversarial reviews it (multi-provider), you fix, you write tests, adversarial reviews tests, you fix. Measures YOUR quality as an agent. Run in different models (Opus, Sonnet, Haiku) and compare results.

6 Updated 4 days ago
greglas75
AI & Automation Listed

tune-agent-docs

Reviews every markdown file in a repository that steers an AI coding agent — CLAUDE.md, AGENTS.md, GEMINI.md, `.cursor/rules/*.mdc`, `.clinerules`, `.windsurfrules`/`.windsurf/rules`, `.github/copilot-instructions.md`, Kiro's `.kiro/steering/*.md`, and similar — together as one corpus, then tightens them. Use whenever the user wants to audit, tune, reconcile, or clean up the instructions a repo gives its AI agents, or asks why an agent keeps missing or contradicting its own steering docs, or burning tokens on them — even when they only name one file, since the value is in reading it alongside its neighbors. Checks consistent terminology, leading-word front-loading, and each doc's token budget — size against its own format's stated limits, content loaded unconditionally that the harness could scope instead, guidance duplicated across multiple docs — applies the repo's own doc-authoring conventions where they hold generally, and calibrates each instruction's degrees of freedom to the fragility of what it govern

0 Updated today
azborgonovo