trajectory-review

Solid

Post-hoc diagnosis of a failed agent trajectory. Classifies the first unrecoverable step into one of nine failure categories (plan adherence, hallucinated information, invalid tool call, misread tool output, intent–plan mismatch, under-specified intent, unsupported intent, guardrail trigger, system failure) and produces an evidence-backed root-cause report.

Code & Development 41 stars 7 forks Updated today MIT

Install

View on GitHub

Quality Score: 82/100

Stars 20%
54
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Trajectory Review ## Overview When an agent run fails, the failure mode is rarely "the model is bad". It is usually one of a small set of recurring problems on the trajectory: the agent skipped a planned step, invented a fact, called a tool wrong, misread a tool's output, or pursued the wrong subgoal entirely. Output-only review cannot distinguish these — they all surface as "the answer was wrong". This skill takes a captured trajectory (tool calls, intermediate outputs, final response) and locates the **first unrecoverable step**, classifies it into one of nine categories, and reports the root cause with citations into the trajectory. The taxonomy and four-stage diagnostic procedure are adapted from Microsoft Research's AgentRx framework (2026-04). The classes themselves are stable across agent stacks; the diagnostic stages are how this skill operates inside a Claude Code or Codex session. ## When to Activate - An agent run produced a wrong or incomplete result and you have the trajectory. - A user reports "the agent is broken" and a postmortem is needed. - A regression appeared after a model upgrade and you need to know whether it is a model issue or a harness issue. - A new capability shipped and you want to characterize the failure modes that remain. - A user says "review the trajectory", "diagnose this run", or "why did the agent fail". Do **not** use this skill to evaluate a passing run. For "evals pass but I don't trust it", use the W-18 three-axis evaluation ...

Details

Author
majiayu000
Repository
majiayu000/vibeguard
Created
6 months ago
Last Updated
today
Language
Shell
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Listed

agent-trajectory-evaluator

Evaluate a multi-step AI agent's whole run — tool calls, intermediate steps, and final result — not just final-answer correctness, so you can pinpoint WHERE it went wrong. Use when building or debugging a tool-using or multi-step agent, when final-answer-only evals can't explain failures, or when a prompt/model change quietly makes the agent less efficient or more error-prone even though the answer still looks right.

1 Updated today
imtiazrayhan
AI & Automation Listed

trajectory

Discover, run, interpret, and iteratively tune a project's existing agent trajectory evaluation capability. Use when the user asks to evaluate agent trajectories, compare effect and cost across runs, find trajectory smells or the next trajectory problem, or identify optimizations for a known trajectory issue. Operate project-owned trajectory evidence and canonical entrypoints; do not invent missing evidence, verdicts, or a parallel evaluation framework.

4 Updated today
compforge
AI & Automation Listed

agent-failure-diagnosis

Diagnose why an AI agent behaved badly, using operationalised criteria that two independent reviewers can apply to the same evidence and reach the same answer. Use this whenever someone describes an agent that misbehaved, went off track, did something unexpected, ignored instructions, made things up, went beyond its scope, got stuck in a loop, lied about what it did, or "went rogue" — and whenever reviewing agent traces, logs, or transcripts to work out what went wrong. Use it for post-incident analysis, for design reviews asking "how could this fail?", and when someone needs to classify agent failures consistently enough to spot patterns across many incidents. Reach for this even when the user just wants an explanation rather than a formal report, because the classification is what makes the explanation defensible later.

0 Updated today
PKusch