jev-judgment-designlisted

How to hand over judgments and decide accept/reject when moving closed judgments an LLM used to make (is this source relevant, is it new, how strong is the evidence) to TypeSafe's Jev. Putting what the judgment is made against into state, the bottom tier of a Score, Jev answers and code decides, the unit of judgment, sources that must pass (canary), calling from Pydantic AI. Use when — "I want to replace the LLM's judgment with Jev", "build a relevance judgment with Jev", "Jev's judgment lets everything through", or before designing Jev into screening / triage / rerank. NOT for — Jev's API, docs, or general guidance on choosing primitives (→ plugin skill `typesafe:typesafe-ai`; read that first), the Claude Code skill-selection hook (→ `jev-skill-router`), designing an LLM as the judge (→ `llm-as-judge`).
shimo4228/claude-harness · ★ 3 · AI & Automation · score 79

Install: claude install-skill shimo4228/claude-harness

# Jev judgment design Jev answers closed questions with probabilities and does not write prose. Moving an LLM pipeline's judgments to Jev makes them cheap and parallel, but Jev answers only within the state it is handed. This skill owns the design of that state and of the side that decides accept/reject from the answers. The general design guidance for Jev (state / instructions / criteria, choosing Choice, Noul, or Score, no-match, set thresholds on your own data) is canonical in plugin skill `typesafe:typesafe-ai`, so read that first and read only the judgments layered on top here. Sources: the implementation in jev-research-pipeline (https://github.com/shimo4228/jev-research-pipeline) and the article "Moving the research judgments I left to an LLM to Jev, a judgment-only model" (https://zenn.dev/shimo4228/articles/jev-research-judgment-offload, scheduled for publication 2026-09-25). ## 1. Move only closed judgments Move to Jev the judgments whose answer can be stated as a yes/no probability, one pick from a set of options, or a point on a graded scale. Keep work with an open-ended answer shape, such as where to search next or how to write, with the LLM. After the move, watch the instrument "number of LLM judgments during a run", not "number of questions to Jev" — Jev's question count may grow by as many judgments as were moved. ## 2. Put what the judgment is made against into state "Is it relevant" lets through anything that shares vocabulary when the comparison targe