evaluating-model-guardrails

# Evaluating model guardrails: measure the boundary, don't collect one jailbreak A guardrail is a claim: "the model will refuse this class of request." A single successful jailbreak is an anecdote; what you need is a measurement, the rate at which the boundary holds across a structured battery of attacks. Evaluating guardrails means grounding probes in the actual policy, attacking each along known bypass axes, and scoring outcomes reproducibly, so you can state how strong the boundary is, not just that someone once beat it. ## When to use - You are reviewing a model deployment, a system prompt, or a content-safety layer. - Before shipping an assistant that must refuse or constrain a defined set of requests. - You need a defensible bypass rate, not a single proof-of-concept transcript. ## Scope check Evaluate models and deployments you own or are authorized to test. Use benign, clearly-scoped probes against a defined policy; do not generate real harmful output against systems you do not control. If you can't name the authorization, stop. ## The loop 1. **Ground the probes in the stated policy.** Get the actual list of what this deployment must refuse or constrain (its safety policy, its system prompt's rules, its allowed scope). Every probe targets a specific rule, so a result maps to a policy line, not a vibe. An undefined policy is the first finding: you cannot evaluate a boundary no one has drawn. 2. **Build a baseline probe set.** For each rule, write...