add-middleware

Solid

Add a new guardrail or intercept type to the NeMo Relay middleware pipeline

AI & Automation 122 stars 57 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 86/100

Stars 20%
70
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Add a Middleware Type ## Companion Guidance Use `karpathy-guidelines` alongside this skill for implementation or review work. Keep changes scoped, surface assumptions, and define focused validation before editing. NeMo Relay supports guardrails (validate/gate) and intercepts (transform) at various pipeline stages. Adding a new middleware type requires changes across all layers. Use this skill when introducing a new middleware registration surface or adding middleware behavior to a new pipeline stage. ## Lock The Design First Decide these before editing code: - Is this for tools, LLMs, marks, scope events, or a combination? - Is it a conditional guardrail, sanitize guardrail, request intercept, or execution intercept? - Does it run on request input, inner callable execution, stream chunks, or final response output? - Is the callback fallible, and how should callback failures propagate? - Does it need both global and scope-local registration? - What should subscribers and exporters observe in the event payload after this middleware runs? - If this is an event sanitizer, which of `data`, `category_profile`, and `metadata` can change, and is the event used only as immutable context? ## Pipeline Order Refer to `docs/about-nemo-relay/concepts/middleware.mdx` for the full diagrams. - **Tool execute**: conditional guardrails -> request intercepts -> sanitize request (for events) | execution intercept chain(callable) -> sanitize response - **LLM execute**: co...

Details

Author
NVIDIA
Repository
NVIDIA/NeMo-Relay
Created
4 months ago
Last Updated
today
Language
Rust
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category