add-middleware

Featured

Add a new NeMo Relay guardrail or intercept type, registration surface, or pipeline stage. Do not use for changing an existing middleware implementation without a new middleware contract.

AI & Automation 177 stars 75 forks Updated today Apache-2.0

Install

View on GitHub

Quality Score: 90/100

Stars 20%
75
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Add a Middleware Type NeMo Relay supports guardrails (validate/gate) and intercepts (transform) at various pipeline stages. Adding a new middleware type requires checking every layer that exposes the new contract. Use this skill when introducing a new middleware registration surface or adding middleware behavior to a new pipeline stage. ## Lock The Design First Decide these before editing code: - Is this for tools, LLMs, marks, scope events, or a combination? - Is it a conditional guardrail, sanitize guardrail, request intercept, or execution intercept? - Does it run on request input, inner callable execution, stream chunks, or final response output? - Is the callback fallible, and how should callback failures propagate? - Does it need both global and scope-local registration? - What should subscribers and exporters observe in the event payload after this middleware runs? - If this is an event sanitizer, which of `data`, `category_profile`, and `metadata` can change, and is the event used only as immutable context? ## Pipeline Order Refer to `docs/about-nemo-relay/concepts/middleware.mdx` for the full diagrams. - **Tool execute**: conditional guardrails -> request intercepts -> sanitize request (for events) | execution intercept chain(callable) -> sanitize response - **LLM execute**: conditional guardrails -> request intercepts -> sanitize request (for events) | execution intercept chain(callable) -> sanitize response - **Mark and scope events**: s...

Details

Author
NVIDIA
Repository
NVIDIA/NeMo-Relay
Created
5 months ago
Last Updated
today
Language
Rust
License
Apache-2.0

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category