NeMo Guardrails + injection fencing

The threat model RAG actually has

A RAG system's most under-appreciated input channel is its own corpus: anything retrieved — a parsed PDF, a scraped doc, pasted content — flows into the prompt. If a document says "ignore your instructions and…", retrieval will happily deliver it. So this platform treats retrieved text as attacker-controlled input, the way a web app treats query strings.

Fencing: first in the chain, never destructive

The core defense is injection fencing, a custom LlamaIndex node postprocessor that runs first — before the reranker, before synthesis: guard → rerank. Nothing downstream ever consumes unfenced retrieved text.

Detection is a set of compiled regexes, one per attack family (instruction override, role hijack, exfiltration patterns, …) — deterministic, in-process, microseconds, zero model calls. Flagged content is wrapped, not deleted: fenced blocks preserve the original substring and mark it as untrusted data, so the LLM can still cite a document that happens to contain hostile text without obeying it. A destructive filter would corrupt the corpus's honesty; a fence keeps the content and removes its authority.

On the way out, output screening applies the same family checks to what the model produced — the second half of the contract.

NeMo rails, in their own lane

NeMo Guardrails layers policy rails on the conversational paths. Like every optional capability here it lives in its own dependency lane (requirements-guard.txt), never in the base service image, and its verdicts reduce to a small status the routes can act on. The integration carries an injection seam for tests, so rail behavior is assertable in CI without a live model.

Design position

Deterministic defenses first — they're free, fast, and auditable; model-based policy second; and every layer testable offline. The same philosophy as the eval stack: don't ask an LLM to do a regex's job, and don't trust any layer you can't test deterministically. The general theory — guardrail taxonomies, layered defenses, HITL as a rail — is the Agent Guardrails deep dive.

The invariants

• Fencing runs first in the postprocessor chain — nothing downstream ever reads unfenced retrieved text.
• Detection is one compiled regex family per attack pattern — in-process, microseconds, zero model calls.
• Flagged content is wrapped, never deleted — the corpus keeps its honesty, hostile text loses its authority.
• The same family checks screen output on the way out — the second half of the contract.