There is no guardrails_config top-level block. Earlier revisions of this
page described one, along with per-provider blocks for AWS Bedrock Guardrails,
Azure Content Safety, Google Model Armor and others. None of that is read by the
gateway.A config.json containing guardrails_config parses, the gateway boots and the
block is discarded — so a deployment that declares its entire content policy that
way runs with no guardrails. As of this release the gateway logs every
top-level section it does not read at startup, so the situation is at least
visible. Configure guardrails through the plugins array as shown below.
config object.
Fields
Top level
A provider
api_key is a plain string. Unlike provider credentials it cannot yet be given
as "env.VAR_NAME" or read from a secret manager.
Scoping a rule
A rule with no scope applies to everything. Naming virtual keys, teams or models
narrows it:
Dimensions are ANDed, values within one are ORed — the example applies to
the legal team, and only on OpenAI models. Virtual keys match on id or name,
because operators think in names and automation carries ids.
This exists because one policy for the whole gateway ends up set to whatever the
most permissive consumer can tolerate, leaving the strict consumer unprotected.
A scoped rule applies only when the scope is known. A request carrying no
virtual key escapes every scoped rule and is covered only by the unscoped ones.That is safe when client.enforce_auth_on_inference is set, because there is
then no such thing as a request without a virtual key. With it unset, an
anonymous caller was already reaching the model with no governance at all.The alternative — applying scoped rules when identity is unknown — sounds safer
and is worse: one team’s rule would start blocking every other team’s traffic the
moment identity resolution failed.
Redaction
action: "redact" masks the matched span instead of refusing the exchange.
The provider receives [CUSTOMER PII REDACTED] in place of the address; the rest
of the message is untouched.
This exists because blocking alone is blunt in the way that gets a rule
disabled: a support transcript containing one customer email is refused
entirely, so the team switches the PII rule off and ends with no coverage rather
than partial coverage. A rule people leave on is worth more than a stricter one
they turn off.
The mask is a fixed marker, not a same-length run of asterisks: preserving the
length leaks how long the secret was, which narrows a search for a credential and
is most of the information for a phone number.
redact is rejected at validation on llm and injection rules. Those judge the
whole text rather than locating a span, so redacting them would delete everything
or nothing — and a policy that says “redact” while blocking is doing something
other than what it says.
Prompt-injection detection
The injection type scores weighted signals — instruction override, role
hijack, named jailbreaks, policy subversion, system-prompt extraction, chat
delimiter injection — and blocks when they pass a threshold.
It normalises before matching, so zero-width characters, full-width forms and
base64-encoded payloads do not evade it. It also inspects tool results, since
extract walks every message regardless of role — which is the MCP attack where
a malicious server returns instructions rather than data.
This is a detector, not a solution. Prompt injection is not solvable by pattern
matching: an attacker who knows the rules can phrase around them. It catches the
large majority of real attempts — copied jailbreak prompts, encoded payloads,
injected tool results — and raises the cost of the rest. It does not remove the
need for least privilege on tools, or the rule that model output is never
authorization.
system: is how people paste a transcript. A test suite of 24
legitimate messages that talk about instructions, roles and prompts guards the
false-positive rate, and a held-out corpus checks that it generalises.
A full example
Validation
A policy the gateway could not fully enforce is rejected rather than saved:
an invalid RE2 pattern, an unknown PII category, a rule left with nothing to
match, an llm rule missing its endpoint or model, or an unrecognised
apply_to. Over the API that is a 400; at startup the affected element is
dropped, the rest of the policy runs, and the plugin reports status degraded.
See also
- Guardrails — what the policy does, what it covers,
and what it does not do.
