Prompt guard¶

Warning

You are using an EXPERIMENTAL processor! Experimental processors:

May have bugs or stability issues
May experience breaking API changes
May not produce the expected results

By using this experimental processor you acknowledge:

It should NOT be used in a production context
It is NOT covered under F5 support agreements
Some experiments are not successful - the functionality could be retired.

Before you begin¶

Follow the steps in the Install with Helm topic to run F5 AI Gateway.

This processor is not part of the default installation and must be enabled like this:

processorLabs:
  promptGuard:
    enable: true

Overview¶

The F5 prompt guard processor runs in the AI Gateway processors container. This processor detects and optionally blocks prompt injection attacks.

Processor details	Supported
Deterministic	No
GPU acceleration support	Yes
Base Memory Requirement	655 MB
Input stage	Yes
Response stage	No
Recommended position in stage	Beginning
Supported language(s)	English, French, German, Hindi, Italian, Portuguese, Spanish, Thai.

Required processor order¶

The prompt-guard processor only supports English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai language prompts; prompt injection attacks crafted in any other language will not be detected.

Configuration¶

processors:
  - name: prompt-guard
    type: external
    config:
      endpoint: https://aigw-processors-f5.ai-gateway.svc.cluster.local
      namespace: f5-processor-labs
      version: 1
    params:
      experimental: true
      reject: true
      threshold: 0.95

Parameters¶

Parameters	Description	Type	Required	Defaults	Examples
Common parameters
`experimental`	This flag acts as an acknowledgement that you are using an experimental processor. The processor will not run unless this is set to `true`.	boolean	Yes	`false`	`true`
`threshold`	Minimum confidence score required to treat the prompt as an injection attack. Lower values will make the processor more strict, but more likely to trigger false-positives.	float `0.0` to `1.0`	No	`0.5`	`0.5`

When reject is set to true, this processor will reject the request when an injection attack is detected, otherwise it will add to the attacks-detected tag.

Tags¶

Tag key	Description	Example values
`detected`	Added if reject is set to false and prompt injection is detected.	`{'detected': {'malicious': 0.64}}`

Chunking input and batch processing¶

The prompt guard processor will split inputs and responses into overlapping chunks and perform inference on these chunks in batches. Chunks are designed to overlap so that context is preserved across boundaries; this ensures that if a prompt injection occurs at the edge of one chunk, the overlapping region with the next chunk will still capture it.

Note

Always perform empirical tests on hardware with real or representative data. Profiling is the best way to see how changing chunk and/or batch sizes impacts performance.

Chunking input¶

Chunk size controls how much of the input is processed at a time. It’s based on the model’s maximum input limit and how much context the task needs. Larger chunks use more memory and may slow things down, while smaller chunks can be faster but might miss important context.

The prompt guard processor splits its input into chunks of a variable number of tokens, between 32 and 512 ( default: 512). The number of tokens is configurable by setting PROMPT_GUARD_PROCESSOR_CHUNK_SIZE in the processors.f5.env section of the AI Gateway Helm chart.

The prompt guard processor implements a sliding window (overlap) for chunking input. A sliding window refers to the practice of dividing longer text into overlapping chunks so that a model can capture context that spans chunk boundaries. During inference, each chunk is fed separately into the classification model. Because each chunk is passed through the model (a forward pass), the process can increase memory usage as more chunks are generated and processed. Too much overlap can lead to repeated processing of the same tokens, which might not improve prediction efficacy and could even introduce redundancy in the predictions. Decreased overlap reduces redundancy in the processed data but with little or no overlap, the model might miss contextual cues that lie near the chunk boundaries, potentially reducing prediction consistency across segments.

The default chunk overlap size in tokens is half the value of the chunk size setting; to disable overlapping set the environment variable PROMPT_GUARD_PROCESSOR_CHUNK_OVERLAP: 0. This value must not be set to a value larger than chunk_size - 1.

Previous Next