Exporting transactions for auditing¶
Warning
This feature is experimental and may change in future releases.
F5 AI Gateway supports exporting transaction data (or transactions, for brevity) for audit purposes.
A transaction corresponds to a client request handled by the AI Gateway. Every transaction gets a unique ID (a 16 byte UUID). If telemetry is enabled, AI Gateway will create an OTel trace per transaction with the trace ID matching the transaction ID.
The exported transaction data includes:
The content of the client request.
The response from the service.
Any modifications made by the processors to the request/response.
The metadata about the request/response and how the processors handled it.
AI Gateway supports exporting transactions to:
The standard output (for debugging/troubleshooting purposes).
AWS S3 or an AWS S3-compatible storage.
Warning
AI Gateway does not guarantee that it will export all transactons. As a consequence, it is possible that some transaction data will be lost.
Before you begin¶
Exporting transactions requires the exporter values in the Helm chart.
Exporting to standard output¶
Caution
This method is suited for debugging or troubleshooting purposes only.
To export transactions to the standard output, configure the values.yaml to enable storage
and the stdout exporter:
aigw:
exporter:
enabled: true # Enables the exporter
type: stdout # Exports to stdout
See an example transaction of an exported transaction.
Exporting to AWS S3 or AWS S3-compatible storage¶
To export transactions to AWS S3 or an AWS S3-compatible storage, configure the values.yaml:
aigw:
exporter:
enabled: true # Enables the exporter
type: s3 # Exports to s3
s3Bucket: my-bucket
where my-bucket is the name of the bucket where AI Gateway will export
the transactions. The bucket must exist.
Additionally, configure the authentication to S3. You can use environmental variables or other methods supported by AWS Go SDK. For example:
aigw:
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: aws-keys
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: aws-keys
key: AWS_SECRET_ACCESS_KEY
- name: AWS_REGION
value: "us-east-1"
- name: AWS_ENDPOINT_URL # Required for an S3-compatible storage that isn't Amazon S3; omit for Amazon S3.
value: https://my-s3-compatible-storage.example.com:8080
If your S3-compatible storage requires path-style addressing,
set s3UsePathStyle to true:
aigw:
exporter:
s3UsePathStyle: true
You can customize the timeout for exporting a transaction to S3 (default is 30s). For example:
aigw:
exporter:
s3UploadTimeout: 60s
AI Gateway creates a file per transaction in the bucket. The file name is in
the format <transaction-id>.json.
For example, 106e82cd377188b8d6c9d24c118cb76c.json. See an
example transaction of the content of the file.
Retrying of failed operations¶
If an upload fails, AI Gateway will try again using the AWS Go SDK defaults. For details, see the AWS Go SDK documentation.
If an upload continues to fail after multiple retries, AI Gateway marks the transaction as failed.
See the Error handling section for more information.
Extra configuration¶
Additional configuration, such as worker pool size can be set. For more details, see the configuration documentation.
Error handling¶
If AI Gateway fails to export a transaction, it will:
Log the corresponding error.
Transaction format¶
The format of the exported transaction is described in the JSON schema.
Warning
The transaction data includes the contents of all headers from the client request. Make sure the headers do not contain any sensitive data.
Example transaction¶
Below is an example of an exported transaction. For readability, we have added line breaks. Also, some strings in the OpenAI response were replaced with the string “example”.
Example:
{
"request": {
"request_id": "0195254a-093e-72bb-9ff9-4a29f7211152",
"tx_map": {
"request_id": "0195254a-093e-72bb-9ff9-4a29f7211152",
"input_chain": [
{
"steps": [
{
"id": "0195254a-093e-7687-9b86-9fce3af4e3d0",
"processor_id": "system-prompt",
"input_meta": {
"request_id": "0195254a-093e-72bb-9ff9-4a29f7211152",
"step_id": "0195254a-093e-7687-9b86-9fce3af4e3d0",
"params": {
"modify": true,
"rules": [
"reply in French"
]
}
},
"result_meta": {
"request_id": "",
"step_id": "",
"processor_id": "f5:system-prompt",
"processor_version": "v1",
"rejected": false,
"error": false
}
}
]
}
],
"response_chain": [
{
"steps": [
{
"id": "0195254a-093e-7693-a9ca-78354eefe102",
"processor_id": "language-id",
"input_meta": {
"request_id": "0195254a-093e-72bb-9ff9-4a29f7211152",
"step_id": "0195254a-093e-7693-a9ca-78354eefe102"
},
"result_meta": {
"request_id": "",
"step_id": "",
"processor_id": "f5:language-id",
"processor_version": "v1",
"processor_result": {
"detected_languages": {
"fr": 0.9925468564033508
}
},
"tags": {
"language": [
"fr"
]
},
"rejected": false,
"error": false
}
}
]
}
]
},
"metadata": {
"schema": "v1/chat_completions",
"original_headers": {
"Accept": [
"application/json"
],
"Content-Length": [
"118"
],
"Content-Type": [
"application/json"
],
"User-Agent": [
"curl/8.7.1"
]
}
},
"tags": {
"language": [
"fr"
]
}
},
"inputSequences": [
{
"name": "__input__",
"content": "{\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": \"explain TCP in one sentence\"\n }\n ]\n }"
},
{
"name": "0195254a-093e-7687-9b86-9fce3af4e3d0",
"content": "{\"messages\":[{\"content\":\"reply in French\",\"role\":\"system\"},{\"content\":\"explain TCP in one sentence\",\"role\":\"user\"}]}"
}
],
"responseSequences": [
{
"name": "__response__",
"content": "{\n \"id\": \"chatcmpl-example\",\n \"object\": \"chat.completion\",\n \"created\": 1740087364,\n \"model\": \"gpt-4o-mini-2024-07-18\",\n \"choices\": [\n {\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"Le TCP (Transmission Control Protocol) est un protocole de communication qui garantit une transmission fiable et ordonnée des données sur un réseau en établissant une connexion entre l'expéditeur et le destinataire.\",\n \"refusal\": null\n },\n \"logprobs\": null,\n \"finish_reason\": \"stop\"\n }\n ],\n \"usage\": {\n \"prompt_tokens\": 20,\n \"completion_tokens\": 41,\n \"total_tokens\": 61,\n \"prompt_tokens_details\": {\n \"cached_tokens\": 0,\n \"audio_tokens\": 0\n },\n \"completion_tokens_details\": {\n \"reasoning_tokens\": 0,\n \"audio_tokens\": 0,\n \"accepted_prediction_tokens\": 0,\n \"rejected_prediction_tokens\": 0\n }\n },\n \"service_tier\": \"default\",\n \"system_fingerprint\": \"fp_example\"\n}\n"
},
{
"name": "0195254a-093e-7693-a9ca-78354eefe102",
"content": ""
}
]
}