Exporting transactions for auditing

F5 AI Gateway supports exporting transaction data (or transactions, for brevity) for audit purposes.

A transaction corresponds to a client request handled by the AI Gateway. Every transaction gets a unique ID (a 16 byte UUID). If telemetry is enabled, the AI Gateway will create an OTel trace per transaction with the trace ID matching the transaction ID.

The exported transaction data includes:

  • The content of the client request.

  • The response from the service.

  • Any modifications made by the processors to the request/response.

  • The metadata about the request/response and how the processors handled it.

AI Gateway supports exporting transactions to:

  • The standard output.

  • AWS S3 or an AWS S3-compatible storage.

Additionally, transactions can be retained on disk.

Before you begin

Exporting transactions as well as retaining them on disk requires enabling the --file-storage flag.

Exporting to standard output

Caution

This method is suited for debugging or troubleshooting purposes only.

To export transactions to the standard output, use the following flags:

--exporter-enable
--exporter-type=stdout

You can omit -exporter-type-stdout, as the default value of that flag is already stdout.

See an example transaction of an exported transaction.

Read the Error handling section to learn what happens if the AI Gateway fails to export a transaction.

Exporting to AWS S3 or AWS S3-compatible storage

To export transactions to AWS S3 or an AWS S3-compatible storage, use the following flags:

--exporter-enable
--exporter-type=s3
--exporter-s3-bucket=my-bucket

where my-bucket is the name of the bucket, where the AI Gateway will export the transactions. The bucket must exist.

Additionally, configure the authentication to S3. You can use environmental variables or other methods supported by AWS Go SDK.

The AI Gateway creates a file per transaction in the bucket. The file name is in the format <transaction-id>.json. For example, 106e82cd377188b8d6c9d24c118cb76c.json. See an example transaction of the content of the file.

Retrying of failed operations

If an upload operation fails, the AI Gateway will retry it using the AWS Go SDK defaults. See AWS Go SDK documentation.

If uploading fails after retries, the AI Gateway will consider the transaction as failed to be exported. Read the Error handling section to learn what happens in this case.

Extra S3 configuration

Use the --exporter-s3-upload-timeout flag to specify a timeout for an upload operation, if necessary.

Extra configuration

  • You can batch exporting transactions with -exporter-batch-size flag. For example, if you set it to 1024, the AI Gateway will export 1024 transactions at a time.

  • You can parallelize exporting transactions by specifying the number of workers using --exporter-workers. The AI Gateway will distribute the batch among those workers for parallel export.

  • Upon a successful export, the AI Gateway will remove the transaction files (see the Retain on disk section to learn about the transaction files). Set --exporter-remove-files to false to disable removing files.

  • To control how many pending for export transactions the AI Gateway can have, use --exporter-max-pending-transactions flag.

Error handling

If the AI Gateway fails to export a transaction, it will:

  • Log the corresponding error.

  • Keep the transaction files on disk. See Retain on disk section to learn where to look for it.

Retain on disk

If --file-storage flag is enabled, the AI Gateway will retain transactions on disk. However, if you enable exporting and also the flag --exporter-remove-files is set to true (which is the default), the AI Gateway will remove transaction files upon a successful export. At the same time, failed to be exported transactions are always retained, no matter the value of the --exporter-remove-files flag.

The transactions are located in the folder configured by the --storage-path flag. For each transaction, the AI Gateway creates a subdirectory with the name that matches the transaction id. Here is an example of the contents of the directory of a transaction:

ls -1 106e82cd377188b8d6c9d24c118cb76c
0191b94d-080e-7bc3-a7fe-106060c89339.data
__input__.data
__response__.data
transaction.json

Note that the content is similar to the example transaction, except inputSequences and responseSequencessequences are stored as separate files without any escaping of their content.

An active transaction (the AI Gateway has not yet sent the response to the client) has the "closed": false entry in the transaction.json file.

Example transaction

Below is an example of an exported transaction. For readability, we have added line breaks.

Example:

{
   "request": {
   "request_id": "106e82cd377188b8d6c9d24c118cb76c",
   "tx_map": {
      "request_id": "106e82cd377188b8d6c9d24c118cb76c",
      "input_chain": [
      {
         "steps": [
         {
            "id": "0191b94d-080e-7bac-8316-72d37d8e71e9",
            "processor_id": "language-id",
            "input_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bac-8316-72d37d8e71e9",
            "body_size": 31,
            "response_size": 0
            },
            "result_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bac-8316-72d37d8e71e9",
            "processor_id": "language:language-id",
            "processor_version": "",
            "processor_result": {
               "detected_language": "en"
            },
            "tags": [
               "en"
            ]
            }
         },
         {
            "id": "0191b94d-080e-7bb4-b691-4779cf227e17",
            "processor_id": "persona/detect-persona",
            "input_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bb4-b691-4779cf227e17",
            "body_size": 31,
            "response_size": 0
            },
            "result_meta": {
            "request_id": "",
            "step_id": "",
            "processor_id": "detect-persona",
            "processor_version": "v1"
            }
         }
         ]
      },
      {
         "steps": [
         {
            "id": "0191b94d-080e-7bbf-965a-51da46615e09",
            "processor_id": "prompt-injection",
            "input_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bbf-965a-51da46615e09",
            "params": {
               "allow_rejection": true
            },
            "body_size": 31,
            "response_size": 0
            },
            "result_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bbf-965a-51da46615e09",
            "processor_id": "xcdf:prompt-injection",
            "processor_version": "",
            "processor_result": {
               "predictions": [
               {
                  "label": "NORMAL",
                  "score": 0.919903576374054
               }
               ]
            }
            }
         }
         ]
      }
      ],
      "response_chain": [
      {
         "steps": [
         {
            "id": "0191b94d-080e-7bc3-a7fe-106060c89339",
            "processor_id": "pii",
            "input_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bc3-a7fe-106060c89339",
            "params": {
               "allow_rewrite": true
            },
            "body_size": 31,
            "response_size": 382
            },
            "result_meta": {
            "request_id": "106e82cd377188b8d6c9d24c118cb76c",
            "step_id": "0191b94d-080e-7bc3-a7fe-106060c89339",
            "processor_id": "xcdf:pii",
            "processor_version": "",
            "processor_result": {
               "predictions": [
               {
                  "end": 14,
                  "entity_group": "PHONE_NUMBER",
                  "score": 0.4880453944206238,
                  "start": 0,
                  "word": " 123-456-7890."
               }
               ]
            }
            }
         }
         ]
      }
      ]
   },
   "metadata": {
      "schema": {
      "Name": "raw",
      "Short": "raw",
      "Custom": false
      }
   },
   "tags": [
      "en"
   ],
   "closed": true
   },
   "inputSequences": [
   {
      "name": "__input__.data",
      "content": "schema:raw\nan example of a US phone number"
   }
   ],
   "responseSequences": [
   {
      "name": "__response__.data",
      "content": "schema:oai\n{\"model\":\"phi3\",\"choices\":[{\"message\":{\"role\":\"assistant\",\"content\":\" 123-456-7890. This is a simple, fictional representation that follows the standard format for U.S. telephone numbers which includes three digits (the area code) followed by seven more numerical digits to complete the eight-digit sequence required for an actual phone number in the United States.\"}}]}\n"
   },
   {
      "name": "0191b94d-080e-7bc3-a7fe-106060c89339.data",
      "content": "schema:oai\n{\"model\":\"phi3\",\"choices\":[{\"message\":{\"role\":\"user\",\"content\":\" ****** This is a simple, fictional representation that follows the standard format for U.S. telephone numbers which includes three digits (the area code) followed by seven more numerical digits to complete the eight-digit sequence required for an actual phone number in the United States.\"}}]}\n"
   }
   ]
}