Changelog¶

AI Gateway Core¶

✨ Highlights:

• Export OpenTelemetry Metrics

• Simplify Route schema configuration.

• Entitlement and Visibility tracking enabled (for usage-based pricing compliance)

🐛 Bug fixes:

• Fixed an issue with OpenTelemetry tracing that sometimes resulted in duplicate spans.

• Improve OpenTelemetry Tracing configuration to avoid performance degradation under load

• Exit on TLS misconfiguration

• Fix various Selector evaluation and configuration bugs.

• Support path style addressing when connecting to AWS S3 for transaction exporting.

AI Gateway Processors¶

✨ Highlights:

• F5 processor set revised to:

  • prompt-injection

  • system-prompt

  • language-id

  • repetition-detect

• prompt-injection and language-id processors now process the full conversation history

• system-prompt processor adds the ability to strip existing system prompts. Changes made to ordering of inserted prompts to fit model expectations

• language-id processor now allows rejection

• Processors now accept common parameters reject, annotate, modify for more predictability in processor actions

• Support for enabling GPU support for model-based processors added to helm chart.

✅ Compatibility:

• Kubernetes version 1.25.0-0 or higher

📦 Container images:

• private-registry.nginx.com/aigw/aigw:v1.0.0

• private-registry.nginx.com/aigw/aigw-processors-f5:v1.0.0

AI Gateway Core¶

🆕 Features

• Secure communications between core, processors, and upstream LLMs using private TLS certificates.

• Authenticate processors using mTLS; private certificates are supported here as well.

• Configure core application loging level, verbosity, and contextual output.

• Model-based LLM request routing.

• List models API for OpenAI

🐛 Bug fixes:

• Transaction exporter “busy looping” causing unnecessary CPU utilization

• OTEL Exporter fails to shut down cleanly when receiving SIGINT

✅ Compatibility:

• Kubernetes version 1.25.0-0 or higher

📦 Container images:

• private-registry.nginx.com/aigw/aigw:v0.2.0

AI Gateway Core¶

🆕 Features:

• Support the following LLM APIs: OpenAI, Microsoft Azure OpenAI Service, Anthropic, Ollama (only between AI Gateway and Ollama)

• Authentication of client requests using JWT and TLS

• Authorization of client requests using JWT claims and HTTP headers

• Executing a chain of processors for client requests and LLM responses.

• Routing client requests based on request attributes and metadata generated by processors.

• Conversion between LLM API schemas when the request schema and the LLM backend schema do not match.

• Recording every request/response on disk with an ability to export to AWS S3 or S3-compatible storage.

• Generating and exporting observability data via OpenTelemetry.

• Providing a configuration interface via an HTTP API and a config file.

✅ Compatibility:

• Kubernetes >= 1.25.0-0

📦 Container images:

• private-registry.nginx.com/aigw/aigw:v0.1.0

AI Gateway Processors¶

🆕 Features:

• Added the following processors:

  • Language ID

  • Prompt enforcement

  • Prompt injection detection (PI Processor)

  • Repetition detection

• Efficacy tests for above processors

• Added GPU support for model-based processors

• TLS support added to the processor server

• Added support for system messages in the processor with new API revision

✅ Compatibility:

• Kubernetes >= 1.25.0-0

📦 Container images:

• private-registry.nginx.com/aigw/aigw-processors-f5:v0.1.0