Enable telemetry¶

AI Gateway supports telemetry via OpenTelemetry.

Traces
Metrics (Note: the processors don’t support metrics at this point)

AI Gateway exports the collected traces and metrics to a gRPC endpoint.

To enable telemetry, configure the tracing and metrics parameters of the Helm chart. For example, the following values file enables telemetry and configures exporting both traces and metrics to a TLS gRPC endpoint at https://otel-collector-opentelemetry-collector:4317:

tracing:
  endpoint: "https://otel-collector-opentelemetry-collector:4317"
metrics:
  endpoint: "https://otel-collector-opentelemetry-collector:4317"

Supported metrics¶

AI Gateway core exposes the following metrics:

Metrics related to requests from the core to services and processors¶

Metric Name	Description	Unit	DataType	IsMonotonic	Aggregation Temporality	Attributes
http.client.request.size	Measures the size of HTTP request messages.	Bytes	Sum	true	Cumulative	`http.method`, `http.status_code`, `net.peer.name`
http.client.response.size	Measures the size of HTTP response messages.	Bytes	Sum	true	Cumulative	`http.method`, `http.status_code`, `net.peer.name`
http.client.duration	Measures the duration of outbound HTTP requests.	Milliseconds	Histogram	N/A	Cumulative	`http.method`, `http.status_code`, `net.peer.name`

Metrics related to requests from clients to the core¶

Metric Name	Description	Unit	DataType	IsMonotonic	Aggregation Temporality	Attributes
http.server.request.size	Measures the size of HTTP request messages.	Bytes	Sum	true	Cumulative	`http.method`, `http.scheme`, `http.status_code`, `net.host.name`, `net.host.port`, `net.protocol.name`, `net.protocol.version`
http.server.response.size	Measures the size of HTTP response messages.	Bytes	Sum	true	Cumulative	`http.method`, `http.scheme`, `http.status_code`, `net.host.name`, `net.host.port`, `net.protocol.name`, `net.protocol.version`
http.server.duration	Measures the duration of inbound HTTP requests.	Milliseconds	Histogram	N/A	Cumulative	`http.method`, `http.scheme`, `http.status_code`, `net.host.name`, `net.host.port`, `net.protocol.name`, `net.protocol.version`

Additional configuration¶

See the additional telemetry configuration-related environmental variables

For AI Gateway core:
For the processors, OpenTelemetry Python SDK

To configure how tracing data is sampled, see the Sampler configuration.

For example, the following values file configures a CA for connections to the tracing and metrics endpoints (the CA needs to be available in the ConfigMap collector-ca (same namespace as the AI Gateway namespace) in the key ca.crt). Additionally, we configure the metrics export interval to 10s.

aigw:
  env:
    - name: OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE
      value: "/etc/collector/ca.crt"
    - name: OTEL_EXPORTER_OTLP_METRICS_CERTIFICATE
      value: "/etc/collector/ca.crt"
    - name: OTEL_METRIC_EXPORT_INTERVAL
      value: "10000" # 10s
  volumes:
    - name: collector-ca-volume
      configMap:
        name: collector-ca
  volumeMounts:
    - name: collector-ca-volume
      readOnly: true
      mountPath: "/etc/collector"
      
tracing:
  endpoint: "https://otel-collector-opentelemetry-collector:4317"
 
metrics:
  endpoint: "https://otel-collector-opentelemetry-collector:4317"
  
processors:
  f5:
    env:
      - name: OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE
        value: "/etc/collector/ca.crt"
    volumes:
      - name: collector-ca-volume
        configMap:
          name: collector-ca
    volumeMounts:
      - name: collector-ca-volume
        readOnly: true
        mountPath: "/etc/collector"

Previous Next