Expose with NGINX Ingress Controller¶

AI Gateway is intended to be used with a fronting HTTP reserve proxy which should:

Load balance among multiple replicas of horizontally scaled AI Gateway core instances (replicas) to increase availability.
Implement security controls for client requests to prevent unauthorized and unrestricted access.
Use TLS for client connections and mTLS for connections between the proxy and the core replicas.

This guide uses NGINX Ingress Controller as proxy and implements the above recommendations.

Install NGINX Ingress Controller¶

Follow the NGINX Ingress Controller installation documentation.

Configuration¶

Note: the configuration below assumes:

AI Gateway is deployed in the ai-gateway namespace.
All the configuration resources and secrets below are deployed in that namespace too.

The following VirtualServer exposes the AI Gateway core through NGINX Ingress Controller:

apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
  name: aigw
  namespace: ai-gateway
spec:
  host: aigw.example.com
  tls:
    secret: aigw-nic-tls
    redirect:
      enable: true
  upstreams:
    - name: aigw
      service: aigw
      port: 80
      tls:
        enable: true
      client-max-body-size: 512k
  routes:
    - path: /
      policies:
        - name: aigw-mtls
        - name: aigw-rate-limit
        - name: aigw-jwt
        - name: aigw-access-control
      action:
        proxy:
          upstream: aigw
          requestHeaders:
            set:
              - name: nic-aud
                value: ${jwt_claim_aud}
              - name: token # clear the header with the JWT
                value: ""

By using this VirtualServer:

The AI Gateway core is exposed on ports 80 for HTTP and port 443 for HTTPS. Note: The TLS secret aigw-nic-tls with a cert and key must exist. Also, while ports 80 and 443 are not explicitly defined in the VirtualServer above, those are the NGINX Ingress Controller defaults.
AI Gateway is available at https://aigw.example.com.
NGINX redirects all HTTP requests to an HTTPS URL. For example, http://aigw.example.com -> https://aigw.example.com
NGINX proxies client requests to the AI Gateway core replicas (pods) belonging to the Kubernetes service aigw in the namespace ai-gateway and use mTLS for connections to those pods, which is defined in the aigw-mtls policy below.
NGINX applies the following security controls to client requests:
- Rejects requests with a body greater than 512k.
- Enforces rate-limiting, configured in the aigw-rate-limit policy.
- Implements JWT authentication, configured in the aigw-jwt policy. Additionally, NGINX:
  - Sets the header nic-aud to the value of the aud claim extracted from the JWT from a client request.
  - Clears the header token so that the JWT isn’t passed to AI Gateway.
- Provides allow-list access control in the aigw-access-control policy.

For more information, see VirtualServers.

Policies¶

Below are the policies referenced by the VirtualServer.

MTLS¶

The EgressMTLS policy below secures connections between NGINX and the core with mTLS. Note that the core will verify the client certificate of NGINX (see also the AI Gateway configuration below):

apiVersion: k8s.nginx.org/v1
kind: Policy
metadata:
  name: aigw-mtls
  namespace: ai-gateway
spec:
  egressMTLS:
    tlsSecret: aigw-nic-client-tls
    trustedCertSecret: internal-ca
    verifyServer: on
    serverName: on
    sslName: aigw.ai-gateway.svc.cluster.local
    protocols: TLSv1.3

The client certificate of NGINX must exist in the secret aigw-nic-client-tls, and the internal CA for NGINX to validate the core TLS certificate must exist in the secret internal-ca.

For more information, see EgressMTLS policy.

Rate-limiting¶

The following rate-limiting policy limits one request per second per a remote address (IP:port):

apiVersion: k8s.nginx.org/v1
kind: Policy
metadata:
  name: aigw-rate-limit
  namespace: ai-gateway
spec:
  rateLimit:
    rate: 1r/s
    key: ${binary_remote_addr}
    zoneSize: 10M

For more information, see RateLimit policy.

JWT authentication¶

The following policy ensures that a client request include a JWT in the header token and verifies it using JWKs from https://example.com/keys/jwks:

apiVersion: k8s.nginx.org/v1
kind: Policy
metadata:
  name: aigw-jwt
  namespace: ai-gateway
spec:
  jwt:
    realm: MyProductAPI
    token: $http_token
    jwksURI: https://example.com/keys/jwks
    keyCache: 1h

Note

This feature requires a version of NGINX Ingress Controller with NGINX Plus.

For more information, see JWT Using Local Kubernetes Secret and JWT Using JWKS From Remote Location policies.

Access control¶

The following policy rejects any connections to NGINX from IPs not from the subnet 10.0.0.0/24 (used as an example trusted subnet):

apiVersion: k8s.nginx.org/v1
kind: Policy
metadata:
  name: aigw-access-control
  namespace: ai-gateway
spec:
  accessControl:
    allow:
      - 10.0.0.0/24

For more information, see AccessControl policy.

Additional configuration¶

We also recommend disabling the emission of the NGINX version in the Server header returned to clients. For that, set the server-tokens to "False" in the global ConfigMap configuration.

AI Gateway configuration¶

The following config enables mTLS for incoming connections from NGINX on port 4141. The client cert used by NGINX will be verified using the CA in /etc/internal-ca/ca.crt.

Additionally, AI Gateway will select the profile based on the value of the nic-aud header, which was set by NGINX via JWT authentication policy above.

mode: standalone

server:
  address: :4141
  tls:
    enabled: true
    serverCertPath: /etc/tls/tls.crt
    serverKeyPath: /etc/tls/tls.key
  mtls:
    enabled: true
    clientCertPath: /etc/internal-ca/ca.crt

profiles:
  - name: my-profile 
    services:
      - name: my-service

policies:
  - name: my-policy 
    profiles:
      - name: my-profile 
        selector:
          type: header
          key: nic-aud
          values:
            - some:aud:value

# the rest of the configuration is omitted

For more information about configuration, see Configuration file.

To propagate TLS secrets, key, and the CA, you can use the volumes and volume mounts in the values file when installing via Helm. For example:

aigw:
  volumes:
    - name: internal-ca-volume
      secret:
        secretName: internal-ca
    - name: tls-volume
      secret:
        secretName: aigw-tls
  volumeMounts:
    - name: internal-ca-volume
      readOnly: true
      mountPath: "/etc/internal-ca"
    - name: tls-volume
      readOnly: true
      mountPath: "/etc/tls"

Previous Next