Configure routes¶

The routes section defines the endpoints that are exposed by the AI Gateway connects them a policy.

Each route includes the following settings:

Setting	Description	Required	Examples
`path`	The path that will be the entrypoint for the request. User-defined and must be unique among the routes. Trailing slashes are not allowed.	Yes	`path: /llama3`
`policy`	The `name` of the policy that applies to the requests to this route.	Yes	`policy: llama3`
`schema`	The input and output schema for the route. Supported values: `v1/chat_completions`: The AI Gateway common schema for chat completions that is compliant with a subset of the OpenAI Chat Completions API. See the `v1/chat_completions` API reference for details. `v1/models`: The AI Gateway common schema for listing available models. Compliant with the OpenAI v1 List Models API endpoint. See the `v1/models` API reference for details.	No	`schema: v1/chat_completions`
`timeoutSeconds`	The number of seconds before requests to this route time out.	No	`timeoutSeconds: 30`

Example routes section YAML:

routes:
  - path: /llama3
    schema: v1/chat_completions
    policy: llama3
    timeoutSeconds: 60

Models API¶

If the routes schema is v1/models models, it will serve the OpenAI v1 List Models API.

routes:
  - path: /openai/v1/models
    schema: v1/models
    policy: demo

The policy MUST route to a profile which defines the list of models. See the Configure profiles topic and the Selecting service based on model example.

Previous Next