Providers

A provider is an upstream LLM endpoint. Relay ships three adapters, all written on raw HttpClient (no vendor SDKs):

The openai adapter also implements embeddings.

#Azure Foundry: two APIs, one resource

One Foundry resource serves several inference APIs on different paths, and a provider's base URL is what decides which one it talks to. This is the only place in Relay where the family and the wire format come apart, so it is worth setting out.

Keep the family as Azure Foundry for both. Relay picks the adapter from the URL: an AzureFoundry provider whose path contains an /anthropic/ segment routes to the Anthropic adapter. Do not relabel the provider's family to Anthropic to force the body shape — the family is also what enables the Azure discovery fields in the panel and what makes model discovery authenticate the Azure way, so relabelling silently costs you those.

Because a provider row holds exactly one base URL, serving both families from one resource means two provider rows pointing at the same host with different paths — and the same key, since a Cognitive Services account shares one key across its openai.azure.com, services.ai.azure.com and cognitiveservices.azure.com hostnames.

A worked example:

Type       Azure Foundry
Base URL   https://<resource>.services.ai.azure.com/anthropic/v1
API key    the Azure resource key (portal → Keys and Endpoint)
Model      public alias  →  provider model id  claude-haiku-4-5

Relay appends messages and authenticates with x-api-key — that route is Anthropic-compatible and takes Anthropic's header, not Azure's api-key.

#When it returns 401 or 404

Both of Azure's rejections here are unhelpfully generic, so match on what changes rather than on the text:

• 401 "invalid subscription key or wrong API endpoint" — Azure returns this for a wrong key and for a credential header it does not recognise, so the two cannot be told apart by reading it. Check the key first: a provider key is what Relay presents upstream, so it must be the Azure resource key. An app_live_… value is a Relay gateway key — the credential your apps use to call Relay — and pointing it upstream is the common mix-up.
• 404 api_not_supported — the route does not serve that model. The endpoint is right for some models and wrong for this one; check which of the paths above serves it.

To test a route without involving Relay, POST to it directly. A 401 means the route exists and only the credential was refused; a genuine 404 (Resource not found) means the path is not there at all. Watching a 401 turn into a 400 or 200 as you fix the key is how you confirm the header is right.

#How adapters translate

Your apps always speak the OpenAI wire format. Each adapter maps that neutral shape to and from its provider:

• OpenAI — largely passthrough; rewrites the model id, sets the credential, relays SSE. Preserves query strings (so Azure's ?api-version= survives) and forwards unknown params (tools, response_format, …).
• Anthropic — hoists system messages to the top-level system field, defaults the required max_tokens, maps stop_reason → finish_reason, and folds Anthropic's typed SSE events into OpenAI-shaped chunks.
• Gemini — maps the request/response and streaming to Gemini's format.

Because Anthropic and Gemini require a user message, Relay's agent pipeline promotes a self-contained system prompt to a user turn when no user message is present — so the same agent runs on every provider.

#Secrets

A provider's API key is not stored in the database. The Provider row holds a SecretRef (a config key name); ConfigurationProviderSecretResolver reads config[secretRef] at call time. Rotate keys via configuration. See Configuration & secrets.

#Managing providers

Add and edit providers in the panel under Providers: give it a name, base URL, family, and the SecretRef. Then create Models whose public alias maps to a provider model id. Health and rolling metrics are at GET /v1/providers/health and GET /v1/models/metrics.