Relay docs

Architecture

Relay is a .NET 9 solution made of focused projects. The public API and the control panel are separate ASP.NET Core apps that share the same domain, data and provider libraries.

#Projects

ProjectKindResponsibility
ApplicationBlazor Web App (host)The control panel host + api/* control-plane controllers + Entra cookie auth. Serves the WebAssembly panel.
Application.ClientBlazor WASMThe interactive admin panel (Dashboard, Playground, Prompts, Agents, …).
Application.ApiASP.NET CoreThe public gateway API (/v1/*), API-key auth, the routing pipeline and all background services.
Application.SharedClass libraryDomain models, both DbContexts, EF configuration, and shared services (prompts, agents, skills, etc.).
Application.ProvidersClass libraryLLM provider adapters (OpenAI, Anthropic, Gemini). Raw HttpClient, no vendor SDKs.
Application.TelemetryClass libraryTelemetry read/write against ClickHouse (custom table or the standard OTel schema).
Application.DatabaseSQL projectThe SQL Server schema as a DACPAC (tables authored as .sql).
Application.DocASP.NET CoreThis documentation site.

#Data stores

  • SQL Server — the relational store, split across two DbContexts:
    • GatewayDbContext (database relay_gateway) — providers, models, API keys, teams, budgets, prompts, agents, skills, tools, MCP servers, knowledge-base metadata, batches, evals, conversations, audit.
    • UserManagementDbContext — ASP.NET Identity (users, roles, companies).
    • Naming is snake_case; rows carry soft-delete + audit columns via a shared base model.
  • ClickHouse — the telemetry store (request-level usage; or the standard otel_* tables when running the OpenTelemetry backend).
  • Qdrant — the vector store for knowledge bases (RAG).

#Request lifecycle (chat)

A POST /v1/chat/completions flows through the gateway pipeline:

  1. Authenticate the API key (ApiKey scheme) and load the key + its team.
  2. Route the requested model through ModelRouter → a concrete provider adapter + invocation context, applying policy (explicit → team default → auto strategies), the per-key model allow-list, and budget hard-stops.
  3. Execute via GatewayChatService: transient retry, provider-health circuit breaking, single-hop fallback, and SSE streaming when requested.
  4. Record telemetry (tokens, cost, latency, status) — written to ClickHouse and/or exported over OTLP.

#Deliberate constraints

Two constraints shape how Relay is built and extended:

  • No new NuGet packages. The build environment cannot restore packages beyond what is already cached. Every new capability is written with the BCL only — raw HttpClient, System.Text.Json, hand-written parsers (the OTLP exporter, the RAG reranker, the PDF/DOCX/HTML extractors, even this docs site's Markdown renderer are all BCL-only).
  • DACPAC-only schema. New tables/columns are authored as .sql in Application.Database and added to the DbContext model — not via EF migrations. Deploy schema by publishing the DACPAC.

See Configuration & secrets and Deployment for how these play out in practice.