Relay docs

Batches API

Submit large workloads that Relay processes asynchronously through the normal routing pipeline. Useful for bulk classification, summarisation or generation over many rows.

See Batch processing for how the processor works (priority, cron, resume, dead-letter, webhooks).

#Submit a batch

POST /v1/batches

{
  "model": "gpt-4o-mini",
  "rows": [
    { "messages": [{ "role": "user", "content": "Classify: great product!" }] },
    { "messages": [{ "role": "user", "content": "Classify: terrible service" }] }
  ],
  "priority": 5,
  "webhookUrl": "https://my-app/relay-webhook",
  "cron": null
}
  • priority — higher runs first.
  • cron — optional schedule for a recurring batch (the template spawns due runs).
  • webhookUrl — optional; Relay POSTs a completion notification when the job finishes.

#Check status

GET /v1/batches/{id}

{ "id": "batch_…", "status": "running", "total": 2, "completed": 1, "failed": 0 }

#Get results

GET /v1/batches/{id}/results — the per-row outputs as JSON, once the job has produced them.

#Notes

  • The processor picks the highest-priority queued job, runs rows with bounded concurrency (Gateway:BatchConcurrency), checkpoints progress so it can resume, and routes failed rows to a dead-letter record.
  • You can also create/upload batches from the panel under Batch (CSV one-prompt-per-line or JSONL).