Skip to content
An old telephone switchboard with patch cords plugged across its panel, lit from one side.

Product · AI Gateway

One governed API for the models you use

Your teams keep their SDKs. You get one governed API in front of the models they use, with spend and policy in one place.

Book a consultation
  • Integrate once, switch providers in config
  • Budget reserved before each call
  • Policy checks in the request path
  • Prompts and responses not stored

Example record

One request, through the gateway

Each request meets the same stages, in the same order.

The prompt and the response are not stored.

gateway · request traceSample log

  • Caller identified

    project key, auth tier

    Checked

  • Model allowed for this project

    model and workflow allowlist

    Allowed

  • Budget reserved before the call

    prepaid reservation per project

    Reserved

  • Prompt injection screen

    in-line, before the provider

    Clear

  • Personal data in the prompt

    redacted before it leaves

    Redacted

  • Routed by intent: chat-fast

    ranked on health, cost tier, context

    Routed

  • Primary backend degraded

    circuit open, next candidate

    Failover

  • Cost attributed

    per project, per request

    Logged

Spend, data and outages, answered

Most organisations adopt LLMs one application at a time, each with its own provider and key. Within a year, 3 questions go unanswered.

  • What are we spending, and on what?

  • What data is leaving the building?

  • What happens when a provider has a bad day?

The gateway puts one governed layer between your applications and your providers. So switching a provider or tightening a policy is a config change in one place.

Request path

The request path

So you change a policy without touching an application. Each stage is configuration, versioned and canaried before full rollout.

01Unified APIOpenAI-compatible chat, embeddings and responses endpoints, plus Anthropic-native Messages with extended thinking, tool use and streaming.
02Access controlPer-project model, alias and workflow allowlists, tier defaults, operator and project auth tiers, a role-scoped admin console.
03Rate limitingFleet-wide shared counters, per project and per key.
04Cost controlPer-project budgets, a prepaid reservation before spend and cost per request. You get usage reports and digests.
05GovernanceInjection guard, PII redaction, tool governance, risk-based holds for human review, a tamper-evident audit trail.
06Intent routingSemantic aliases, named workflows, task-type routing, capability and cost-tier filtering.
07ResiliencePer-backend circuit breakers on failure streaks, rate-limit streaks and a rolling error ratio, with health shared across the fleet.

Providers

Behind one API

Your applications name a model, or an intent, and the gateway chooses.

  • OpenAI: Chat, embeddings and responses.
  • Anthropic: Native Messages, with extended thinking and tool use.
  • Azure OpenAI: Behind the same endpoint as the rest.
  • Amazon Bedrock: Models on AWS, under the same policy.
  • Self-hosted: OpenAI-compatible endpoints, open-weight models included.

Private by default, run by your team

So your prompts and responses stay yours. The gateway does not persist them.

  • Config versioned and hot-reloaded

  • Changes canaried to a traffic share or named projects first

  • Circuit breakers shed load from failing backends

  • OpenTelemetry, Prometheus and Grafana wired in

  • An operator CLI and an admin console

  • Plugins in-process or remote, without forking

There are 2 narrow, documented exceptions, governance review holds and tool-authorisation records. Both are time-bounded and auditable.

Related

Where to go next

Next step

Put your first team behind it.

Start with one team, self-hosted, in front of the providers it already uses. What you get: its spend per request, and a policy on what leaves.

Book a consultation →