Reliability

Failover & Reliability

Metriqual automatically fails over across your provider chain. If a provider errors, times out, or hits its usage cap, the request is retried against the next provider in your chain — which can even be a different model. Every response tells you exactly what happened, you can subscribe to failover and recovery webhooks, and you can query live provider health.

Response headers

Every POST /v1/chat/completions response — both streaming and non-streaming — includes a set of x-mql-* headers describing which provider actually served the request and whether a failover occurred.

Response Headers

x-mql-provider-usedstring

The provider that actually served the request (e.g. openai, anthropic).

x-mql-is-fallbackboolean

true if a fallback provider served the request, false if the primary served it.

Options: true, false

x-mql-fallback-levelinteger

Depth of the serving provider in the chain. 0 = entry/primary provider, 1+ = fallback depth.

x-mql-fallback-alertjson string

Present ONLY when a failover occurred. A JSON string describing the failover, e.g. {"message": "...", "exhausted_provider": "openai", "active_provider": "anthropic", "fallback_level": 1}.

x-mql-request-idstring

Correlate this request with usage logs and webhook events.

Response body
The JSON response body also carries a non-standard provider field naming the serving provider, so you can read it without inspecting headers.

Inspecting headers with curl

Dump response headers
curl -D - https://api.metriqual.com/v1/chat/completions \
  -H "Authorization: Bearer $MQL_PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{ "role": "user", "content": "Hello" }]
  }'

# Response headers
# x-mql-provider-used: anthropic
# x-mql-is-fallback: true
# x-mql-fallback-level: 1
# x-mql-fallback-alert: {"message":"Primary provider exhausted","exhausted_provider":"openai","active_provider":"anthropic","fallback_level":1}
# x-mql-request-id: mql_9f2c1a7b

Reading headers in Node.js

Node.js (fetch)
const response = await fetch('https://api.metriqual.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.MQL_PROXY_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello' }],
  }),
});

const providerUsed = response.headers.get('x-mql-provider-used');
const isFallback = response.headers.get('x-mql-is-fallback') === 'true';

if (isFallback) {
  console.log(`Request served by fallback provider: ${providerUsed}`);
}

Failover webhooks

Subscribe to push events so your systems are notified the moment a failover or recovery happens. Manage your endpoints through the webhook management API.

POST/v1/user/webhooks
GET/v1/user/webhooks
PATCH/v1/user/webhooks/:id
DELETE/v1/user/webhooks/:id
POST/v1/user/webhooks/:id/test
Plan requirement
Failover webhooks require the Growth plan or higher.

Create an endpoint with a url and the list of events you want to receive. If events is omitted it defaults to ["failover"].

Create a webhook endpoint
curl -X POST https://api.metriqual.com/v1/user/webhooks \
  -H "Authorization: Bearer $MQL_PROXY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-app.com/webhooks/metriqual",
    "events": ["failover", "provider_recovered"]
  }'

Event types

Events

failoverevent

Fired when a request is served by a non-entry (fallback) provider.

provider_recoveredevent

Fired when a previously-failing provider's circuit breaker recovers — a probe succeeds and it returns to healthy.

Delivery request headers

Metriqual sends these headers with every delivery to your endpoint:

Delivery Headers

X-MQL-Eventstring

The event type (failover or provider_recovered).

X-MQL-Timestampinteger

Unix time in seconds at which the event was sent.

X-MQL-Signaturestring

HMAC signature of the payload, formatted as sha256=<hex hmac>.

Payload shape

Webhook payload
{
  "id": "evt_<uuid>",
  "event": "failover",
  "timestamp": "2026-07-06T02:18:04Z",
  "data": {
    "proxy_key": "pk_...",
    "model": "gpt-4o",
    "exhausted_provider": "openai",
    "active_provider": "anthropic",
    "fallback_level": 1,
    "latency_ms": 842,
    "request_id": "mql_..."
  }
}
provider_recovered payloads
For provider_recovered events, exhausted_provider, fallback_level, and latency_ms may be null. active_provider names the provider that recovered.

Signature verification

The signature is computed as HMAC_SHA256(endpoint_secret, "{X-MQL-Timestamp}.{raw_request_body}"), hex-encoded, and sent as X-MQL-Signature: sha256=<hex>. To verify, recompute the HMAC over the string `${timestamp}.${rawBody}` — the timestamp, then a literal dot, then the exact raw JSON body — and constant-time compare the hex digest against the value after sha256=.

Use the raw body
You must sign and verify the raw request body bytes, not a re-serialized parsed object. Re-serializing can change whitespace or key order and will break the signature.
Node.js (Express + crypto)
const express = require('express');
const crypto = require('crypto');

const app = express();
const ENDPOINT_SECRET = process.env.MQL_WEBHOOK_SECRET;

// Capture the raw body so we can verify the signature over exact bytes.
app.use('/webhooks/metriqual', express.raw({ type: 'application/json' }));

app.post('/webhooks/metriqual', (req, res) => {
  const rawBody = req.body.toString('utf8');
  const timestamp = req.get('X-MQL-Timestamp');
  const signatureHeader = req.get('X-MQL-Signature') || '';
  const received = signatureHeader.replace(/^sha256=/, '');

  const expected = crypto
    .createHmac('sha256', ENDPOINT_SECRET)
    .update(`${timestamp}.${rawBody}`)
    .digest('hex');

  const a = Buffer.from(received, 'hex');
  const b = Buffer.from(expected, 'hex');
  if (a.length !== b.length || !crypto.timingSafeEqual(a, b)) {
    return res.status(401).send('Invalid signature');
  }

  const event = JSON.parse(rawBody);
  console.log('Verified event:', event.event, event.data);
  res.status(200).send('ok');
});

app.listen(3000);
Python (Flask + hmac/hashlib)
import hmac
import hashlib
import os
from flask import Flask, request, abort

app = Flask(__name__)
ENDPOINT_SECRET = os.environ["MQL_WEBHOOK_SECRET"].encode()


@app.route("/webhooks/metriqual", methods=["POST"])
def metriqual_webhook():
    raw_body = request.get_data()  # exact raw bytes, not the parsed object
    timestamp = request.headers.get("X-MQL-Timestamp", "")
    signature_header = request.headers.get("X-MQL-Signature", "")
    received = signature_header.removeprefix("sha256=")

    signed_payload = f"{timestamp}.".encode() + raw_body
    expected = hmac.new(ENDPOINT_SECRET, signed_payload, hashlib.sha256).hexdigest()

    if not hmac.compare_digest(received, expected):
        abort(401, "Invalid signature")

    event = request.get_json()
    print("Verified event:", event["event"], event["data"])
    return "ok", 200

Provider health API

GET/v1/providers/health

Authenticate with Authorization: Bearer <proxy_key> to fetch live circuit-breaker health for the providers in that key's failover chain.

Query provider health
curl https://api.metriqual.com/v1/providers/health \
  -H "Authorization: Bearer $MQL_PROXY_KEY"
Response
{
  "overall": "operational",
  "providers": [
    { "provider": "openai",    "status": "healthy", "consecutive_failures": 0, "cooldown_remaining_secs": 0 },
    { "provider": "anthropic", "status": "down",    "consecutive_failures": 5, "cooldown_remaining_secs": 22 }
  ]
}

overall is one of operational, degraded, or down. Each provider reports a status:

Provider status values

healthystatus

Normal operation — serving requests.

degradedstatus

Some recent failures, but still serving requests.

recoveringstatus

Circuit is half-open and probing to confirm recovery.

downstatus

Circuit is open, so the provider is temporarily skipped. cooldown_remaining_secs counts down until the next probe.

Powering a status page
Poll this endpoint to drive a public status page, or run it as a pre-flight check before dispatching latency-sensitive traffic.

How failover works

Your proxy key holds an ordered provider chain. The requested model picks the entry step — the highest-priority step that serves that model. On an error, timeout, or usage cap, the request proceeds down the chain, and each fallback step sends its own configured model. That means a chain like gpt-4o → gpt-4o-mini → claude is possible, failing over to a different model at each step.

A circuit breaker trips a provider after 5 consecutive failures and skips it for roughly 30 seconds, then probes for recovery. While a provider is skipped it reports down; once a probe succeeds it returns to healthy and a provider_recovered webhook fires.