Failover & Reliability
Metriqual automatically fails over across your provider chain. If a provider errors, times out, or hits its usage cap, the request is retried against the next provider in your chain — which can even be a different model. Every response tells you exactly what happened, you can subscribe to failover and recovery webhooks, and you can query live provider health.
Response headers
Every POST /v1/chat/completions response — both streaming and non-streaming — includes a set of x-mql-* headers describing which provider actually served the request and whether a failover occurred.
Response Headers
x-mql-provider-usedstringThe provider that actually served the request (e.g. openai, anthropic).
x-mql-is-fallbackbooleantrue if a fallback provider served the request, false if the primary served it.
Options: true, false
x-mql-fallback-levelintegerDepth of the serving provider in the chain. 0 = entry/primary provider, 1+ = fallback depth.
x-mql-fallback-alertjson stringPresent ONLY when a failover occurred. A JSON string describing the failover, e.g. {"message": "...", "exhausted_provider": "openai", "active_provider": "anthropic", "fallback_level": 1}.
x-mql-request-idstringCorrelate this request with usage logs and webhook events.
provider field naming the serving provider, so you can read it without inspecting headers.Inspecting headers with curl
curl -D - https://api.metriqual.com/v1/chat/completions \
-H "Authorization: Bearer $MQL_PROXY_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{ "role": "user", "content": "Hello" }]
}'
# Response headers
# x-mql-provider-used: anthropic
# x-mql-is-fallback: true
# x-mql-fallback-level: 1
# x-mql-fallback-alert: {"message":"Primary provider exhausted","exhausted_provider":"openai","active_provider":"anthropic","fallback_level":1}
# x-mql-request-id: mql_9f2c1a7bReading headers in Node.js
const response = await fetch('https://api.metriqual.com/v1/chat/completions', {
method: 'POST',
headers: {
Authorization: `Bearer ${process.env.MQL_PROXY_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Hello' }],
}),
});
const providerUsed = response.headers.get('x-mql-provider-used');
const isFallback = response.headers.get('x-mql-is-fallback') === 'true';
if (isFallback) {
console.log(`Request served by fallback provider: ${providerUsed}`);
}Failover webhooks
Subscribe to push events so your systems are notified the moment a failover or recovery happens. Manage your endpoints through the webhook management API.
/v1/user/webhooks/v1/user/webhooks/v1/user/webhooks/:id/v1/user/webhooks/:id/v1/user/webhooks/:id/testCreate an endpoint with a url and the list of events you want to receive. If events is omitted it defaults to ["failover"].
curl -X POST https://api.metriqual.com/v1/user/webhooks \
-H "Authorization: Bearer $MQL_PROXY_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://your-app.com/webhooks/metriqual",
"events": ["failover", "provider_recovered"]
}'Event types
Events
failovereventFired when a request is served by a non-entry (fallback) provider.
provider_recoveredeventFired when a previously-failing provider's circuit breaker recovers — a probe succeeds and it returns to healthy.
Delivery request headers
Metriqual sends these headers with every delivery to your endpoint:
Delivery Headers
X-MQL-EventstringThe event type (failover or provider_recovered).
X-MQL-TimestampintegerUnix time in seconds at which the event was sent.
X-MQL-SignaturestringHMAC signature of the payload, formatted as sha256=<hex hmac>.
Payload shape
{
"id": "evt_<uuid>",
"event": "failover",
"timestamp": "2026-07-06T02:18:04Z",
"data": {
"proxy_key": "pk_...",
"model": "gpt-4o",
"exhausted_provider": "openai",
"active_provider": "anthropic",
"fallback_level": 1,
"latency_ms": 842,
"request_id": "mql_..."
}
}provider_recovered events, exhausted_provider, fallback_level, and latency_ms may be null. active_provider names the provider that recovered.Signature verification
The signature is computed as HMAC_SHA256(endpoint_secret, "{X-MQL-Timestamp}.{raw_request_body}"), hex-encoded, and sent as X-MQL-Signature: sha256=<hex>. To verify, recompute the HMAC over the string `${timestamp}.${rawBody}` — the timestamp, then a literal dot, then the exact raw JSON body — and constant-time compare the hex digest against the value after sha256=.
const express = require('express');
const crypto = require('crypto');
const app = express();
const ENDPOINT_SECRET = process.env.MQL_WEBHOOK_SECRET;
// Capture the raw body so we can verify the signature over exact bytes.
app.use('/webhooks/metriqual', express.raw({ type: 'application/json' }));
app.post('/webhooks/metriqual', (req, res) => {
const rawBody = req.body.toString('utf8');
const timestamp = req.get('X-MQL-Timestamp');
const signatureHeader = req.get('X-MQL-Signature') || '';
const received = signatureHeader.replace(/^sha256=/, '');
const expected = crypto
.createHmac('sha256', ENDPOINT_SECRET)
.update(`${timestamp}.${rawBody}`)
.digest('hex');
const a = Buffer.from(received, 'hex');
const b = Buffer.from(expected, 'hex');
if (a.length !== b.length || !crypto.timingSafeEqual(a, b)) {
return res.status(401).send('Invalid signature');
}
const event = JSON.parse(rawBody);
console.log('Verified event:', event.event, event.data);
res.status(200).send('ok');
});
app.listen(3000);import hmac
import hashlib
import os
from flask import Flask, request, abort
app = Flask(__name__)
ENDPOINT_SECRET = os.environ["MQL_WEBHOOK_SECRET"].encode()
@app.route("/webhooks/metriqual", methods=["POST"])
def metriqual_webhook():
raw_body = request.get_data() # exact raw bytes, not the parsed object
timestamp = request.headers.get("X-MQL-Timestamp", "")
signature_header = request.headers.get("X-MQL-Signature", "")
received = signature_header.removeprefix("sha256=")
signed_payload = f"{timestamp}.".encode() + raw_body
expected = hmac.new(ENDPOINT_SECRET, signed_payload, hashlib.sha256).hexdigest()
if not hmac.compare_digest(received, expected):
abort(401, "Invalid signature")
event = request.get_json()
print("Verified event:", event["event"], event["data"])
return "ok", 200Provider health API
/v1/providers/healthAuthenticate with Authorization: Bearer <proxy_key> to fetch live circuit-breaker health for the providers in that key's failover chain.
curl https://api.metriqual.com/v1/providers/health \
-H "Authorization: Bearer $MQL_PROXY_KEY"{
"overall": "operational",
"providers": [
{ "provider": "openai", "status": "healthy", "consecutive_failures": 0, "cooldown_remaining_secs": 0 },
{ "provider": "anthropic", "status": "down", "consecutive_failures": 5, "cooldown_remaining_secs": 22 }
]
}overall is one of operational, degraded, or down. Each provider reports a status:
Provider status values
healthystatusNormal operation — serving requests.
degradedstatusSome recent failures, but still serving requests.
recoveringstatusCircuit is half-open and probing to confirm recovery.
downstatusCircuit is open, so the provider is temporarily skipped. cooldown_remaining_secs counts down until the next probe.
How failover works
Your proxy key holds an ordered provider chain. The requested model picks the entry step — the highest-priority step that serves that model. On an error, timeout, or usage cap, the request proceeds down the chain, and each fallback step sends its own configured model. That means a chain like gpt-4o → gpt-4o-mini → claude is possible, failing over to a different model at each step.
A circuit breaker trips a provider after 5 consecutive failures and skips it for roughly 30 seconds, then probes for recovery. While a provider is skipped it reports down; once a probe succeeds it returns to healthy and a provider_recovered webhook fires.