Kestrel Labs / Production / Models
Model performance
Fleet · Requests
1.62M
+29.2%
Error rate
0.50%
+12.5%
P50
1.01 s
+1.9%
P95
2.75 s
+3.8%
P99
4.12 s
+5.6%
$ / 1K requests
$8.55
+5.5%
Tokens / min
89.5K
+32.0%
4 models · Last 30 days · latency on a shared log scale
Trend
Percentiles come from a request-weighted mixture of daily latency distributions. Error rate and cost per request count every attempted request. Highlighted points in the trend mark incident episodes.
01
Latency anatomy
Each model’s full response-time distribution on one log axis, or one percentile per day. Exclude incident days to see how much of the tail they cause.
Exclude incident days (6)
Share of requests by response time, log scale. Each curve is that model's request-weighted mixture of daily distributions.
Claude Sonnet 4.5
Mean
1.64 s
P50
1.43 s
P95
3.35 s
P99
4.81 s
Tail
3.4×
Gemini 2.5 Flash
Mean
716 ms
P50
615 ms
P95
1.48 s
P99
2.27 s
Tail
3.7×
DeepSeek V3
Mean
956 ms
P50
827 ms
P95
1.97 s
P99
2.93 s
Tail
3.5×
GPT-4o
Mean
1.10 s
P50
964 ms
P95
2.25 s
P99
3.25 s
Tail
3.4×
Fleet
Mean
1.21 s
P50
1.01 s
P95
2.75 s
P99
4.12 s
Tail
4.1×
The mean sits above the median for every model: a few slow requests pull it up, which is why the average alone understates what users wait for.
02
Efficiency frontier
Latency against cost per request, with each model’s path through the range. Models on the dashed frontier are not beaten on both axes.
On the frontier: Gemini 2.5 Flash, DeepSeek V3. Claude Sonnet 4.5 is slower and costlier than Gemini 2.5 Flash; GPT-4o is slower and costlier than Gemini 2.5 Flash (P95 latency against cost per 1K requests).
Bubble area shows request volume; small dots trace 6 equal slices of last 30 days, oldest faintest. Answer quality is not measured, so a model off the frontier can still be the right choice for its workload.
03
Performance signals
What the data shows, kept apart from what you might do about it. Each finding links to its evidence.
error rate vs baseline · daily, last 30 days · marked days are the finding
Observed
1.5% of requests failed at the peak, against a 14-day median of 0.50%. 43% of the 215 failures were timed out (Client 30 s). P95 rose to 3.80 s (2.1× baseline) on the same day.
Possible action
Timeouts dominated while latency rose. This data cannot separate provider slowness from network faults; review the 30 s deadline and the retry budget for this route.
Inspect Oct 15 in the DeepSeek V3 dossier
04
Head-to-head
Pick two models. Each bar points to the side that wins a measure, scaled by the margin. The like-for-like table compares them inside the same workloads.
Side A
Side B
Latency
Gemini 2.5 Flash
Gemini 2.5 Flash ahead on 3 of 4 measures
Latency
Reliability
Cost
Request shape
Same workload, side by side
Their workload mixes overlap 50%. Part of the gap in tokens and cost per request comes from the work each model is given, so compare within a workload.
Workload
Share of each model’s requests
A · $ / 1K
B · $ / 1K
A · out tok
B · out tok
Support copilot
Customer experience
39%
24%
$15.30
$1.28
738
395
Code review agent
Developer platform
41%
5%
$17.71
$1.51
493
312
Search and retrieval
Core product
10%
43%
$9.70
$0.813
274
173
Nightly summaries
Data
11%
28%
$17.89
$1.58
719
450
05
Reliability and failures
Failures per day by response type or by model, and each model’s failure profile per 10K requests. Select a day to carry it into the dossier and request log.
Failures per 10K requests
Last 30 days
Provider
Unclear
Capacity
Client
Model
All
Claude Sonnet 4.5
5.5
16
5.4
13
6.9
46.4
Gemini 2.5 Flash
7.5
2.9
9.0
11
6.3
36.8
DeepSeek V3
8.6
6.0
15
16
4.6
50.9
GPT-4o
29
4.0
10
23
9.4
76.1
Provider and overloaded errors point to the provider; rate limits point to account limits; timeouts can be either side. Categories describe the response, not a confirmed cause.
06
Model dossier
One model in depth: six daily views on one crosshair, and the inspected day measured against the model’s own 14-day baseline.
Claude Sonnet 4.5
Stable
Anthropic · $3.00 in / $15.00 out per 1M tokens · claude-sonnet-4-5
Requests
679.69K
Error rate
0.46%
P95
3.35 s
$ / 1K req
$15.98
Requests
17.6K
Latency · P50, P95–P99 band
1.43 s · 3.29 s · 4.66 s
Error rate
0.43%
$ / 1K requests
$15.45
Output tokens / request
558
Input tokens / request
2,357
Episodes
Day inspector
Sun, Oct 18, 2026
Requests
17,635
Failed
75
P50 · P99
1.43 s · 4.66 s
Tokens / min
35.7K
vs 14-day median
Day
Base
Error rate
0.43%
0.47%
0.91×
P95
3.29 s
3.23 s
1.02×
$ / 1K req
$15.45
$15.79
0.98×
Failures by type
Provider error
9
Overloaded
23
Timed out
9
Rate limited
23
Invalid request
11
Workloads that day
Support copilot
7.4K
$13.73/1K
Code review agent
4.0K
$18.72/1K
Search and retrieval
2.2K
$10.14/1K
Nightly summaries
4.0K
$18.27/1K
07
Request inspector
A tail-based sample of individual requests, each carrying its sampling weight. Search, filter and open any request; no prompts or completions are stored.
898 sampled requests · represents ≈ 1.6M requests
Estimated failure share 0.47%
Kept 1 in 2,000 successful and 1 in 100 failed requests · no prompts or completions stored