Elevated latency on AI Gateway openai/gpt-oss-120b routed to Baseten

We observed a significant latency increase for AI Gateway requests to:

Model: openai/gpt-oss-120b Resolved provider: baseten Approximate affected window: 2026-06-29 15:00-15:05 UTC

This was during a repeatable internal probe/validation run using the same model path and similar structured-output prompts as prior runs. The requests completed successfully, but latency increased across the full probe set.

Affected run summary:

Requests: 31
Successful responses: 31
Failed/errored responses: 0
Total tokens: about 64.7k
Input tokens: about 56.9k
Output tokens: about 7.8k
Median latency: about 6.95s
Average latency: about 6.88s
p95 latency: about 10.97s

For comparison, recent prior runs on the same route were typically:

Average latency: 1.0-1.3s
Median latency: 0.9-1.1s
p95 latency: 1.5-2.4s

Sample slow generation IDs from the affected window, with two faster prior same-route samples for comparable validation prompts:

Affected generation Affected latency Prior fast samples ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ gen_01KW9YDE2B45WPG0MMKBDB4MYY about 19.1s gen_01KW5471SGENBV90ZXBJSDMYFX about 2.1s; gen_01KW58JFB1RZ1HWVR2Y24B98FP about 2.2s ──────────────────────────────── ────────────────── ────────────────────────────────────────────────────────────────────────────────────── gen_01KW9YFR6MCPZRV4N8481TY642 about 16.8s gen_01KW8JTMDDC5GB9Z2R39DN63GK about 1.5s; gen_01KW7M99N4TX9E0BCBXNKHVMED about 1.5s ──────────────────────────────── ────────────────── ────────────────────────────────────────────────────────────────────────────────────── gen_01KW9YGSG7QAGJ75EYEFAVRWGV about 11.0s gen_01KW58JZE2S7VNCQGN3AD2J3KS about 1.6s; gen_01KW7M9EDHE6XH1DNKAADCAJPP about 1.7s ──────────────────────────────── ────────────────── ────────────────────────────────────────────────────────────────────────────────────── gen_01KW9YKG2RFPV8H82AT0XMSBPC about 10.0s gen_01KW572V409E6ERNTBDXG2T24W about 0.9s; gen_01KW58KEQP23201ZBQD430APK1 about 1.0s ──────────────────────────────── ────────────────── ────────────────────────────────────────────────────────────────────────────────────── gen_01KW9YEWCVFT5BBBDRQRNTESDS about 9.2s gen_01KW5478W9MSZFG6AEK293BMYC about 0.7s; gen_01KW7M96BZT1DAN9NY9MC859FJ about 0.9s

Conformance-style probe requests immediately before the full validation were also slower than usual:

Affected probe generation Affected latency Prior fast samples ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ gen_01KW9YAXCW87NQX6KRTCSSSSGG about 2.2s gen_01KW7M6D9M5AAVX2MYJ832V84C about 0.7s; gen_01KW58GFS0QCK8NCF605KY133T about 0.9s ──────────────────────────────── ────────────────── ────────────────────────────────────────────────────────────────────────────────────── gen_01KW9YAZFH3HFX17V80NK0HM5E about 2.8s gen_01KW544VS93S8R7C5JH3613VKE about 0.4s; gen_01KW58GGHT66ACF59WT3HPH496 about 0.4s ──────────────────────────────── ────────────────── ────────────────────────────────────────────────────────────────────────────────────── gen_01KW9YB27GM3WAMXCB9NZQ3949 about 3.2s gen_01KW544W8AVD6E0B8Z54TQQ2SW about 0.5s; gen_01KW7M6EHYH6KGDXZ74SQGEPQ6 about 0.5s

Can smbd help whether there was elevated latency, capacity pressure, routing behavior, or backend degradation for openai/gpt-oss-120b via Baseten around 2026-06-29 15:00-15:05 UTC?