Hi Flo,

I’d avoid treating 300–700ms as expected Gateway overhead until you confirm the Gateway request is hitting the same upstream path as your direct Vertex request.

For a narrow benchmark, I’d first pin the provider route so you’re comparing:

1
direct Vertex
2
vs
3
AI Gateway → Vertex
4

For example:

1
const result = await generateText({
2
  model: "google/gemini-...",
3
  prompt,
4
  providerOptions: {
5
    gateway: {
6
      only: ["vertex"],
7
    },
8
  },
9
})
10

11
console.log(JSON.stringify(result.providerMetadata?.gateway, null, 2))
12

If the metadata shows a different resolved provider, a fallback attempt, or more than one provider attempt, that could explain the added latency.

I’d also measure first-token latency separately from full completion time, because total duration can hide whether the delay is before the provider starts responding or during generation. For the comparison, keep these identical:

1
- same model ID / model variant
2
- same prompt and max output settings
3
- same Vercel Function region/runtime
4
- same streaming vs non-streaming mode
5
- same Vertex location / EU routing assumptions
6

The routing options are here if you want to pin or order providers during testing:

https://vercel.com/docs/ai-gateway/models-and-providers/provider-options

Can you share the redacted providerMetadata.gateway.routing output from one slow Gateway request?