#ai-sdk

Jonoroboto Sick thumbnail - love it. Can't wait to build a full blown accounting tool with Jev

tope love to see this, welldone

Amy Egan so cute

Skindowshao 3161 very well . me too.

Ryu Hi Pallav, I wouldn’t take the display `name` mismatch alone as proof that the backend is routing to the wrong model, but it is definitely worth reporting because the model catalog/API should not label `openai/gpt-oss-20b` as `GPT OSS 120B`. A useful way to separate “display metadata bug” from “routing bug” would be to compare the two model IDs directly: ``` curl -s https://ai-gateway.vercel.sh/v1/models \ | jq '.data[] | select(.id == "openai/gpt-oss-20b" or .id == "openai/gpt-oss-120b")' ``` Then, if you have an AI Gateway key, compare the endpoint metadata too: ``` curl -s https://ai-gateway.vercel.sh/v1/models/openai/gpt-oss-20b/endpoints \ -H "Authorization: Bearer $AI_GATEWAY_API_KEY" curl -s https://ai-gateway.vercel.sh/v1/models/openai/gpt-oss-120b/endpoints \ -H "Authorization: Bearer $AI_GATEWAY_API_KEY" ``` If only the `name` field is wrong but the IDs, endpoints, pricing/context fields, and generation records differ correctly, then it is probably a catalog/display bug. If the 20B ID returns the same endpoint/pricing/usage behavior as 120B, then that is stronger evidence of a routing/config issue. For a clean bug report, I’d include: ``` model id: openai/gpt-oss-20b incorrect name returned: GPT OSS 120B timestamp: output from /v1/models for both 20B and 120B: generationId from one tiny test request, if available: ``` Don’t include the AI Gateway key.

Ryu Hi Sbullocks, Since static assets work but every function hangs until timeout, I’d separate “Vercel can route to a function” from “your new runtime code is waiting forever.” First, deploy one tiny endpoint that does not import anything shared from the app: ``` // app/api/ping/route.ts export const dynamic = "force-dynamic" export async function GET() { return Response.json({ ok: true, time: Date.now() }) } ``` If that works, the platform/function routing is fine and I’d look at the changes from the rate-limiting commit: middleware, shared auth helpers, Redis/KV/database clients, external fetches, or env vars that are now awaited before returning a response. If that tiny endpoint also hangs, check whether it is really included in the latest deployment and inspect logs outside the dashboard: ``` vercel inspect <deployment-url> --logs vercel logs <deployment-url> --since 1h ``` For the affected routes, I’d also add timing around each awaited operation: ``` console.time("rate-limit") await checkRateLimit() console.timeEnd("rate-limit") console.time("upstream") const res = await fetch(url, { signal: AbortSignal.timeout(10000) }) console.timeEnd("upstream") ``` A very common cause after adding rate limiting is a server-side call waiting on a missing/incorrect Production env var or an upstream service that is reachable locally but not from the deployed function. The key signal is whether the function logs stop before or after the rate-limit / external API call. Vercel has a useful slow-function debugging command here: https://vercel.com/docs/functions/debug-slow-functions Can you share the first log line before the hang and whether the isolated `/api/ping` route returns immediately?

Frdwings Successful control: gen_01M2CQE67VEZVYMBTG88XF787H GPT-5.4-mini → OpenAI → HTTP 200 → inference completed Failed current control: gen_01M2EE8Y8GE0B21BARH3R3XW7T GPT-5.4-mini → Azure → HTTP 403 → pre-inference Gateway API Key ID: 2anmn74Bq2bVwWbkAWWEP09dres3z3ha85VOzqvUT9jJM2M0

Tadeu Tupinambá OBSERVATIONS 1. The largest inline payload ever returned to us is 16,123 KB. Re-running the dropped prompts dropped them again, and the delivered ones came through again, which points to a fixed size threshold rather than a transient fault. 2. On dropped jobs `expired` is already true at completion time, while `expiredAt` is completion + 24 h, the same window a hosted result gets. The result appears to be discarded rather than stored. 3. The status is "completed" with empty `warnings`. There is no error, no 413, and the webhook payload carries no bytes and sends the receiver back to the same status call. 4. The documentation mentions only the 300 KiB cap on the persisted start request and the `expiresAt` / `expiredAt` / `partial` / `unretained` keys. Neither `inlineTooLarge` nor a result-size limit is documented. 5. Providers that hand back a hosted URL are unaffected: 18 of 18 Seedance jobs in the same period delivered with result: { expiresAt }. EXPECTED Either the result is hosted with an `expiresAt`, as URL providers' results are, or the job ends in an explicit error so a client can retry at a smaller size. A "completed" job with no output and no warning is indistinguishable from success. REQUESTS a) Host inline results above the cap, or raise the cap for the async job store. b) Until then, return an error status, or at least a warning, when a result is dropped. c) Document the limit and the `inlineTooLarge` key. d) Credit for the seven generations billed without an output. RAW STATUS RESPONSE for job_68M78EC73RVNDWQBD5XKBJ37WA (queried again at 2026-09-03 00:53 UTC) { "status": "completed", "videos": [], "warnings": [], "providerMetadata": { "gateway": { "asyncJob": { "jobId": "job_68M78EC73RVNDWQBD5XKBJ37WA", "status": "completed", "completedAt": 1788396479596, "result": { "expired": true, "expiredAt": 1788482879550, "inlineTooLarge": true } }, "generationId": "gen_01M1JBDZ9FB4MC8X02M709VGZW" } }, "response": { "modelId": "google/veo-3.1-generate-001", "headers": { "x-matched-path": "/v4/ai/video-model/status", "x-vercel-id": "iad1::iad1::cznpt-1788396828104-fe62f2166fbb" } } } Happy to share more job ids or re-run a controlled case if it helps. Thanks, Tadeu Tupinambá

Udayganguru2000 1232 May be I have to switch to lang graph, this is not working for some reason. advise me it it's a better idea.

Ryu Hi Nekomiya, Since the same request works with `only: ['anthropic']` but fails with `only: ['bedrock']`, I’d treat this as a Gateway → Bedrock request translation/schema issue first, not an auth issue. `providerOptions.gateway.only` is the right kind of option for restricting routing, but Bedrock seems to be rejecting an extra field inside the tool definition: ``` tools.0.custom.eager_input_streaming: Extra inputs are not permitted ``` A useful isolation test would be to run three tiny requests against the same model/provider route: 1. no tools at all 2. one minimal tool with only `name`, `description`, and `input_schema` / `parameters` 3. your real tool payload If #1 works and #2/#3 fail only on Bedrock, that narrows it to tool schema serialization rather than model access or Gateway auth. I’d also check whether your HTTP request is manually including any `custom` object under `tools`. If it is, remove that for Bedrock. If it is not in your payload and Gateway is adding it internally, then the best bug report would include: ``` status code redacted request body redacted response body model ID providerOptions.gateway.only value timestamp x-vercel-id / request ID if present ``` The Gateway provider options docs are here for comparison: https://vercel.com/docs/ai-gateway/models-and-providers/provider-options As a temporary workaround, I’d route tool-calling requests away from Bedrock or remove provider-specific/custom tool metadata when forcing Bedrock, then keep the smallest failing repro for Vercel to inspect.

Ryu Hi Wyatt, I would split this into two stored representations instead of trying to make one MongoDB document serve both jobs. For example: ``` 1. UI/history record - what you show in the chat UI - can include reasoning parts, metadata, timestamps, model name, token usage, etc. 2. Model context record - only the clean ModelMessage[] you plan to send back into generateText/streamText - no UI-only fields - no provider response metadata - no fields copied blindly from the provider response ``` `outputs.response.messages` is useful for continuing a conversation, but I would avoid persisting the full object and replaying it forever without normalizing it first. If a provider adds metadata or reasoning-specific fields that are valid as output but not valid as future input for that provider, you can get exactly this kind of “invalid prompt” problem after loading. A safer save step is to whitelist the fields you actually want to send back: ``` function cleanModelMessages(messages: ModelMessage[]): ModelMessage[] { return messages.map((message) => ({ role: message.role, content: message.content, })) } ``` Then store reasoning separately for display/debugging, not necessarily as part of the next model prompt. In many apps, the previous assistant’s final answer is what you resend as context, while reasoning traces are stored for audit/UI and not replayed to the model on every turn. So the flow would be: ``` const context = await loadCleanModelContext(userId) context.push({ role: "user", ModelContext(userId) context.push({ role: "user", content: [{ type: "text", text: prompt }], }) const result = await generateText({ model: openrouter.chat("google/gemini-3-flash-preview", { reasoning: { enabled: true, effort: "medium", }, }), system: SYSTEM_PROMPT, messages: context, }) // store clean context for the next call await saveCleanModelContext(userId, [ ...context, ...cleanModelMessages(result.response.messages), ]) // store richer UI/debug data separately await saveUiTranscript(userId, { text: result.text, responseMessages: result.response.messages, providerMetadata: result.providerMetadata, }) ``` Also, you’re right that `convertToModelMessages()` is mainly for converting `UIMessage[]` from `useChat` into model messages. I would not use it on data that is already stored as `ModelMessage[]`. The AI SDK docs mention `response.messages` for conversation history here: https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#response-messages If removing `providerOptions` makes the prompt valid, that is a good signal that the loaded object contains provider-specific data that should not be replayed as-is. I’d keep that data in your UI/audit collection, but only end a normalized `ModelMessage[]` back to the model.

Ryu Hi MountainMan, Yes, I’d expect the same Bedrock `providerOptions` shape to work with `streamText`. The docs show `generateText`, but the guardrail config is a provider option on the Bedrock model call, not something specific to `generateText`. Example: ``` import { bedrock, type AmazonBedrockLanguageModelOptions } from "@ai-sdk/amazon-bedrock" import { streamText } from "ai" const result = streamText({ model: bedrock("anthropic.claude-3-sonnet-20240229-v1:0"), prompt: "Write a story about space exploration.", providerOptions: { bedrock: { guardrailConfig: { guardrailIdentifier: "1abcd2ef34gh", guardrailVersion: "1", trace: "enabled", streamProcessingMode: "async", }, } satisfies AmazonBedrockLanguageModelOptions, }, }) return result.toTextStreamResponse() ``` The main thing I’d watch is `streamProcessingMode`. For streaming responses, `async` is usually the setting I’d try first because it lets streaming continue while Bedrock evaluates guardrails. If you need stricter blocking behavior before output reaches the client, test the other mode and confirm the latency/behavior matches what you want. The Bedrock provider docs are here: https://ai-sdk.dev/providers/ai-sdk-providers/amazon-bedrock#guardrails If it does not behave the same with `streamText`, I’d log the Bedrock error body and the AI SDK / `@ai-sdk/amazon-bedrock` versions, because that would likely be a provider-specific streaming issue rather than a different top-level API shape.