The AI SDK Production Checklist: Streaming, Tools, and the Parts That Fail

Building an LLM feature with the Vercel AI SDK feels finished after about twenty minutes. You call streamText, wire the result to a text box, and tokens start painting on the screen one at a time. It looks like production. It demos beautifully. Everyone nods.

Then you ship it, and the parts nobody demos start showing up. A provider rate-limits you mid-stream and the user sees nothing. Someone pastes a 40-page PDF into the chat and your function 400s while still billing you for the tokens. Your bill triples in a week and you cannot explain why. The happy path was never the hard part. The hard part is everything that happens when the happy path doesn't hold.

This is the checklist I wish someone had handed me before the first deploy. It's built around the AI SDK because that's what most people reach for, and because it's genuinely good. The SDK shipped version 6.0 on December 22, 2025 with over 20 million monthly downloads, up from the 2 million weekly downloads it reported when version 5 launched on July 31, 2025.^[1] That kind of adoption means the sharp edges are well-documented. It also means a lot of people are hitting them.

The stream that fails without telling you

This is the one that cost me the most time, so it goes first. When you use streamText or createDataStreamResponse, the response has already sent HTTP 200 and started flowing before the model does most of its work. So when something goes wrong partway through, the SDK can't hand you a clean server error. The connection is open. The status is already 200. There's nowhere to put the failure.

The result is documented across multiple open issues on the vercel/ai repo (#4726, #4720, #3893): a bad model name, a wrong API key, or a mid-stream provider rate-limit can produce no server-side error at all, and the client gets a generic “An error occurred.” message with nothing in your logs to explain it.^[2]You find out from a support ticket, not from your monitoring. That's the worst way to find out anything.

The fix is to stop assuming a thrown error will reach you. You pass an onErrorcallback into the stream call and do your own logging there. That's where the real error object lives. Without it, the error exists for a few milliseconds inside the SDK and then evaporates.

Retries: two by default, and that's a decision

The AI SDK retries failed provider calls twice by default. maxRetries is 2, and the backoff is exponential and respects the provider's Retry-After and retry-after-ms headers, which is exactly what you want.^[3] When the attempts run out, you get a typed RetryError, and the SDK surfaces other typed errors too: APICallError, NoTextGeneratedError, and friends. Typed errors are a gift. Use them to branch instead of string-matching on messages.

Two retries is a reasonable default, but treat it as a decision, not a law. For a user-facing chat where someone is staring at a spinner, two retries with backoff can stack into several seconds of dead air before you even get to fail. Sometimes you want maxRetries: 0and a fast, honest “try again” instead of a long silent wait. For a background job that nobody is watching, crank it up. The point is that the retry count interacts with your latency budget, and the default was chosen without knowing yours.

Timeouts: the 25-second wall nobody mentions

Streaming and serverless timeouts have a nasty interaction, and it's the single most common reason a working local build breaks on Vercel. With Fluid compute, which is the default for new projects, function max duration defaults to 300 seconds across all plans, reaches an 800-second maximum on Pro and Enterprise, and has a 1,800-second extended maximum in beta.^[4] That sounds like plenty. It usually is.

The trap is the Edge runtime, which has a second, tighter clock. On Edge you must begin sending a response within 25 seconds or the stream is killed. You can then keep streaming for up to 300 seconds, but only if you cleared that first-byte window.^[5] A function that overruns returns a 504 FUNCTION_INVOCATION_TIMEOUT. So if you do slow setup work, fetching context, running a retrieval query, calling a tool, before you open the stream, you can blow the 25-second budget and die before the first token ever ships.

The fix is ordering. Open the stream early and do slow work inside it, or move the heavy lifting off the request path entirely. The model's first token is your heartbeat. Get it out the door fast, and the rest of the budget is yours.

The 4.5 MB and six-connection walls

Two more platform limits that turn into confusing bugs. Vercel Functions cap request and response body payloads at 4.5 MB and return HTTP 413 FUNCTION_PAYLOAD_TOO_LARGE when you exceed it.^[4] A chat that accumulates a long history, or a feature that lets users upload content into the prompt, can quietly grow past that ceiling and start 413ing for exactly the power users you least want to lose.

The other one is pure browser behavior: browsers cap HTTP/1.1 connections at six per domain. Open more than six concurrent SSE streams to the same domain and the rest queue silently, with no error, just streams that never start. If you've ever had a dashboard with several live AI panels where some randomly hang, this is a prime suspect. The answer is usually HTTP/2, which multiplexes, or simply not opening that many streams at once.

The paste-bomb: context windows and the 400 you still pay for

Here's the one that motivated this whole piece. Someone will paste something enormous into your input. A full contract, a log dump, an entire codebase file. When the prompt exceeds the model's context window, the provider returns a 400 error with no partial response, and, this is the part that stings, the input tokens are still billed.^[6] You pay for the privilege of being rejected.

And the window you have to respect depends entirely on the model. As of 2026, Claude Opus 4.6 through 4.8 and GPT-5.4 offer 1M-token context windows, while GPT-5's API context tops out at 400,000 tokens.^[7]That's a huge spread. A single pasted document that sails through Opus 4.8 can blow straight past GPT-5's limit. If your app lets users switch models, the safe input size is a moving target.

Production code cannot trust user input to fit. You pre-count the tokens before the call and truncate the oldest messages first, keeping the system prompt and the most recent turns. It's not glamorous. It's the difference between a graceful “I trimmed some older context” and a hard 400 that also charged you.

Cost control is an architecture decision, not a setting

The bill is where LLM features quietly go wrong, because the cost is invisible until the invoice. GPT-5 launched in August 2025 at $1.25 per million input tokens and $10.00 per million output tokens.^[8]Output is 8x the price of input, which tells you where to aim: shorter, more structured responses are not just nicer, they're cheaper.

Two levers move the number a lot. Prompt caching makes the cached prefix roughly 90 percent cheaper, so if you put your stable system prompt and shared context first, repeated calls ride the cache and the marginal cost collapses. And the Batch endpoint runs at 50 percent of standard pricing for anything that doesn't need a real-time answer.^[8]Summarizing yesterday's support tickets overnight has no business paying real-time rates. Structure the work so the non-urgent half goes through Batch.

Tools and the agentic loop

If you're on AI SDK 5 or later, the tool story changed and it's worth knowing before you copy old code. Version 5 renamed the tool field parameters to inputSchema and result to output, added Zod 4 compatibility, and made tool-call inputs stream by default with lifecycle hooks: onInputStart, onInputDelta, and onInputAvailable.^[1]If you paste a version-4 tool definition into a version-5 project, it won't line up, and the mismatch is easy to miss.

The bigger addition is loop control. Version 5 introduced stopWhen, so you cap the agent with something like stepCountIs(5) or hasToolCall('finalAnswer'), plus a prepareStep hook to change the model, system prompt, tools, or tool choice per step, and an Agent class wrapping generateText and streamText.^[1] Version 6 went further with a ToolLoopAgent class whose production tool-execution loop defaults to a maximum of 20 steps.^[9]

That default matters. An agent without a hard stop is a runaway cost and latency risk. A confused model can loop, calling tools, reconsidering, calling again, until something cuts it off. The step cap is the thing that cuts it off. Set it deliberately for your use case instead of trusting a loop to end on its own, because a loop that doesn't end is billing you the whole time.

The structured-output tax

One last failure mode, because it surprises people who think schemas make LLM output safe. Practitioner reports put structured-output validation failure at roughly 2 to 5 percent on complex schemas, and the reason is timing: Zod validation runs only after the full response is generated.^[10]So when the parse fails, the tokens are already spent. You paid for output that didn't validate.

The same reports find that array outputs above about 15 items get unreliable, and that batching to 10 to 15 items at a time cuts failures to under 1 percent.^[10]So if you're asking a model to return 50 structured records in one shot, you're fighting the grain. Chunk it. Ask for smaller batches, validate each, and retry the ones that fail rather than re-rolling the whole set and paying for all of it again.

The actual checklist

Strip away the explanation and here's what I actually check before an AI SDK feature goes out:

• onError on every stream. Assume streamed errors never reach your normal handler. Log them in onError and be deliberate about what the client sees.
• Retries matched to the wait. maxRetries: 0 or low when a human is watching, higher for background jobs. Branch on the typed errors.
• First byte before 25 seconds. On Edge, open the stream before slow work, or a 504 kills you regardless of your 300-second budget.
• Token pre-count with oldest-first truncation. Never trust input to fit the window. A 400 for overflow still bills the tokens.
• Payloads under 4.5 MB.Watch growing chat histories and uploads, and don't open more than six concurrent SSE streams per domain on HTTP/1.1.
• Caching and Batch wired in. Stable prefix first for the 90 percent cache discount, Batch endpoint for anything not real-time.
• Hard step cap on agents. The ToolLoopAgent 20-step default exists for a reason. Set yours on purpose.
• Batched, validated structured output. Arrays under ~15 items, expect a 2 to 5 percent re-roll on complex schemas.

None of this is exotic. It's the gap between a thing that works in a demo and a thing that survives contact with real users, real bills, and real providers having a bad day. The AI SDK gives you good tools for all of it, typed errors, retry controls, lifecycle hooks, loop caps. It just doesn't turn them on for you, because it can't know your latency budget or your cost tolerance. That part is the engineering. The demo was never the job.