Custom SSE for LangGraph Chat in Next.js

If you are shipping chat on Next.js, ai + streamText + useChat is the correct default. Fast to wire, solid UX helpers, a known protocol.

I did not use it for production chat.

Not because the SDK is bad. Because my streaming problem was not "call a model and paint tokens." It was "run a LangGraph RAG agent, give the conversation a durable URL before the first token, and treat the socket as optional."

That is a different product shape. So I used LangGraph's streamEvents, a native ReadableStream SSE response, and a small custom client hook.

When the default stops fitting

useChat assumes a fairly linear turn: user message in, assistant tokens out. My agent is a graph:

generateQueryOrRespond -> retrieve -> grade -> (rewrite?) -> generate -> END

Internal nodes are non-streaming. The UI should only see on_chat_model_stream chunks from the final model. Everything else (retrieval, grading, rewrites) is server-side noise.

I also needed three product behaviors the SDK does not own for me:

1. A real /conversation/{id} URL before generation starts
2. Partial answers persisted while tokens are still arriving
3. Client disconnect stops the live view, not the write

Once those are requirements, a custom SSE lane is simpler than bending the AI SDK around an agent runtime it does not control.

Architecture

Stack:

Protocol is intentionally dumb JSON over SSE:

data: {"assistantMessageId":"..."}

data: {"content":"token..."}

data: {"done":true}

No AI SDK data-stream prefixes. No EventSource (I need authenticated POST bodies).

Decision 1: Persist first, stream second

The hard part of "new chat" is not streaming. It is navigation.

If you start the SSE on / and then route to /conversation/{id}, you are streaming across a page transition. That is fragile.

Instead:

1. POST /api/conversations inserts the user message and an empty assistant row (content: "")
2. Client navigates with a real id
3. The conversation page SSR-hydrates both messages
4. The hook detects an empty assistant shell after a user message and opens SSE into that row

The URL exists before token one. The DB already knows which message to fill. Follow-ups do the optimistic version of the same idea: temp assistant id on the client, swap it when the first SSE event returns assistantMessageId.

Decision 2: The socket is a viewer, not the job

Abort means "stop writing to this response," not "stop generating."

abortSignal.addEventListener("abort", () => {
  isClientConnected = false;
});

On each on_chat_model_stream chunk:

• if connected, enqueue SSE content
• always append locally
• flush to Postgres about every 50 characters
• on finish, persist the final message and clear isGenerating

Client stop aborts the fetch only. Reload after disconnect reads whatever was saved. No resume protocol.

Caveat, said plainly: this is "continue for the life of the request," not a guaranteed background worker. On serverless, the runtime can still kill the function after disconnect. Periodic DB writes reduce how much you lose. They do not invent a job queue.

Decision 3: Stream the answer, not the agent

LangGraph emits many events. The route only forwards:

event === "on_chat_model_stream" && data.chunk.content

The UI never sees tool chatter unless I choose to surface it later. That keeps the bubble clean and the client parser tiny:

const reader = response.body?.getReader();
// decode -> split on "\n\n" -> JSON.parse(line.slice(6))

useSSEChat mirrors the familiar surface (messages, input, handleSubmit, isLoading, isStreaming, stop) without importing ai. The SDK can stay out of the hot path entirely.

Tradeoffs

Worth it when

• The model call is really an agent graph you already own
• Conversation identity and persistence matter before streaming
• You want disconnect to degrade to "reload later," not "lost answer"
• You can maintain a small protocol on both sides

Not worth it when

• You are doing linear chat with one provider call
• You want AI SDK UI primitives and ecosystem defaults
• You need true out-of-request background generation (use a queue/worker)

I am not arguing against Vercel's packages. I am arguing that streaming is not the product. Durable generation with an optional live view is. Once you design for that, native SSE over LangGraph is enough.

Mental model

Treat chat streaming as a live subscription to a durable write:

1. Create the rows
2. Start generation
3. Attach SSE if someone is watching
4. Keep writing to the database either way

That is the whole design.