SSE · Troubleshooting
Troubleshoot interrupted AI API streams: SSE parsing, timeouts and proxies
Last updated：
A streaming request may return HTTP 200 and then fail halfway through. Receiving some text and completing the request are separate states. Check completion events first, then distinguish output limits, network timeouts and client parsing errors.
1. Record a minimal reproduction
Record protocol, model ID, request time/ID, first-event time and last-event type. Keep keys and private prompts out of public reports. Reproduce with a short prompt, then compare with a long conversation.
2. Check how each protocol finishes
| Protocol | What to inspect |
|---|---|
| OpenAI Chat Completions stream | Deltas, finish_reason and the endpoint’s [DONE] marker. |
| Anthropic Messages stream | Content-block deltas, message_delta, message_stop and error events. |
| Gemini stream | Its SSE format, candidate content and finishReason; an OpenAI parser is unsuitable. |
Some finish/stop reasons signal an output cap or tool request rather than a network failure. Connection closure alone does not prove completion, and [DONE] is not a universal marker for all protocols.
3. Network chunks are not complete SSE events
One read can split a UTF-8 character or contain several events. Keep a buffer across reads, separate events at blank lines and support multiline data:. The Node.js fetch and SSE guide includes an offline parser demo.
If JSON parsing fails in a network chunk, check whether the client incorrectly treated that chunk as an entire event before concluding the server sent malformed data.
4. Distinguish proxy buffering from timeouts
| Symptom | Checks |
|---|---|
| Long wait followed by all text at once | Streaming reads, proxy buffering and SSE content type. |
| Stops after roughly the same duration | Client total timeout, proxy read timeout and server task limits. |
| Only one IDE or desktop client fails | That process’s network, proxy and environment. |
| Only long output fails | Output limit, long-lived connections and in-stream errors. |
| Error event after text | Preserve the partial result and mark the request incomplete. |
For your own Nginx proxy, inspect buffering and read-timeout settings for streaming routes. Identify the layer closing the connection before changing timeouts; use values suited to that route.
5. Decide whether to retry
Check request records and response state first. Missing a completion event at the client does not establish that no server generation or billing occurred. Separate partial from completed results and, where appropriate, let the user choose to continue, retry or keep partial output. Avoid silently repeating tasks.
For cost reconciliation, see token billing. Handle authentication issues using the 401 guide.