How to build advanced features for AI chatbots on SSE
The article discusses how to implement advanced features like resumable streams, cancellations, and multi-device synchronization for AI chatbots using Server-Sent Events (SSE). It explains the challenges of maintaining durable and responsive chatbot sessions when transitioning from synchronous to background agent models. While these features are feasible with SSE and proper handling of token streams, the process is more complex than commonly assumed.
- ▪Resumable streams allow users to refresh a page mid-response and continue receiving tokens without waiting for the full response to be stored.
- ▪Cancellation enables stopping an LLM-generated response mid-stream, even if the connection has dropped and reconnected.
- ▪Multi-device support lets users view and interact with the same chat session across different devices in real time.
- ▪SSE can support these features using mechanisms like Last-Event-ID, but implementation complexity is often underestimated.
- ▪Different LLM APIs structure their streaming responses differently, but all include start, delta, and end events for parsing generated content.
Opening excerpt (first ~120 words) tap to expand
Apr 23, 2026 · 9 minHow to make SSE token streams resumable, cancellable, and multi-deviceAgents used to be a thing you talked to synchronously. Now they’re a thing that runs in the background while you work. When you make that change, the transport breaks.But a lot of folks are saying: “No, you can just use Server-Sent Events (SSE) with Last-Event-ID to get a durable stream, it’s easy”. And yes, all of this is do-able. But I contest that it’s easy. So let’s walk through how to do it, and you can decide for yourself.Catch up on the previous article and discussionhttps://news.ycombinator.com/item?id=47832720The advanced chatbot features I want to walk through are:Resumable streams — refresh the page mid-response and get the in-progress tokens back, instead of waiting for the full response…
Excerpt limited to ~120 words for fair-use compliance. The full article is at /dev/knill.
Discussion
0 commentsMore from /dev/knill
