With HTTP streaming the connection to clients stays open, allowing a server to push data to connected clients. Popular use cases are when there is large data, and because of its size is sent in smaller part or a server needs to push updates to connected clients. For example sending the content of a very large log file or sending stock updates to connected Apps.
HTTP Chunked data
To send data as chunk the server must send Transfer Encoding: chunked
header in the first response.
app.get("some-big-file", (req, res) => {
// Indicate we're sending a multipart response
res.writeHead(200, {
Connection: "keep-alive",
"Content-Type": 'multipart/mixed; boundary="-"',
"Transfer-Encoding": "chunked",
});
// If the request is closed by the client we can do some cleanup
req.on("close", () => {
//...
});
res.write("---");
// Subscribe and send back each result as a separate chunk. We await the readData
// call. Once we're done executing the request and there are no more results to send
// to the client, the Promise returned by subscribe will resolve and we can end the response.
await readData((result) => {
const chunk = Buffer.from(JSON.stringify(result), "utf8");
const data = [
"",
"Content-Type: application/json; charset=utf-8",
"Content-Length: " + String(chunk.length),
"",
chunk,
];
if (result.hasMoreData) {
data.push("---");
}
res.write(data.join("\r\n"));
});
res.write("\r\n-----\r\n");
res.end();
})
Note: One downside of this solution is that it is not supported in HTTP 2. It is an HTTP 1 only feature.
Server Sent Events
Server Sent Events(SSE) are another streaming method. SSE also has a standardized HTML5 API called EventSource. With SSE, the data is encoded as text/event-stream
in the header.
app.get("subscribe-to-events", (req, res) => {
// Indicate we're sending an event stream to the client
res.writeHead(200, {
"Content-Type": "text/event-stream",
Connection: "keep-alive",
"Cache-Control": "no-cache",
});
// If the request is closed by the client, we do some cleanup
req.on("close", () => {
// ... some cleanup code
});
// We subscribe to the event stream and push any new events to the client
onSomeEvent((result) => {
res.write(`data: ${JSON.stringify(result)}\n\n`);
});
})