I have an ExpressJs endpoint returning chunks of data using res.write() like this:

process.stdout.on("data", (data) => {
    const progress = getDownloadProgress(data.toString());
    res.write(JSON.stringify(progress));
  });

process.stdout.on("end", (data) => {
    res.end();
  });

Then at my front-end side, I struggle to consume those chunks of progress values like 10%, 66%. Here is what I have so far used ReadableStream with the Fetch API:

fetch("http://localhost:4000/download")
    .then((response) => response.body)
    .then((rb) => {
      const reader = rb.getReader();

      return new ReadableStream({
        start(controller) {
          // The following function handles each data chunk
          function push() {
            // "done" is a Boolean and value a "Uint8Array"
            reader.read().then(({ done, value }) => {
              // If there is no more data to read
              if (done) {
                console.log("done", done);
                controller.close();
                return;
              }
              // Get the data and send it to the browser via the controller
              controller.enqueue(value);
              // Check chunks by logging to the console
              console.log(done, value); // I'm seeing what I assume to be a stream of my chunks here, but its in array buffer format
              push();
            });
          }

          push();
        },
      });
    })
    .then((stream) => {
      // Respond with our stream
      return new Response(stream, {
        headers: { "Content-Type": "application/json" },
      }).json();
    })
    .then((result) => {
      // Do things with result
      console.log(result);
    });

It's just a snippet from MDN.

From the console.log, I'm getting a stream of what I assume to be my chunks. But they are in array buffer format - not something I can use to update my UI.

I guess my basic question is: How to have my front-end consume the streamed response sent through res.write() from my ExpressJs?

CodePudding user response：

According to the response.write() method documentation, it takes three parameters:

response.write(chunk[, encoding][, callback]);

So, try to add encoding(which is usually utf-8) in your write method:

process.stdout.on("data", (data) => {
    const progress = getDownloadProgress(data.toString());
    res.write(JSON.stringify(progress), "utf-8");
  });

CodePudding user response：

I decided to just decode the array buffer using TextDecoder:

const decoder = new TextDecoder();

const response = await fetch("http://localhost:4000/download");
  const reader = response.body.getReader();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;

    try {
      // decodedChunk will have your literal values
      const decodedChunk = JSON.parse(decoder.decode(value));
    } catch (e) {}
  }