JavaScript NodeJS How to use stream/promises with async functions?-CodePudding

I have a JS async function in Node. Say, it downloads a file from a URL and does something with it, eg. unzip it. I wrote it like this, it works, but eslint showed me there is a thick code smell: error Promise executor functions should not be async no-async-promise-executor. Async was required because of await fetch in body function.

I am not skilled enough with streams nor async/await to correct it by myself. I would like to get rid of the Promise and fully use async/await. The module stream/promises seems the way to go from Node-15 on as commented here how-to-use-es8-async-await-with-streams. How to use await pipeline(...) in this context? Maybe there's a better and shorter way?

Here's the function:

function doSomething(url) {
  return new Promise(async (resolve, reject) => {
    try {
      const fileWriteStream = fs.createWriteStream(someFile, {
        autoClose: true,
        flags: 'w',
      });

      const res = await fetch(url);
      const body = res.body;
      body
        .pipe(fileWriteStream)
        .on('error', (err) => {
          reject(err);
        })
        .on('finish', async () => {
          await doWhatever();
          resolve('DONE');
        });
    } catch (err) {
      reject(err);
    }
  });
}

CodePudding user response：

You can use the fs/promises in NodeJS and trim your code down to the following:

import { writeFile } from 'fs/promises'

async function doSomething(url) {
    const res = await fetch(url);
    if (!res.ok) throw new Error('Response not ok');
    await writeFile(someFile, res.body, { encoding: 'utf-8'})
    await doWhatever();
    return 'DONE';
  });
}

CodePudding user response：

You could simply perform the await before getting to the executor:

async function doSomething(url) {
  
  const fileWriteStream = fs.createWriteStream(someFile, { autoClose: true, flags: 'w' });
  
  let { body } = await fetch(url);
  
  body.pipe(fileWriteStream);
  
  return new Promise((reject, resolve) => {
    body.on('error', reject);
    body.on('finish', resolve);
  });
  
};

My advice in general is to remove as much code from within your promise executors as possible. In this case the Promise is only necessary to capture resolution/rejection.

Note that I've also removed doWhatever from within doSomething - this makes doSomething much more robust. You can simply do:

doSomething('http://example.com').then(doWhatever);

Lastly I recommend you set someFile as a parameter of doSomething instead of referencing it from some broader context!

CodePudding user response：

To use the pipeline function you're looking for, it would be

const { pipeline } = require('stream/promises');

async function doSomething(url) {
  const fileWriteStream = fs.createWriteStream(someFile, {
    autoClose: true,
    flags: 'w',
  });

  const res = await fetch(url);
  await pipeline(res.body, fileWriteStream);
  await doWhatever();
  return 'DONE';
}