in an async function i, want to main function to return only in a specific point and not to return void at the end of the function

const fun = () => {

  const list = [];
  let streamFinished = 0;
  let streamCount = files.length;

  await fs.readdir(JSON_DIR, async(err, files) => {
    await files.forEach((filename) => {

      const readStream = fs.createReadStream(path.join("directory", filename));
      const parseStream = json.createParseStream();

      await parseStream.on('data', async(hostlist: Host[]) => {
        hostlist.forEach(async host => {
          list.push(host);
        });
      });

      parseStream.on('end', () => {
        streamFinished  ;
        if (streamFinished === streamCount) {
          // End of all streams...
          return list; //this should be the only point when the function return something    
        }
      })

      readStream.pipe(parseStream);
    })
  });
};

CodePudding user response：

There are lots of things wrong with this code. It's a big mix of event-driven streams, callback-driven functions and promises. The first order of business is to make the main business logic just be promise-driven (as you can see in the new parseData() function and switch all control flow outside of that to just use promises. Here's one way to do it:

const fsp = fs.promises;

function parseData(filename) {
    return new Promise((resolve, reject) => {
        const readStream = fs.createReadStream(path.join("directory", filename));
        const parseStream = json.createParseStream();
        const list = [];

        parseStream.on('data', async (hostlist: Host[]) => {
            list.push(...hostlist);
        }).on('end', () => {
            resolve(list);
        }).on('error', reject);

        readStream.pipe(parseStream);
    });
}

const fun = async () => {

    const list = [];
    const files = await fsp.readdir(JSON_DIR);
    for (let filename of files) {
        const listData = await parseData(filename);
        list.push(...listData);
    }
    return list;
};

fun().then(result => {
    console.log(result);
}).catch(err => {
    console.log(err);
});

A few thoughts here:

1. The easiest way to "wait" for a stream operation to complete is to encapsulate it in a promise which you can then use await or .then() with. That's the purpose of the parseData() function. In addition, this also enables error handling by hooking stream errors to the promise rejection.

2. For processing a loop of asynchronous operations, you can either do them one at a time, using await on each asynchronous operation in the loop or you can run them in parallel by collecting an array of promises and using let data = await Promise.all(arrayOfPromises); on that array of promises.

3. It is only useful to use await if the thing you're awaiting is a promise that is connected to your asynchronous operation. So, things like await parseStream.on('data', ...) and await files.forEach(...) are pointless because neither of those return promises. Do NOT just stick await in places unless you KNOW you are awaiting a promise.

4. You will generally NOT want to use .forEach() with an asynchronous operation in the loop. Because .forEach() has no return value and no loop control, you can't really control much. Use a regular for loop instead which gives you full control. I consider .forEach() pretty much obsolete for asynchronous programming.

5. Don't mix promises and callbacks and don't mix promises and streams. If you have those, then "promisify" the stream or callback so all your main logic/control flow can be promises. This will vastly simplify your code and make error handling possible/practical.