NodeJs Csv parser async operations-CodePudding

I'm trying to read a csv file, fetch data for every record on csv do some operations and at the end (after all of the async operations are complete) write a new csv file here is the code:

async function readInputCsv() {

  results = [];
  fs.createReadStream('./example_short.csv')
.pipe(csv())
.on('data', async function(data){
        console.log("trying for name: ", data.username);
        let edu = await getLastEducation(data.username);
        results.push({
          name: data.username,
          level: edu
        })
        console.log(data.username,": ", edu)
}).on("end", () => {
  console.log("end ! ");
  writeCsv(results);

});
}

What happens is "end !" gets printed before the getLastEducation function returns the responses

CodePudding user response：

fs.createReadStream().on() does not accept Promises as second argument.

on(event: string, listener: (...args: any[]) => void): this;

CodePudding user response：

First of all the results in this case is a global variable. I would suggest you to use let keyword to declare it within the scope of the function.

What is happening in this case is the time taken by the getLastEducation() for the async operation is more than the time required to the example_short.csv file.

To avoid this, add await keyword before fs.createReadStream() and move the writeCsv() function (I am assuming you are using csv-writer inside this function) after the entire read operation. The await keyword will make it wait until the async fetching operation is done.

async function readInputCsv() {
  let results = [];

  await fs
    .createReadStream('./example_short.csv')
    .pipe(csv())
    .on('data', async function (data) {
      console.log('trying for name: ', data.username);
      let edu = await getLastEducation(data.username);
      results.push({
        name: data.username,
        level: edu,
      });
      console.log(data.username, ': ', edu);
    })
    .on('end', () => {
      console.log('end ! ');
    });

  writeCsv(results);
}