Synchronicity problem: Wait until multiple fs.readFile calls complete-CodePudding

I want to read columns from different CSV files, and combine those columns into an array. I'm using fs.readFile to read the CSV files and a callback which processes the data and pushed a new element onto the array of columns. This array of columns is then sent on to another part of the software (I'm building an electron app, so it's sent to the render process).

The issue I'm having is that 'fs.readFile' is asynchronous, so my columns array is sent off before any of the fs.readFile calls finish, resulting in an empty array.

What's the best way to solve this issue? It is to simply use fs.readFileSync? Is there a way to do it without blocking execution?

Minimal example code below:

//Process each column, reading the file and extracting the data one at a time
let columns: (number[] | undefined)[] = []; //Somewhere to store the processed columns
for (const dataHandle of dataHandles)
{
  //read the file as a raw string
  fs.readFile(dataHandle.filePath, (error: any, data: any) => {
    if (error) {
      console.log("Error reading file:", error);
    } else {
      data = data.toString();
      const newColumn = parseColumnFromStringDataframe(data, dataHandle.columnName);
      columns.push(newColumn);
    }
  })
}
//Finished processing each column, so send response.
//BUT... readfile is non-blocking! Sends the response before anything is pushed to columns! How can we wait in a smart way?
console.log(columns); // []
mainWindow?.webContents.send("readDataProductColumnsResponse", columns); //Sends response

CodePudding user response：

This was answered here: https://stackoverflow.com/a/34642827/7603434

Basically you have to create an array of promises and then call Promise.all(promises);

const fs = require("fs");
const files = ["app.js", "index.html", "script.js"];

const readAllFiles = async () => {
  let promises = [];
  for (const f of files) {
    promises.push(fs.promises.readFile(f, "utf8"));
  }
  return Promise.all(promises);
};

async function run() {
  readAllFiles()
    .then((fileContents) => {
      console.log("done", fileContents);
      // fileContents is an array that contains all the file contents as strings
    })
    .catch((err) => {
      console.error(err);
    });
}

run();