How to get data out of a stream?-CodePudding

I would have expected that I got all the CSV data in the array, but for some reason I don't get anything.

Can someone explain why a is empty at the end?

const fs = require('fs');
const { parse } = require('csv-parse');

let a = [];

fs.createReadStream('./example.csv')
   .pipe(parse({ delimiter: ';', from_line: 2 }))
   .on('data', function (row) {
      a.push(row);
   })
   .on('end', function () {
      console.log('finished');
   })
   .on('error', function (error) {
      console.log(error.message);
   });

console.log(a);

CodePudding user response：

createReadStream() runs parallel. So after calling this function, the program continues and the next process is your console.log(a). Thats the idea behind streams because the program can continue running while the big file is read out.

The code inside the .on() functions is called later. Thats why they are called "callbacks".

So you are logging the result before any data is read...

CodePudding user response：

As explained before the headstream runs in the background. You can wrap the stream in a promise to get the result. Here is one possible solution:

const fs = require('fs');
const { parse } = require('csv-parse');

const readFile = async () => new Promise((resolve, reject) => {
    let a = [];
    fs.createReadStream('./example.csv')
        .pipe(parse({ delimiter: ';', from_line: 2 }))
        .on('data', function (row) {
            a.push(row);
        })
        .on('end', function () {
            resolve(a);
        })
        .on('error', function (error) {
            reject(error)
        });
});

readFile().then(data => console.log(data))

CodePudding user response：

Your a is undefined because the code is executed asynchronously. The on() function is called after your console.log(a) is executed.

You can read the value of a inside the data event or the end event:

fs.createReadStream('./example.csv')
   .pipe(parse({ delimiter: ';', from_line: 2 }))
   .on('data', function (row) {
      a.push(row);
      console.log(a) // <------- here
   })
   .on('end', function () {
      console.log('finished');
      console.log(a) // <------- here
   })
   .on('error', function (error) {
      console.log(error.message);
   });

One solution may be to use a Promise

const readCSVData = new Promise((resolve, reject) => {
let a = []
fs.createReadStream('./example.csv')
   .pipe(parse({ delimiter: ';', from_line: 2 }))
   .on('data', function (row) {
      a.push(row);
   })
   .on('end', function () {
      // return the a value 
      resolve(a) 
   })
   .on('error', function (error) {
      reject(error)
   });
})

// ..... 

const foo = async () => {
  const data = await readCSVData()
  console.log(data)
}