I want to read a string that is encoded in CSV format synchronously in Node.js. (I don't want to read a CSV file asynchronously).

To illustrate, below is asynchronous code to read a CSV file in Node.js. This code works well.

const fs = require("fs");
// https://www.npmjs.com/package/csv
const { parse } = require("csv-parse");

const readCSV = () => {
  fs.createReadStream("./testfile.csv").pipe(
    parse({ delimiter: ",", from_line: 1 })
      .on("data", function (row) {
        console.log(row);
      })
      .on("end", function () {
        console.log("finished");
      })
      .on("error", function (error) {
        console.log(error.message);
      })
  );
};

Below is the content of testfile.csv. Although it's just one line, it is not an easy one (notice the the multiple special characters and commas, arbitrarily placed for the purposes of this test), and it is best handled with a proper CSV parser.

Guatemala,United States,"Congo, ""Dem."" Rep. 'of' `the` (Kinshasa)",other country name

How do I read the same content as an encoded string in Node.js?

Below is the same example as an encoded string in Node.js which I wish to read synchronously. Notice that I had to escape the ' characters given the syntax of the language.

  const encodedString = 'Guatemala,United States,"Congo, ""Dem."" Rep. \'of\' `the` (Kinshasa)",other country name';

I came across this problem for two reasons:

1. I wish to encode a Node.js array as a string that can handle special characters and commas (and decode it as needed); and

2. I couldn't figure how to use the CSV package (or any other package, I really looked at a lot of them) for strings. That is, they seem to always assume that you will be reading a CSV file with the fs package asynchronously; while my use case assumes that I already have the data encoded as a string variable. Thus, I also wish to do it synchronously, for there's no reason to do it differently since I am not getting the data from a stream.

I've already looked a lot of Stack Overflow questions (How do I read the contents of a Node.js stream into a string variable?, Is there a way to read CSV in node.js synchronously?, etc.), and all I could find is examples of how to read a CSV file asynchronously. So the solution to this basic question is still missing.

To summarize

A successful answer to this question will be the one that provides a function to decode the encodedString variable into an array like the one below. An even more ideal answer would also provide the corresponding function to encode the Node.js array into the same original encoded string.

[
  'Guatemala',
  'United States',
  'Congo, "Dem." Rep. \'of\' `the` (Kinshasa)',
  'other country name',
]

CodePudding user response：

The csv-parse expects to act on streams not strings. But you can easily create your own ReadableStream and push your string into it.

const Readable = require('stream').Readable
const { parse } = require("csv-parse")

const stream = new Readable()
const encodedString = 'Guatemala,United States,"Congo, ""Dem."" Rep. \'of\' `the` (Kinshasa)",other country name'
stream.push(encodedString)
stream.push(null)

stream.pipe(
    parse({ delimiter: ",", from_line: 1 })
    .on("data", (row) => {
        console.log(row);
    })
    .on("error", (error) => {
        console.log(error.message);
    })
)

Output:

[
  'Guatemala',
  'United States',
  'Congo, "Dem." Rep. \'of\' `the` (Kinshasa)',
  'other country name'
]

CodePudding user response：

I learned how to use the csv-parse package synchronously. It is in fact supported by the package. You just need to import the synchronous version of the module. See documentation here https://csv.js.org/parse/api/sync/.

Below is a script that can handle the encoded string I provided as an example.

// https://csv.js.org/parse/api/sync/
const { parse } = require("csv-parse/sync");

const decodeCSVStringSync = () => {
  const encodedString =
    'Guatemala,United States,"Congo, ""Dem."" Rep. \'of\' `the` (Kinshasa)",other country name';

  const records = parse(encodedString, { delimiter: ",", from_line: 1 });

  console.log(records[0]);
};

decodeCSVStringSync();

Output:

[
  'Guatemala',
  'United States',
  'Congo, "Dem." Rep. \'of\' `the` (Kinshasa)',
  'other country name'
]

And below is the corresponding function to encode an array of data into an string that follows the CSV format. (The output is the original encoded string.)

//https://csv.js.org/stringify/options/cast/#field-level-options
const { stringify } = require("csv-stringify/sync");

const encodeArrayToCSVStringSync = () => {
  const arrayData = [
    "Guatemala",
    "United States",
    "Congo, \"Dem.\" Rep. 'of' `the` (Kinshasa)",
    "other country name",
  ];

  const encodedString = stringify([arrayData]);

  console.log(encodedString);
};

encodeArrayToCSVStringSync();

Output:

Guatemala,United States,"Congo, ""Dem."" Rep. 'of' `the` (Kinshasa)",other country name