I have to run a python script from my node project that converts a .csv file to a .txt file.

The command for running the python script on the terminal is

python3 csv_to_rdf.py "pathToFile" "date" > "nameOfNewFile.txt"
eg. python3 csv_to_rdf.py modifiedCSV.csv 08122022 > test.txt

I am using the child_process.spawn() method in nodeJS to run the script but I can't figure out the args.

When I use the following code snippet:

I get the following error message:

My question is how do I send in "> test.txt" in my spawn() method?

CodePudding user response：

This is incredible, I am a semantic developer, and I had an extremely similar file (a python script that parsed a .csv and then converted it to .txt and then to .TriG) and I needed it run in my Node.js backend.

Short answer

A very simple solution I thought of after writing everything below is:

Python is better at writing to files than bash. Use python to write to your file and then do not pass a redirect into your spawn() and you'll be good. Escaped characters are difficult to handle in the child_process because you have to deal with JavaScript and Python both trying to handle escaped characters at the same time. If python doesn't have write access, then below may server you better.

Long answer

I moved away from child processes because they cause security issues. Any child_process with user input can be exploited. If you could move away from a child process and re-write your python file in JS/TypeScript, that would be best. If you are confident in the use of the spawn, then here is a solution:

Firstly, you cannot add a redirect to your spawn because the command is python3 and, as the error suggests, python3 takes the redirect as an unrecognized argument. It literally takes it as an argument to the python file, so sys.argv is the array you passed in the spawn. Python will take it in but without seeing your python code, I couldn't say how the extra arguments are being handled. Obviously an error is being thrown.

Remember the child_process.spawn() happens asynchronously, so the stdout and stderr can be extracted right after. Read child_process.spawn() docs. So remove the redirect in the spawns argument array and just run another child_process or use fn to write the output to a file when the stdout occurs:

const { spawn } = require('node:child_process');
const childPython = spawn('python3', ['modifiedCSV.csv', '08182022']);

childPython.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

Or you could use exec instead.

const { exec } = require('node:child_process');

const path = "modifiedCSV.csv";
const date = "08182022";

exec(`python3 ${path} ${date} > test.txt`, (error, stdout, stderr) => {
  if (error) {
    console.error(`exec error: ${error}`);
    return;
  }
  console.log(`stdout: ${stdout}`);
  console.error(`stderr: ${stderr}`);
})

The main issue is, if the path to the modified csv file is in anyway connected to a user's input, you have a security risk, because if python3 is run as sudo, which I believe in node, it is, a user can inject python code in-place of the file path and re-write and encrypt all the files that it can touch.