Home > other >  Call an external application within mapPartitions problem
Call an external application within mapPartitions problem

Time:09-25

I hope in the following data processing within the RDD mapPartition function:
1. Each partition in the executor of RDD worker node local files,
2. Call the worker node command line program mycmd.!!!!! With 1 for the command-line program generated file for input, output to the specified file
This is I call scala. Sys. In the process. _
"My command program".!!!!!
Under the scala, this is no problem of
3. Read the output files to the RDD back to the driver in analytic 2

The above problem in the second step, exitCode=50

Consult a friend have encountered the same problem?
  • Related