Home > other >  Written in python udf's problems
Written in python udf's problems

Time:10-19

Great god answers to a question of written in python, a PMT formula reimbursement amount of udf:
Found that when using the can only be run when the string field can be converted to success : select the transform (cast (capital as string), cast (interest as string), cast (period_num as string)) USING 'python PMT. Py' as (capital, interest, period_num, repay_amount) from dw_tmp. Calculate_repay_amount;
If you don't convert capital, interest is a floating point number, period_num is integer, is always an error, is this why?
An error content is as follows:
Error: org. Apache. Spark. SparkException: Job aborted due to stage a failure: Task 1 in stage 2.0 failed 4 times, the most recent failure: Lost Task in stage 1.3 2.0 (10, dar data14, executor (3) : ExecutorLostFailure (executor 3 exited under caused by one of the running tasks) Reason: the Container marked as failed: container_1555125275242_853949_01_000004 on host: data14. Exit status: 50. Diagnostics: Exception from the Container - launch.
The Container id: container_1555125275242_853949_01_000004
The Exit code: 50
The Stack trace: ExitCodeException exitCode=50:
At org, apache hadoop. Util. Shell. RunCommand (545) Shell. Java:
At org, apache hadoop. Util. Shell. The run (456) Shell. Java:
At org, apache hadoop. Util. Shell $ShellCommandExecutor. Execute (722) Shell. Java:
At org, apache hadoop. Yarn. Server. The nodemanager. DefaultContainerExecutor. LaunchContainer (DefaultContainerExecutor. Java: 212)
At org, apache hadoop. Yarn. Server. The nodemanager. Containermanager. The launcher. ContainerLaunch. Call (302) ContainerLaunch. Java:
At org, apache hadoop. Yarn. Server. The nodemanager. Containermanager. The launcher. ContainerLaunch. Call (82) ContainerLaunch. Java:
The at Java. Util. Concurrent. FutureTask. Run (FutureTask. Java: 266)
The at Java. Util. Concurrent. ThreadPoolExecutor. RunWorker (ThreadPoolExecutor. Java: 1142)
The at Java. Util. Concurrent. ThreadPoolExecutor $Worker. The run (ThreadPoolExecutor. Java: 617)
The at Java. Lang. Thread. The run (Thread. Java: 745)


The Container exited with a non - zero exit code 50

Driver stacktrace:
SQLState: null
ErrorCode: 0
  • Related