Home > database >  Kettle to extract data output HDFS error: Could not close the output stream for the file
Kettle to extract data output HDFS error: Could not close the output stream for the file

Time:10-01

, every brother please take a look at what problems, running environment is in the Linux (Redhat6.5), account permissions are root, with the kettle to extract data output to the hive outer table (HDFS), is TXT format, data volume is about 600000 +, the data can be extracted and output, but the middle can appear the following error, then the task over

2017/12/11 01:25:56 - JSON Input. 0 - Finished processing (I=1, O=0, R=1, W=1, U=0, E=0)
2017/12/11 01:25:56 - Hadoop File Output. 0 - ERROR (version 6.1.0.1-196, build 1 from the 2016-04-07 12.08.49 by buildguy) : Exception trying to close the File: org.apache.com mons. Vfs2. FileSystemException: Could not close the Output stream for the File "HDFS://HDFS: * * * @ 10.38.100.192:8020/NBGJ itf_ins_result_sum/itf_ins_result_sum. TXT".
2017/12/11 01:25:56 - Hadoop File Output. 0 - Finished processing (I=0, O=1, R=1, W=1, U=0, E=1)
2017/12/11 01:25:56 nbgj_result_sum - Transformation detected one or more steps with errors.
2017/12/11 01:25:56 nbgj_result_sum - Transformation is before the other steps!
2017/12/11 01:25:56 nbgj_result_sum - ERROR (version 6.1.0.1-196, build 1 from the 2016-04-07 12.08.49 by buildguy) : Errors detected!
2017/12/11 01:25:56 nbgj_for_result_sum_each - Finished job entry [nbgj_result_sum] (result=(false))
2017/12/11 01:25:56 nbgj_for_result_sum_each - Finished job entry [nbgj_set_each_result_sum_id] (result=(false))
2017/12/11 01:25:56 nbgj_job_result_sum - Finished job entry [nbgj_for_result_sum_each] (result=(false))
2017/12/11 01:25:56 nbgj_job_result_sum - Finished job entry [nbgj_result_sum_id] (result=(false))
2017/12/11 01:25:56 nbgj_result_sum_daily_job - Finished job entry [nbgj_job_result_sum] (result=(false))
2017/12/11 01:25:56 nbgj_result_sum_daily_job - Finished job entry [nbgj_get_result_sum_daily_id] (result=(false))
2017/12/11 01:25:56 nbgj_result_sum_daily_job - Finished job entry [get_parameters] (result=(false))
2017/12/11 01:25:56 nbgj_get_total_daily_job - Finished job entry [nbgj_total_job] (result=(false))
2017/12/11 01:25:56 nbgj_get_total_daily_job - Job execution finished
2017/12/11 01:25:56 - Kitchen - Finished!
2017/12/11 01:25:56 - Kitchen - ERROR (version 6.1.0.1-196, build 1 from the 2016-04-07 12.08.49 by buildguy) : Finished with errors
2017/12/11 01:25:56 - Kitchen - Start=2017/12/11 01:00:03. 987, the Stop=2017/12/11 01:25:56. 506
2017/12/11 01:25:56 - Kitchen - Processing ended after 25 minutes and 52 seconds 1552 seconds (total).
  • Related