Home > other >  Spark RDD call saveToES error, how to solve?
Spark RDD call saveToES error, how to solve?

Time:09-29

15/05/15 11:45:21 INFO TaskSetManager: Starting task in stage 0.0 1.0 (dar 1, localhost, ANY, 1327 bytes)
15/05/15 11:45:21 INFO Executor: Running task in stage 0.0 1.0 (dar) 1
15/05/15 11:45:21 INFO HadoopRDD: Input the split: HDFS://192.168.105.226:9000/LesterDemoTest/Data/draft/patent. The db: 0 + 48437
15/05/15 11:45:22 ERROR Executor: Exception in task 0.0 in stage 1.0 1 (dar)
Org. Elasticsearch. Hadoop. Rest. EsHadoopTransportException: Invalid target URI fe80:0:0:0:20 c: 29 ff: feac: b7e2%2:9 200
The at org.elasticsearch.hadoop.rest.commonshttp.Com monsHttpTransport. & lt; init> (CommonsHttpTransport. Java: 173)
At org.elasticsearch.hadoop.rest.Net workClient. SelectNextNode (NetworkClient. Java: 86)
At org.elasticsearch.hadoop.rest.Net workClient. & lt; init> (NetworkClient. Java: 69)
The at org. Elasticsearch. Hadoop. Rest. RestClient. & lt; init> (RestClient. Java: 80)
The at org. Elasticsearch. Hadoop. Rest. InitializationUtils. DiscoverEsVersion (InitializationUtils. Java: 81)
The at org. Elasticsearch. Hadoop. Rest. RestService. CreateWriter (RestService. Java: 348)
The at org. Elasticsearch. Spark. RDD. EsRDDWriter. Write (31) EsRDDWriter. Scala:
The at org. Elasticsearch. Spark. RDD. EsSpark $$$saveToEs anonfun $1. Apply (EsSpark. Scala: 34)
The at org. Elasticsearch. Spark. RDD. EsSpark $$$saveToEs anonfun $1. Apply (EsSpark. Scala: 34)
The at org. Apache. Spark. The scheduler. ResultTask. RunTask (ResultTask. Scala: 61)
At org. Apache. Spark. The scheduler. Task. Run (64) Task. Scala:
The at org. Apache. Spark. Executor. $TaskRunner executor. Run (executor. Scala: 203)
The at Java. Util. Concurrent. ThreadPoolExecutor. RunWorker (ThreadPoolExecutor. Java: 1145)
The at Java. Util. Concurrent. ThreadPoolExecutor $Worker. The run (ThreadPoolExecutor. Java: 615)
The at Java. Lang. Thread. The run (Thread. Java: 722)
Under Caused by: org.apache.com mons. Httpclient. URIException: invalid port number
At org.apache.com mons. Httpclient. URI. ParseAuthority (URI) Java: 2248)
At org.apache.com mons. Httpclient. URI. ParseUriReference (URI) Java: 1978)
At org.apache.com mons. Httpclient. URI. & lt; Init> (URI. Java: 167)
The at org.elasticsearch.hadoop.rest.commonshttp.Com monsHttpTransport. & lt; Init> (CommonsHttpTransport. Java: 171)
... 14 more
15/05/15 11:45:22 WARN TaskSetManager: Lost task 0.0 in stages (dar 1, localhost) : 1.0 org. Elasticsearch. Hadoop. Rest. EsHadoopTransportException: Invalid target URI fe80:0:0:0:20 c: 29 ff: feac: b7e2%2:9 200
The at org.elasticsearch.hadoop.rest.commonshttp.Com monsHttpTransport. & lt; Init> (CommonsHttpTransport. Java: 173)
At org.elasticsearch.hadoop.rest.Net workClient. SelectNextNode (NetworkClient. Java: 86)
At org.elasticsearch.hadoop.rest.Net workClient. & lt; Init> (NetworkClient. Java: 69)
The at org. Elasticsearch. Hadoop. Rest. RestClient. & lt; Init> (RestClient. Java: 80)
The at org. Elasticsearch. Hadoop. Rest. InitializationUtils. DiscoverEsVersion (InitializationUtils. Java: 81)
The at org. Elasticsearch. Hadoop. Rest. RestService. CreateWriter (RestService. Java: 348)
The at org. Elasticsearch. Spark. RDD. EsRDDWriter. Write (31) EsRDDWriter. Scala:
The at org. Elasticsearch. Spark. RDD. EsSpark $$$saveToEs anonfun $1. Apply (EsSpark. Scala: 34)
The at org. Elasticsearch. Spark. RDD. EsSpark $$$saveToEs anonfun $1. Apply (EsSpark. Scala: 34)
The at org. Apache. Spark. The scheduler. ResultTask. RunTask (ResultTask. Scala: 61)
At org. Apache. Spark. The scheduler. Task. Run (64) Task. Scala:
The at org. Apache. Spark. Executor. $TaskRunner executor. Run (executor. Scala: 203)
The at Java. Util. Concurrent. ThreadPoolExecutor. RunWorker (ThreadPoolExecutor. Java: 1145)
The at Java. Util. Concurrent. ThreadPoolExecutor $Worker. The run (ThreadPoolExecutor. Java: 615)
The at Java. Lang. Thread. The run (Thread. Java: 722)
Under Caused by: org.apache.com mons. Httpclient. URIException: invalid port number
At org.apache.com mons. Httpclient. URI. ParseAuthority (URI) Java: 2248)
At org.apache.com mons. Httpclient. URI. ParseUriReference (URI) Java: 1978)
At org.apache.com mons. Httpclient. URI. & lt; Init> (URI. Java: 167)
The at org.elasticsearch.hadoop.rest.commonshttp.Com monsHttpTransport. & lt; Init> (CommonsHttpTransport. Java: 171)
. 14 more

15/05/15 11:45:22 ERROR TaskSetManager: Task 0 in stage 1.0 1 times failed; Aborting job
15/05/15 11:45:22 INFO TaskSchedulerImpl: Removed the TaskSet 1.0, whose tasks have all completed, the from the pool
15/05/15 11:45:22 INFO TaskSchedulerImpl: Cancelling stage 1
15/05/15 11:45:22 INFO DAGScheduler: Job 1 failed: runJob at EsSpark. Scala: 34, took 0.653925 s
The Exception in the thread "main" org. Apache. Spark. SparkException: Job aborted due to stage a failure: Task 0 1.0 failed in stage 1 times, most recent failure: Lost Task in stage 1.0 0.0 (dar 1, localhost) : org. Elasticsearch. Hadoop. Rest. EsHadoopTransportException: Invalid target URI fe80:0:0:0:20 c: 29 ff: feac: b7e2%2:9 200
The at org.elasticsearch.hadoop.rest.commonshttp.Com monsHttpTransport. & lt; init> (CommonsHttpTransport. Java: 173)
At org.elasticsearch.hadoop.rest.Net workClient. SelectNextNode (NetworkClient. Java: 86)
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull
  • Related