Home > other >  Spark a good environment, spark - shell can be used, the spark - submit submit example program error
Spark a good environment, spark - shell can be used, the spark - submit submit example program error

Time:09-18

Cluster
Three machines, one is 2 g memory is done advocate node, the other two are made from node 1 gb of memory,
Want to ask next great god is my memory is too small to

Thanks..
An error is as follows:

[hadoop @ master spark - 1.3.0 - bin - hadoop2.4] $./bin/spark - submit - class SimpleApp has - master spark://172.21.7.182: ~ 7077/spark_wordcount/target/scala - 2.10/simple - project_2. 10-1.0. Jar
Spark an assembly has had been built with Hive, o Datanucleus jars on the classpath
Using the Spark 's default log4j profile: org/apache/Spark/log4j - defaults. The properties
15/04/22 14:30:30 WARN TaskSchedulerImpl: Initial job has not accepted any resources; Check your cluster UI to ensure that workers are registered and have sufficient resources
15/04/22 14:30:32 INFO SparkDeploySchedulerBackend: Registered executor: Actor [akka. TCP://sparkExecutor @ bananapi: 38979/user/Executor# 266790798] with ID 1
15/04/22 14:30:32 INFO TaskSetManager: Starting task in stage 0.0 0.0 (dar 0, bananapi PROCESS_LOCAL, 1389 bytes)
15/04/22 14:30:32 INFO TaskSetManager: Starting task in stage 1.0 0.0 (dar 1, bananapi PROCESS_LOCAL, 1389 bytes)
15/04/22 14:30:32 INFO SparkDeploySchedulerBackend: Registered executor: Actor [akka. TCP://sparkExecutor @ bananapi: 43806/user/Executor# - 850130035] with ID 0
15/04/22 14:30:33 INFO BlockManagerMasterActor: Registering block manager bananapi: 60321 with 267.3 MB of RAM, BlockManagerId (60321) 1, bananapi,
15/04/22 14:30:33 INFO BlockManagerMasterActor: Registering block manager bananapi: 51018 with 267.3 MB of RAM, BlockManagerId (0, bananapi, 51018)
15/04/22 14:30:34 WARN TaskSetManager: Lost task in stage 1.0 0.0 (dar 1, bananapi) : Java. IO. IOException: Java. Lang. Reflect. InvocationTargetException
The at org. Apache. Spark. Util. Utils $. TryOrIOException (Utils. Scala: 1155)
The at org. Apache. Spark. Broadcast. TorrentBroadcast. ReadBroadcastBlock (TorrentBroadcast. Scala: 164)
The at org. Apache. Spark. Broadcast. TorrentBroadcast. _value $lzycompute (TorrentBroadcast. Scala: 64)
The at org. Apache. Spark. Broadcast. TorrentBroadcast. _value (TorrentBroadcast. Scala: 64)
The at org. Apache. Spark. Broadcast. TorrentBroadcast. GetValue (TorrentBroadcast. Scala: 87)
At org. Apache. Spark. Broadcast. Broadcast. The value (70) broadcast. Scala:
The at org. Apache. Spark. The scheduler. ResultTask. RunTask (58) ResultTask. Scala:
At org. Apache. Spark. The scheduler. Task. Run (64) Task. Scala:
The at org. Apache. Spark. Executor. $TaskRunner executor. Run (executor. Scala: 203)
The at Java. Util. Concurrent. ThreadPoolExecutor. RunWorker (ThreadPoolExecutor. Java: 1145)
The at Java. Util. Concurrent. ThreadPoolExecutor $Worker. The run (ThreadPoolExecutor. Java: 615)
The at Java. Lang. Thread. The run (Thread. Java: 745)
Under Caused by: Java. Lang. Reflect. InvocationTargetException
At sun. Reflect. NativeConstructorAccessorImpl. NewInstance0 (Native Method)
At sun. Reflect. NativeConstructorAccessorImpl. NewInstance (NativeConstructorAccessorImpl. Java: 57)
At sun. Reflect. DelegatingConstructorAccessorImpl. NewInstance (DelegatingConstructorAccessorImpl. Java: 45)
The at Java. Lang. Reflect. Constructor. NewInstance (526) Constructor. Java:
At org.apache.spark.io.Com pressionCodec $. CreateCodec (CompressionCodec. Scala: 68)
At org.apache.spark.io.Com pressionCodec $. CreateCodec (CompressionCodec. Scala: 60)
The at $$$$TorrentBroadcast broadcast spark apache org.apache.spark.broadcast.TorrentBroadcast.org $$setConf (TorrentBroadcast. Scala: 73)
The at org. Apache. Spark. Broadcast. TorrentBroadcast $$$readBroadcastBlock anonfun $1. Apply (TorrentBroadcast. Scala: 166)
The at org. Apache. Spark. Util. Utils $. TryOrIOException (Utils. Scala: 1152)
. 11 more
Under Caused by: Java. Lang. IllegalArgumentException
The at org. Apache. Spark. IO. SnappyCompressionCodec. & lt; init> (CompressionCodec. Scala: 152)
. 20 more

15/04/22 14:30:34 INFO TaskSetManager: Lost task in stage 0.0 0.0 (dar) on executor bananapi: Java. IO. IOException (Java. Lang. Reflect. InvocationTargetException) [1] duplicate
15/04/22 14:30:34 INFO TaskSetManager: Starting task in stage 0.1 0.0 (TID 2, bananapi, PROCESS_LOCAL, 1389 bytes)
15/04/22 14:30:34 INFO TaskSetManager: Starting task in stage 1.1 0.0 (dar 3, bananapi, PROCESS_LOCAL, 1389 bytes)
15/04/22 14:30:34 INFO TaskSetManager: Lost task in stage 0.1 0.0 (dar) 2 on executor bananapi: Java. IO. IOException (Java. Lang. Reflect. InvocationTargetException) [2] duplicate
15/04/22 14:30:34 INFO TaskSetManager: Starting task in stage 0.2 0.0 (dar 4, bananapi, PROCESS_LOCAL, 1389 bytes)
15/04/22 14:30:35 INFO TaskSetManager: Lost task in stage 1.1 0.0 (dar) on executor bananapi: Java. IO. IOException (Java. Lang. Reflect. InvocationTargetException) [3] duplicate
15/04/22 14:30:35 INFO TaskSetManager: Starting task in stage 1.2 0.0 (dar 5, bananapi PROCESS_LOCAL, 1389 bytes)
15/04/22 14:30:35 INFO TaskSetManager: Lost task in stage 0.2 0.0 (dar) on executor bananapi: Java. IO. IOException (Java. Lang. Reflect. InvocationTargetException) [4] duplicate
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull