The environment is java1.8 hadoop2.7.3 spark1.7.3
Attach a yarn log
ubuntu @ Master:/usr/soft/hadoop - 2.7.3/logs $spark - submit - class org. Apache. Spark. Examples. SparkPi - Master yarn - deploy - mode client - driver - the memory 1 g - executor - memory 1 g - executor - cores 1/usr/soft/spark - 1.6.3/examples/target/spark - examples_2. 10-1.6.3. Jar 10
17/02/27 16:11:37 INFO util. Utils: Successfully started service 'sparkDriverActorSystem on port 46860.
17/02/27 16:11:37 INFO spark. SparkEnv: Registering MapOutputTracker
17/02/27 16:11:37 INFO spark. SparkEnv: Registering BlockManagerMaster
17/02/27 16:11:37 INFO storage. DiskBlockManager: Created local directory at/TMP/blockmgr b35587e 4-6 ebb4-4468 - a7ba - e8b734982a99
17/02/27 16:11:37 INFO storage. MemoryStore: MemoryStore started with capacity of 511.1 MB
17/02/27 16:11:37 INFO spark. SparkEnv: Registering OutputCommitCoordinator
17/02/27 16:11:37 INFO for server AbstractConnector: Started [email protected]:4040
17/02/27 16:11:37 INFO util. Utils: Successfully started service 'SparkUI on port 4040.
17/02/27 16:11:37 INFO UI. SparkUI: Started SparkUI at http://172.16.239.1:4040
17/02/27 16:11:37 INFO spark. HttpFileServer: HTTP File server directory is/TMP/spark - cbd08d7c - d16 fc96-4-9 f06c896 e2edb3728/HTTPD CBB - 257-4-7401-420 - a - 8 eb1-6 e0be82e7dd6
17/02/27 16:11:38 INFO spark. HttpServer: Starting the HTTP Server
17/02/27 16:11:38 INFO server. Server: jetty - 8 y.z - the SNAPSHOT
17/02/27 16:11:38 INFO for server AbstractConnector: Started [email protected]:45684
17/02/27 16:11:38 INFO util. Utils: Successfully started service 'HTTP file server on port 45684.
17/02/27 16:11:38 INFO spark. SparkContext: Added the JAR file:/usr/soft/spark - 1.6.3/examples/target/spark - examples_2. 10-1.6.3. JAR at http://172.16.239.1:45684/jars/spark-examples_2.10-1.6.3.jar with the timestamp 1488183098062
17/02/27 16:11:38 INFO client. RMProxy: Connecting to the ResourceManager at Master/172.16.239.1:8032
17/02/27 16:11:38 INFO yarn. The Client: Requesting a new application from cluster with 3 NodeManagers
17/02/27 16:11:38 INFO yarn. The Client: Verifying our application from the requested wining the maximum memory capability of the cluster (8192 MB per container)
17/02/27 16:11:38 INFO yarn. The Client: Will the allocate AM container, with 896 MB o 384 MB memory overhead
17/02/27 16:11:38 INFO yarn. The Client: Setting up the container launch the context for our AM
17/02/27 16:11:38 INFO yarn. The Client: Setting up the launch environment for our AM container
17/02/27 16:11:38 INFO yarn. The Client: Preparing the resources for our AM container
17/02/27 16:11:40 INFO yarn. The Client: Uploading the resource file:/usr/soft/spark - 1.6.3/assembly/target/scala - 2.10/spark - assembly - 1.6.3 - hadoop2.7.3. Jar - & gt; HDFS://Master: 9000/user/ubuntu/. SparkStaging/application_1488181713860_0003/spark - assembly - 1.6.3 - hadoop2.7.3. Jar
17/02/27 16:11:47 INFO yarn. The Client: Uploading the resource file:/TMP/spark - cbd08d7c - d16 fc96-4-9 CBB - 257 e2edb3728/__spark_conf__7095116935778215646. Zip - & gt; HDFS://Master: 9000/user/ubuntu/. SparkStaging/application_1488181713860_0003/__spark_conf__7095116935778215646.zip
17/02/27 16:11:47 INFO spark. SecurityManager: Changing the view acls to: ubuntu
17/02/27 16:11:47 INFO spark. SecurityManager: Changing the modify acls to: ubuntu
17/02/27 16:11:47 INFO spark. SecurityManager: SecurityManager: authentication disabled; The UI acls disabled; The users with the view permissions: Set (ubuntu); The users with the modify permissions: Set (ubuntu)
17/02/27 16:11:47 INFO yarn. The Client: date application 3 to the ResourceManager
17/02/27 16:11:47 INFO impl. YarnClientImpl: Submitted application application_1488181713860_0003
17/02/27 16:11:48 INFO yarn. The Client: Application report for application_1488181713860_0003 (state: ACCEPTED)
17/02/27 16:11:48 INFO yarn. The Client:
The client token: N/A
Diagnostics: N/A
ApplicationMaster host: N/A
1
ApplicationMaster RPC port: -Queue: default
Start time: 1488183107661
Final status: UNDEFINED
Tracking URL: http://Master:8088/proxy/application_1488181713860_0003/
User: ubuntu
17/02/27 16:12:05 INFO yarn. The Client: Application report for application_1488181713860_0003 (state: ACCEPTED)
17/02/27 16:12:06 INFO yarn. The Client: Application report for application_1488181713860_0003 (state: ACCEPTED)
17/02/27 16:12:07 INFO yarn. The Client: Application report for application_1488181713860_0003 (state: FAILED)
17/02/27 16:12:07 INFO yarn. The Client:
The client token: N/A
Diagnostics: Application application_1488181713860_0003 failed 2 times due to AM Container for appattempt_1488181713860_0003_000002 exited with exitCode: - 103
The Container killed on request. The Exit code is 143
The Container exited with a non - zero exit code 143
Failing this attempt. Failing to the application. The
ApplicationMaster host: N/A
1
ApplicationMaster RPC port: -nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull