Home > other >  Use cloudera manager - building of spark hadoop cluster what are useful, spark - shell appear this p
Use cloudera manager - building of spark hadoop cluster what are useful, spark - shell appear this p

Time:09-21


[root @ master bin] #./spark - shell
Setting the default log level to "WARN".
To adjust logging level use sc. SetLogLevel (newLevel.)
Welcome to
____ __
/__/__ _____ _____//__
/_ \ \ _ \ _ `/__/__/
'/___/__/\ _, _/_/_ \ _ \ version 1.6.0
/_

Using the Scala version 2.10.5 (Java HotSpot (TM), 64 - Bit VM Server, Java 1.8.0 comes with _144)
Type in expressions and them evaluated.
Type: help for more information.
17/09/04 09:55:59 ERROR spark. SparkContext: ERROR initializing SparkContext.
Java. Lang. IllegalArgumentException: Required executor memory (1024 + 384 MB) is above the Max threshold (1024 MB) of this cluster. Both Please check the values of 'yarn. The scheduler. The maximum allocation - MB' and/or 'yarn. The nodemanager. Resource. The memory - MB'.
The at org. Apache. Spark. Deploy. Yarn. Client. VerifyClusterResources (284). The Client scala:
The at org. Apache. Spark. Deploy. Yarn. Client. SubmitApplication (140). The Client scala:
The at org. Apache. Spark. The scheduler. Cluster. YarnClientSchedulerBackend. Start (YarnClientSchedulerBackend. Scala: 57)
The at org. Apache. Spark. The scheduler. TaskSchedulerImpl. Start (TaskSchedulerImpl. Scala: 144)
The at org. Apache. Spark. SparkContext. & lt; init> (SparkContext. Scala: 541)
The at org. Apache. Spark. Repl. SparkILoop. CreateSparkContext (SparkILoop. Scala: 1022)
The at $line3. $$$read the iwC $$iwC. & lt; init> (& lt; Console> :
15)The at $line3. $read $$iwC. & lt; init> (& lt; Console> : 25)
The at $line3. $read. & lt; init> (& lt; Console> : 27)
The at $line3. $read $. & lt; init> (& lt; Console> : 31)
The at $line3. $read $. & lt; Clinit> (& lt; Console>)
The at $line3. $eval $. & lt; init> (& lt; Console> : 7)
The at $line3. $eval $. & lt; Clinit> (& lt; Console>)
The at $line3. $eval. $print (& lt; Console>)
At sun. Reflect. NativeMethodAccessorImpl. Invoke0 (Native Method)
At sun. Reflect. NativeMethodAccessorImpl. Invoke (NativeMethodAccessorImpl. Java: 62)
At sun. Reflect. DelegatingMethodAccessorImpl. Invoke (43) DelegatingMethodAccessorImpl. Java:
The at Java. Lang. Reflect. Method. Invoke (498) Method. The Java:
The at org. Apache. Spark. Repl. SparkIMain $ReadEvalPrint. Call (1045). SparkIMain scala:
The at org. Apache. Spark. Repl. SparkIMain $Request. LoadAndRun (SparkIMain. Scala: 1326)
The at org. Apache. Spark. Repl. SparkIMain. LoadAndRunReq $1 (821). SparkIMain scala:
The at org. Apache. Spark. Repl. SparkIMain. Interpret (SparkIMain. Scala: 852)
The at org. Apache. Spark. Repl. SparkIMain. Interpret (SparkIMain. Scala: 800)
The at org. Apache. Spark. Repl. SparkILoop. ReallyInterpret $1 (857). SparkILoop scala:
The at org. Apache. Spark. Repl. SparkILoop. InterpretStartingWith (SparkILoop. Scala: 902)
Mand at org.apache.spark.repl.SparkILoop.com (SparkILoop. Scala: 814)
The at org. Apache. Spark. Repl. SparkILoopInit $$$initializeSpark anonfun $1. Apply (SparkILoopInit. Scala: 125)
The at org. Apache. Spark. Repl. SparkILoopInit $$$initializeSpark anonfun $1. Apply (SparkILoopInit. Scala: 124)
The at org. Apache. Spark. Repl. SparkIMain. BeQuietDuring (SparkIMain. Scala: 305)
The at org. Apache. Spark. Repl. SparkILoopInit $class. InitializeSpark (SparkILoopInit. Scala: 124)
The at org. Apache. Spark. Repl. SparkILoop. InitializeSpark (SparkILoop. Scala: 64)
At org. Apache. Spark. Repl. SparkILoop $$$$$$repl spark apache org anonfun $SparkILoop $$process $1 $$$$MCZ apply anonfun $sp $5. The apply $MCV $sp (SparkILoop. Scala: 974)
The at org. Apache. Spark. Repl. SparkILoopInit $class. RunThunks (SparkILoopInit. Scala: 160)
The at org. Apache. Spark. Repl. SparkILoop. RunThunks (SparkILoop. Scala: 64)
The at org. Apache. Spark. Repl. SparkILoopInit $class. PostInitialization (SparkILoopInit. Scala: 108)
The at org. Apache. Spark. Repl. SparkILoop. PostInitialization (SparkILoop. Scala: 64)
The at org. Apache. Spark. Repl. SparkILoop $$$$$$repl spark apache org anonfun $SparkILoop $$process $1. Apply $MCZ $sp (SparkILoop. Scala: 991)
The at org. Apache. Spark. Repl. SparkILoop $$$$$$repl spark apache org anonfun $SparkILoop $$process $1. Apply (SparkILoop. Scala: 945)
The at org. Apache. Spark. Repl. SparkILoop $$$$$$repl spark apache org anonfun $SparkILoop $$process $1. Apply (SparkILoop. Scala: 945)
At the scala. Tools. NSC. Util. ScalaClassLoader $. SavingContextLoader (ScalaClassLoader. Scala: 135)
The at $$$$SparkILoop repl spark apache org.apache.spark.repl.SparkILoop.org $$process (SparkILoop. Scala: 945)
The at org. Apache. Spark. Repl. SparkILoop. Process (SparkILoop. Scala: 1064)
The at org. Apache. Spark. Repl. Main $. The Main (31). The Main scala:
The at org. Apache. Spark. Repl. Main. The Main (Main. Scala)
At sun. Reflect. NativeMethodAccessorImpl. Invoke0 (Native Method)
At sun. Reflect. NativeMethodAccessorImpl. Invoke (NativeMethodAccessorImpl. Java: 62)
At sun. Reflect. DelegatingMethodAccessorImpl. Invoke (43) DelegatingMethodAccessorImpl. Java:
The at Java. Lang. Reflect. Method. Invoke (498) Method. The Java:
The at org.apache.spark.deploy.SparkSubmit$.org $$$$$$runMain SparkSubmit deploy spark apache (SparkSubmit. Scala: 731)
The at org. Apache. Spark. Deploy. SparkSubmit $. DoRunMain $1 (181). SparkSubmit scala:
The at org. Apache. Spark. Deploy. SparkSubmit $. Submit (SparkSubmit. Scala: 206)
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull
  • Related