Home > other >  Spark on yarn: when starting an Error Error initializing SparkContext, please everyone a great god.
Spark on yarn: when starting an Error Error initializing SparkContext, please everyone a great god.

Time:09-19

[datamining @ FTP spark - 2.2.1 - bin - hadoop2.6] $bin/spark - the shell - master yarn - deploy - mode client
Setting the default log level to "WARN".
To adjust logging level use sc. SetLogLevel (newLevel.) For SparkR, use setLogLevel (newLevel.)
18/02/01 17:10:24 ERROR spark. SparkContext: ERROR initializing SparkContext.
Java. Lang. IllegalArgumentException: java.net.URISyntaxException: Expected scheme - specific part at the index 5: HDFS:
At org, apache hadoop. Fs. Path. The initialize (206) Path. Java:
At org, apache hadoop. Fs. Path. & lt; init> (172) Path. Java:
At org, apache hadoop. Fs. Path. & lt; init> (94) Path. Java:
At org, apache hadoop. Fs. Globber. Glob (Globber. Java: 211)
At org, apache hadoop. Fs. FileSystem. GlobStatus (1625) FileSystem. Java:
The at org. Apache. Spark. Deploy. Yarn. The Client $$$prepareLocalResources anonfun $6. Apply (527). The Client scala:
The at org. Apache. Spark. Deploy. Yarn. The Client $$$prepareLocalResources anonfun $6. Apply (523). The Client scala:
At the scala. Collection. Mutable. ArraySeq. Foreach (ArraySeq. Scala: 74)
The at org. Apache. Spark. Deploy. Yarn. Client. PrepareLocalResources (523). The Client scala:
The at org. Apache. Spark. Deploy. Yarn. Client. CreateContainerLaunchContext (832). The Client scala:
The at org. Apache. Spark. Deploy. Yarn. Client. SubmitApplication (170). The Client scala:
The at org. Apache. Spark. The scheduler. Cluster. YarnClientSchedulerBackend. Start (YarnClientSchedulerBackend. Scala: 56)
The at org. Apache. Spark. The scheduler. TaskSchedulerImpl. Start (TaskSchedulerImpl. Scala: 173)
The at org. Apache. Spark. SparkContext. & lt; init> (SparkContext. Scala: 509)
The at org. Apache. Spark. SparkContext $. GetOrCreate (SparkContext. Scala: 2516)
The at org. Apache. Spark. SQL. SparkSession $Builder $$anonfun $6. Apply (SparkSession. Scala: 918)
The at org. Apache. Spark. SQL. SparkSession $Builder $$anonfun $6. Apply (SparkSession. Scala: 910)
At the scala. Option. GetOrElse (121) Option. The scala:
The at org. Apache. Spark. SQL. SparkSession $Builder. GetOrCreate (SparkSession. Scala: 910)
The at org. Apache. Spark. Repl. Main $. CreateSparkSession (101). The Main scala:
At $line3. $read $$$$iw iw. & lt; init> (& lt; Console> :
15)The at $line3. $read $$iw. & lt; init> (& lt; Console> : 42)
The at $line3. $read. & lt; init> (& lt; Console> : 44)
The at $line3. $read $. & lt; init> (& lt; Console> : 48)
The at $line3. $read $. & lt; Clinit> (& lt; Console>)
The at $line3. $eval $. $print $lzycompute (& lt; Console> : 7)
The at $line3. $eval $. $print (& lt; Console> : 6)
The at $line3. $eval. $print (& lt; Console>)
At sun. Reflect. NativeMethodAccessorImpl. Invoke0 (Native Method)
At sun. Reflect. NativeMethodAccessorImpl. Invoke (NativeMethodAccessorImpl. Java: 62)
At sun. Reflect. DelegatingMethodAccessorImpl. Invoke (43) DelegatingMethodAccessorImpl. Java:
The at Java. Lang. Reflect. Method. Invoke (498) Method. The Java:
At the scala. Tools. NSC. Interpreter. IMain $ReadEvalPrint. Call (786). IMain scala:
At the scala. Tools. NSC. Interpreter. IMain $Request. LoadAndRun IMain. Scala: (1047)
At the scala. Tools. NSC. Interpreter. IMain $$$$loadAndRunReq anonfun WrappedRequest $1. Apply (IMain. Scala: 638)
At the scala. Tools. NSC. Interpreter. IMain $$$$loadAndRunReq anonfun WrappedRequest $1. Apply (IMain. Scala: 637)
At the scala. Reflect. Internal. Util. ScalaClassLoader $class. AsContext (31) ScalaClassLoader. Scala:
At the scala. Reflect. Internal. Util. AbstractFileClassLoader. AsContext (AbstractFileClassLoader. Scala: 19)
At the scala. Tools. NSC. Interpreter. IMain $WrappedRequest. LoadAndRunReq IMain. Scala: (637)
At the scala. Tools. NSC. Interpreter. IMain. Interpret (IMain. Scala: 569)
At the scala. Tools. NSC. Interpreter. IMain. Interpret (IMain. Scala: 565)
At the scala. Tools. NSC. Interpreter. ILoop. InterpretStartingWith (ILoop. Scala: 807)
Mand at scala.tools.nsc.interpreter.ILoop.com (ILoop. Scala: 681)
At the scala. Tools. NSC. Interpreter. ILoop. ProcessLine (ILoop. Scala: 395)
The at org. Apache. Spark. Repl. SparkILoop $$$initializeSpark anonfun $1. Apply $MCV $sp (SparkILoop. Scala: 38)
At org. Apache. Spark. Repl. SparkILoop $$$initializeSpark anonfun $1. Apply (37) SparkILoop. Scala:
At org. Apache. Spark. Repl. SparkILoop $$$initializeSpark anonfun $1. Apply (37) SparkILoop. Scala:
At the scala. Tools. NSC. Interpreter. IMain. BeQuietDuring IMain. Scala: (214)
The at org. Apache. Spark. Repl. SparkILoop. InitializeSpark (37) SparkILoop. Scala:
The at org. Apache. Spark. Repl. SparkILoop. LoadFiles (SparkILoop. Scala: 98)
At the scala. Tools. NSC. Interpreter. ILoop $$anonfun $process $1. Apply $MCZ $sp (ILoop. Scala: 920)
At the scala. Tools. NSC. Interpreter. ILoop $$anonfun $process $1. Apply (ILoop. Scala: 909)
At the scala. Tools. NSC. Interpreter. ILoop $$anonfun $process $1. Apply (ILoop. Scala: 909)
At the scala. Reflect. Internal. Util. ScalaClassLoader $. SavingContextLoader (ScalaClassLoader. Scala: 97)
At the scala. Tools. NSC. Interpreter. ILoop. Process (909) ILoop. Scala:
The at org. Apache. Spark. Repl. Main $. DoMain (74). The Main scala:
The at org. Apache. Spark. Repl. Main $. The Main (Main) scala: 54)
The at org. Apache. Spark. Repl. Main. The Main (Main. Scala)
At sun. Reflect. NativeMethodAccessorImpl. Invoke0 (Native Method)
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull
  • Related