Everyone a great god, and I now, please write a local test connection server spark example,
SparkSession spark=SparkSession. Builder (). The config (conf.) getOrCreate (); Perform this sentence Java program and server error error above


If the Java code setmaster address instead of local [*] is no problem, please teach various Pal

CodePudding user response:

No one? Why didn't anyone?

CodePudding user response:

The spark is distributed computing framework
Distributed require a cluster
Local [*] is based on simulated CPU auditing into cluster
If the configuration of specific cluster
You need to put your spark program exported executable jar package
Then put the jar package uploaded to cluster in
Within the cluster through/spark/bin/spark - submit shell command commit
Makes little sense to spark program debug debugging
Because everything is on the driver program debug
A central part of the RDD conversion program are distributed, unable to debug
Because you don't know these programs (job) specific be scheduled to which the worker node
After the spark started the default provides a WEB - UI for you to view the specific job log
Specific UI path can check it on the spark startup log

The hope can help you

CodePudding user response:

First to make sure the following question whether made:
1. SSH secret landing free
2. The firewall is shut down
3. Whether the/etc/hosts file with the host name
I also met this problem before, so I solve