Python wrote a very simple program:
The conf=Sparkconf (.) setMaster (" spark://123.43.54.12:7077 "). The setAppname (" test ")
Sc=SparkContext (conf=conf)
Def ct (p) :
Temp=0
For I in range (10000) :
Tempi=math. Sin (I) + math.h cos (I)
Temp=temp + tempi
Return temp
P_list=list (range (100000))
P_rdd=sc. Parallelize (p_list, 4)
gt_value=https://bbs.csdn.net/topics/p_rdd.map (lambda p: ct (p)). Coalesce (144)
Sc. Stop ()
Every execution, found that only run on a executor, perform at a time, and the worker may change the address, can't use all of the actuator,
Submit the grammar:
Submit - master spark://123.43.54.12:7077 - deploy - mode client - num - executors 2 - executor - cores 1
Is very strange, the operating system's logical cpu3,,