My recent research about the spark of graph model, the spark recently contact, scala language also is not very well, in the BBS have expert help answer about the shortest path, the official document I also saw, but seemingly not on the shortest path, the shortestpaths source, what also didn't understand, but also the implementation measures just shows the source to the target point distance, I want to use the spark to make the shortest path, it is best to have code case, the results with intermediate nodes, I do not know what a great god had done the research, help

CodePudding user response:

I don't know the building Lord solved, I this give you an example:

 import org. Apache. Spark. {SparkConf, SparkContext} 
The import org. Apache. Spark. The graphx. {EdgeDirection VertexId, Graph} 
The import org. Apache. Spark. The graphx. Util. GraphGenerators 


The object Pregel_SSSP {
Def main (args: Array [String]) {
Val conf=new SparkConf (). SetAppName (" Pregel_SSSP ") 
Val sc=new SparkContext (conf) 
//A graph with edge attributes containing distances 
Val graph: a graph [Long, Double]=
GraphGenerators. LogNormalGraph (sc, numVertices=5). MapEdges (e=& gt; E.a TTR event. ToDouble) 
Graph. Edges. Foreach (println) 
Val sourceId: VertexId=0//The ultimate source 

//Initialize the graph to that all are the except the root have short infinity. 
Val initialGraph: Graph [(Double, List [VertexId]), Double]=Graph. MapVertices ((id, _)=& gt; If (id==sourceId) (0.0, List [VertexId] (sourceId)) else (Double PositiveInfinity, List [VertexId] ())) 

Val SSSP=initialGraph. Pregel ((Double) PositiveInfinity, List [VertexId] ()), Int. J MaxValue, EdgeDirection. Out) (

//Vertex Program 
(id, dist, newDist)=& gt; If (dist. _1 & lt; NewDist. _1) dist else newDist, 

//Send Message 
Triplet=& gt; {
If (triplet. SrcAttr. _1 & lt; Triplet. DstAttr. _1 - triplet. Attr) {
The Iterator ((triplet. DstId (triplet. SrcAttr. _1 + triplet. Attr, triplet. SrcAttr. _2: + triplet. DstId))) 
} else {
The Iterator. Empty 
} 
}, 
//Merge Message 
(a, b)=& gt; If (a. _1 & lt; B. _1) else b) 
Println (SSSP. Are. Collect. MkString (" \ n ")) 
} 
} 

CodePudding user response:

Based on the spark 1.3

CodePudding user response:

Hello, this is monophyletic shortest paths, parallel multi-source path, how can I try to write method calls, he then needs to point to production RDD, call monophyletic method to the map, but cannot nested RDD RDD, on the other hand, I can feel he must be parallel, because for every nodes monophyletic path and other irrelevant, just don't know how to parallel, urgent!!!

CodePudding user response:

reference sinat_35206416 reply: 3/f

hello, this is monophyletic shortest paths, parallel multi-source path, how can I try to write method calls, he then needs to point to production RDD, call monophyletic method to the map, but RDD cannot be nested RDD, on the other hand, I can feel he must be parallel, because for every nodes monophyletic path and other irrelevant, just don't know how to parallel, urgent!!!

Pay attention to the implementation approach, placing the source is the shortest distance to record the current iteration process, is now a multi-source, you need to record multiple paths relative to the shortest distance, respectively, in the attr requires a mapping relationship, and is not a simple parallel implementation for each point of monophyletic algorithms and data structures,