Spark novice, how to receive more than one stream in the driver, for example, the spark in the example JavaNetworkWordCount example

 
SparkConf SparkConf=new SparkConf (.) setAppName (" JavaNetworkWordCount "); 
JavaStreamingContext SSC=new JavaStreamingContext (sparkConf, new Duration (10000)); 
JavaReceiverInputDStream Lines=SSC. SocketTextStream (
The host, IP, StorageLevels. MEMORY_ONLY); 

The only received a data flow, check it on the website have can support multiple source to receive, but did not give an example, this thought can be realized according to the following

 
SparkConf SparkConf=new SparkConf (.) setAppName (" JavaNetworkWordCount "); 
JavaStreamingContext SSC=new JavaStreamingContext (sparkConf, new Duration (10000)); 
JavaReceiverInputDStream Lines=SSC. SocketTextStream (
Host1, ip1, StorageLevels. MEMORY_ONLY); 
JavaReceiverInputDStream Lines2=SSC. SocketTextStream (
Host2, ip2, StorageLevels. MEMORY_ONLY); 

JavaDStream Words=lines. The union (lines1);//use the union to merge 

But can't find this way, the union after the transform operations are no longer perform,
Also found the spark module on the CSDN problem is very little, there is no other BBS can communicate, stackoverflow need over the wall to ask, dt!

CodePudding user response:

Found the transform operation only deals with one of the flow, untreated, another flow action failed