I want to use the Spark to build a real-time data processing platform, front end user request, according to the request of the different content, background, Spark corresponding calculation, then the calculation result is sent to the front to display,
I have written a version is written by Spark + + scala Play framework, but only with set into standalone mode (local), Play the advantages of no Spark distributed,
I currently configured cluster, a spark is four machine, theoretically can accelerate a lot, but I checked on the net a lot of play and spark cluster integration solution, seemingly speak not too clear,
So I want to know the now is how to build the industry based on the application of the Spark, development framework to Spark send the request? (I haven't work,,,)
Git on currently have several open source, Spark Server and Spark Job Server, these two? Suitable for my development? thank you

CodePudding user response:

Probably understand the Play framework, the Play is a Full - Stack of Web application development framework, use it to quickly write your own Web application, also can use it to write a RESTful API, compared with the bucket is now very popular Spring family, Play slightly small,
I side with SpringBoot, write RESTful API, front page to invoke the API,
1) front end call API, back-end persistence of calculation of Spark request
(2) the Spark receives the request, and complete the request, the small data set, the results generally directly through the toJSON () collect () in the form of JSON string is returned to the front end
(3) the front to receive as a result, parsing JSON, show,