Such as title, the company want to develop such a small program, but don't know where to start now, spark and provided no interface can be called directly in my web application, it means that every time I query, the SQL statement together with the corresponding scala code via SSH to submit to the cluster host running on spark - shell command, but there are a few places don't understand

1. The results of the so how can I take back shows on the web interface? Still have other way to achieve?
2. We have a function point is can use machine learning so MLLib, every process point need to submit the query time, take the part back to the user, and the results as the input of a process node, then how my web applications and cluster communication

CodePudding user response:

Your problem is how to solve, we also want to do so now, how to can give younger brother a guide

CodePudding user response:

Consult the apache open source project: apache lens

CodePudding user response:

How I got to share

CodePudding user response:

Write the request on the web to kafka, Spark read kafka after data processing will be written to the cache database,

CodePudding user response:

http://blog.csdn.net/linchunhua/article/details/52228730

CodePudding user response:

Words don't need it, spark not provide Java API, directly in the web background introduction of spark package and then call the API can submit things if something more local not put, on HDFS, also can call to get, not necessarily into package, I'm not talking about local mode, is also considered a local node, although no, but local stress should not small,

CodePudding user response:

The original poster is how to solve? Company has also had the same problem, all say online can open Spark thrift - server port, but the CDH version of Spark didn't start the thrift - the file server

CodePudding user response:

My side to make the building Lord, you can do, try akka - HTTP