Quote: refer to the second floor WmxL56 response:

Quote: refer to 1st floor LinkSe7en response:

From the perspective of a few to answer:
1, what is your current school, we haven't seen the real "big data," imagine yourself, taobao operated for more than ten years, all users order how many? 0.1 trillion level? That can carry a MYSQL database? Carry not to live, so they will break up history and hot spot data, and will be hot again into divided according to certain rules, spread to a number of database, such as not check your order in a day or two, but a few milliseconds have result,
If Mr Ma want to know how many single under taobao, how each region, and so on various dimensions of time, what do you think of the SQL can also run on mysql?
2, based on the above, the business data break-up in the history of business-oriented, and analysis and need them together, so, the business concept came out of the database and data warehouse (OLTP) database for business affairs, and data warehouse oriented analysis (OLAP), generally, is a data warehouse will spread out on business data, and through certain technical means to list, so, taobao's data development idea to meet Mr. Ma, you can write such a SQL database, to analyse the data of the scattered here and there,
3, OLTP can do most of the task of OLAP, such as you find flink can implement SQL, mysql can write, but their specialization is to deal with affairs, and OLAP, in view of the analysis, there are a lot of OLTP no function, such as the HIVE against window analysis of SQL dialect, so your doubts, is found at the intersection between them, they still have each other do not have completely,

thank you, it's good to write

Can ask you a question, take now flink average batch is what kind of statistical analysis, can you give some examples? Now I, for one, I am now in school, just want to try the learn to look, don't know do what, now I baidu as if no one to discuss this we all know (should be), I only know that now can spark can adjust some machine learning repository to calculate that those algorithms, flink, too? If I want to now I do some big events purpose of data analysis, can analyze point what, mainly and car networking, user when driving behavior data

Flink best practices in the industry, that is, from kafka consumption log data in the message queue, such as simple understanding is only added in mysql not to delete data, most of them are based on some logic statistics, also have trained machine learning model, through the real-time forecast analysis flink,
Spark for real-time processing, basic it is based on the window, that is to define a good window after a certain period of time and sliding distance, statistical analysis and model prediction for the window, also known as the small batch, spark began to support continuous flow, of course, that is a a message processing,
Basic is offline hive presto spark impala veteran of the world, such as flink also interested in side direction, let's wait and see,

You want for some of the data analysis, you must first dimensional model is modeled according to the fact, it is recommended that you first take a look at several CangJianMo related reference books, such as "toolbox" data warehouse, dimensional model to construct the facts, according to a variety of dimensions are combined, will be able to find something new,