Home > database >  Big data operation
Big data operation

Time:09-25

For huge amounts of data, the processing of hundreds, and the calculation method of special but how to deal with parallel computing, how to distribute data, how to handle errors? 这些问题使得原本简单的运算变得难以处理,为了解决上诉的复杂问题,设计了一个新的抽象模型,使用这个模型只需要进行简单的运算,可以不用关心复杂细节,容易实现大规模并行化计算,由于我的专业是国际经济与贸易,这一点和专业结合思考的话,我能想到,在国际贸易中,买卖双方需要通过交易磋商,用合同来明确约定货物交付的地点,运输中的风险划分,运输手续和货运保险手续由谁办理,取得各种进出口所需要的文件和进出口手续由谁办理,以及在办理这些手续中所要支出的费用,如运费,保险费,进出口环节关税和其他费用由谁支付,等等,如果贸易双方就这些事项进行逐一商讨,不仅会耗费大量时间和精力,还可能会因考虑不周而使合同无法履行,甚至产生争议和纠纷,因此,在长期的国际贸易实践中,人们用一些简短的概念和外文字母缩写来明确买卖双方责任,费用和风险划分界限,约定买卖双方交接货物中各自应尽的义务,尽管我们在国际贸易中有很多方法和手段进行磋商,但由于上文提到的,注意磋商每一细节得不尝试,于是人们改用贸易术语,不同的贸易术语有着不同的含义,贸易术语就像是结果,在实践中确定贸易术语的过程就像是计算,容错,数据分布,负载均衡等复杂细节,简化复杂细节,不仅在Mapreduce中有应用,在国际贸易以及生活在很多方面都有应用,而简化复杂细节,也是我们在工作学习中需要用到的技巧,
Graphs model can have a variety of different ways of implementation, how to correctly choose depends on the specific environment, for example, an implementation approach is suitable for the small way of Shared memory machine, another kind of way is suitable for large NUMA architecture multi-processor host, and some implementation approach is more suitable for large network cluster, in international trade, both sides have reached a cooperation, in order to in international business negotiation, negotiation skills and subject to change depending on the specific environment, the number of groups in the course of the negotiation, division of labor is also changed, according to the negotiation details according to the specific environment to choose different graphs of different implementation methods, according to the different environment to determine the different negotiating team and negotiation skills, in the process of work and study, we also is to arrange my plan according to the priorities of different tasks,
Through the input data to Map calls will be automatically divided into M pieces of data collection, the Map calls are distributed to multiple machines, input data fragments can be on different machines parallel processing, in international business negotiations, each member of the negotiating groups have different roles, points is given priority to, complementary to talk, and so on, but the negotiation process is more than one member for the common goal together, this with "input data fragments can be on different machines parallel processing" has the same place,
Through to the graphs in Chinese reading, found that many of these ideas can be combined with I learned the professional knowledge of the thought, the process of learning is endless, combined with different disciplines is also wonderful and interesting, thanks to "big data and artificial intelligence" opened my eyes, this course also let yourself from another perspective, the international economic and trade professional knowledge, (finally, want to be, this is a late substance, also don't know the teacher will search again, but very thank the teacher leads us from this course the perspective of their own professional and life,)
  • Related