First of all to be clear, only when the processing capacity of a single node cannot meet the increasing computation, storage tasks, and the ascension of the hardware (add memory, disk, using better CPU) high to do more harm than good, the application cannot be further optimized, we just need to consider a distributed system, because of the distributed system to solve the problem is the same as stand-alone systems, and because the multi-node distributed system, through the topology of the network communication, will introduce a lot of stand-alone systems without problems, in order to solve these problems and will introduce more mechanism, protocol, bring more problems,
Mainly about distributed systems can be divided into distributed computing (computation) and distributed storage (storage), computing and storage is complementary to each other, need data calculation, either from the real-time data (data flow), either from the data stored; Also need to store and calculation results, in the operating system, has a very detailed discussion to the computing and storage, distributed system just put these theory to multiple nodes,
Then distributed system how to distribute the task to the computer nodes, very simple idea, divide and rule, namely the shard (partition), the calculation, it is to switch of computing tasks, each node calculates some, the final summary, this is the thought of graphs; For storage, better understand the data of each node, when the data size bigger, Partition is the only choice, at the same time also can bring some benefits:
(1) improve performance and concurrency, operations are distributed to different fragmentation, independent
(2) improve the system availability, even if some fragmentation can't use, other fragmentation will not be affected
Ideally, a shard to go, but the facts are not ideal, the reason is that there are a large number of nodes in the distributed system, and through the network communication, a single node fault (process crash, power outages, disk damage) is a small probability event, but the failure rate of the whole system will increase exponentially with the increase of nodes, network communication may also appear offline, because of the situation, in this will be the "abnormal" cases, the distributed system still need to continue the stable foreign service, which need strong fault tolerance, the easiest way to do that is redundant or duplicate set (Replication), namely, multiple node is responsible for the same task, the most common is distributed storage, multiple nodes complex to store the same data, in order to increase availability and reliability, at the same time, the Replication may cause performance improvements, such as the locality of data can reduce the waiting time for users,
CodePudding user response:
Fragmentation distribution server? To learn,,,CodePudding user response:
Fragmentation distribution server, to learn,,,