Look at what Redis is a

Official introduction explained: Redis is a based on the BSD open source project, is a structured data on the memory of a storage system, you can use it as a database, cache and message middleware,

Support both strings, lists, hashes, sets, sorted sets, bitmaps, hyperloglogs and geospatial indexes data types, such as

It has a built-in copy, lua scripts, LRU, transactions, and other functions, through the redis sentinel to achieve high availability, via redis cluster has realized automatic subdivision,

And transaction, publish/subscribe, automatic failover, and so on,
To sum up, redis provides rich functionality, first met may feel dazzling,

These functions are used?

What is the problem is solve?

What circumstance can use the corresponding function?

The evolution of the following starting from scratch, step by step to the interpretation of the rough,

1 from scratch

The initial demand is very simple, we have a list to provide hot news API:http://api.xxx.com/hot-news, API consumers complain that every request to 2 seconds or so to return a result,

Then we will work on how to improve the API consumers perceived performance, the most simple and crude first plan soon:

API based on the HTTP response and cache control cache-control: Max - age=600, which let the consumer can cache the response ten minutes,

If the API consumers if effective use of the cache control information in the response, can effectively improve the performance of perception (10 minutes),

But there are two disadvantages:

The first is in the cache into effect in 10 minutes, API consumers may get old data;

The second is that if the API client ignore the cache direct access to the API is still a need for 2 seconds, take temporary solution not effect a permanent cure,

Based on the native memory cache
2
In order to solve the problem of call API still need to 2 seconds, after screening, the main reason is that in the process of using SQL access to hot news consumes nearly two seconds,

As a result, we have thought of a simple solution, namely the SQL query results are cached in the current API server directly in memory (Settings cache valid time for 1 minute),

Subsequent requests in one minute read cache directly, no longer spend 2 seconds to execute SQL,

If the API 100 received requests per second, a minute is 6000, that is only the first 2 seconds, the request of the crowded come over will takes 2 seconds, all the requests in subsequent 58 seconds can do even response, without having to wait for 2 seconds,

Other apis friend found that this is a good way, so we soon found the API server memory is full,

3 server-side Redis

The API server memory were cache with time, we found that had to think solution,

The most direct ideas that we put these cache is lost to a dedicated server, it's big memory configuration,

Then we took redis,

As for how to configure the deployment of redis here don't explain, redis official has a detailed introduction,

Then we use a single server as redis server, API server memory pressure,

3.1 Persistence (personal)

Single redis server a month there is always a few days in a bad mood, bad mood strikes, lead to lost all cache (redis data is stored in memory),

Though to the redis server comes back online, but due to the loss of data memory, the cache avalanche, the pressure of the API server and database or suddenly came up,

So redis persistence will come in handy at this time, can alleviate the impact of the cache avalanche,

Redis persistence refers to Redis will put the data in the memory written to the hard disk, in Redis restart when loading the data, thus maximum limit to reduce the effects of cache miss,

3.2 the sentry (Sentinel) and copy (Replication)

Redis server without warning strike is a hassle,

So what can I do?

Answer: a backup, you hang up on it,

So how do you know a redis server hang up and how to switch, how to ensure the backup machine is a full backup of the original server?

At this time requires Sentinel and played the Replication,

Sentinel can manage multiple redis server, it provides monitoring, reminding and automatic failover function; Replication is responsible for a redis server can be equipped with multiple backup servers,

Redis is also using these two functions to ensure high availability of Redis,

In addition, the Sentinel function is the function of a publish and subscribe to redis use,

3.3 the Cluster (Cluster)

There is always a limit of single server resources, CPU and I/o resources we can through the master-slave replication, and separation, the part of the CPU and IO pressure moved from the server,

But what to do with memory resources, master-slave mode to do just the same data backup, not horizontal expansion of memory; Single machine memory also can increase the processing, but there is always a limit,

So we need a solution that allows us to scale horizontally,

Ultimate goal is to each server responsible for only part of the integrated to let all the server and the rest of the world consumers, this group of the distributed server is like a centralized server,

Before redis official distributed scheme out a twemproxy and codis two scheme, the two schemes in general are dependent on the proxy for distributed, i.e. redis itself does not care about distributed, but rather to twemproxy and codis to responsible,

While official redis cluster scheme is put this part of the distributed things done every redis server, make it no longer need other components can be independent to complete the requirement of the distributed,

We don't care about these solutions of optimal slightly, but what we focus on the distributed here to deal with those things?

Namely twemproxy and codis independent processing this part of the distributed logic and the processing of this part of the cluster integration to redis service logic to deal with the problem of what exactly is it?

As we said earlier, a distributed service to the outside world is like a centralized service,

So have to do this will face a problem to be solved:

Is not only the increase or decrease the amount of server in distributed service, the service on consumption in terms of the client should be non-inductive;

Also means that the client cannot penetrate distributed services, tied his death to a particular set of servers, because if so, you can no longer be new server, can't fault replacement,

To solve this problem, there are two way:

The first way is the most direct, that's why I add a layer to isolate the specific dependence,

Namely twemproxy adopts the way, let all the client can consumption redis service through it, through which to isolate this dependence (but you will find twermproxy becomes a single point), in this case each redis servers are independent, they do not know each other between each other;

The second way is to make the redis server knew of each other, through the redirection mechanism to guide the client to complete her need operation,

Such as links to the client a redis server, said I to perform this operation, redis server found himself can't do this, then you can finish this operation to the client and server information to let the client to request another server,

At that time you will find that every redis server need to keep a full distributed server information of a material, or it how do you know which server for the client to try to find other to perform the operation, the client want

This section explains so much, don't know find whether the first way is the second way, there is a common thing, that is all the server in the distributed service and the service can provide information,

This information also to exist anyway, the difference is:

The first way is to separate this part of the information management, use this information to coordinate the backend multiple independent redis server;

The second way is to let every redis server with this information, know each other, each other to achieve, as well as the first way, the purpose of advantage is no longer requires an additional component to deal with this part,

Redis Cluster specific implementation details is using the concept of Hash slot, namely pre-allocated out 16384 slots:

On the client side through the study of the CRC16 of Key (Key) % 16384 operation get corresponding slot is which one;

In redis server is each server is responsible for part of the tank, when there is a new server to add or remove to migrate these slots and its corresponding data,

Each server at the same time have a full tank and its corresponding server information, which makes the server can be redirected to the client's request processing,

4 the client Redis

Above are mainly introduced in the third section redis server evolution steps, explained how redis from a single service, evolved into a highly available, decentralized, distributed storage system,

Under this section is focused on the redis service client can consumption,

4.1 data type

Redis support rich data types, from the most basic string is commonly used to complex to the data structure of support:

String: the most basic data type, the binary security string, 512 m, the largest
List: in order to keep the order of the list of strings,
Set: unordered collection of string, there is no repetitive element,
Sorted set: string sorted collection,
Hash: key - the value of a collection of
Bitmap: a more detailed operation, to bit,
Hyperloglog: based on the probability of the data structure,
Many of those data types, mainly in order to support the needs of the various scenarios, of course, each type has different time complexity,

Actually before these complex Data structure is equivalent to my blog in this series "unscramble REST" based on the network application in the architectural style of introduction to the Remote Data Access (Remote Data Access=(RDA) of the specific implementation,

Through execution on the server in a standard set of commands, in between the service side want to reduce the result set, after simplifying the client use, also can improve the network performance,

If there is no list this kind of data structure, for example, you can only deposit the list into a string, the client to get the complete list, complete again after the operation to redis, will produce a lot of waste,

4.2 transaction

nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull