Cache is a kind of quantitative data would be preserved for catering to the treatment of subsequent capture requirements, aimed at increasing the speed of data acquisition, data generation process may need to be calculated, neat, remote access to operation, if it is the same data needs to be used multiple times, each time to regenerate the greatly waste of time, so, if the calculation or remote request operation so as to obtain the data cache, can accelerate the data acquisition of subsequent demand,
First a simple example to understand the concepts of caching mechanism:
# - * - coding: utf-8 - * -
Import the random
Import a datetime
The class MyCache:
"" "cache class "" "
Def __init__ (self) :
# use a dictionary structure cached data in the form of kv
The self. The cache={}
# limit the size of the cache, because of the cache space is limited
# when the cache is too big, so need to old cached discard
Self. Max_cache_size=10
Def __contains__ (self key) :
"" "according to the key exists in the cache returns True or False "" "
The return key in the self. The cache
Def get (self key) :
"" get data from the cache" "" "
[key]
data=https://bbs.csdn.net/topics/self.cacheData [" date_accessed "]=datetime. Datetime. Now ()
Return the data (" value ")
Def add (self, key, value) :
"" "updates the cache dictionary, if the cache is too big, first remove the earliest entry "" "
If the key not in self. The cache and len (self) cache) & gt;=self. Max_cache_size:
Self. Remove_oldest ()
The self. The cache [key]={
'date_accessed: datetime. Datetime. Now (),
'value: the value
}
Def remove_oldest (self) :
"" "delete have access the earliest date of input data "" "
Oldest_entry=None
For the key in the self. The cache:
If oldest_entry is None:
Oldest_entry=key
The continue
Curr_entry_date=self. Cache [key] [' date_accessed]
Oldest_entry_date=self. Cache [oldest_entry] [' date_accessed]
If curr_entry_date & lt; Oldest_entry_date:
Oldest_entry=key
The self. The cache. Pop (oldest_entry)
@ property
Def size (self) :
"" "return to the cache size "" "
Return len (self) cache)
If __name__=="__main__ ':
# test cache function
Cache=MyCache ()
Cache. The add (" test ", the sum (range (100000)))
Assert cache. Get (" test ")==cache. Get (" test ")
Keys=[
'red', 'fox', 'fence', 'junk', 'other', 'alpha' and 'bravo', 'CAL'
'devo', 'ele'
]
S='abcdefghijklmnop'
For I, the key in enumerate (keys) :
If the key in the cache:
The continue
The else:
valuehttps://bbs.csdn.net/topics/='. Join ([the random choice (s) for I in range (20)])
Cache. The add (key, value)
Assert "test" not in the cache
Print (cache. The cache)
The above example shows only simple caching mechanism principle, the data by means of using key/value pair in the dictionary, if next time we need to value can be directly obtained in the dictionary, the realization of the sample in the delete old data is not efficient, practical application can be used in other ways,
In Python version 3.2, introduced a very elegant cache mechanism, namely functool module of lru_cache decorators, can be directly to a function or class method the results of the cache, subsequent calls will return the cached results directly, lru_cache prototype is as follows:
@ functools. Lru_cache (maxsize=None, typed=False)
USES functools module lur_cache decorator, most can cache maxsize a call result of this function, so as to improve the efficiency of the program execution, particularly suitable for time-consuming function, parameter maxsize to cache the number of times at most if to None, is unlimited, is set to a power of 2, the best performance; If typed=True (note that there is no this parameter in functools32), different parameter types call will cache respectively, such as f (3) and f (3.0),
LRU (further Recently, informs, the Least Recently Used) algorithm is a cache elimination strategy, the weeding, according to access data in the history of the core idea is that "if the data has been Recently visited, then be accessed at higher risk of" in the future, this algorithm first for the operating system page replacement algorithm of a kind of memory management, is mainly Used to find out the memory of a long time didn't use the memory block, to move it out of the memory so as to provide space for new data, the principle as a simple example,
Be lru_cache illuminative function will have cache_clear and cache_info two methods, used to clear the cache and check the cache information respectively, the use effect of the following is a simple lru_cache:
The from functools import lru_cache
@ lru_cache (None)
Def add (x, y) :
Print (" + calculating: % s % s "% (x, y))
Return x + y
The print (add (1, 2))
The print (add (1, 2))
The print (add (2, 3))
Output:
Calculating: 1 + 2
3
3
Calculating: 2 + 3
5
As can be seen from the results, when a second call the add (1, 2), and there is no real executive function body, but the result of the direct return to cache,
If you want to use in Python 2 functools32 lru_cahce need to install the third party module, there is a use of C language implementation, faster, and at the same time compatible Python2 and Python3 fastcache third-party modules can realize the same function, and it can support the TTL,
Lru_cahce is from the data cached in memory, in fact can also be data cache to disk, try the following example implements a disk-based cache decorator:
The import OS
The import uuid
The import pickled
The import shutil
The import tempfile as expected
The from functools import wraps as func_wraps
The class DiskCache (object) :
"" "cache data to disk
Instantiate the parameters:
-- -- -- -- --
Cache_path: the path of the cache file
"" "
_NAMESPACE=uuid. Uuid (" c875fb30 - a8a8-402 - d - a796-225 a6b065cad ")
Def __init__ (self, cache_path=None) :
If cache_path:
Self. Cache_path=OS. Path. Abspath (cache_path)
The else:
Self. Cache_path=OS. Path. Join (tempfile as expected. The gettempdir (), ". Diskcache ")
Def __call__ (self, func) :
"" "return a packaging function after
If the disk does not cache the function and cache the results returned again after
If the disk cache, the direct return to cache the results of the
"" "
@ func_wraps (func)
Def wrapper (* args, * * kw) :
Params_uuid=uuid. Uuid5 (self) _NAMESPACE, "-". Join (map (STR, (args, kw))))
Key='{} - {}. The cache'. The format (func __name__, STR (params_uuid))
Cache_file=OS. Path. Join (self) cache_path, key)
If not OS. Path. The exists (self cache_path) :
OS. Makedirs (self. Cache_path)
Try:
With the open (cache_file, 'rb') as f:
Val=pickle. The load (f)
Except the Exception:
Val=func (* args, * * kw)
Try:
With the open (cache_file, 'wb) as f:
Pickle. Dump (val, f)
Except the Exception:
Pass
Return val
Return wrapper
Def the clear (self, func_name) :
"" "clean up the specified function call cache "" "
For cache_file in OS. Listdir (self. Cache_path) :
If cache_file. Startswith (func_name + "-") :
OS. Remove (OS) path) join (self. Cache_path cache_file))
Def clear_all (self) :
"" "clean all the cache "" "
nullnullnullnullnullnullnullnullnullnullnull
