Redis 缓存失效和回收机制

时间 2019-12-04

标签 redis 缓存失效回收机制栏目 Redis 繁體版

原文原文链接

本文及后续文章，Redis版本均是v3.2.8redis

1、内存回收策略算法

maxmemory配置用于配置Redis存储数据时指定限制的内存大小。咱们能够通过redis.conf配置或者使用CONFIG SET命令来进行运行时配置。数据库

例如在redis.conf文件中，配置内存限制为100mb缓存

maxmemory 100mbdom

设置maxmemory为0表明没有内存限制。对于64位的系统这是个默认值，对于32位的系统默认内存限制为3GB。ui

当目前使用的内存超过了设置的最大内存，就要进行内存释放了，当须要进行内存释放的时候，须要用某种策略对保存的的对象进行删除。this

redis中当内存超过限制时，按照配置的策略，淘汰掉相应的key-value，使得内存能够继续留有足够的空间保存新的数据。redis 在肯定了要驱逐某个键值对后，会删除这个数据，并将这个数据变动消息发布到本地（AOF 持久化）和从机（主从链接）。lua

当maxmemory限制达到的时候，Redis采起内存回收的策略由Redis的maxmemory-policy配置来进行决定的。有六种策略：spa

volatile-lru：默认的策略，尝试回收最少使用的键（LRU），但仅限于在过时集合的键,使得新添加的数据有空间存放。code

volatile-random：回收随机的键使得新添加的数据有空间存放，但仅限于在过时集合的键。
volatile-ttl：回收在过时集合的键，而且优先回收存活时间（TTL）较短的键,使得新添加的数据有空间存放。
allkeys-lru：尝试回收最少使用的键（LRU），使得新添加的数据有空间存放。
allkeys-random：回收随机的键使得新添加的数据有空间存放。
no-enviction：禁止淘汰数据

若是没有键知足回收的前提条件的话，策略volatile-lru, volatile-random,volatile-ttl就和no-eviction 差很少了。

选择正确的回收策略是很是重要的，这取决于你的应用的访问模式，不过你能够在运行时进行相关的策略调整，而且监控缓存命中率和没命中的次数，经过Redis INFO命令输出以便调优。

通常的经验规则:

使用allkeys-lru策略：当你但愿你的请求符合一个幂定律分布，也就是说，你但愿部分的子集元素将比其它其它元素被访问的更多。
使用allkeys-random：若是你是循环访问，全部的键被连续的扫描，或者你但愿请求分布正常（全部元素被访问的几率都差很少）。
使用volatile-ttl：若是你想要经过建立缓存对象时设置TTL值，来决定哪些对象应该被过时。
allkeys-lru 和 volatile-random策略：当你想要单一的实例实现缓存及持久化一些键时颇有用。不过通常运行两个实例是解决这个问题的更好方法。

为键设置过时时间也是须要消耗内存的，因此使用allkeys-lru这种策略更加高效，由于没有必要为键取设置过时时间当内存有压力时。

除此以外还有一个配置项，就是maxmemory-samples，默认值是3，由于上面的策略代码实现的都是近似算法，因此无论是lru算法，仍是ttl，都并非在数据库中全部的数据为基础的算法，由于当数据库的数据不少的时候，这样效率过低，因此代码中都是基于maxmemory-samples个数据的近似算法。

近似LRU的算法，经过对少许keys进行取样，而后回收其中一个最被访问时间较早的key。咱们能够经过调整每次回收时检查的采样数量，以实现调整算法的精度。Redis为何不使用真实的LRU实现是由于这须要太多的内存。不过近似的LRU算法对于应用而言应该是等价的。

最后，咱们看下命令在执行前，Redis server作了什么事情？

/* If this function gets called we already read a whole

* command, arguments are in the client argv/argc fields.

* processCommand() execute the command or prepare the

* server for a bulk read from the client.

*

* If C_OK is returned the client is still alive and valid and

* other operations can be performed by the caller. Otherwise

* if C_ERR is returned the client was destroyed (i.e. after QUIT). */

int processCommand(client *c) {

/* The QUIT command is handled separately. Normal command procs will

* go through checking for replication and QUIT will cause trouble

* when FORCE_REPLICATION is enabled and would be implemented in

* a regular command proc. */

if (!strcasecmp(c->argv[0]->ptr,"quit")) {

addReply(c,shared.ok);

c->flags |= CLIENT_CLOSE_AFTER_REPLY;

return C_ERR;

}

/* Now lookup the command and check ASAP about trivial error conditions

* such as wrong arity, bad command name and so forth. */

c->cmd = c->lastcmd = lookupCommand(c->argv[0]->ptr);

if (!c->cmd) {

flagTransaction(c);

addReplyErrorFormat(c,"unknown command '%s'",

(char*)c->argv[0]->ptr);

return C_OK;

} else if ((c->cmd->arity > 0 && c->cmd->arity != c->argc) ||

(c->argc < -c->cmd->arity)) {

flagTransaction(c);

addReplyErrorFormat(c,"wrong number of arguments for '%s' command",

c->cmd->name);

return C_OK;

}

/* Check if the user is authenticated */

if (server.requirepass && !c->authenticated && c->cmd->proc != authCommand)

{

flagTransaction(c);

addReply(c,shared.noautherr);

return C_OK;

}

/* If cluster is enabled perform the cluster redirection here.

* However we don't perform the redirection if:

* 1) The sender of this command is our master.

* 2) The command has no key arguments. */

if (server.cluster_enabled &&

!(c->flags & CLIENT_MASTER) &&

!(c->flags & CLIENT_LUA &&

server.lua_caller->flags & CLIENT_MASTER) &&

!(c->cmd->getkeys_proc == NULL && c->cmd->firstkey == 0 &&

c->cmd->proc != execCommand))

{

int hashslot;

int error_code;

clusterNode *n = getNodeByQuery(c,c->cmd,c->argv,c->argc,

&hashslot,&error_code);

if (n == NULL || n != server.cluster->myself) {

if (c->cmd->proc == execCommand) {

discardTransaction(c);

} else {

flagTransaction(c);

}

clusterRedirectClient(c,n,hashslot,error_code);

return C_OK;

}

}

/* Handle the maxmemory directive.

*

* First we try to free some memory if possible (if there are volatile

* keys in the dataset). If there are not the only thing we can do

* is returning an error. */

if (server.maxmemory) {

int retval = freeMemoryIfNeeded();

/* freeMemoryIfNeeded may flush slave output buffers. This may result

* into a slave, that may be the active client, to be freed. */

if (server.current_client == NULL) return C_ERR;

/* It was impossible to free enough memory, and the command the client

* is trying to execute is denied during OOM conditions? Error. */

if ((c->cmd->flags & CMD_DENYOOM) && retval == C_ERR) {

flagTransaction(c);

addReply(c, shared.oomerr);

return C_OK;

}

}

/* Don't accept write commands if there are problems persisting on disk

* and if this is a master instance. */

if (((server.stop_writes_on_bgsave_err &&

server.saveparamslen > 0 &&

server.lastbgsave_status == C_ERR) ||

server.aof_last_write_status == C_ERR) &&

server.masterhost == NULL &&

(c->cmd->flags & CMD_WRITE ||

c->cmd->proc == pingCommand))

{

flagTransaction(c);

if (server.aof_last_write_status == C_OK)

addReply(c, shared.bgsaveerr);

else

addReplySds(c,

sdscatprintf(sdsempty(),

"-MISCONF Errors writing to the AOF file: %s\r\n",

strerror(server.aof_last_write_errno)));

return C_OK;

}

/* Don't accept write commands if there are not enough good slaves and

* user configured the min-slaves-to-write option. */

if (server.masterhost == NULL &&

server.repl_min_slaves_to_write &&

server.repl_min_slaves_max_lag &&

c->cmd->flags & CMD_WRITE &&

server.repl_good_slaves_count < server.repl_min_slaves_to_write)

{

flagTransaction(c);

addReply(c, shared.noreplicaserr);

return C_OK;

}

/* Don't accept write commands if this is a read only slave. But

* accept write commands if this is our master. */

if (server.masterhost && server.repl_slave_ro &&

!(c->flags & CLIENT_MASTER) &&

c->cmd->flags & CMD_WRITE)

{

addReply(c, shared.roslaveerr);

return C_OK;

}

/* Only allow SUBSCRIBE and UNSUBSCRIBE in the context of Pub/Sub */

if (c->flags & CLIENT_PUBSUB &&

c->cmd->proc != pingCommand &&

c->cmd->proc != subscribeCommand &&

c->cmd->proc != unsubscribeCommand &&

c->cmd->proc != psubscribeCommand &&

c->cmd->proc != punsubscribeCommand) {

addReplyError(c,"only (P)SUBSCRIBE / (P)UNSUBSCRIBE / PING / QUIT allowed in this context");

return C_OK;

}

/* Only allow INFO and SLAVEOF when slave-serve-stale-data is no and

* we are a slave with a broken link with master. */

if (server.masterhost && server.repl_state != REPL_STATE_CONNECTED &&

server.repl_serve_stale_data == 0 &&

!(c->cmd->flags & CMD_STALE))

{

flagTransaction(c);

addReply(c, shared.masterdownerr);

return C_OK;

}

/* Loading DB? Return an error if the command has not the

* CMD_LOADING flag. */

if (server.loading && !(c->cmd->flags & CMD_LOADING)) {

addReply(c, shared.loadingerr);

return C_OK;

}

/* Lua script too slow? Only allow a limited number of commands. */

if (server.lua_timedout &&

c->cmd->proc != authCommand &&

c->cmd->proc != replconfCommand &&

!(c->cmd->proc == shutdownCommand &&

c->argc == 2 &&

tolower(((char*)c->argv[1]->ptr)[0]) == 'n') &&

!(c->cmd->proc == scriptCommand &&

c->argc == 2 &&

tolower(((char*)c->argv[1]->ptr)[0]) == 'k'))

{

flagTransaction(c);

addReply(c, shared.slowscripterr);

return C_OK;

}

/* Exec the command */

if (c->flags & CLIENT_MULTI &&

c->cmd->proc != execCommand && c->cmd->proc != discardCommand &&

c->cmd->proc != multiCommand && c->cmd->proc != watchCommand)

{

queueMultiCommand(c);

addReply(c,shared.queued);

} else {

call(c,CMD_CALL_FULL);

c->woff = server.master_repl_offset;

if (listLength(server.ready_keys))

handleClientsBlockedOnLists();

}

return C_OK;

}

未完待续