redis 主从复制

时间 2021-08-13

标签 node redis 缓存服务器网络并发 app 负载均衡 less socket 栏目 Redis 繁體版

原文原文链接

一，redis主从复制原理

一、主从复制过程大致能够分为3个阶段
1.链接创建阶段（即准备阶段）
2.数据同步阶段
3.命令传播阶段

二、在从节点执行 slaveof 命令后，复制过程便开始按下面的流程运做
1.保存主节点信息：配置slaveof以后会在从节点保存主节点的信息。
2.主从创建socket链接：定时发现主节点以及尝试创建链接。
3.发送ping命令：从节点定时发送ping给主节点，主节点返回PONG。若主节点没有返回PONG或因阻塞没法响应致使超时，则主从断开，在下次定时任务时会重新ping主节点。
4.权限验证：若主节点开启了ACL或配置了requirepass参数，则从节点须要配置masteruser和masterauth参数才能保证主从正常链接。
5.同步数据集：首次链接，全量同步。
6.命令持续复制：全量同步完成后，保持增量同步。

三、当节点被当作从节点的时候，须要注意！！
当节点被当作从节点的时候，须要将从节点设置为只读，这样作的目的是保证集群的数据一致性。

主从复制原理图

二，主从复制做用

1.数据冗余：主从复制实现了数据的热备份，是持久化以外的一种数据冗余方式。
2.故障恢复：当主节点出现问题时，能够由从节点提供服务，实现快速的故障恢复；其实是一种服务的冗余。
3.负载均衡：在主从复制的基础上，配合读写分离，能够由主节点提供写服务，由从节点提供读服务，分担服务器负载；尤为是在写少读多的场景下，经过多个从节点分担读负载，能够大大提升Redis服务器的并发量。
4.读写分离：主库写、从库读，读写分离不只能够提升服务器的负载能力，同时可根据需求的变化，改变从库的数量。
高可用基石：除了上述做用之外，主从复制仍是哨兵和集群可以实施的基础。node

2.1 主从部分复制

当master和slave断开链接时，master会将期间所作的操做记录到复制缓存区当中（能够当作是一个队列，其大小默认1M）。待slave重连后，slave会向master发送psync命令并传入offset和runId，这时候，若是master发现slave传输的偏移量的值，在缓存区队列范围中，就会将从offset开始到队列结束的数据传给slave，从而达到同步，下降了使用全量复制的开销。redis

2.2 全量复制的开销

1.bgsave的开销，每次bgsave须要fork子进程，对内存和CPU的开销很大
2.RDB文件网络传输的时间（网络带宽）
3.从节点清空数据的时间
4.从节点加载RDB的时间
5.可能的AOF重写时间（若是咱们的从节点开启了AOF，则加载完RDB后会对AOF进行一个重写，保证AOF是最新的

三，部署主从复制

1.1 环境准备

角色	主机	IP端口
主库	redis01	172.16.1.81:6379
从库	redis02	172.16.1.82:6379
从库	redis03	172.16.1.83:6379

1.2 配置

# 81 redis配置
[root@redis01 ~]# egrep -v '^#|^$' /usr/local/redis/conf/redis.conf 
bind 127.0.0.1 172.16.1.81
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "/usr/local/redis/conf/redis.log"
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir "/usr/local/redis/data"
masterauth 123
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100
acllog-max-len 128
requirepass 123
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
oom-score-adj no
oom-score-adj-values 0 200 800
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
jemalloc-bg-thread yes

# 82 配置
[root@redis02]# egrep -v '^#|^$' /usr/local/redis/conf/redis.conf
bind 127.0.0.1 172.16.1.82
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "/usr/local/redis/conf/redis.log"
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir "/usr/local/redis/data"
masterauth 123
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100
acllog-max-len 128
requirepass 123
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
oom-score-adj no
oom-score-adj-values 0 200 800
appendonly yes
appendfilename "appendonly.aof"
appendfsync always
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
jemalloc-bg-thread yes


# 83 配置
[root@redis03 /usr/local/redis]# egrep -v '^#|^$' /usr/local/redis/conf/redis.conf
bind 127.0.0.1 172.16.1.83
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile "/usr/local/redis/conf/redis/log"
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir "/usr/local/redis/data"
masterauth 123
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100
acllog-max-len 128
requirepass 123
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
oom-score-adj no
oom-score-adj-values 0 200 800
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes
jemalloc-bg-thread yes

四，启动查看主从复制

# 启动每台redis
[root@redis01 /usr/local/redis/conf]# ../bin/redis-server redis.conf
[root@redis02 /usr/local/redis/conf]# ../bin/redis-server redis.conf
[root@redis03 /usr/local/redis/conf]# ../bin/redis-server redis.conf 

# 开启主从配置以81为主库
[root@redis02 /usr/local/redis/conf]# ../bin/redis-cli 
127.0.0.1:6379> auth 123
OK
127.0.0.1:6379> SLAVEOF 172.16.1.81 6379
OK

[root@redis03 /usr/local/redis/conf]# ../bin/redis-cli 
127.0.0.1:6379> auth 123
OK
127.0.0.1:6379> SLAVEOF 172.16.1.81 6379
OK

# 81查看主从状态
[root@redis01 /usr/local/redis/conf]# ../bin/redis-cli 
127.0.0.1:6379> auth 123
OK
127.0.0.1:6379> INFO replication
# Replication
role:master
connected_slaves:2
slave0:ip=172.16.1.82,port=6379,state=online,offset=224,lag=1
slave1:ip=172.16.1.83,port=6379,state=online,offset=224,lag=1
master_replid:36be025a09e8ef0983dbc69dd69b697f465cc868
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:224
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:224

五，主从复制存在的问题

1.手动故障转移
2.写能力和存储能力受限缓存