redis的cluster集群模式
我们知道主从复制和哨兵模式解决了Redis的单机问题，但是Redis仍然存在着以下两个问题：
1)所有的写操作都集中到主服务器上，主服务器CPU压力比较大
2)不管是主服务器还是从服务器，它们都同样保存了redis的所有数据，随着数据越来越多，可能会出现内存不够用的问题
在Redis3.0版本之后支持Cluste集群,它是一个由多个主从节点群组成的分布式服务器群，具有复制、高可用和分片特性，每个主存储的数据通过槽位来划分存储不通的数据。redis集群不需要sentinel哨兵也能完成节点移除和故障转移的功能，需要将每个节点设置成集群模式，这个集群模式没有中心节点，可水平扩展，据官方文档称可以线性扩展到1000节点。下面按下图架构部署集群：
集群配置
[[email protected] soft]$ pwd
/home/redis/soft
[[email protected] soft]$ ll
total 4
drwxrwxr-x 2 redis redis 6 Feb 12 03:56 cluster
drwxrwxr-x 6 redis redis 4096 Feb 12 03:53 redis-3.2.1
[[email protected] cluster]$ more ./7777/redis.conf |grep -v '^$'
include /home/redis/soft/redis-3.2.1/redis.conf
daemonize yes
loglevel notice
logfile "/home/redis/soft/cluster/7777/redis.log"
port 7777
bind 0.0.0.0
dir "/home/redis/soft/cluster/7777"
appendonly yes #持久化使用AOF
appendfilename "appendonly.aof"
appendfsync everysec
maxmemory 100000000 #最大内存设置100M
cluster-enabled yes #开启集群模式
cluster-node-timeout 15000 #节点超时时间15秒
maxclients 4064 #最大客户端连接数
tcp-backlog 0
cluster-config-file "/home/redis/soft/cluster/7777/nodes.conf" #集群内部配置文件，自动生成
pidfile "/home/redis/soft/cluster/7777/redis.pid"
masterauth "123456"
requirepass "123456"
[[email protected] cluster]$ more ./8888/redis.conf |grep -v '^$'
include /home/redis/soft/redis-3.2.1/redis.conf
daemonize yes
loglevel notice
logfile "/home/redis/soft/cluster/8888/redis.log"
port 8888
bind 0.0.0.0
dir "/home/redis/soft/cluster/8888"
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
maxmemory 100000000
cluster-enabled yes
cluster-node-timeout 15000
maxclients 4064
tcp-backlog 0
cluster-config-file "/home/redis/soft/cluster/8888/nodes.conf"
pidfile "/home/redis/soft/cluster/8888/redis.pid"
masterauth "123456"
requirepass "123456"
[[email protected] cluster]$
[[email protected] cluster]$ ll
total 0
drwxrwxr-x 2 redis redis 23 Feb 12 04:10 7777
drwxrwxr-x 2 redis redis 23 Feb 12 04:12 8888
[[email protected] cluster]$ pwd
/home/redis/soft/cluster
[[email protected] cluster]$ scp -r * [email protected]:/home/redis/soft/cluster/ #配置文件拷贝到其他机器
The authenticity of host '192.168.23.100 (192.168.23.100)' can't be established.
ECDSA key fingerprint is 64:e7:8c:8d:6a:e1:21:c5:04:78:a1:4e:7a:c3:e3:8d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.23.100' (ECDSA) to the list of known hosts.
[email protected]'s password:
redis.conf 100% 512 0.5KB/s 00:00
redis.conf 100% 512 0.5KB/s 00:00
[[email protected] cluster]$ scp -r * [email protected]:/home/redis/soft/cluster/ #配置文件拷贝到其他机器
The authenticity of host '192.168.23.101 (192.168.23.101)' can't be established.
ECDSA key fingerprint is 64:e7:8c:8d:6a:e1:21:c5:04:78:a1:4e:7a:c3:e3:8d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.23.101' (ECDSA) to the list of known hosts.
[email protected]'s password:
redis.conf 100% 512 0.5KB/s 00:00
redis.conf 100% 512 0.5KB/s 00:00
[[email protected] cluster]$
启动redis服务
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/7777/redis.conf
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/8888/redis.conf
[[email protected] ~]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/7777/redis.conf
[[email protected] ~]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/8888/redis.conf
[[email protected] ~]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/7777/redis.conf
[[email protected] ~]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/8888/redis.conf
构进集群关系
[[email protected] src]$ ./redis-cli -h 192.168.23.102 -p 7777 -a 123456
192.168.23.102:7777> cluster info #redis实例是孤立的，它们之间还没有建立集群的关系
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
192.168.23.102:7777> cluster nodes
72a85ee1b384421e031f59e9e2a24d7eb82fe737 :7777 myself,master - 0 0 0 connected
192.168.23.102:7777>
192.168.23.102:7777> cluster meet 192.168.23.102 8888 #通过cluster meet命令分别连接其他节点，使集群节点之间相关联
OK
192.168.23.102:7777> cluster meet 192.168.23.101 8888
OK
192.168.23.102:7777> cluster meet 192.168.23.101 7777
OK
192.168.23.102:7777> cluster meet 192.168.23.100 8888
OK
192.168.23.102:7777> cluster meet 192.168.23.100 7777
OK
192.168.23.102:7777>
192.168.23.102:7777> cluster nodes #集群关系构进完成
72a85ee1b384421e031f59e9e2a24d7eb82fe737 192.168.23.102:7777 myself,master - 0 0 1 connected
c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 192.168.23.101:7777 master - 0 1581500730968 3 connected
681b511c9ea52b032b197e8e39dc25e6f9edc8ce 192.168.23.100:8888 master - 0 1581500736006 4 connected
554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 192.168.23.101:8888 master - 0 1581500731976 2 connected
e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce 192.168.23.102:8888 master - 0 1581500733991 0 connected
584fcfaa343e1777d8a123dc18ed120a44f13422 192.168.23.100:7777 master - 0 1581500734998 5 connected
192.168.23.102:7777> cluster info #虽然连接各个节点连接成功但是还不能使用，因为还没有将16384个槽分配到集群节点中
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:0
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_sent:123
cluster_stats_messages_received:123
192.168.23.102:7777>
虚拟槽的分配
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.100 -p 7777 -a 123456 cluster addslots {0..5461}
OK
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.101 -p 7777 -a 123456 cluster addslots {5462..10922}
OK
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.102 -p 7777 -a 123456 cluster addslots {10923..16383}
OK
[[email protected] src]$
192.168.23.102:7777> cluster info #槽位分配后，集群状态正常
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_sent:718
cluster_stats_messages_received:718
192.168.23.102:7777> cluster nodes #显示槽位
72a85ee1b384421e031f59e9e2a24d7eb82fe737 192.168.23.102:7777 myself,master - 0 0 1 connected 10923-16383
c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 192.168.23.101:7777 master - 0 1581501043432 3 connected 5462-10922
681b511c9ea52b032b197e8e39dc25e6f9edc8ce 192.168.23.100:8888 master - 0 1581501041416 4 connected
554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 192.168.23.101:8888 master - 0 1581501044441 2 connected
e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce 192.168.23.102:8888 master - 0 1581501043431 0 connected
584fcfaa343e1777d8a123dc18ed120a44f13422 192.168.23.100:7777 master - 0 1581501042424 5 connected 0-5461
192.168.23.102:7777>
192.168.23.102:7777> cluster slots #显示槽位
1) 1) (integer) 10923
2) (integer) 16383
3) 1) "192.168.23.102"
2) (integer) 7777
3) "72a85ee1b384421e031f59e9e2a24d7eb82fe737"
2) 1) (integer) 5462
2) (integer) 10922
3) 1) "192.168.23.101"
2) (integer) 7777
3) "c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a"
3) 1) (integer) 0
2) (integer) 5461
3) 1) "192.168.23.100"
2) (integer) 7777
3) "584fcfaa343e1777d8a123dc18ed120a44f13422"
192.168.23.102:7777>
为主设置从节点
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.100 -p 8888 -a 123456 cluster replicate 584fcfaa343e1777d8a123dc18ed120a44f13422 #登录从节点进行关联主
OK
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.101 -p 8888 -a 123456 cluster replicate c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a
OK
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.102 -p 8888 -a 123456 cluster replicate 72a85ee1b384421e031f59e9e2a24d7eb82fe737
OK
[[email protected] src]$
192.168.23.102:7777> cluster nodes #主从配置正常
72a85ee1b384421e031f59e9e2a24d7eb82fe737 192.168.23.102:7777 myself,master - 0 0 1 connected 10923-16383
c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 192.168.23.101:7777 master - 0 1581501321613 3 connected 5462-10922
681b511c9ea52b032b197e8e39dc25e6f9edc8ce 192.168.23.100:8888 slave 584fcfaa343e1777d8a123dc18ed120a44f13422 0 1581501322620 5 connected
554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 192.168.23.101:8888 slave c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 0 1581501316573 3 connected
e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce 192.168.23.102:8888 slave 72a85ee1b384421e031f59e9e2a24d7eb82fe737 0 1581501323626 1 connected
584fcfaa343e1777d8a123dc18ed120a44f13422 192.168.23.100:7777 master - 0 1581501319595 5 connected 0-5461
192.168.23.102:7777> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_sent:1318
cluster_stats_messages_received:1318
192.168.23.102:7777>
192.168.23.102:7777> cluster slots
1) 1) (integer) 10923
2) (integer) 16383
3) 1) "192.168.23.102"
2) (integer) 7777
3) "72a85ee1b384421e031f59e9e2a24d7eb82fe737"
4) 1) "192.168.23.102"
2) (integer) 8888
3) "e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce"
2) 1) (integer) 5462
2) (integer) 10922
3) 1) "192.168.23.101"
2) (integer) 7777
3) "c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a"
4) 1) "192.168.23.101"
2) (integer) 8888
3) "554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f"
3) 1) (integer) 0
2) (integer) 5461
3) 1) "192.168.23.100"
2) (integer) 7777
3) "584fcfaa343e1777d8a123dc18ed120a44f13422"
4) 1) "192.168.23.100"
2) (integer) 8888
3) "681b511c9ea52b032b197e8e39dc25e6f9edc8ce"
192.168.23.102:7777>
验证集群可用性
[[email protected] src]$ ./redis-cli -h 192.168.23.102 -p 7777 -a 123456 #普通方式登录，不能进行写操作，需要用-c集群方式登录操作
192.168.23.102:7777> set name zhaiky
(error) MOVED 5798 192.168.23.101:7777
192.168.23.102:7777>
[[email protected] src]$ ./redis-cli -c -h 192.168.23.102 -p 7777 -a 123456
192.168.23.102:7777> set name zhaiky
-> Redirected to slot [5798] located at 192.168.23.101:7777
OK
192.168.23.101:7777> get name
"zhaiky"
192.168.23.101:7777>
192.168.23.102:7777> cluster keyslot name
(integer) 5798
192.168.23.102:7777>
模拟主宕机故障
192.168.23.102:7777> cluster nodes #当前集群状态正常，其中192.168.23.101:7777为主，192.168.23.101:8888为备
72a85ee1b384421e031f59e9e2a24d7eb82fe737 192.168.23.102:7777 myself,master - 0 0 1 connected 10923-16383
c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 192.168.23.101:7777 master - 1581505306750 1581505302518 7 disconnected 5462-10922
681b511c9ea52b032b197e8e39dc25e6f9edc8ce 192.168.23.100:8888 slave 584fcfaa343e1777d8a123dc18ed120a44f13422 0 1581505307558 5 connected
554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 192.168.23.101:8888 slave c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 0 1581505306551 7 connected
e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce 192.168.23.102:8888 slave 72a85ee1b384421e031f59e9e2a24d7eb82fe737 0 1581505310587 1 connected
584fcfaa343e1777d8a123dc18ed120a44f13422 192.168.23.100:7777 master - 0 1581505309577 5 connected 0-5461
192.168.23.102:7777>
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-cli -h 192.168.23.101 -p 7777 -a 123456 #停止192.168.23.101:7777，模拟宕机
192.168.23.101:7777> shutdown
not connected>
192.168.23.102:7777> cluster nodes #192.168.23.101:7777停止，192.168.23.101:8888正常接管
72a85ee1b384421e031f59e9e2a24d7eb82fe737 192.168.23.102:7777 myself,master - 0 0 1 connected 10923-16383
c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 192.168.23.101:7777 master,fail - 1581505306750 1581505302518 7 disconnected
681b511c9ea52b032b197e8e39dc25e6f9edc8ce 192.168.23.100:8888 slave 584fcfaa343e1777d8a123dc18ed120a44f13422 0 1581505326748 5 connected
554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 192.168.23.101:8888 master - 0 1581505327759 8 connected 5462-10922
e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce 192.168.23.102:8888 slave 72a85ee1b384421e031f59e9e2a24d7eb82fe737 0 1581505325742 1 connected
584fcfaa343e1777d8a123dc18ed120a44f13422 192.168.23.100:7777 master - 0 1581505324733 5 connected 0-5461
192.168.23.102:7777>
[[email protected] src]$ /home/redis/soft/redis-3.2.1/src/redis-server /home/redis/soft/cluster/7777/redis.conf #恢复192.168.23.101:7777
192.168.23.102:7777> cluster nodes #192.168.23.101:7777为从，192.168.23.101:8888为主，不抢占
72a85ee1b384421e031f59e9e2a24d7eb82fe737 192.168.23.102:7777 myself,master - 0 0 1 connected 10923-16383
c1e0f4ede71cdfa94db63fbc8dbc0d03024e598a 192.168.23.101:7777 slave 554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 0 1581505509554 8 connected
681b511c9ea52b032b197e8e39dc25e6f9edc8ce 192.168.23.100:8888 slave 584fcfaa343e1777d8a123dc18ed120a44f13422 0 1581505510565 5 connected
554fe235bfe4f1f5490bc1c56cc3aff88cb6aa1f 192.168.23.101:8888 master - 0 1581505508544 8 connected 5462-10922 e8c026d7bf2d82e95d9498cf56ad3ac45ab6e6ce 192.168.23.102:8888 slave 72a85ee1b384421e031f59e9e2a24d7eb82fe737 0 1581505505525 1 connected 584fcfaa343e1777d8a123dc18ed120a44f13422 192.168.23.100:7777 master - 0 1581505507538 5 connected 0-5461 192.168.23.102:7777>