Redis 集群安装配置及调研记录

时间 2019-11-05

标签 redis 集群安装配置调研记录栏目 Redis 繁體版

原文原文链接

Redis简介

REmote DIctionary Server(Redis) Redis是一个开源的使用ANSI C语言编写、遵照BSD协议、支持网络、可基于内存亦可持久化的日志型、Key-Value数据库，并提供多种语言的API。一般被称为数据结构服务器，值（value）能够是字符串(String), 哈希(Map), 列表(list), 集合(sets) 和有序集合(sorted sets)等类型。支持主从模式的数据备份。拥有极高的读写性能，丰富的数据类型，原子性操做，还支持pub/sub，通知，key过时等特性。node

与其余k-v存储的区别，redis有更为复杂的数据结构而且对他们提供原子性操做，redis运行在内存但支持持久化，持久化是以追加的方式产生的。mysql

Redis是一个开源的高性能键值对数据库.它经过提供多种键值数据类型来适应不一样场景下的存储需求,而且借助许多高层级的接口使其能够胜任,如缓存、队列系统的不一样角色.将键值对数据类型存放在内存中的一个数据库。c++

单机安装

#挂载ISO操做系统光盘
[root@MySQLNODE02 /]# mount -t iso9660 /dev/cdrom  /mnt/
#确认是否挂载成功
[root@MySQLNODE02 /]# df -h
Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/centos_mysql-root   48G  7.7G   40G  17% /
devtmpfs                       903M     0  903M   0% /dev
tmpfs                          913M     0  913M   0% /dev/shm
tmpfs                          913M   17M  896M   2% /run
tmpfs                          913M     0  913M   0% /sys/fs/cgroup
/dev/sda1                      497M  125M  373M  25% /boot
tmpfs                          183M     0  183M   0% /run/user/0
/dev/sr0                       4.1G  4.1G     0 100% /mnt
#yum源配置，按如下步骤修改文件内容
[root@MySQLNODE02 /]#vi /etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
baseurl=file:///mnt/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
#清理yum配置
[root@MySQLNODE02 /]#yum  clean all
#安装基础依赖软件包
yum -y install cpp binutils glibc glibc-kernheaders glibc-common glibc-devel gcc make gcc-c++ libstdc++-devel tcl
#解压
[root@MySQLNODE02 local]# tar -zxf redis-3.2.3.tar.gz
#编译安装前
[root@MySQLNODE02 local]# cd /usr/local/redis-3.2.3
[root@MySQLNODE02 redis-3.2.3]# ls src/redis-*
src/redis-benchmark.c  src/redis-check-aof.c  src/redis-check-rdb.c  src/redis-cli.c  src/redis-trib.rb [root@MySQLNODE02 redis-3.2.3]# make && make install
……
    INSTALL install
    INSTALL install
    INSTALL install
    INSTALL install
    INSTALL install
make[1]: Leaving directory `/usr/local/redis-3.2.3/src'
#编译安装后
[root@MySQLNODE02 redis-3.2.3]# ls src/redis-*
src/redis-benchmark    src/redis-check-aof    src/redis-check-rdb    src/redis-cli    src/redis-sentinel
src/redis-benchmark.c  src/redis-check-aof.c  src/redis-check-rdb.c  src/redis-cli.c  src/redis-server
src/redis-benchmark.o  src/redis-check-aof.o  src/redis-check-rdb.o  src/redis-cli.o  src/redis-trib.rb
#利用默认配置启动redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server
15968:C 25 Aug 11:19:34.214 # Warning: no config file specified, using the default config. In order to specify a config file use ./src/redis-server /path/to/redis.conf
15968:M 25 Aug 11:19:34.216 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 15968
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

15968:M 25 Aug 11:19:34.229 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
15968:M 25 Aug 11:19:34.230 # Server started, Redis version 3.2.3
15968:M 25 Aug 11:19:34.230 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
15968:M 25 Aug 11:19:34.230 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
15968:M 25 Aug 11:19:34.230 * The server is now ready to accept connections on port 6379
#后台进程启动并简单测试
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  >/dev/null  2>&1 &
[5] 16060
[root@MySQLNODE02 redis-3.2.3]# netstat -nlap | grep 6379
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      16060/./src/redis-s
tcp6       0      0 :::6379                 :::*                    LISTEN      16060/./src/redis-s
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli
127.0.0.1:6379> set foo bar
OK
127.0.0.1:6379> get foo
"bar"

简单集群配置

本集群规模为两个redis节点和一个哨兵节点，配置两个redis节点，过程基本一致，确认两节点的主从状态。redis

#节点222上的7000
[root@MySQLNODE02 redis-3.2.3]# cp redis.conf  clusters/redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# vi clusters/redis_7000.conf
port 7000
daemonize yes
bind 192.168.0.222
save ""
#save 900 1
#save 300 10
#save 60 10000
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
#节点224上的7000
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7000.conf
port 7000
daemonize yes
bind 192.168.0.224
save ""
#save 900 1
#save 300 10
#save 60 10000
slaveof 192.168.0.222 7000
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
#分别在两个机器上检查各自的主从状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.0.224,port=7000,state=online,offset=841,lag=1
master_repl_offset:841
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:840
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7000
# Replication
role:slave
master_host:192.168.0.222
master_port:7000
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:1023
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

哨兵

Redis的哨兵(sentinel) 系统用于管理多个 redis 服务器,该系统执行如下三个任务:算法

监控(Monitoring): 哨兵(sentinel) 会不断地检查你的Master,Slave,other sentinel是否运做正常；
提醒(Notification):当被监控的某个 Redis出现问题时, 哨兵(sentinel) 能够经过 API 向管理员或者其余应用程序发送通知；
自动故障迁移(Automatic failover):当一个Master不能正常工做时，哨兵(sentinel) 会开始一次自动故障迁移操做,它会将集群中一个Slave升级为新的Master,原master作为slave; 当客户端试图链接失效的Master时,集群也会向客户端返回新Master的地址,使得集群可使用新Master代替失效Master；

哨兵(sentinel) 是一个分布式系统,你能够在一个架构中运行多个哨兵(sentinel) 进程,这些进程使用流言协议(gossipprotocols)来接收关于Master是否下线的信息,并使用投票协议(agreement protocols)来决定是否执行自动故障迁移,以及选择哪一个Slave做为新的Master。sql

每一个哨兵(sentinel) 会向其它sentinel、master、slave定时发送消息,以确认对方是否”活”着,若是发现对方在指定时间(可配置)内未回应,则暂时认为对方已挂(所谓的”主观认为宕机” Subjective Down,简称sdown)；若“哨兵群”中的多数sentinel,都报告某一master没响应,系统才认为该master “完全死亡”(即:客观上的真正down机,Objective Down,简称odown),经过必定的vote算法,从剩下的slave节点中,选一台提高为master,而后自动修改相关配置。数据库

虽然哨兵(sentinel) 是一个单独的可执行文件 redis-sentinel ,但实际上它只是一个运行在特殊模式下的 Redis 服务器，你能够在启动一个普通 Redis 服务器时经过给定 --sentinel 选项来启动哨兵(sentinel)。哨兵(sentinel) 的一些设计思路和zookeeper很是相似。json

哨兵配置文件在安装reids过程当中已经建立，只须要修改相应的配置文件启动便可。主要关注sentinel monitor配置项。centos

[root@MySQLNODE02 redis-3.2.3]# cp sentinel.conf  sentinel.conf.bak
[root@MySQLNODE02 redis-3.2.3]# vi sentinel.conf
port 27000
protected-mode no
sentinel monitor mymaster 192.168.0.222 7000 1
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-sentinel  sentinel.conf
18844:X 25 Aug 14:54:38.995 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 27000
 |    `-._   `._    /     _.-'    |     PID: 18844
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

18844:X 25 Aug 14:54:38.997 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
18844:X 25 Aug 14:54:38.999 # Sentinel ID is 94dd02d5be1379ade7b1fca1de96e9ad412dbe68
18844:X 25 Aug 14:54:38.999 # +monitor master mymaster 192.168.0.222 7000 quorum 1
18844:X 25 Aug 14:54:39.001 * +slave slave 192.168.0.224:7000 192.168.0.224 7000 @ mymaster 192.168.0.222 7000

实际配置中能够将daemonize yes配置为no，在启动服务的时候用>/filename.log 2>&1 &的方式实如今后台运行同时将对应运行日志写入指定文件，方便查看服务运行状态。待系统运行稳定后能够设置daemonize yes避免日志过多占用OS空间。缓存

测试哨兵模式下的集群状态变化过程，整个集群中的哨兵，master,slave的配置文件在集群状态发生变化时内容会自动更新，因此从新启动时若是要保持原始集群的主从配置状态，须要从新配置这些文件（特别是哨兵的配置文件sentinel.conf，不从新配置启动时只能监控master节点）。在主从切换的过程当中，哨兵也会根据集群的最新状态更新相应的redis.conf配置文件。

复杂集群配置

本集群环境中配置5个Redis服务，3个哨兵，Redis的安装参照单机安装过程，如下为本集群各节点的配置过程。

#master 7000
[root@MySQLNODE02 redis-3.2.3]# cp redis.conf  clusters/redis_7000.conf
port 7000
daemonize yes
bind 192.168.0.222
save ""
#save 900 1
#save 300 10
#save 60 10000
#slave 7001
[root@MySQLNODE02 redis-3.2.3]# cp redis.conf  clusters/redis_7001.conf
port 7001
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.222
save ""
#save 900 1
#save 300 10
#save 60 10000
#slave 7002
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7002.conf
port 7002
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.224
save ""
#save 900 1
#save 300 10
#save 60 10000
#slave 7003
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7003.conf
port 7003
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.224
#slave 7004
[root@MySQLNODE04 redis-3.2.3]# cp redis.conf  clusters/redis_7004.conf
port 7004
slaveof 192.168.0.222 7000
daemonize yes
bind 192.168.0.224
save ""
#save 900 1
#save 300 10
#save 60 10000
如下为各sentinel配置过程。
#sentinel 27000
[root@MySQLNODE02 redis-3.2.3]# cp sentinel.conf  clusters/sentinel_27000.conf
port 27000
protected-mode no
#2表示最低经过票数
sentinel monitor mymaster 192.168.0.222 7000 2
#sentinel 27001
[root@MySQLNODE02 redis-3.2.3]# cp sentinel.conf  clusters/sentinel_27001.conf
port 27001
protected-mode no
sentinel monitor mymaster 192.168.0.222 7000 2
#sentinel 27002
[root@MySQLNODE04 redis-3.2.3]# cp sentinel.conf  clusters/sentinel_27002.conf
port 27002
protected-mode no
sentinel monitor mymaster 192.168.0.222 7000 2
启动各个redis节点，并经过客户端工具查看集群的状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7001.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root      20104      1  0 16:27 ?        00:00:00 ./src/redis-server 192.168.0.222:7000
root      20110      1  0 16:27 ?        00:00:00 ./src/redis-server 192.168.0.222:7001
root      20132  15583  0 16:28 pts/0    00:00:00 grep --color=auto redis
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7002.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7003.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7004.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     117642      1  0 16:28 ?        00:00:00 ./src/redis-server 192.168.0.224:7002
root     117646      1  0 16:28 ?        00:00:00 ./src/redis-server 192.168.0.224:7003
root     117650      1  0 16:28 ?        00:00:00 ./src/redis-server 192.168.0.224:7004
root     117654   6219  0 16:28 pts/0    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
192.168.0.222:7000> info replication
# Replication
role:master
connected_slaves:4
slave0:ip=192.168.0.222,port=7001,state=online,offset=197,lag=1
slave1:ip=192.168.0.224,port=7002,state=online,offset=197,lag=1
slave2:ip=192.168.0.224,port=7003,state=online,offset=197,lag=0
slave3:ip=192.168.0.224,port=7004,state=online,offset=197,lag=1
master_repl_offset:197
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:196
#依次在新的窗口启动各个哨兵，启动一会监听到两哨兵关掉
20938:X 25 Aug 17:24:14.116 # Sentinel ID is 1056bcf2bb85895e72338752e8a0fe7f1a233449
20938:X 25 Aug 17:24:14.116 # +monitor master mymaster 192.168.0.222 7000 quorum 2
20938:X 25 Aug 17:24:14.117 * +slave slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.119 * +slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.120 * +slave slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.122 * +slave slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.596 * +sentinel sentinel 8b66a575e24286486637d081030a49154436c5e6 192.168.0.224 27002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:15.138 * +sentinel sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.222 7000
验证过程，先关闭其中一个slave节点，再启动该节点；关闭master节点，再启动该节点；关闭一个sentinel节点；收集个验证过程当中集群哨兵的日志信息。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-sentinel  clusters/sentinel_27001.conf
20938:X 25 Aug 17:24:14.111 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.3 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 27001
 |    `-._   `._    /     _.-'    |     PID: 20938
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

20938:X 25 Aug 17:24:14.114 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
20938:X 25 Aug 17:24:14.116 # Sentinel ID is 1056bcf2bb85895e72338752e8a0fe7f1a233449
20938:X 25 Aug 17:24:14.116 # +monitor master mymaster 192.168.0.222 7000 quorum 2
20938:X 25 Aug 17:24:14.117 * +slave slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.119 * +slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.120 * +slave slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.122 * +slave slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:14.596 * +sentinel sentinel 8b66a575e24286486637d081030a49154436c5e6 192.168.0.224 27002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:24:15.138 * +sentinel sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.222 7000
#整个集群启动状态
20938:X 25 Aug 17:29:10.069 # +sdown slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:29:46.571 * +reboot slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:29:46.622 # -sdown slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
#关闭一个slave，再启动该slave节点的过程
20938:X 25 Aug 17:31:07.727 # +sdown master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.780 # +odown master mymaster 192.168.0.222 7000 #quorum 2/2
20938:X 25 Aug 17:31:07.780 # +new-epoch 1
20938:X 25 Aug 17:31:07.780 # +try-failover master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.781 # +vote-for-leader 1056bcf2bb85895e72338752e8a0fe7f1a233449 1
20938:X 25 Aug 17:31:07.783 # 2944fe768c00ff33c0dd2bbad24f432f384d946f voted for 1056bcf2bb85895e72338752e8a0fe7f1a233449 1
20938:X 25 Aug 17:31:07.783 # 8b66a575e24286486637d081030a49154436c5e6 voted for 1056bcf2bb85895e72338752e8a0fe7f1a233449 1
20938:X 25 Aug 17:31:07.845 # +elected-leader master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.845 # +failover-state-select-slave master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.938 # +selected-slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:07.938 * +failover-state-send-slaveof-noone slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.030 * +failover-state-wait-promotion slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.748 # +promoted-slave slave 192.168.0.224:7002 192.168.0.224 7002 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.749 # +failover-state-reconf-slaves master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:08.830 * +slave-reconf-sent slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.769 * +slave-reconf-inprog slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.769 * +slave-reconf-done slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.845 * +slave-reconf-sent slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:09.923 # -odown master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:10.814 * +slave-reconf-inprog slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:10.814 * +slave-reconf-done slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:10.870 * +slave-reconf-sent slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.817 * +slave-reconf-inprog slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.817 * +slave-reconf-done slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.875 # +failover-end master mymaster 192.168.0.222 7000
20938:X 25 Aug 17:31:11.875 # +switch-master mymaster 192.168.0.222 7000 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.224:7003 192.168.0.224 7003 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.222:7001 192.168.0.222 7001 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.224:7004 192.168.0.224 7004 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:11.876 * +slave slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002
20938:X 25 Aug 17:31:41.903 # +sdown slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002
#关闭Redis 的master节点，在多sentinel模式下会先选举一个sentinel的leader节点
20938:X 25 Aug 17:32:31.091 # -sdown slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002
#启动关闭的master节点
20938:X 25 Aug 17:32:41.044 * +convert-to-slave slave 192.168.0.222:7000 192.168.0.222 7000 @ mymaster 192.168.0.224 7002

哨兵关闭一个，收集哨兵的日志信息。被关闭的哨兵窗口会关闭，其余两个窗口输出以下信息仅此而已，哨兵之间是平等关系的。

117740:X 25 Aug 17:53:34.956 # +sdown sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.224 7002

20938:X 25 Aug 17:53:35.183 # +sdown sentinel 2944fe768c00ff33c0dd2bbad24f432f384d946f 192.168.0.222 27000 @ mymaster 192.168.0.224 7002

Redis命令操做

Hash

Redis的哈希是键值对的集合。 Redis的哈希值是字符串字段和字符串值之间的映射，所以它们被用来表示对象。

#在master节点上进行操做：
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> hset user:1 username zhangsan
(integer) 1
192.168.0.224:7002> hset user:1 password zhangsan123
(integer) 1
192.168.0.224:7002> hset user:2 username zhangsan
(integer) 1
192.168.0.224:7002> hdel user:2 username
(integer) 1
192.168.0.224:7002> hset user:2 username lisi
(integer) 1
192.168.0.224:7002> hset user:2 password lisi123
(integer) 1
192.168.0.224:7002> hmset user:3 username wangwu password wangwu123
OK
192.168.0.224:7002> hget user:1 username
"zhangsan"
192.168.0.224:7002> hmget user:1 username password
1) "zhangsan"
2) "zhangsan123"
192.168.0.224:7002> hgetall user:1
1) "username"
2) "zhangsan"
3) "password"
4) "zhangsan123"
192.168.0.224:7002> hgetall user:2;
(empty list or set)
192.168.0.224:7002> hgetall user:2
1) "username"
2) "lisi"
3) "password"
4) "lisi123"
#在slave节点上操做，slave只能查
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7001
192.168.0.222:7001> hgetall user:1
1) "username"
2) "zhangsan"
3) "password"
4) "zhangsan123"
192.168.0.222:7001> hgetall user:4
(empty list or set)
192.168.0.222:7001> hset user:4 username zhaoliu
(error) READONLY You can't write against a read only slave.

默认状况下redis数据库充当slave角色时是只读的不能进行写操做。可经过修改配置项使从库也具备写功能。
slave-read-only yes

列表

一个简单的字符串列表,排序插入顺序,添加到头部或者尾部,通常咱们都是存放json序列化后的数据。他的最大长度为2^32 – 1。

192.168.0.224:7002> lpush list 1
(integer) 1
192.168.0.224:7002> rpush list 2
(integer) 2
192.168.0.224:7002> lpush list 3 4 5
(integer) 5
192.168.0.224:7002> llen list
(integer) 5
192.168.0.224:7002> lpop list
"5"
192.168.0.224:7002> rpop list
"2"
192.168.0.224:7002> llen list
(integer) 3

Redis持久化

RDB

RDB 持久化能够在指定的时间间隔内生成数据集的时间点快照（point-in-time snapshot），在Redis配置文件中已经预置了3个条件知足其中任何一个条件就会进行快照。

默认的RDB的文件路径是在当前目录, 文件名是 : dump.rdb,能够再配置文件中修改路径和文件名,分别是dir和dbfilenameRedis启动后会读取RDB快照文件,将数据从硬盘中载入到内存,通常状况下1GB的快照文件载入到内存的时间约为20~30分钟.(不一样服务器会存在差别)。

BD文件是经过压缩的, 能够经过配置rdbcompression参数来禁用压缩。

rdbcompression yes

若是要禁止Redis的RDB的持久化方式那么只用吧上面的触发条件禁止掉就OK了。

RDB的持久化过程

Redis是使用fork函数复制一份当前进程(父进程)的副本(子进程);
父进程继续接收并处理客户端发来的命令,而子进程开始将内存中的数据写入到磁盘的临时文件中;
当子进程写入完全部的数据后会用该临时文件替换掉旧的RDB文件;

因为Redis使用fork来复制一份当前进程,那么子进程就会占有和主进程同样的内存资源,若是主进程是8G内存,那么在备份的时候必须保证有16GB的内存,要否则会启用虚拟内存,性能很是的差,若是数据集很是巨大,也可能会影响到CPU。

AOF

AOF 持久化记录服务器执行的全部写操做命令,并在服务器启动时,经过从新执行这些命令来还原数据集。AOF 文件中的命令所有以 Redis 协议的格式来保存,新命令会被追加到文件的末尾. Redis 还能够在后台对 AOF 文件进行重写（rewrite）,使得 AOF 文件的体积不会超出保存数据集状态所需的实际大小。

AOF持久化经过修改appendonly参数来设置默认是不开启的

AOF文件的位置和RDB文件的位置是相同的,都是经过dir参数设置,默认的文件名是apendonly.aof,也能够经过apendfilename参数修改。

# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
# appendfsync always
appendfsync everysec
# appendfsync no

auto-aof-rewrite-percentage：表示当前AOF文件大小超过上一次重写时的AOF文件大小的百分之多少时会再次进行重写,若是以前没有重写过,则以启动时的AOF文件大小为依据。上面的表示百分之一百。

uto-aof-rewrite-min-size：表示限制容许重写的最小AOF文件大小,一般在AOF文件很小的时候即便其中有些冗余的命令也是能够忽略的。

文件写入默认状况下会先写入到系统的缓存中,系统每30秒同步一次,才是真正的写入到硬盘中,若是在这30秒服务器宕机的话,那么数据也会丢失,能够经过修改Reids 配置参数来修改同步策略。

Redis主从架构

主从复制(读写分离)

redis的主从复制功能很是强大，一个master能够拥有多个slave,而一个slave又能够拥有多个slave,如此下去,造成了强大的多级服务器集群架构. 能够避免Redis单点故障,构建读写分离架构,知足读多写少的应用场景。

一个Master能够有多个Slave；
Redis使用异步复制。从2.8开始，Slave会周期性（每秒一次）发起一个Ack确认复制流（replication stream）被处理进度；
不只主服务器能够有从服务器，从服务器也能够有本身的从服务器，多个从服务器之间能够构成一个图状结构；
复制在Master端是非阻塞模式的，这意味着即使是多个Slave执行首次同步时，Master依然能够提供查询服务；
复制在Slave端也是非阻塞模式的：若是你在redis.conf作了设置，Slave在执行首次同步的时候仍可使用旧数据集提供查询；你也能够配置为当Master与Slave失去联系时，让Slave返回客户端一个错误提示；
当Slave要删掉旧的数据集，并从新加载新版数据时，Slave会阻塞链接请求（通常发生在与Master断开重连后的恢复阶段）；
复制功能能够单纯地用于数据冗余（dataredundancy），也能够经过让多个从服务器处理只读命令请求来提高扩展性（scalability）：好比说，繁重的 SORT 命令能够交给附属节点去运行。
能够经过修改Master端的redis.config来避免在Master端执行持久化操做（Save），由Slave端来执行持久化。

Redis的主从配置能够经过修改从库的Redis.conf文件增长slaveof配置项，也能够在全部节点都启动后，在从库的redis.cli窗口执行slaveof命令配置，但此种配置重启后会丢失配置信息。

主从从架构，的配置将slaveof配置为上层从库的IP和端口便可。在主库set的数据在从库都能get到。

数据同步过程

当从库和主库创建MS关系后，会向主数据库发送SYNC命令(同步命令);

主库接收到SYNC命令后会开始在后台保存快照（RDB持久化过程），并将期间接收到的写命令缓存起来;因此就算关掉了RDB持久化方式,在他们同步的时候也会产生RDB文件

以上主库只作了2件事情将接收的命令进行RDB持久化；在RDB持久化中,将接收的命令缓存起来

当快照完成后，主Redis会将快照文件和全部缓存的写命令发送给从Redis;

从Redis接收到后，会载入快照文件而且执行收到的缓存的命令;

以后，主Redis每当接收到写命令时就会将命令发送从Redis，从而保证数据的一致;

在这个过程当中主从同步是经过RDB来同步数据, 即便禁用了RDB也没有用,那么就会产生IO问题,在这个复制过程可能就会出现瓶颈. Redis在2.8.18版本开始实现了无磁盘复制功能

Redis在与从库进行复制初始化时将不会吧快照储存到磁盘,而是直接经过网络发送给从库,避免了IO性能问题

修改repl-diskless-sync 为 yes

# With slow disks and fast (large bandwidth) networks, diskless replication
# works better.
repl-diskless-sync no

宕机处理

在主从架构中出现了宕机的状况

Slave 宕机

在Redis中,从库从新启动会自动加入到主从架构中,自动完成同步数据；

Redis 2.8以后,在从库有作持久化的前提下,若是从库在断开的期间,主库的变化不大,从库再次启动后,主库不会将或有的数据进行RDB操做,而是进行增量复制；

Master 宕机

在Slave中执行SLAVEOF NO ONE 命令,断开主从关系而且提高为主库继续服务；

将主库从新启动后,执行 SLAVEOF命令,将其设置为其余Redis的从库,这个时候数据就同步回来了；

Redis3.0集群

虽然咱们搭建了一个主从架构,可是每一个Redis都要保存相同的数据,这样容易形成水桶效应。并且主从架构频繁TCP链接断开也可能会对服务器和网络带来很大负担。Redis 3.0版本最大的更新就是支持集群。集群具备如下特征。

节点自动发现

slave->master 选举,集群容错

Hot resharding:在线分片

进群管理:clusterxxx

基于配置(nodes-port.conf)的集群管理

ASK转向/MOVED 转向机制

何时整个集群不可用(cluster_state:fail),当集群不可用时,全部对集群的操做作都不可用，收到((error)CLUSTERDOWN The cluster is down)错误；若是集群任意master挂掉,且当前master没有slave.集群进入fail状态,也能够理解成进群的slot映射[0-16383]不完成时进入fail状态；若是进群超过半数以上master挂掉，不管是否有slave集群进入fail状态。

集群搭建

搭建不带slave的三节点master集群。修改各节点的配置文件。

#若是配置了slaveof须要关闭
#7000节点
cluster-enabled yes
cluster-config-file nodes-7000.conf
port 7000
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
#7001节点
cluster-enabled yes
cluster-config-file nodes-7001.conf
port 7001
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
#7002节点
cluster-enabled yes
cluster-config-file nodes-7002.conf
port 7002
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
依次启动确认各节点的状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  clusters/redis_7001.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root      76990      1  0 17:36 ?        00:00:00 ./src/redis-server 192.168.0.222:7000 [cluster]
root      76996      1  0 17:36 ?        00:00:00 ./src/redis-server 192.168.0.222:7001 [cluster]
root      77006  70424  0 17:36 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  clusters/redis_7002.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     121497      1  0 17:37 ?        00:00:00 ./src/redis-server 192.168.0.224:7002 [cluster]
root     121501 121037  0 17:37 pts/1    00:00:00 grep --color=auto redis
Redis3.0之后版本配置集群时使用redis-trib.rb工具，该工具的使用须要安装依赖的基础软件包，此软件包（redis-3.2.1.gem）在ISO光盘中并不存在，须要下载上传服务器后安装。
[root@MySQLNODE02 redis-3.2.3]# yum -y install zlib ruby rubygems
[root@MySQLNODE02 redis-3.2.3]# gem install -l ./redis-3.2.1.gem
Successfully installed redis-3.2.1
Parsing documentation for redis-3.2.1
Installing ri documentation for redis-3.2.1
1 gem installed
集群的配置要求各个Redis节点都是最新的，不然会报一下错误，能够从新安装配置Redis再配置（也能够删除对应node-port.conf文件）。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb create --replicas 0 192.168.0.222:7000 192.168.0.222:7001 192.168.0.224:7002
>>> Creating cluster
[ERR] Node 192.168.0.222:7000 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
经过redis-trib.rb工具配置集群环境
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb create --replicas 0 192.168.0.222:7000 192.168.0.222:7001 192.168.0.224:7002
>>> Creating cluster
>>> Performing hash slots allocation on 3 nodes...
Using 3 masters:
192.168.0.222:7000
192.168.0.224:7002
192.168.0.222:7001
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 192.168.0.222:7000)
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
由于配置Redis集群以后每次设置值的时候都要先计算key的值获取该key 的插槽值,而后在设置到集群中对应插槽值的Redis实例中，若是没有换一个key试试，在set 和 get 的时候都报错。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
192.168.0.222:7000> set test test
(error) MOVED 6918 192.168.0.224:7002
192.168.0.222:7000> get test
(error) MOVED 6918 192.168.0.224:7002
#能够去指定节点上操做
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> get test
(nil)
192.168.0.224:7002> set test AAAA
OK
192.168.0.224:7002> get test
"AAAA"
redis-cli 提供了一个参数–c，指定了这个参数以后,redis-cli会根据插槽值作一个重定向,链接到指定的redis实例上面。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> get test
-> Redirected to slot [6918] located at 192.168.0.224:7002
"AAAA"
#查看集群的节点信息
192.168.0.224:7002> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 master - 0 1503915810894 1 connected 0-5460
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 myself,master - 0 0 3 connected 5461-10922
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1503915811914 2 connected 10923-16383

Redis集群的数据是根据插槽值来设置进具体的节点中的.可是若是这个key的插槽值不是在当前redis实例的话,他就须要进行重定向.这样一次操做就变成了2次，这个就是Redis3.0里面所说的ASK 转向/MOVED 转向机制。

整个Redis提供了16384个插槽，也就是说集群中的每一个节点分得的插槽数总和为16384。redis-trib.rb脚本实现了是将16384个插槽平均分配给了N个节点。

key的有效部分使用CRC16算法计算出哈希值，再将哈希值对16384取余，获得插槽值。

新增节点

新增一个节点的配置文件，并结合集群中各节点的配置修改相应配置项。

7003
cluster-enabled yes
cluster-config-file nodes-7003.conf
port 7003
daemonize yes
#save 900 1
#save 300 10
#save 60 10000
#启动该节点
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7003.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     124626      1  0 18:18 ?        00:00:01 ./src/redis-server 192.168.0.224:7002 [cluster]
root     124640      1  0 18:27 ?        00:00:00 ./src/redis-server 192.168.0.224:7003 [cluster]
root     124644 121037  0 18:27 pts/1    00:00:00 grep --color=auto redis
在集群中的任意节点上利用redis-trib.rb工具配置新增节点，
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-trib.rb add-node 192.168.0.224:7003 192.168.0.224:7002
>>> Adding node 192.168.0.224:7003 to cluster 192.168.0.224:7002
>>> Performing Cluster Check (using node 192.168.0.224:7002)
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
   0 additional replica(s)
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.0.224:7003 to make it join the cluster.
[OK] New node added correctly.
添加成功后，查看集群个节点的状态信息，发现新增长的节点没有分配插槽值。
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 master - 0 1503916248543 1 connected 0-5460
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 myself,master - 0 0 3 connected 5461-10922
f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003 master - 0 1503916249565 0 connected
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1503916250585 2 connected 10923-16383
经过命令手动给新增长节点分配插槽值
[root@MySQLNODE02 redis-3.2.3]# ./redis-trib.rb reshard 192.168.0.224:7003
#指定要转移的插槽数
#接收插槽的节点ID
#指定插槽转移方式，all(所有重分配),done(从指定节点转移)
#yes确认转移
#转移完成后查看集群节点的状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1503916592510 2 connected 10923-16383
f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003 master - 0 1503916590480 4 connected 5461-6460
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 master - 0 1503916591496 3 connected 6461-10922

删除节点

若是要删除某个节点的话, 咱们要先把该节点上面的插槽数给转移到别的节点上面,而后在经过redis-trib.rb脚原本删除节点,否则这个插槽数就没有咯,而对应的key也失效。

#转移要删除节点的插槽
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb reshard 192.168.0.224:7003
>>> Performing Cluster Check (using node 192.168.0.224:7003)
M: f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003
   slots:5461-6460 (1000 slots) master
   0 additional replica(s)
M: 279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
M: 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002
   slots:6461-10922 (4462 slots) master
   0 additional replica(s)
M: a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
#指定要转移插槽数
How many slots do you want to move (from 1 to 16384)? 1000
#指定接收者
What is the receiving node ID? 6a57ae40bb995453248ff0f0fbd76551fb6e3c0a
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
#指定要转移的节点
Source node #1:f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac
Source node #2:done  
......
#确认转移
Do you want to proceed with the proposed reshard plan (yes/no)? yes
#查看转移后的集群状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1504058169010 2 connected 10923-16383
f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac 192.168.0.224:7003 master - 0 1504058170031 4 connected
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 master - 0 1504058167991 5 connected 5461-10922
#删除节点
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb   del-node 192.168.0.224:7003 f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac
>>> Removing node f0d4f695e647ed95b1ab7223ccc2e2f8617bf7ac from cluster 192.168.0.224:7003
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
[root@MySQLNODE02 redis-3.2.3]#
#查看删除后的状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
a795b16768779cd8cc453347b114a5f42a7a0710 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
279af53b2f3eed9323fab3287844e8a54293347c 192.168.0.222:7001 master - 0 1504058318129 2 connected 10923-16383
6a57ae40bb995453248ff0f0fbd76551fb6e3c0a 192.168.0.224:7002 master - 0 1504058317110 5 connected 5461-10922

高可用集群

能够搭建一个只有master节点的集群,可是若是咱们其中的一个master宕机了,那么他对应插槽值的key所有都会失效,集群就不可用.Redis集群为咱们提供了故障机制，能够为集群中的每一个master节点指定一个slave节点实现高可用集群环境。

集群中的每一个节点都会按期的向其它节点发送PING命令，而且经过有没有收到回复判断目标节点是否下线

集群中每一秒就会随机选择几个节点，而后选择其中最久没有响应的节点放PING命令

若是必定时间内目标节点都没有响应，那么该节点就认为目标节点疑似下线

当集群中的节点超过半数认为该目标节点疑似下线，那么该节点就会被标记为下线

当集群中的任何一个节点下线，就会致使插槽区有空档，不完整，那么该集群将不可用

若是该下线的节点使用了主从模式,那么该节点(master)宕机后,集群会将该节点的从库(slave)提高为(master)继续完成集群服务。

按照集群模式准备好6个redis节点的配置文件，清理原来启用过得节点文件（不然启动时会提示节点不为空），依次启动各节点。

[root@MySQLNODE02 redis-3.2.3]# ll redis_*
-rw-r--r-- 1 root root 46727 Aug 30 10:23 redis_7000.conf
-rw-r--r-- 1 root root 46727 Aug 28 17:32 redis_7001.conf
-rw-r--r-- 1 root root 46727 Aug 30 10:25 redis_7005.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7000.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7001.conf
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7005.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111838      1  0 10:31 ?        00:00:00 ./src/redis-server 192.168.0.222:7000 [cluster]
root     111844      1  0 10:31 ?        00:00:00 ./src/redis-server 192.168.0.222:7001 [cluster]
root     111848      1  0 10:31 ?        00:00:00 ./src/redis-server 192.168.0.222:7005 [cluster]
root     111854  70424  0 10:32 pts/1    00:00:00 grep --color=auto redis

[root@MySQLNODE04 redis-3.2.3]# ll redis_*
-rw-r--r-- 1 root root 46727 Aug 28 17:34 redis_7002.conf
-rw-r--r-- 1 root root 46727 Aug 28 18:26 redis_7003.conf
-rw-r--r-- 1 root root 46755 Aug 30 10:29 redis_7004.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7002.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7003.conf
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7004.conf
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     126824      1  0 10:32 ?        00:00:00 ./src/redis-server 192.168.0.224:7002 [cluster]
root     126828      1  0 10:32 ?        00:00:00 ./src/redis-server 192.168.0.224:7003 [cluster]
root     126832      1  0 10:32 ?        00:00:00 ./src/redis-server 192.168.0.224:7004 [cluster]
root     126836 121037  0 10:32 pts/1    00:00:00 grep --color=auto redis
利用redis-trib.rb文件配置高可用集群，redis会自动指定主从节点，经过实际配置大体应该是从小到大依次分配主从节点。
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-trib.rb  create --replicas 1 192.168.0.222:7000 192.168.0.222:7001 192.168.0.222:7005 192.168.0.224:7002 192.168.0.224:7003 192.168.0.224:7004
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.0.222:7000
192.168.0.224:7002
192.168.0.222:7001
Adding replica 192.168.0.224:7003 to 192.168.0.222:7000
Adding replica 192.168.0.222:7005 to 192.168.0.224:7002
Adding replica 192.168.0.224:7004 to 192.168.0.222:7001
M: 60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
S: 91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005
   replicates e154234f624b5f4f44cd3dc029b87ecabb29e564
M: e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
S: e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003
   replicates 60b53e792f10a1fab087e73713180aa56554858a
S: db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004
   replicates 3ef887b183685eb4687085bea40352af018f07ea
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 192.168.0.222:7000)
M: 60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000
   slots:0-5460 (5461 slots) master
M: 3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001
   slots:10923-16383 (5461 slots) master
M: 91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005
   slots: (0 slots) master
   replicates e154234f624b5f4f44cd3dc029b87ecabb29e564
M: e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002
   slots:5461-10922 (5462 slots) master
M: e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003
   slots: (0 slots) master
   replicates 60b53e792f10a1fab087e73713180aa56554858a
M: db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004
   slots: (0 slots) master
   replicates 3ef887b183685eb4687085bea40352af018f07ea
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
#查看集群的状态
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504060793611 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504060790556 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master - 0 1504060789541 4 connected 5461-10922
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 slave e154234f624b5f4f44cd3dc029b87ecabb29e564 0 1504060791568 4 connected
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504060792591 5 connected
192.168.0.222:7000>
测试集群环境下的数据操做
#操做时仍是须要重定向
192.168.0.222:7000> set one aaaa
(error) MOVED 9084 192.168.0.224:7002
192.168.0.222:7000> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> set one AAAAAA
-> Redirected to slot [9084] located at 192.168.0.224:7002
OK
192.168.0.224:7002> get one
"AAAAAA"
192.168.0.224:7002> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.224 -p 7002
192.168.0.224:7002> get one
"AAAAAA"
192.168.0.224:7002> exit
#7002的从节点上没法直接获取数据，但有该key信息
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli  -h 192.168.0.222 -p 7005
192.168.0.222:7005> get one
(error) MOVED 9084 192.168.0.224:7002
192.168.0.222:7005> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7005
192.168.0.222:7005> get one
-> Redirected to slot [9084] located at 192.168.0.224:7002
"AAAAAA"
192.168.0.224:7002> keys *
1) "one"
测试集群中节点的启停对整个集群的影响状况。
#一个slave故障对集群的操做无影响
[root@MySQLNODE02 redis-3.2.3]# kill -9 111848
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111838      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7000 [cluster]
root     111844      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7001 [cluster]
root     112628  70424  0 11:24 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504063464906 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504063467964 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master - 0 1504063465922 4 connected 5461-10922
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 slave,fail e154234f624b5f4f44cd3dc029b87ecabb29e564 1504063447033 1504063442442 4 disconnected
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504063466942 5 connected
192.168.0.222:7000> set two BBBBB
OK
192.168.0.222:7001> get two
-> Redirected to slot [2127] located at 192.168.0.222:7000
"BBBBB"
#重启slave节点，对集群的操做无影响
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-server  redis_7005.conf
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111838      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7000 [cluster]
root     111844      1  0 10:31 ?        00:00:07 ./src/redis-server 192.168.0.222:7001 [cluster]
root     112663      1  0 11:26 ?        00:00:00 ./src/redis-server 192.168.0.222:7005 [cluster]
root     112667  70424  0 11:26 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504063622035 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504063624074 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master - 0 1504063620413 4 connected 5461-10922
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 slave e154234f624b5f4f44cd3dc029b87ecabb29e564 0 1504063625093 4 connected
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504063623054 5 connected
192.168.0.222:7000> set three CCCCCC
-> Redirected to slot [13861] located at 192.168.0.222:7001
OK
192.168.0.222:7001> get three
"CCCCCC"
#关闭7002master节点
[root@MySQLNODE04 redis-3.2.3]# ps -ef | grep redis
root     126824      1  0 10:32 ?        00:00:08 ./src/redis-server 192.168.0.224:7002 [cluster]
root     126828      1  0 10:32 ?        00:00:09 ./src/redis-server 192.168.0.224:7003 [cluster]
root     126832      1  0 10:32 ?        00:00:09 ./src/redis-server 192.168.0.224:7004 [cluster]
root     126888 121037  0 11:30 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE04 redis-3.2.3]# kill -9 126824
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
192.168.0.222:7000> cluster nodes
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 myself,master - 0 0 1 connected 0-5460
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504063851577 2 connected 10923-16383
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504063846500 6 connected
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 master,fail - 1504063834998 1504063829584 4 disconnected
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 master - 0 1504063853607 7 connected 5461-10922
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504063852599 5 connected
#原7002上的数重定向到7005新的master上
192.168.0.222:7000> get one
-> Redirected to slot [9084] located at 192.168.0.222:7005
"AAAAAA"
#重启7002，做为7005的slave节点
[root@MySQLNODE04 redis-3.2.3]# ./src/redis-server  redis_7002.conf
192.168.0.222:7005> cluster nodes
e40e2cc28b3ab2f4da546a2249b0bf515f998911 192.168.0.224:7003 slave 60b53e792f10a1fab087e73713180aa56554858a 0 1504064034546 5 connected
91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 192.168.0.222:7005 myself,master - 0 0 7 connected 5461-10922
3ef887b183685eb4687085bea40352af018f07ea 192.168.0.222:7001 master - 0 1504064031480 2 connected 10923-16383
e154234f624b5f4f44cd3dc029b87ecabb29e564 192.168.0.224:7002 slave 91ff4893725aa7f44dfa935d3df6dfb41e43ec9e 0 1504064035568 7 connected
db9f8e823febb67fe25a1bc150283f598bce3dd4 192.168.0.224:7004 slave 3ef887b183685eb4687085bea40352af018f07ea 0 1504064036591 6 connected
60b53e792f10a1fab087e73713180aa56554858a 192.168.0.222:7000 master - 0 1504064037615 1 connected 0-5460
#当其中一个节点挂掉，那么久没法经过该节点端口链接到redis集群
[root@MySQLNODE02 redis-3.2.3]# kill -9 111838
[root@MySQLNODE02 redis-3.2.3]# ps -ef | grep redis
root     111844      1  0 10:31 ?        00:00:25 ./src/redis-server 192.168.0.222:7001 [cluster]
root     112663      1  0 11:26 ?        00:00:17 ./src/redis-server 192.168.0.222:7005 [cluster]
root     114158  70424  0 13:22 pts/1    00:00:00 grep --color=auto redis
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.222 -p 7000
Could not connect to Redis at 192.168.0.222:7000: Connection refused
Could not connect to Redis at 192.168.0.222:7000: Connection refused
not connected> exit
[root@MySQLNODE02 redis-3.2.3]# ./src/redis-cli -c  -h 192.168.0.224 -p 7003
192.168.0.224:7003> get two
"BBBBB"

多键的命令操做（如MGET、MSET），若是每一个键都位于同一个节点，则能够正常支持，不然会提示错误。

集群中的节点只能使用0号数据库，若是执行SELECT切换数据库会提示错误。

Codis

Codis 是一个分布式 Redis 解决方案, 对于上层的应用来讲, 链接到 CodisProxy 和链接原生的 Redis Server 没有明显的区别 (不支持的命令列表), 上层应用能够像使用单机的 redis 同样使用, Codis 底层会处理请求的转发, 不停机的数据迁移等工做, 全部后边的一切事情, 对于前面的客户端来讲是透明的, 能够简单的认为后边链接的是一个内存无限大的 Redis 服务。

Codis 由四部分组成:

Codis Proxy (codis-proxy) 实现redis协议，因为自己是无状态的，所以能够部署不少个节点
Codis Manager (codis-config) 是codis的管理工具，包括添加/删除redis节点添加删除proxy节点，发起数据迁移等操做，自带httpserver，支持管理后台方式管理配置
Codis Redis (codis-server) 是codis维护的redis分支，基于2.8.21分支，加入了slot的支持和原子的数据迁移命令;codis-proxy和 codis-config只能和这个版本的redis交互才能正常运行
ZooKeeper 用于codis集群元数据的存储,维护codis集群节点

Codis优缺点

优势

对客户端透明，与codis交互方式和redis自己交互同样
支持在线数据迁移，迁移过程对客户端透明
有简单的管理和监控界面
支持高可用，不管是redis数据存储仍是代理节点
自动进行数据的均衡分配
最大支持1024个redis实例，存储容量海量
高性能

缺点

采用自有的redis分支，不能与原版的redis保持同࠵
若是codis的proxy只有一个的状况下，redis的性能会降低20%左右
某些命令不支持，好比事务命令muti
国内开源产品，活跃度相对弱一些

Codis架构

参考资料：

http://blog.csdn.net/a67474506/article/details/51418728