redis高可用方案之sentinel(哨兵集群）

时间 2019-12-18

原文原文链接

今天给你们介绍下redis高可用方案，redis自带的sentinel,简单的口语描述下什么是sentinel（哨兵集群），哨兵集群可以将redis主从封装起来，实现redis主宕机，redis备将接管主，再次恢复的时候自动升级为从。网上有不少介绍redis+vip的方法，这个很简单就是当主宕机以后，实现vip的漂移。这种方法确实是能够的，可是没有办法实现主从资源利用最大化，我要给你们介绍的是如何使用sentinel+redis+python(宕机报警之后在添加)实现redis高可用以及redis读写分离。node

1.安装redis(我这边使用了2个，生产上建议是用3个)
具体安装redis就不介绍了。我这是使用的是ansible安装
python

我这里安装的是2个，192.168.7.30准备用来的作主redis用来写数据，192.168.7.31是从用来读数据。redis

2.配置redis
由于使用的是ansible，因此配置文件是默认配置好的，不用作修改，能够直接启动就好了。vim

172.16.10.30主上的配置app

[root@redis-master etc]# cat redis_16399.conf 
bind 172.16.10.30
protected-mode yes
port 16399
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /usr/local/redis/run/redis_16399.pid
loglevel notice
logfile "/usr/local/redis/logs/redis_16399.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /data/redis/16399
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
[root@redis-master etc]# /etc/init.d/redis_16399 start 
Starting Redis server...
[root@redis-master etc]# netstat -nltp |grep 16399 
tcp        0      0 172.16.10.30:16399          0.0.0.0:*                   LISTEN      5521/redis-server 1 
[root@redis-master etc]#

172.16.10.31上的配置less

[root@master-slave etc]# grep -rn slaveof redis_16399.conf  
16:slaveof 172.16.10.30 16399   #从配置文件比主多这行
[root@master-slave etc]# /etc/init.d/redis_16399 start 
Starting Redis server...
[root@master-slave etc]# netstat -nltpa |grep 16399 
tcp        0      0 172.16.10.31:16399          0.0.0.0:*                   LISTEN      10466/redis-server          
tcp        0      0 172.16.10.31:18936          172.16.10.30:16399          ESTABLISHED 10466/redis-server                     
[root@master-slave etc]#

从上面就能够看出主从已经配置完成。
3.配置sentinel
sentinel的配置文件全部的节点基本是同样的，能够部署在不是redis的节点上，进行更加全面的监控socket

[root@redis-master etc]# cat sentinel_5001.conf  
port 5001
pidfile "/usr/local/redis/run/sentinel_5001.pid"
loglevel notice
logfile "/usr/local/redis/logs/sentinel_5001.log"
daemonize yes
protected-mode no
sentinel monitor mymaster 172.16.10.30 16399 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000

[root@redis-master etc]#

4.启动sentinel
172.16.10.30启动sentineltcp

[root@master-slave etc]# cat sentnel_5001.conf 
port 5001
pidfile "/usr/local/redis/run/sentinel_5001.pid"
loglevel notice
logfile "/usr/local/redis/logs/sentinel_5001.log"
daemonize yes
protected-mode no
sentinel monitor mymaster 172.16.10.30 16399 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000

[root@master-slave etc]# 
[root@redis-master etc]# /usr/local/redis/bin/redis-sentinel /usr/local/redis/etc/sentinel_5001.conf  
[root@redis-master etc]# tail -30f /usr/local/redis/logs/sentinel_5001.log 
5704:X 08 Dec 22:46:44.977 # Not listening to IPv6: unsupproted
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.11 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 5001
 |    `-._   `._    /     _.-'    |     PID: 5704
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

5704:X 08 Dec 22:46:44.986 # Sentinel ID is 453837ef09fc1a41332df32487882f2e42233762
5704:X 08 Dec 22:46:44.986 # +monitor master mymaster 172.16.10.30 16399 quorum 2
5704:X 08 Dec 22:46:44.987 * +slave slave 172.16.10.31:16399 172.16.10.31 16399 @ mymaster 172.16.10.30 16399

到这里原本就已经监控起来了，上面咱们配置只有2个节点的时候才会发生切换，咱们如今将的172.16.10.31上的sentinel启动起来。ide

[root@master-slave etc]# vim sentnel_5001.conf 
[root@master-slave etc]# /usr/local/redis/bin/redis-sentinel /usr/local/redis/etc/sentnel_5001.conf 
[root@master-slave etc]# tail -30f /usr/local/redis/logs/sentinel_5001.log 
10615:X 08 Dec 22:51:05.232 # Not listening to IPv6: unsupproted
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.11 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 5001
 |    `-._   `._    /     _.-'    |     PID: 10615
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

10615:X 08 Dec 22:51:05.234 # Sentinel ID is 0ac7df95b3e057a0ccb4a10f1cc8cc91e071526e
10615:X 08 Dec 22:51:05.234 # +monitor master mymaster 172.16.10.30 16399 quorum 2
10615:X 08 Dec 22:51:05.235 * +slave slave 172.16.10.31:16399 172.16.10.31 16399 @ mymaster 172.16.10.30 16399
10615:X 08 Dec 22:51:06.697 * +sentinel sentinel 453837ef09fc1a41332df32487882f2e42233762 172.16.10.30 5001 @ mymaster 172.16.10.30 16399

到这里咱们已经完成了sentinel集群的搭建
5.测试
测试主从
测试

测试sentinel
停掉 172.16.10.30的redis

[root@redis-master init.d]# /etc/init.d/redis_16399 stop 
Stopping ...
Redis stopped

sentinel fallover日志

10615:X 08 Dec 22:52:08.497 # +sdown master mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:08.553 # +odown master mymaster 172.16.10.30 16399 #quorum 2/2
10615:X 08 Dec 22:52:08.553 # +new-epoch 1
10615:X 08 Dec 22:52:08.553 # +try-failover master mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:08.554 # +vote-for-leader 0ac7df95b3e057a0ccb4a10f1cc8cc91e071526e 1
10615:X 08 Dec 22:52:08.556 # 453837ef09fc1a41332df32487882f2e42233762 voted for 0ac7df95b3e057a0ccb4a10f1cc8cc91e071526e 1
10615:X 08 Dec 22:52:08.655 # +elected-leader master mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:08.655 # +failover-state-select-slave master mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:08.710 # +selected-slave slave 172.16.10.31:16399 172.16.10.31 16399 @ mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:08.710 * +failover-state-send-slaveof-noone slave 172.16.10.31:16399 172.16.10.31 16399 @ mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:08.782 * +failover-state-wait-promotion slave 172.16.10.31:16399 172.16.10.31 16399 @ mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:09.715 # +promoted-slave slave 172.16.10.31:16399 172.16.10.31 16399 @ mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:09.715 # +failover-state-reconf-slaves master mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:09.785 # +failover-end master mymaster 172.16.10.30 16399
10615:X 08 Dec 22:52:09.785 # +switch-master mymaster 172.16.10.30 16399 172.16.10.31 16399
10615:X 08 Dec 22:52:09.785 * +slave slave 172.16.10.30:16399 172.16.10.30 16399 @ mymaster 172.16.10.31 16399
10615:X 08 Dec 22:52:14.833 # +sdown slave 172.16.10.30:16399 172.16.10.30 16399 @ mymaster 172.16.10.31 16399

172.16.10.30恢复日志

10615:X 08 Dec 22:56:50.802 # -sdown slave 172.16.10.30:16399 172.16.10.30 16399 @ mymaster 172.16.10.31 16399
10615:X 08 Dec 22:57:00.751 * +convert-to-slave slave 172.16.10.30:16399 172.16.10.30 16399 @ mymaster 172.16.10.31 16399

6.python client 实现读写分离。

[root@cmdb redis-sentinel]# tree 
.
├── eg.py
└── redis_cluster
    ├── __init__.py
    ├── __init__.pyc
    ├── RedisSentinel.py
    ├── RedisSentinel.pyc
    ├── settings.py
    └── settings.pyc

1 directory, 7 files
[root@cmdb redis-sentinel]#

settings.py是配置文件

[root@cmdb redis-sentinel]# cat redis_cluster/settings.py
#default timeout
SOCKET_TIMEOUT = 0.1

#sentinel cluster nodes list
SENTINEL_NODES = [('192.168.7.30',5001),('192.168.7.31',5001)]

#sentinel cluster name
CLUSTER_NAME = "mymaster"

#default select db 
CLUSTER_DB= 15 
[root@cmdb redis-sentinel]#

RedisSentinel.py是处理方法

[root@cmdb redis-sentinel]# cat redis_cluster/RedisSentinel.py
#!/usr/bin/env python 
#encoding:utf-8
#
from redis.sentinel import Sentinel
from settings import SENTINEL_NODES,SOCKET_TIMEOUT,CLUSTER_NAME,CLUSTER_DB

class RedisSentinelClient:
    def __init__(self,sentinel_nodes,socket_timeout,cluster_name,db):
        self.sentinel=Sentinel(sentinel_nodes,socket_timeout=socket_timeout)
        self.master_name=cluster_name
        self.socket_timeout=socket_timeout
        self.db=db
    def master(self,socket_timeout=None):
        return self.sentinel.master_for(self.master_name,db=self.db,socket_timeout=self.socket_timeout)
    def slave(self,socket_timeout=None):
        return self.sentinel.slave_for(self.master_name,db=self.db,socket_timeout=self.socket_timeout)
sentinel_client=RedisSentinelClient(SENTINEL_NODES,SOCKET_TIMEOUT,CLUSTER_NAME,CLUSTER_DB)
master_redis=sentinel_client.master()
slave_redis=sentinel_client.slave()
[root@cmdb redis-sentinel]#

eg.py是使用的方法：

[root@cmdb redis-sentinel]# cat eg.py 
#!/usr/bin/env python 
#encoding:utf-8
from redis_cluster.RedisSentinel import master_redis,slave_redis
master_redis.set('ceshi','nihaoma')   #写入数据
print slave_redis.get('ceshi')  #读取数据的。此处读取数据是从库上读取的
[root@cmdb redis-sentinel]#

到此就实现了python对 sentinel哨兵集群的读写分离操做