Redis分布式锁，集群搭建（哨兵模式，cluster）

时间  2019-11-18
标签 redis 分布式 集群 搭建 哨兵 模式 cluster 栏目 Redis 繁體版
原文   原文链接

分布式锁

  1. 什么是分布式锁？java

    1）分布式锁是控制分布式系统中或不一样系统之间共同访问共享资源的一种锁实现。
2）若是不一样系统或同一系统的不一样主机之间共享了某个资源时，每每经过互斥来防止彼此之间的干扰。
3）不会发生死锁，即便一个server在持有锁时间出现问题没能主动解锁，也能保证后续其余server能正常加锁。

  2. 分布式锁的目的？node

    能够保证分布式部署的应用集群中，同一个资源在同一时刻只能被一台机器上的一个线程执行。

  3. Redis分布式锁可能出现的问题？redis

    在setnx和setex中间发生了服务宕机
解决方案：1）该问题须要setnx和setex连用，能够采用lua脚本
     2）Redis从2.6以后支持setnx、setex连用

当server1执行任务时间大于setex设置的过时时间时，可能在解锁时解了server2的锁
解决方案：每一个server解锁时须要判断当前要remove掉的锁的value是否是该server的value，再去解锁，可采用lua脚本作解锁流程

主从节点搭建

  1. 在redis目录下建立redis-replication目录spring

    mkdir redis-replication

  2. 在redis-replication目录下建立目录6380 6381centos

    mkdir 6380
mkdir 6381

  3. 将src目录下的redis-server拷贝到redis-replication目录下springboot

    [root@bogon redis-replication]# cp ../src/redis-server redis-server

  4. 将redis.conf分别拷贝到6380 6381,目录下ruby

    [root@bogon redis-replication]# cp ../redis.conf 6380
[root@bogon redis-replication]# cp ../redis.conf 6381

  5. 修改6380和6381中redis.conf的配置bash

    daemonize yes #守护进程模式开启
port 6380
slaveof 127.0.0.1 6379 #模拟环境三个redis都在同一台机器上
    daemonize yes #守护进程模式开启
port 6381
slaveof 127.0.0.1 6379 #模拟环境三个redis都在同一台机器上

  6. 分别启动三个节点服务服务器

    [root@bogon redis-replication]# ./redis-server ../redis.conf
[root@bogon redis-replication]# ./redis-server 6380/redis.conf
[root@bogon redis-replication]# ./redis-server 6381/redis.conf

  7. 在6379节点redis-cli查看info，并set，而后分别在两个从节点get，get到即集群搭建完毕app

    [root@bogon /]# redis-cli
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6380,state=online,offset=1639,lag=0
slave1:ip=127.0.0.1,port=6381,state=online,offset=1639,lag=0
master_replid:5f13eaede714511b727b6058ed7299c317c66d8b
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1639
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:1639
127.0.0.1:6379>

    主从复制的原理

    **全量复制**
 实现原理：创建主从关系时，从机会给主机发送sync命令，主机接收命令，后台启动的存盘进程，同时收集全部用于修改命令，传送给从机。
 **增量复制**
 实现原理：主机会继续将新收集到的修改命令依次传给从机，实现数据的同步效果

    主从复制的缺点

    Redis的主从复制最大的缺点就是延迟，主机负责写，从机负责备份，这个过程有必定的延迟，当系统很繁忙的时候，延迟问题会更加严重，从机器数量的增长也会使这个问题更加严重

    故障转移

  8. 将redis-5.0.5-1目录下sentinel.conf拷贝到redis-replication

    [root@bogon redis-5.0.5-1]# cp sentinel.conf redis-replication/

  9. 将sentinel.conf重命名为sentinel_1.conf并修改配置

    sentinel monitor mymaster 127.0.0.1 6379 1   #指定master节点，1表明1个sentinel认为某节点挂掉就挂掉了
sentinel down-after-milliseconds mymaster 10000   #指定sentinel认为服务器断线所需毫秒数
sentinel failover-timeout mymaster 60000  #执行故障转移确认时间毫秒数
sentinel parallel-syncs mymaster 1  #执行故障转移时，最大能够有多少个从服务器同时对新的主服务器进行同步，这个数字越小，完成故障转移所需时间就越长

  10. 保存后启动redis-sentinel

    [root@bogon redis-5.0.5-1]# ./src/redis-sentinel redis-replication/sentinel_1.conf

19559:X 04 Nov 2019 18:45:56.540 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
19559:X 04 Nov 2019 18:45:56.540 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=19559, just started
19559:X 04 Nov 2019 18:45:56.540 # Configuration loaded
19559:X 04 Nov 2019 18:45:56.541 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.5 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 19559
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

19559:X 04 Nov 2019 18:45:56.542 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
19559:X 04 Nov 2019 18:45:56.543 # Sentinel ID is a3bbbce34899d8534d98f7328ba9d61857a94168
19559:X 04 Nov 2019 18:45:56.543 # +monitor master mymaster 127.0.0.1 6379 quorum 1
19559:X 04 Nov 2019 18:45:56.545 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:45:56.545 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379

  11. 而后shutdown主节点6379，能够看到下面日志，将6381选举为了新的主节点，当6379再次启动时，会做为6381的从节点

    ./redis-cli -p 6379 shutdown

19559:X 04 Nov 2019 18:47:24.078 # +sdown master mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:24.078 # +odown master mymaster 127.0.0.1 6379 #quorum 1/1
19559:X 04 Nov 2019 18:47:24.078 # +new-epoch 1
19559:X 04 Nov 2019 18:47:24.078 # +try-failover master mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:24.079 # +vote-for-leader a3bbbce34899d8534d98f7328ba9d61857a94168 1
19559:X 04 Nov 2019 18:47:24.079 # +elected-leader master mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:24.079 # +failover-state-select-slave master mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:24.133 # +selected-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:24.133 * +failover-state-send-slaveof-noone slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:24.191 * +failover-state-wait-promotion slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:25.120 # +promoted-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:25.120 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:25.172 * +slave-reconf-sent slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:26.128 * +slave-reconf-inprog slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:26.128 * +slave-reconf-done slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:26.222 # +failover-end master mymaster 127.0.0.1 6379
19559:X 04 Nov 2019 18:47:26.222 # +switch-master mymaster 127.0.0.1 6379 127.0.0.1 6381
19559:X 04 Nov 2019 18:47:26.222 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381
19559:X 04 Nov 2019 18:47:26.222 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381

高可用Sentinel故障转移原理

  1. Sentinel是如何工做的呢？

    主观下线：单个Sentinel实例对服务器作出下线判断
主观下线特色：若是一个服务器没有在master-down-after-milliseconds所指时间内，对发送PING命令的Sentinel获得一个有效回复，那么将会标记这个服务器为主观下线。

客观下线：多个Sentinel投票对一个服务作出判断。

客观下线条件只适用于主节点，主观下线适用于从节点。

SpringBoot整合Sentinel

  1. 直接在yml文件加入配置便可

    spring:
  redis:
    sentinel:
      master: mymaster
      nodes: 192.168.47.129:26379 #多个sentinel以逗号分隔

  2. 启动后set

    redisTemplate.opsForValue().set(key, value);
return redisTemplate.opsForValue().get(key);

  3. 可能会遇到问题

    io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /127.0.0.1:6379
 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
 at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
 at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:327)
 at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:670)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:617)
 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:534)
 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
 at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:906)
 at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
 at java.lang.Thread.run(Thread.java:748)

  4. 致使问题的缘由是Sentinel.conf中的配置

    sentinel monitor mymaster 127.0.0.1 6379 1
改为
sentinel monitor mymaster 192.168.47.129 6379 1

  5. 问题解决

集群搭建（三主三从）

  1. 在local目录下建立redis-cluster文件夹，在redis-cluster下建6个节点分别拷入redis.conf

    cd /usr/local/
mkdir redis_cluster
cd redis_cluster
mkdir 7000 7001 7002 7003 7004 7005
cp /usr/local/redis/redis.conf  /usr/local/redis/7000

  2. 修改6个节点的redis.conf

    daemonize    yes                     //redis后台运行
port  7000                           //端口7000,7001,7002,7003,7004,7005
cluster-enabled  yes                 //开启集群  把注释#去掉
cluster-config-file  nodes-xxx.conf  //集群的配置  配置文件首次启动自动生成
cluster-node-timeout  5000           //请求超时  设置5秒够了
appendonly  yes                      //aof日志开启  有须要就开启，它会每次写操做都记录一条日志
bind 127.0.0.1 192.168.80.129(此处为本身内网的ip地址，centos7下面采用ip addr来查看，其余系统试一下ifconfig查看)

  3. 启动全部节点

    cd /usr/local/redis_cluster/7000 ../redis-server ./redis.conf

cd /usr/local/redis-cluster/7001 ../redis-server ./redis.conf

cd /usr/local/redis-cluster/7002 ../redis-server ./redis.conf

cd /usr/local/redis-cluster/7003 ../redis-server ./redis.conf

cd /usr/local/redis-cluster/7004 ../redis-server ./redis.conf

cd /usr/local/redis-cluster/7005 ../redis-server ./redis.conf

  4. 注意：Redis 5.X前的版本安装ruby。Redis 5.X版本请看下面第8点

    yum -y install ruby ruby-devel rubygems rpm-build

gem install redis

  5. gem install redis可能会遇到以下错误

    ERROR: Error installing redis:

redis requires Ruby version >= 2.3.0.

CentOS7 库中的支持到2.0.0，可ruby安装Redis（我用的redis版本是5.0.5）的须要最低是2.3.0

  6. 解决办法

    # 下载
curl -L get.rvm.io | bash -s stable

cd /usr/local/rvm/archives

#解压
tar xvzf rvm-1.29.9.tgz

cd rvm-1.29.9

./install

source /usr/local/rvm/scripts/rvm

#查看全部版本
rvm list known

#安装2.3.3
rvm install 2.3.3

rvm use 2.3.3

ruby --version

gem install redis

  7. Redis 5.X前的版本，进入redis的src目录，执行以下命令

    ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

  8. Redis 5.X版本，进入redis的src目录，执行以下命令

    ./redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1

  9. 出现下面信息表示启动成功

    >>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7004 to 127.0.0.1:7000
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7003 to 127.0.0.1:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
M: 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
M: 815fe658394702b8e38498918510aa9ea75ff73d 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
S: 55784ce9df135a51efeba908643018b13e33d11a 127.0.0.1:7003
   replicates ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba
S: b630b0dae92977f1447623ad34981a1086bfdbfa 127.0.0.1:7004
   replicates 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc
S: 7a383e5d8a3b1c22c7b47c6ce9f92f820f44f9d5 127.0.0.1:7005
   replicates 815fe658394702b8e38498918510aa9ea75ff73d
Can I set the above configuration? (type 'yes' to accept): yes

  10. Can I set the above configuration? (type 'yes' to accept): yes (是否这样配置？ 是)

    >>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba 127.0.0.1:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 7a383e5d8a3b1c22c7b47c6ce9f92f820f44f9d5 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 815fe658394702b8e38498918510aa9ea75ff73d
M: 815fe658394702b8e38498918510aa9ea75ff73d 127.0.0.1:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: b630b0dae92977f1447623ad34981a1086bfdbfa 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc
M: 842bbbcbfd9c7b98067a8cf8cff09cb54817c8cc 127.0.0.1:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 55784ce9df135a51efeba908643018b13e33d11a 127.0.0.1:7003
   slots: (0 slots) slave
   replicates ebb3d01bd2578d5b38400f5330ba7c9a5bdaefba
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

  11. 测试，redis-cli -c -p 7002，-c 表明链接集群结点，以下结果说明没问题

    [root@bogon src]# redis-cli -c -p 7002
127.0.0.1:7002> set w x
-> Redirected to slot [3696] located at 127.0.0.1:7000
OK
127.0.0.1:7000> get w
"x"
[root@bogon src]# redis-cli -c -p 7003
127.0.0.1:7003> get w
-> Redirected to slot [3696] located at 127.0.0.1:7000
"x"
127.0.0.1:7000>
相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程