redis集群(Cluster)模式部署
1、Redis 集群模式搭建
node
redis集群部署分为三种：主从、哨兵(Sentinel)、集群(Cluster)。
web
前几天因为新应用上线，须要部署一套redis集群，现造成文档，将部署过程进行记录。redis
环境介绍ruby
| ip |
软件包 |
系统版本 |
| 10.1.112.248 | redis-4.0.11.tar.gz | CentOS Linux release 7.6.1810 (Core) |
| 10.1.112.249 | redis-4.0.11.tar.gz | CentOS Linux release 7.6.1810 (Core) |
一、目录结构bash
[root@localhost local]# pwd /usr/local [root@localhost local]# tree redis-cluster/ redis-cluster/ ├── redis-4.0.11-node1 │ ├── data │ └── logs ├── redis-4.0.11-node2 │ ├── data │ └── logs └── redis-4.0.11-node3 ├── data └── logs 9 directories, 0 files
二、下载解压redis-4.0.11
服务器
mkdir /usr/local/software cd /usr/local/software && wget http://download.redis.io/releases/redis-4.0.11.tar.gz tar xf redis-4.0.11.tar.gz
三、安装(两台服务器均安装)app
[root@localhost software]# pwd /usr/local/software [root@localhost software]# tar xf redis-4.0.11.tar.gz [root@localhost software]# cd redis-4.0.1 [root@localhost redis-4.0.11]# make install 出现以下截图便可
四、准备配置文件less
[root@localhost redis-4.0.11]# cp src/redis-trib.rb /usr/local/bin/
[root@localhost redis-4.0.11]# cp redis.conf /usr/local/redis-cluster/redis-4.0.11-node1/
[root@localhost redis-4.0.11]# cp redis.conf /usr/local/redis-cluster/redis-4.0.11-node2/
[root@localhost redis-4.0.11]# cp redis.conf /usr/local/redis-cluster/redis-4.0.11-node3/
[root@localhost redis-4.0.11]# cp src/{redis-cli,redis-server} /usr/local/redis-cluster/redis-4.0.11-node1/
[root@localhost redis-4.0.11]# cp src/{redis-cli,redis-server} /usr/local/redis-cluster/redis-4.0.11-node2/
[root@localhost redis-4.0.11]# cp src/{redis-cli,redis-server} /usr/local/redis-cluster/redis-4.0.11-node3/
[root@localhost redis-4.0.11-node1]# grep -Ev "^#|^$" redis.conf
bind 10.1.112.248 #领一台服务器配置为10.1.112.249
protected-mode yes
port 7000 #其余节点分别配置为7001/7002
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /usr/local/redis-cluster/redis-4.0.11-node1/redis_7000.pid #其余节点配置为redis-4.0.11-node2/redis_7001.pid、redis-4.0.11-node3/redis_7002.pid
loglevel notice
logfile "/usr/local/redis-cluster/redis-4.0.11-node1/logs/redis_7000.log" #同上
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /usr/local/redis-cluster/redis-4.0.11-node1/data #redis-4.0.11-node3/data、redis-4.0.11-node3/data
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
requirepass xxx
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 15000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
五、分别启动3个节点(另外一台服务器一样启动)
curl
cd /usr/local/redis-cluster/redis-4.0.11-node1 ./redis.server redis.conf cd /usr/local/redis-cluster/redis-4.0.11-node2 ./redis.server redis.conf cd /usr/local/redis-cluster/redis-4.0.11-node3 ./redis.server redis.conf 查看进程 [root@localhost redis-4.0.11-node1]# ps -ef |grep redis root 6015 5673 0 17:52 pts/1 00:00:00 grep --color=auto redis weblogic 28246 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.248:7000 [cluster] weblogic 28284 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.248:7001 [cluster] weblogic 28301 1 0 Oct21 ? 00:02:25 ./redis-server 10.1.112.248:7002 [cluster] [root@localhost redis-4.0.11-node1]# ps -ef |grep redis root 6015 5673 0 17:52 pts/1 00:00:00 grep --color=auto redis weblogic 28246 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.249:7000 [cluster] weblogic 28284 1 0 Oct21 ? 00:01:14 ./redis-server 10.1.112.249:7001 [cluster] weblogic 28301 1 0 Oct21 ? 00:02:25 ./redis-server 10.1.112.249:7002 [cluster]
六、使用 redis-trib.rb建立集群tcp
[root@localhost redis-4.0.11-node1]# yum -y install ruby ruby-devel rubygems rpm-build [root@localhost redis-4.0.11-node1]# gem install redis Fetching: redis-4.1.3.gem (100%) ERROR: Error installing redis: redis requires Ruby version >= 2.3.0. 提示ruby版本太低。 升级ruby版本 [root@localhost redis-4.0.11-node1]# curl -L get.rvm.io | bash -s stable 按提示导入秘钥，再次执行"curl -L get.rvm.io | bash -s stable" [root@localhost redis-4.0.11-node1]# source /usr/local/rvm/scripts/rvm 查看可用的ruby版本并安装 [root@localhost redis-4.0.11-node1]# rvm list known [root@localhost redis-4.0.11-node1]# rvm install 2.3.0 #这一步用时较长，耐心等待安装完成。
七、安装gem redis接口
rvm use 2.3.0 ruby --version gem install redis
八、安装rubygems
yum install -y rubygems
至此，Ruby和运行redis-trib.rb须要的环境安装完成了，接下来使用redis-trib.rb建立集群
九、建立集群
redis-trib.rb create --replicas 1 10.1.112.248:7000 10.1.112.248:7001 10.1.112.248:7002 10.1.112.249:7000 10.1.112.249:7001 10.1.112.249:7002
十、查看集群状态
[weblogic@geb-redis1 ~]$ redis-trib.rb check 10.1.112.248:7000 >>> Performing Cluster Check (using node 10.1.112.248:7000) M: 61d94b047e12679886991c7c6d225b471374a018 10.1.112.248:7000 slots:0-5460 (5461 slots) master 1 additional replica(s) S: a582b9d7784d2f1b2e6a20d5352a19062dc29d4a 10.1.112.248:7002 slots: (0 slots) slave replicates 0addc56bce940a659aa2ac4aa6d51677c1f2c523 M: 0addc56bce940a659aa2ac4aa6d51677c1f2c523 10.1.112.249:7000 slots:5461-10922 (5462 slots) master 1 additional replica(s) S: 087de17dafa03c361d87f7a4c4a5b2d7ce0555fc 10.1.112.249:7001 slots: (0 slots) slave replicates fd9f572038a7504e088dde150b89beb2bac39e3c M: fd9f572038a7504e088dde150b89beb2bac39e3c 10.1.112.248:7001 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: 407d6ea2eb3725e142d2e5ae4e1eef4eee1a48ca 10.1.112.249:7002 slots: (0 slots) slave replicates 61d94b047e12679886991c7c6d225b471374a018 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
十一、总体目录结构
[root@geb-redis1 redis-cluster]# tree . ├── redis-4.0.11-node1 │ ├── data │ │ ├── appendonly.aof │ │ └── nodes-7000.conf │ ├── logs │ │ └── redis_7000.log │ ├── redis_7000.pid │ ├── redis-cli │ ├── redis.conf │ └── redis-server ├── redis-4.0.11-node2 │ ├── data │ │ ├── appendonly.aof │ │ └── nodes-7001.conf │ ├── logs │ │ └── redis_7001.log │ ├── redis_7001.pid │ ├── redis-cli │ ├── redis.conf │ └── redis-server └── redis-4.0.11-node3 ├── data │ ├── appendonly.aof │ └── nodes-7002.conf ├── logs │ └── redis_7002.log ├── redis_7002.pid ├── redis-cli ├── redis.conf └── redis-server 9 directories, 21 files [root@geb-redis2 redis-cluster]# tree . ├── redis-4.0.11-node1 │ ├── data │ │ ├── appendonly.aof │ │ └── nodes-7000.conf │ ├── logs │ │ └── redis_7000.log │ ├── redis_7000.pid │ ├── redis-cli │ ├── redis.conf │ └── redis-server ├── redis-4.0.11-node2 │ ├── data │ │ ├── appendonly.aof │ │ └── nodes-7001.conf │ ├── logs │ │ └── redis_7001.log │ ├── redis_7001.pid │ ├── redis-cli │ ├── redis.conf │ └── redis-server └── redis-4.0.11-node3 ├── data │ ├── appendonly.aof │ └── nodes-7002.conf ├── logs │ └── redis_7002.log ├── redis_7002.pid ├── redis-cli ├── redis.conf └── redis-server 9 directories, 21 files
十二、部署过程当中须要调整的系统参数
# echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf # echo "net.core.somaxconn = 2048" >> /etc/sysctl.conf # cat /etc/rc.local if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never > /sys/kernel/mm/transparent_hugepage/defrag fi