一:phpredisadmin工具:相似于mysqladmin管理mysql同样php
安装环境:html
[root@node5 ~]# yum install httpd php php-redis -y
下载管理包:前端
[root@node5 html]# git clone https://github.com/erikdubbelboer/phpRedisAdmin.git
更更名称并解决依赖:java
[root@node5 html]# mv phpRedisAdmin admin [root@node5 html]# cd admin [root@node5 admin]# git clone https://github.com/nrk/predis.git vendor #解决依赖包
启动http,启动前确认80端口没有被占用,若是被占用能够修改监听的默认端口为80觉得的其余端口:node
[root@node5 html]# /etc/init.d/httpd restart Stopping httpd: [FAILED] Starting httpd: [ OK ]
[root@node5 html]# chkconfig httpd on
访问http://ServerIP/admin:python
报错,没法打开,查看apache错误日志以下:mysql
[Mon Mar 21 21:24:47 2016] [error] [client 192.168.10.1] PHP Fatal error: require(): Failed opening required '/var/www/html/admin/includes/../vendor/autoload.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/www/html/admin/includes/common.inc.php on line 2
解决过程:linux
[root@node5 admin]# cd includes/ [root@node5 includes]# ls config.sample.inc.php config.inc.php [root@node5 includes]# vim config.inc.php $config = array( 'servers' => array( array( 'name' => 'local server', // Optional name. 'host' => '192.168.10.205', #配置为本身的主机IP 'port' => 6379, 'filter' => '*', // Optional Redis authentication. //'auth' => 'redispasswordhere' // Warning: The password is sent in plain-text to the Redis server. ),
保存退出后从新启动httpd:
[root@node5 includes]# /etc/init.d/httpd restart
Stopping httpd: [ OK ]
Starting httpd: [ OK ]git
再次测试仍是不能访问,报错以下:github
[Mon Mar 21 21:55:27 2016] [error] [client 192.168.10.1] PHP Fatal error: Call to undefined function mb_internal_encoding() in /var/www/html/admin/includes/common.inc.php on line 59
解决办法:
[root@node5 admin]# vim /var/www/html/admin/includes/common.inc.php mb_internal_encoding('utf-8'); 将此行改成注释,以下: #mb_internal_encoding('utf-8');
访问:
配置管理多个reids Server:
[root@node5 includes]# pwd /var/www/html/admin/includes [root@node5 includes]# vim config.inc.php $config = array( 'servers' => array( array( 'name' => '182.168.10.205', // Optional name. 'host' => '192.168.10.205', 'port' => 6379, 'filter' => '*', // Optional Redis authentication. //'auth' => 'redispasswordhere' // Warning: The password is sent in plain-text to the Redis server. ), array( #一个主机一个配置,名称自定义,IP和端口是Redis的IP和端口 'name' => '7001', // Optional name.name支持中文 'host' => '192.168.10.205', 'port' => 7001, ), array( 'name' => '7002', // Optional name. 'host' => '192.168.10.205', 'port' => 7002, ), array( 'name' => '7003', // Optional name. 'host' => '192.168.10.205', 'port' => 7003, ),
更改完成之后无需重启apache,若是浏览器已经打开管理地址,刷新一下便可:
二:rdbtools管理工具:
使用pip命令安装,若是没有pip命令则先安装pip:
[root@node5 ~]# yum install python-pip
使用pip安装rdbtools: pip install rdbtools
[root@node5 ~]# pip install rdbtools /usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning You are using pip version 7.1.0, however version 8.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. Collecting rdbtools /usr/lib/python2.6/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Downloading rdbtools-0.1.6.tar.gz Installing collected packages: rdbtools Running setup.py install for rdbtools Successfully installed rdbtools-0.1.6
导出rdb中的key信息:
[root@node5 7001]# rdb -c memory dump_7001.rdb > memory.csv #而后将memory.csv文件下载到本地,便可用excel打开分析了,格式以下图:
三:经过saltstack管理redis:
模块名称:redismod
官方文档地址:https://docs.saltstack.com/en/latest/ref/modules/all/index.html
四:经过codis完成redis管理:
1.codis简介:
Codis 是一个分布式 Redis 解决方案, 对于上层的应用来讲, 链接到 Codis Proxy 和链接原生的 Redis Server 没有明显的区别 (不支持的命令列表), 上层应用能够像使用单机的 Redis 同样使用, Codis 底层会处理请求的转发, 不停机的数据迁移等工做, 全部后边的一切事情, 对于前面的客户端来讲是透明的, 能够简单的认为后边链接的是一个内存无限大的 Redis 服务.其架构以下:
Codis 由四部分组成:
codis-proxy 是客户端链接的 Redis 代理服务, codis-proxy 自己实现了 Redis 协议, 表现得和一个原生的 Redis 没什么区别 (就像 Twemproxy), 对于一个业务来讲, 能够部署多个 codis-proxy, codis-proxy 自己是无状态的.
codis-config 是 Codis 的管理工具, 支持包括, 添加/删除 Redis 节点, 添加/删除 Proxy 节点, 发起数据迁移等操做. codis-config 自己还自带了一个 http server, 会启动一个 dashboard, 用户能够直接在浏览器上观察 Codis 集群的运行状态.
2.准备基础环境:
安装go语言并设置环境变量:
[root@node5 opt]# yum install golang -y #codis是基于go语言编写的,所以要安装go语言环境 [root@node5 opt]# mkdir /opt/gopath #保存go环境的路径 [root@node5 opt]# vim /etc/profile export GOPATH=/opt/gopath
[root@node5 opt]# source /etc/profile
安装配置zookeeper,须要有java环境:
[root@node5 ~]# wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz [root@node5 ~]# mv zookeeper-3.4.6 /usr/local/zookeeper #移动到目录 [root@node5 ~]# cd /usr/local/zookeeper/conf/ #进入到目录 [root@node5 conf]# cp zoo_sample.cfg /opt/zoo.cfg #复制配置文件 [root@node5 conf]# cd /opt/ [root@node5 opt]# mkdir zk1 zk2 zk3 #准备zookeeper 服务ID,每一个服务器的ID是不一样的 [root@node5 opt]# echo 1 > zk1/myid [root@node5 opt]# echo 2 > zk2/myid [root@node5 opt]# echo 3 > zk3/myid [root@node5 opt]# vim zoo.cfg #编辑配置文件 dataDir=/opt/zk1 #保存数据的命令 clientPort=2181 #客户端链接的端口 server.1 = 192.168.10.205:2887:3887 server.2 = 192.168.10.205:2888:3888 server.3 = 192.168.10.205:2889:3889 [root@node5 opt]# cp zoo.cfg zk1/zk1.cfg [root@node5 opt]# cp zoo.cfg zk2/zk2.cfg [root@node5 opt]# cp zoo.cfg zk3/zk3.cfg [root@node5 opt]# vim zk2/zk2.cfg #每一个服务对应不一样的目录 dataDir=/opt/zk2
clientPort=2182 [root@node5 opt]# vim zk3/zk3.cfg dataDir=/opt/zk3
clientPort=2183
详细配置:
[root@node5 opt]# grep '^[a-z]' zk3/zk3.cfg tickTime=2000 #服务器和客户端的心跳维持间隔,间隔多久发送心跳 initLimit=10 #选举的时候的时间间隔是10次,10次 * 2000微秒 syncLimit=5 # Leader 与 Follower 之间发送消息,请求和应答时间长度,最长不能超过多少个 tickTime 的时间长度 dataDir=/opt/zk3 #数据保存目录 clientPort=2181 #客户端链接的端口 server.1 = 192.168.10.205:2887:3887 #集群端口和ID配置 server.2 = 192.168.10.205:2888:3888 server.3 = 192.168.10.205:2889:3889 详细解释: tickTime:这个时间是做为 Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔,也就是每一个 tickTime 时间就会发送一个心跳。 dataDir:顾名思义就是 Zookeeper 保存数据的目录,默认状况下,Zookeeper 将写数据的日志文件也保存在这个目录里。 clientPort:这个端口就是客户端链接 Zookeeper 服务器的端口,Zookeeper 会监听这个端口,接受客户端的访问请求。 initLimit:这个配置项是用来配置 Zookeeper 接受客户端(这里所说的客户端不是用户链接 Zookeeper 服务器的客户端,而是 Zookeeper 服务器集群中链接到 Leader 的 Follower 服务器)初始化链接时最长能忍受多少个心跳时间间隔数。当已经超过 5个心跳的时间(也就是 tickTime)长度后 Zookeeper 服务器尚未收到客户端的返回信息,那么代表这个客户端链接失败。总的时间长度就是 5*2000=10 秒 syncLimit:这个配置项标识 Leader 与 Follower 之间发送消息,请求和应答时间长度,最长不能超过多少个 tickTime 的时间长度,总的时间长度就是 2*2000=4 秒 server.A=B:C:D:其中 A 是一个数字,表示这个是第几号服务器;B 是这个服务器的 ip 地址;C 表示的是这个服务器与集群中的 Leader 服务器交换信息的端口;D 表示的是万一集群中的 Leader 服务器挂了,须要一个端口来从新进行选举,选出一个新的 Leader,而这个端口就是用来执行选举时服务器相互通讯的端口。若是是伪集群的配置方式,因为 B 都是同样,因此不一样的 Zookeeper 实例通讯端口号不能同样,因此要给它们分配不一样的端口号。
启动每一个zookeeper 服务:
[root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk1/zk1.cfg JMX enabled by default Using config: /opt/zk1/zk1.cfg Starting zookeeper ... STARTED [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk2/zk2.cfg JMX enabled by default Using config: /opt/zk2/zk2.cfg Starting zookeeper ... STARTED [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh start /opt/zk3/zk3.cfg JMX enabled by default Using config: /opt/zk3/zk3.cfg Starting zookeeper ... STARTED
确认端口启动成功:
[root@node5 opt]# ss -tnl | grep 2181 LISTEN 0 50 :::2181 :::* [root@node5 opt]# ss -tnl | grep 2182 LISTEN 0 50 :::2182 :::* [root@node5 opt]# ss -tnl | grep 2183 LISTEN 0 50 :::2183 :::* [root@node5 opt]#
查看每一个节点的状态:
[root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk1/zk1.cfg JMX enabled by default Using config: /opt/zk1/zk1.cfg Mode: follower [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk2/zk2.cfg JMX enabled by default Using config: /opt/zk2/zk2.cfg Mode: leader #这是主的,另外两个是从的,表示已经启动完成 [root@node5 opt]# /usr/local/zookeeper/bin/zkServer.sh status /opt/zk3/zk3.cfg JMX enabled by default Using config: /opt/zk3/zk3.cfg Mode: follower
链接到zookeeper节点:
[root@node5 opt]# /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.205:2181
[root@node5 opt]# /usr/local/zookeeper/bin/zkCli.sh -server 192.168.10.205:2181Connecting to 192.168.10.205:2181 2016-03-22 01:27:48,325 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2016-03-22 01:27:48,331 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node5.a.com 2016-03-22 01:27:48,332 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.7.0_45 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper/bin/../build/classes:/usr/local/zookeeper/bin/../build/lib/*.jar:/usr/local/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/usr/local/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper/bin/../zookeeper-3.4.6.jar:/usr/local/zookeeper/bin/../src/java/lib/*.jar:/usr/local/zookeeper/bin/../conf: 2016-03-22 01:27:48,334 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2016-03-22 01:27:48,335 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2016-03-22 01:27:48,336 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/opt 2016-03-22 01:27:48,337 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=192.168.10.205:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@6ade299f Welcome to ZooKeeper! 2016-03-22 01:27:48,393 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@975] - Opening socket connection to server 192.168.10.205/192.168.10.205:2181. Will not attempt to authenticate using SASL (unknown error) 2016-03-22 01:27:48,424 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@852] - Socket connection established to 192.168.10.205/192.168.10.205:2181, initiating session JLine support is enabled 2016-03-22 01:27:48,621 [myid:] - INFO [main-SendThread(192.168.10.205:2181):ClientCnxn$SendThread@1235] - Session establishment complete on server 192.168.10.205/192.168.10.205:2181, sessionid = 0x1539a349fc40000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
进入到目录执行编译:
[root@node5 codis]# pwd /opt/gopath/src/github.com/CodisLabs/codis [root@node5 codis]# ls cmd docker genver.sh MIT-LICENSE.txt test config.ini Dockerfile Godeps pkg vitess_license doc extern Makefile README.md wandoujia_licese.txt [root@node5 codis]# make
[root@node5 codis]# make GO15VENDOREXPERIMENT=0 GOPATH=`godep path` godep restore godep: [WARNING]: godep should only be used inside a valid go package directory and godep: [WARNING]: may not function correctly. You are probably outside of your $GOPATH. godep: [WARNING]: Current Directory: /opt/gopath/src/github.com/CodisLabs/codis godep: [WARNING]: $GOPATH: /opt/gopath/src/github.com/CodisLabs/codis/Godeps/_workspace GOPATH=`godep path`:$GOPATH go build -o bin/codis-proxy ./cmd/proxy GOPATH=`godep path`:$GOPATH go build -o bin/codis-config ./cmd/cconfig make -j4 -C extern/redis-2.8.21/ make[1]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21' cd src && make all make[2]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/src' rm -rf redis-server redis-sentinel redis-cli redis-benchmark redis-check-dump redis-check-aof *.o *.gcda *.gcno *.gcov redis.info lcov-html (cd ../deps && make distclean) make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (cd hiredis && make clean) > /dev/null || true (cd linenoise && make clean) > /dev/null || true (cd lua && make clean) > /dev/null || true (cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true (rm -f .make-*) make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (rm -f .make-*) echo STD=-std=c99 -pedantic >> .make-settings echo WARN=-Wall -W >> .make-settings echo OPT=-O2 >> .make-settings echo MALLOC=jemalloc >> .make-settings echo CFLAGS= >> .make-settings echo LDFLAGS= >> .make-settings echo REDIS_CFLAGS= >> .make-settings echo REDIS_LDFLAGS= >> .make-settings echo PREV_FINAL_CFLAGS=-std=c99 -pedantic -Wall -W -O2 -g -ggdb -I../deps/hiredis -I../deps/linenoise -I../deps/lua/src -DUSE_JEMALLOC -I../deps/jemalloc/include >> .make-settings echo PREV_FINAL_LDFLAGS= -g -ggdb -rdynamic >> .make-settings (cd ../deps && make hiredis linenoise lua jemalloc) make[3]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' (cd hiredis && make clean) > /dev/null || true (cd linenoise && make clean) > /dev/null || true (cd lua && make clean) > /dev/null || true (cd jemalloc && [ -f Makefile ] && make distclean) > /dev/null || true (rm -f .make-*) (echo "" > .make-ldflags) (echo "" > .make-cflags) MAKE hiredis cd hiredis && make static MAKE linenoise cd linenoise && make MAKE lua make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise' cc -Wall -Os -g -c linenoise.c cd lua/src && make all CFLAGS="-O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL " MYLDFLAGS="" AR="ar rcu" MAKE jemalloc cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src' cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lapi.o lapi.c make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis' cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb net.c checking for xsltproc... /usr/bin/xsltproc checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb hiredis.c checking whether we are cross compiling... no checking for suffix of object files... make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/linenoise' cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb sds.c o checking whether we are using the GNU C compiler... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lcode.o lcode.c yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... cc -std=c99 -pedantic -c -O3 -fPIC -Wall -W -Wstrict-prototypes -Wwrite-strings -g -ggdb async.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldebug.o ldebug.c yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldo.o ldo.c yes ldo.c: In function ‘f_parser’: ldo.c:496: warning: unused variable ‘c’ cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldump.o ldump.c checking for inttypes.h... yes checking for stdint.h... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lfunc.o lfunc.c yes checking for unistd.h... ar rcs libhiredis.a net.o hiredis.o sds.o async.o cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lgc.o lgc.c yes checking whether byte ordering is bigendian... make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/hiredis' cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o llex.o llex.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmem.o lmem.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lobject.o lobject.c no checking size of void *... 8 checking size of int... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lopcodes.o lopcodes.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lparser.o lparser.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstate.o lstate.c 4 checking size of long... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstring.o lstring.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltable.o ltable.c 8 checking size of intmax_t... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltm.o ltm.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lundump.o lundump.c 8 checking build system type... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lvm.o lvm.c x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking whether pause instruction is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lzio.o lzio.c yes checking whether SSE2 intrinsics is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o strbuf.o strbuf.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o fpconv.o fpconv.c yes checking for ar... ar checking whether __attribute__ syntax is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lauxlib.o lauxlib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lbaselib.o lbaselib.c yes checking whether compiler supports -fvisibility=hidden... yes checking whether compiler supports -Werror... yes checking whether tls_model attribute is compilable... yes checking for a BSD-compatible install... /usr/bin/install -c checking for ranlib... ranlib checking for ld... /usr/bin/ld checking for autoconf... /usr/bin/autoconf checking for memalign... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ldblib.o ldblib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o liolib.o liolib.c yes checking for valloc... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lmathlib.o lmathlib.c yes checking configured backtracing method... N/A checking for sbrk... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loslib.o loslib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o ltablib.o ltablib.c yes checking whether utrace(2) is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lstrlib.o lstrlib.c no checking whether valgrind is compilable... no checking STATIC_PAGE_SHIFT... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o loadlib.o loadlib.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o linit.o linit.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cjson.o lua_cjson.c 12 checking pthread.h usability... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_struct.o lua_struct.c yes checking pthread.h presence... yes checking for pthread.h... yes checking for pthread_create in -lpthread... yes checking for _malloc_thread_cleanup... no checking for _pthread_mutex_init_calloc_cb... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_cmsgpack.o lua_cmsgpack.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua_bit.o lua_bit.c no checking for TLS... yes checking whether a program using ffsl is compilable... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o lua.o lua.c cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o luac.o luac.c yes checking whether atomic(9) is compilable... no checking whether Darwin OSAtomic*() is compilable... no checking whether to force 32-bit __sync_{add,sub}_and_fetch()... no checking whether to force 64-bit __sync_{add,sub}_and_fetch()... cc -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -c -o print.o print.c no checking whether Darwin OSSpin*() is compilable... ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o strbuf.o fpconv.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o lua_cjson.o lua_struct.o lua_cmsgpack.o lua_bit.o # DLL needs all object files ranlib liblua.a no checking for stdbool.h that conforms to C99... cc -o lua lua.o liblua.a -lm yes checking for _Bool... liblua.a(loslib.o): In function `os_tmpname': loslib.c:(.text+0x35): warning: the use of `tmpnam' is dangerous, better use `mkstemp' cc -o luac luac.o print.o liblua.a -lm make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/lua/src' yes configure: creating ./config.status config.status: creating Makefile config.status: creating doc/html.xsl config.status: creating doc/manpages.xsl config.status: creating doc/jemalloc.xml config.status: creating include/jemalloc/jemalloc_macros.h config.status: creating include/jemalloc/jemalloc_protos.h config.status: creating include/jemalloc/internal/jemalloc_internal.h config.status: creating test/test.sh config.status: creating test/include/test/jemalloc_test.h config.status: creating config.stamp config.status: creating bin/jemalloc.sh config.status: creating include/jemalloc/jemalloc_defs.h config.status: creating include/jemalloc/internal/jemalloc_internal_defs.h config.status: creating test/include/test/jemalloc_test_defs.h config.status: executing include/jemalloc/internal/private_namespace.h commands config.status: executing include/jemalloc/internal/private_unnamespace.h commands config.status: executing include/jemalloc/internal/public_symbols.txt commands config.status: executing include/jemalloc/internal/public_namespace.h commands config.status: executing include/jemalloc/internal/public_unnamespace.h commands config.status: executing include/jemalloc/internal/size_classes.h commands config.status: executing include/jemalloc/jemalloc_protos_jet.h commands config.status: executing include/jemalloc/jemalloc_rename.h commands config.status: executing include/jemalloc/jemalloc_mangle.h commands config.status: executing include/jemalloc/jemalloc_mangle_jet.h commands config.status: executing include/jemalloc/jemalloc.h commands =============================================================================== jemalloc version : 3.6.0-0-g46c0af68bd248b04df75e4f92d5fb804c3d75340 library revision : 1 CC : gcc CPPFLAGS : -D_GNU_SOURCE -D_REENTRANT CFLAGS : -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -fvisibility=hidden LDFLAGS : EXTRA_LDFLAGS : LIBS : -lpthread RPATH_EXTRA : XSLTPROC : /usr/bin/xsltproc XSLROOT : /usr/share/sgml/docbook/xsl-stylesheets PREFIX : /usr/local BINDIR : /usr/local/bin INCLUDEDIR : /usr/local/include LIBDIR : /usr/local/lib DATADIR : /usr/local/share MANDIR : /usr/local/share/man srcroot : abs_srcroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/ objroot : abs_objroot : /opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc/ JEMALLOC_PREFIX : je_ JEMALLOC_PRIVATE_NAMESPACE : je_ install_suffix : autogen : 0 experimental : 1 cc-silence : 1 debug : 0 code-coverage : 0 stats : 1 prof : 0 prof-libunwind : 0 prof-libgcc : 0 prof-gcc : 0 tcache : 1 fill : 1 utrace : 0 valgrind : 0 xmalloc : 0 mremap : 0 munmap : 0 dss : 0 lazy_lock : 0 tls : 1 =============================================================================== cd jemalloc && make CFLAGS="-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops " LDFLAGS="" lib/libjemalloc.a make[4]: Entering directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc' gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/jemalloc.o src/jemalloc.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/arena.o src/arena.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/atomic.o src/atomic.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/base.o src/base.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/bitmap.o src/bitmap.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk.o src/chunk.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_dss.o src/chunk_dss.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/chunk_mmap.o src/chunk_mmap.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ckh.o src/ckh.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/ctl.o src/ctl.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/extent.o src/extent.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/hash.o src/hash.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/huge.o src/huge.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mb.o src/mb.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/mutex.o src/mutex.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/prof.o src/prof.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/quarantine.o src/quarantine.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/rtree.o src/rtree.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/stats.o src/stats.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tcache.o src/tcache.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/util.o src/util.c gcc -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops -c -D_GNU_SOURCE -D_REENTRANT -Iinclude -Iinclude -o src/tsd.o src/tsd.c ar crus lib/libjemalloc.a src/jemalloc.o src/arena.o src/atomic.o src/base.o src/bitmap.o src/chunk.o src/chunk_dss.o src/chunk_mmap.o src/ckh.o src/ctl.o src/extent.o src/hash.o src/huge.o src/mb.o src/mutex.o src/prof.o src/quarantine.o src/rtree.o src/stats.o src/tcache.o src/util.o src/tsd.o make[4]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps/jemalloc' make[3]: Leaving directory `/opt/gopath/src/github.com/CodisLabs/codis/extern/redis-2.8.21/deps' CC adlist.o CC ae.o CC anet.o CC dict.o anet.c: In function ‘anetSockName’: anet.c:565: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:563: note: initialized from here anet.c:569: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:567: note: initialized from here anet.c: In function ‘anetPeerToString’: anet.c:543: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:541: note: initialized from here anet.c:547: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:545: note: initialized from here anet.c: In function ‘anetTcpAccept’: anet.c:511: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:509: note: initialized from here anet.c:515: warning: dereferencing pointer ‘s’ does break strict-aliasing rules anet.c:513: note: initialized from here CC redis.o CC sds.o CC zmalloc.o CC lzf_c.o CC lzf_d.o CC pqsort.o CC zipmap.o CC sha1.o CC ziplist.o CC release.o CC networking.o CC util.o CC object.o CC db.o CC replication.o CC rdb.o CC t_string.o db.c: In function ‘scanGenericCommand’: db.c:454: warning: ‘pat’ may be used uninitialized in this function db.c:455: warning: ‘patlen’ may be used uninitialized in this function CC t_list.o CC t_set.o CC t_zset.o CC t_hash.o CC config.o CC aof.o CC pubsub.o CC multi.o CC debug.o CC sort.o CC intset.o CC syncio.o CC migrate.o CC endianconv.o CC slowlog.o CC scripting.o CC bio.o CC rio.o CC rand.o CC memtest.o CC crc64.o CC crc32.o CC bitops.o CC sentinel.o CC notify.o CC setproctitle.o CC hyperloglog.o CC latency.o CC sparkline.o CC slots.o CC redis-cli.o CC redis-benchmark.o CC redis-check-dump.o CC redis-check-aof.o LINK redis-benchmark LINK redis-check-dump LINK redis-check-aof LINK redis-server INSTALL redis-sentinel LINK redis-cli Hint: It's a good idea to run 'make test' ;)
[root@node5 codis]# make gotest GOPATH=`godep path`:$GOPATH go test ./pkg/... ./cmd/... ok github.com/CodisLabs/codis/pkg/models 17.324s ok github.com/CodisLabs/codis/pkg/proxy 6.296s ok github.com/CodisLabs/codis/pkg/proxy/redis 1.338s ok github.com/CodisLabs/codis/pkg/proxy/router 0.654s ? github.com/CodisLabs/codis/pkg/utils [no test files] ? github.com/CodisLabs/codis/pkg/utils/assert [no test files] ? github.com/CodisLabs/codis/pkg/utils/atomic2 [no test files] ok github.com/CodisLabs/codis/pkg/utils/bytesize 0.002s ? github.com/CodisLabs/codis/pkg/utils/errors [no test files] ? github.com/CodisLabs/codis/pkg/utils/log [no test files] ? github.com/CodisLabs/codis/pkg/utils/trace [no test files] ? github.com/CodisLabs/codis/cmd/cconfig [no test files] ? github.com/CodisLabs/codis/cmd/proxy [no test files]
执行所有指令后,会在 bin 文件夹内生成 codis-config、codis-proxy、codis-server三个可执行文件。另外, bin/assets 文件夹是 codis-config 的 dashboard http 服务须要的前端资源, 须要和 codis-config 放置在同一文件夹下)
[root@node5 codis]# cd bin/ [root@node5 bin]# ls assets codis-config codis-proxy codis-server
可执行文件使用方式:
[root@node5 bin]# ./codis-config -h
[root@node5 bin]# ./codis-config -h
usage: codis-config [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>] <command> [<args>...] options: -c #配置文件地址 -L #日志输出文件地址 --log-level=<loglevel> #输出日志级别 (debug < info (default) < warn < error < fatal) commands: server #redis 服务器组管理 slot #slot 管理 dashboard #启动 dashboard 服务 action #事件管理 (目前只有删除历史事件的日志) proxy #proxy 管理
[root@node5 bin]# ./codis-proxy -h
$ bin/codis-proxy -h usage: codis-proxy [-c <config_file>] [-L <log_file>] [--log-level=<loglevel>] [--cpu=<cpu_num>] [--addr=<proxy_listen_addr>] [--http-addr=<debug_http_server_addr>] options: -c #配置文件地址 -L #日志输出文件地址 --log-level=<loglevel> #输出日志级别 (debug < info (default) < warn < error < fatal) --cpu=<cpu_num> #proxy占用的 cpu 核数, 默认1, 最好设置为机器的物理cpu数的一半到2/3左右 --addr=<proxy_listen_addr> #proxy 的 redis server 监听的地址, 格式 <ip or hostname>:<port>, 如: localhost:9000, :9001 --http-addr=<debug_http_server_addr> #proxy 的调试信息启动的http server, 能够访问 http://debug_http_server_addr/debug/vars
[root@node5 bin]# ./codis-server -h
[root@node5 bin]# ./codis-server -h Usage: ./redis-server [/path/to/redis.conf] [options] ./redis-server - (read config from stdin) #指定redis配置文件 ./redis-server -v or --version #显示版本 ./redis-server -h or --help #获取帮助 ./redis-server --test-memory <megabytes> #单位为字节 Examples: ./redis-server (run the server with default conf) ./redis-server /etc/redis/6379.conf ./redis-server --port 7777 ./redis-server --port 7777 --slaveof 127.0.0.1 8888 ./redis-server /etc/myredis.conf --loglevel verbose
开始部署:
[root@node5 bin]# cd ..
[root@node5 codis]# vim config.ini
zk=192.168.10.205:2181
dashboard_addr=192.168.10.205:18087
配置并启动 dashboard:
[root@node5 codis]# cp config.ini bin/ #将修改后的文件复制至bin目录,由于bin里面你的命令默认读取当前目录的config.ini文件,或可使用 -c指定也能够
[root@node5 bin]# ./codis-config dashboard & #该命令会启动 dashboard
启动成功就能够经过 http://ServerIP:18087访问
执行./codis-config slot init
,该命令会在zookeeper上建立slot相关信息,若是提示已经初始化能够加 -f 强制
[root@node5 bin]# ./codis-config slot init -f { "msg": "OK", "ret": 0 }
若是已经启动能够跳过,若是还未启动就执行:
[root@node5 bin]# /etc/init.d/redisd start
每个 Server Group 做为一个 Redis 服务器组存在, 只容许有一个 master, 能够有多个 slave, group id 仅支持大于等于1的整数
添加一个master:
[root@node5 bin]# ./codis-config server add 1 localhost:6379 master { "msg": "OK", "ret": 0 }
为master添加一个slave:
[root@node5 bin]# ./codis-config server add 1 192.168.10.205:7009 slave
[root@node5 bin]# ./codis-config server add 1 192.168.10.205:7009 slave 2016/03/22 03:37:27 main.go:154: [PANIC] run sub-command failed [error]: http status code 500, ERR SLAVEOF not allowed in cluster mode. 4 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/utils.go:66 main.callApi 3 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/server_group.go:94 main.runAddServerToGroup 2 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/server_group.go:54 main.cmdServer 1 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:87 main.runCommand 0 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:152 main.main ... ... [stack]: 0 /opt/gopath/src/github.com/CodisLabs/codis/cmd/cconfig/main.go:154 main.main ... ...
查看列表信息:
[root@node5 bin]# ./codis-config server list [ { "id": 1, "product_name": "test", "servers": [ { "addr": "localhost:6379", "group_id": 1, "type": "master" }, { "addr": "192.168.10.205:7009", "group_id": 1, "type": "slave" } ] } ]
在浏览器验证:
Codis 采用 Pre-sharding 的技术来实现数据的分片, 默认分红 1024 个 slots (0-1023), 对于每一个key来讲, 经过如下公式肯定所属的 Slot Id : SlotId = crc32(key) % 1024 每个 slot 都会有一个且必须有一个特定的 server group id 来表示这个 slot 的数据由哪一个 server group 来提供:
如:
设置编号为[0, 511]的 slot 由 server group 1 提供服务, 编号 [512, 1023] 的 slot 由 server group 2 提供服务
[root@node5 bin]# ./codis-config slot range-set 0 511 1 online { "msg": "OK", "ret": 0 } [root@node5 bin]# ./codis-config slot range-set 512 1023 2 online { "msg": "OK", "ret": 0 }
[root@node5 bin]# ./codis-proxy -c config.ini -L ./log/proxy.log --cpu=8 --addr=0.0.0.0:19000 --http-addr=0.0.0.0:11000 _____ ____ ____/ / (_) _____ / ___/ / __ \ / __ / / / / ___/ / /__ / /_/ / / /_/ / / / (__ ) \___/ \____/ \__,_/ /_/ /____/
刚启动的 codis-proxy 默认是处于 offline状态的, 而后设置 proxy 为 online 状态, 只有处于 online 状态的 proxy 才会对外提供服务
[root@node5 bin]# ./codis-config -c config.ini proxy online proxy_1 { "msg": "OK", "ret": 0 }
注:命令格式为./codis-config -c config.ini proxy online <proxy_name> <---- proxy的id, 如 proxy_1,能够在浏览器看到