20190426 部署Zookeeper集群

kafka是制做消息队列的，与redis类似，可是优于redis，支持消息并列与通知，

实验:部署zookeeper集群：

安装思路及环境：先安装Java环境，再安装zookeeper，后安装kafka，：安装思路及环境：先安装Java环境，再安装zookeeper，后安装kafka，
zookeeper版本下载网址： http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
http://zookeeper.apache.org/doc/current/zookeeperStarted.html中有：
server.1=172.18.9.203:2888:3888 server.2=172.18.9.204:2888:3888 server.3=172.18.9.205:2888:3888
制做zookeeper集群需三台服务器：172.18.9.203,172.18.9.204,172.18.9.205
三台服务器分别配置hosts文件：
#cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.18.9.203 linux-host1.exmaple.com
172.18.9.204 linux-host2.exmaple.com
172.18.9.205 linux-host3.exmaple.com

1、安装Java环境：[root@203/204/205 src]#yum -y install jdk-8u191-linux-x64.rpm

2、安装zookeeper：[root@203/204/205 src]#wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

[root@203/204/205 src]#tar xf zookeeper-3.4.14.tar.gz

3、制做zookeeper配置文件：

[root@203/204/205 src]#cd zookeeper-3.4.14/conf
[root@203/204/205 conf]#ll
total 12
-rw-rw-r--. 1 2002 2002 535 Mar 7 00:50 configuration.xsl
-rw-rw-r--. 1 2002 2002 2161 Mar 7 00:50 log4j.properties
-rw-rw-r--. 1 2002 2002 922 Mar 7 00:50 zoo_sample.cfg
[root@203/204/205 conf]#cp zoo_sample.cfg zoo.cfg #把模板cp成配置文件

4、修改配置文件并作软链接并建立数据存放目录：

[root@203/204/205 conf]#vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/data
clientPort=2181
maxClientCnxns=4096
server.1=172.18.9.203:2888:3888
server.2=172.18.9.204:2888:3888
server.3=172.18.9.205:2888:3888
[root@203/204/205 conf]#mkdir /usr/local/zookeeper/data
[root@203/204/205 conf]#ln -s /usr/local/src/zookeeper-3.4.14 /usr/local/zookeeper

5、开启并查看状态：[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh start

ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
出现此错的缘由是zookeeper的pid没有生成！

6、将server的编号写入myid文件：

[root@203/204/205 zookeeper-3.4.14]#echo 1 2 3 > /usr/local/zookeeper/data/myid
[root@203/204/205 zookeeper-3.4.14]#

7、从新开启服务并查看端口：

[root@203/204/205 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh start
[root@203/204/205 zookeeper-3.4.14]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 50 :::2181 :::*

8、查看集群状态：leader/follower 一目了然：

[root@203 zookeeper-3.4.14]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@204 conf]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@205 conf]#/usr/local/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower

9、将开启服务命令写入脚本、令其开机自启：

[root@203/204/205 zookeeper-3.4.14]#vim /etc/rc.d/rc.local
/usr/local/zookeeper/bin/zkServer.sh start
此时，zookeeper已完成安装，它没有帐号认证，任何普户都能用！

10、安装kafka：

[root@203/204/205 src]#tar xf kafka_2.12-2.2.0.tar.gz
[root@203/204/205 src]#ln -s /usr/local/src/kafka_2.12-2.2.0 /usr/local/kafka
[root@203/204/205 src]#vim /usr/local/kafka/config/server.properties
21 broker.id=1 2 3 #设置每一个代理全局惟一的整数ID
31 listeners=PLAINTEXT://172.18.9.203:9092
103 log.retention.hours=24 #保留指定小时的日志内容
123 zookeeper.connect=172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 #全部的zookeeper地址

11、启动kafka，验证进程“jps”：

[root@203/204/205 ~]#/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties
[root@203 src]#jps
26632 Jps
26602 Kafka
23453 QuorumPeerMain
[root@204 src]#jps
41969 Jps
41939 Kafka
34490 QuorumPeerMain
[root@205 src]#jps
19785 QuorumPeerMain
23066 Kafka
23147 Jps

12、下载安装kafka客户端插件：点击并安装：

http://www.kafkatool.com/download.html Windows 64-Bit download

十3、one：测试建立topic：建立名为logstashtest，partitions(分区)为3，replication(复制)为3的topic(主题)：在任意kafaka服务器操做：

[root@204 src]#/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --partitions 3 --replication-factor 3 --topic logstashtest
Created topic logstashtest.

two：测试获取topic：可在任意一台服务器进行测试：
[root@205 src]#/usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --topic logstashtest
Topic:logstashtest PartitionCount:3 ReplicationFactor:3 Configs:
Topic: logstashtest Partition: 0 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
Topic: logstashtest Partition: 1 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2
Topic: logstashtest Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
状态说明：logstashtest有三个分区分别为0、一、2，分区0的leader是3（broker.id），分区0有三个副本，而且状态都为lsr（ln-sync，表示能够参加选举成为leader）。

three：删除topic：[root@205 src]#/usr/local/kafka/bin/kafka-topics.sh --delete --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181 --topic logstashtest
Topic logstashtest is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

four：获取全部的topic：[root@205 src]#/usr/local/kafka/bin/kafka-topics.sh --list --zookeeper 172.18.9.203:2181,172.18.9.204:2181,172.18.9.205:2181logstashtest #此为topic的内容