centos下搭建kafka集群

一、下载安装zookeeper

一、下载安装包

wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz

二、解压

tar -zxvf apache-zookeeper-3.6.2-bin.tar.gz

三、改文件夹名称，去掉版本号

mv apache-zookeeper-3.6.2-bin zookeeper

四、建立data目录后，进入conf文件夹，修改zoo_sample.cfg

cd  zookeeper
mkdir data
cd conf/
mv zoo_sample.cfg zoo.cfg 
vi zoo.cfg

//修改两处
（1）dataDir=/usr/local/zookeeper/data  (前面刚建立的目录)
（2）在最后添加：
server.0=172.19.130.156:2888:3888
server.1=172.19.130.157:2888:3888
server.2=172.19.130.158:2888:3888

五、建立服务器标识

进入data目录，建立文件myid，内容为0

 

这里单台的zookeeper已经完成，先告一段落，进行kafka的安装后再配置另两台并启动集群

 

二、下载安装kafka

一、回到local目录下载kafka安装包解压、更名

wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.6.0/kafka_2.12-2.6.0.tgz
tar -zxvf kafka_2.12-2.6.0.tgz
mv kafka_2.12-2.6.0 kafka

二、进入kafka目录并修改配置文件

cd kafka/config/
vi server.properties

//须要修改三处地方
broker.id=0
listeners=PLAINTEXT://172.19.130.156:9092
zookeeper.connect=172.19.130.156:2181,172.19.130.157:2181,172.19.130.158:2181

三、配置环境变量

vi /etc/profile


//在文件最下方加上
export KAFKA_HOME=/usr/local/kafka
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:${KAFKA_HOME}/bin:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf


//更新生效
source /etc/profile

　　

三、拷贝zookeeper和kafka至另外两台机器

scp -r zookeeper/  root@172.19.130.157:/usr/local/
scp -r zookeeper/  root@172.19.130.158:/usr/local/

scp -r kafka/  root@172.19.130.157:/usr/local/
scp -r kafka/  root@172.19.130.158:/usr/local/

 

四、登陆第二台服务器

一、修改配置

cd /usr/local/zookeeper/data
vi myid   //内容修改成1

cd /usr/local/kafka/config/
vi server.properties
//修改两处 
broker.id=1 
listeners=PLAINTEXT://172.19.130.157:9092

二、配置环境变量

vi /etc/profile


//在文件最下方加上
export KAFKA_HOME=/usr/local/kafka
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:${KAFKA_HOME}/bin:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf


//更新生效
source /etc/profile

五、登陆第三台服务器

一、修改配置

cd /usr/local/zookeeper/data
vi myid   //内容修改成2

cd /usr/local/kafka/config/
vi server.properties
//修改两处
broker.id=2
listeners=PLAINTEXT://172.19.130.158:9092

二、配置环境变量

vi /etc/profile


//在文件最下方加上
export KAFKA_HOME=/usr/local/kafka
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:${KAFKA_HOME}/bin:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf


//更新生效
source /etc/profile

 

六、启动zookeeper集群

cd /usr/local/zookeeper/bin

zkServer.sh start   //3台服务分别执行

zkServer.sh status  //查看启动成功与否以及选举状态

 

七、启动kafka集群

cd /usr/local/kafka
./bin/kafka-server-start.sh -daemon config/server.properties