Nginx集成Kafka

host：xupan002

 

librdkafka简介

librdkafka 是一个C实现的高性能 Apache Kafka 客户端，为生产环境提供了一个可靠和高性能的客户端。 librdkafka 一样也提供了传统的 C++ 接口。

 

性能

librdkafka 是一个基于现代硬件设计的多线程库， 而且试图保持最少的内存拷贝。 
若是应用程序愿意，生产和消费消息的载体能够不经过任何拷贝实现让消息大小不受限制。librdkafka 一样适用于高吞吐仍是低延时的场景，均可以经过属性配置接口来知足。

 

librdkafka

cd /usr/local/devtools/nginx/plugin
git clone https://github.com/edenhill/librdkafka
cd librdkafka 
./configure
make
make install

 

ngx_kafka_module：

cd /usr/local/devtools/nginx/plugin
git clone https://github.com/brg-liuwei/ngx_kafka_module

cd /usr/local/devtools/nginx/nginx-1.12.2
./configure --add-module=/usr/local/devtools/nginx/plugin/ngx_kafka_module/
make
make install

 

配置nginx.conf

kafka;
kafka_broker_list xupan001:9092 xupan002:9093 xupan003:9093;

location = /kafka/track {
     kafka_topic track;
}

location = /kafka/user {
 kafka_topic user;
}
=================================================

echo "/usr/local/lib" >> /etc/ld.so.conf
ldconfig

 

 

测试

--kafka建立topic
kafka-topics.sh -zookeeper xupan001:2181,xupan002:2181,xupan003:2181 --create --topic track --replication-factor 3 --partitions 3     
kafka-topics.sh -zookeeper xupan001:2181,xupan002:2181,xupan003:2181 --create --topic user --replication-factor 3 --partitions 3      

--kafka建立消费者
./bin/kafka-console-consumer.sh --zookeeper xupan001:2181,xupan002:2181,xupan003:2181 --topic track --from-beginning

curl xupan002/kafka/track  -d "hello ngx_kafka_module"