使用 Exhibitor 监控管理 ZooKeeper

前言

Exhibitor 是 Netflix 开源的一个用于 ZooKeeper 配置监控和管理的系统。如今 Netflix Exhibitor 已经成为社区开源公共维护项目 Soabase Exhibitor。Exhibitor 是 ZooKeeper 实例监控,备份,恢复,清理和可视化工具,是 ZooKeeper 的监控管理系统。html

使用 Exhibitor 监控管理 ZooKeeper

更新历史

2019年01月07日 - 初稿java

阅读原文 - https://wsgzao.github.io/post...node

扩展阅读git

exhibitor - https://github.com/soabase/ex...github


exhibitor简介

ZooKeeper co-process for instance monitoring, backup/recovery, cleanup and visualization.bash

Exhibitor is a Java supervisor system for ZooKeeper. It provides a number of features:服务器

  • Watches a ZK instance and makes sure it is running
  • Performs periodic backups
  • Perform periodic cleaning of ZK log directory
  • A GUI explorer for viewing ZK nodes
  • A rich REST API

https://github.com/soabase/ex...app

exhibitor特性

Exhibitor 主要包括如下特性 / 功能:

实例监控maven

Exhibitor 实例监控在同一服务器上运行的 ZooKeeper 服务器。若是 ZK 没有运行,Exhibitor 会写入 zoo.cfg 文件(请参阅下面的 ZK 集群配置)并启动它。若是 ZooKeeper 因为某种缘由崩溃,Exhibitor 也会从新启动它。ide

日志清理

在 ZooKeeper 3.4.x 以前的版本中,日志文件须要维护,Exhibitor 会负责按期维护。

备份 / 还原

ZooKeeper 集群中的备份比传统数据存储(例如 RDBMS)更复杂。通常来讲,ZooKeeper 中的大部分数据是短暂的。盲目恢复整个 ZooKeeper 数据集可能会形成更大危害,所以,须要选择性的恢复以防止对数据集的子集形成意外损坏。Exhibitor 提供了这一功能。

Exhibitor 会按期备份 ZooKeeper 的事务文件,备份后,就能够对这些事务文件创建索引。

集群配置

Exhibitor 为整个 Zookeeper 集群提供了一个独立的控制台,经过它所作的配置更改会对整个集群有效。如下是一些共享配置值:

Name Description
ZooKeeper Install DirPath to the ZooKeeper server installation
ZooKeeper Data Dir Path where ZooKeeper should store its data
Log Index Dir Path where indexed transaction logs should be kept
Servers List of servers/server-ids in the ensemble
Additional Config Additional fields/values to store in zoo.cfg

集群滚动升级

Exhibitor 能够以滚动方式更新集群中的服务器,以便在进行更改时让 ZooKeeper 集群确保 Quorum 设定的最低服务能力。

自动实例管理

Exhibitor 能够配置为自动向集群中添加新实例,并删除陈旧的实例。这使得 ZooKeeper 集群能够实现 “无接触交钥匙管理”。

可视化

Exhibitor 为 ZooKeeper 提供了 ZNode 层次结构的图形树视图。

ZK 数据维护

启用后,维护人员能够在 ZooKeeper 的存储层次结构中建立 / 更新 / 删除节点。

死锁检测

当使用 Curator 的锁方案(或相似)时,Exhibitor 能够分析一组表示锁的 ZNode,并肯定是否存在潜在的死锁。

Curator 集成

Exhibitor 和 Curator 能够集成工做,当集群中的信息变动时,Curator 实例能够同步更新。

REST API

Exhibitor 提供了一组用于程序集成的 REST API。

重要事项
  1. 使用 Exhibitor 时,不要手动编辑 Zookeeper 的 zoo.cfg 和 myid 文件,由于 Exhibitor 会覆盖它们。
  2. 除了标准的 ZooKeeper 端口,防火墙必须打开 Exhibitor 使用的 HTTP 端口,由于每一个 Exhibitor 实例须要与其余参与者通讯传递状态。
使用

Exhibitor 的部署构件能够从 Maven 仓库获取。构件分为两种:

GroupID/Org ArtifactID/Name Description
com.netflix.exhibitor exhibitor-standalone 自包含的,可执行的 Exhibitor 版本(能够是独立应用,也能够是 War)
com.netflix.exhibitor exhibitor-core 类库的形式,能够嵌入到应用中

exhibitor编译安装

官方分享了Maven和Gradle两种build方法,这里以Maven为例

https://github.com/soabase/ex...

# install maven
yum install -y maven

# build exhibitor
mkdir exhibitor
cd exhibitor/
wget https://github.com/soabase/exhibitor/archive/exhibitor-1.7.1.zip
unzip exhibitor-1.7.1.zip
cd exhibitor-exhibitor-1.7.1/exhibitor-standalone/src/main/resources/buildscripts/standalone/maven
mvn clean package

[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /root/exhibitor/exhibitor-exhibitor-1.7.1/exhibitor-standalone/src/main/resources/buildscripts/standalone/maven/target/exhibitor-1.6.0.jar with /root/exhibitor/exhibitor-exhibitor-1.7.1/exhibitor-standalone/src/main/resources/buildscripts/standalone/maven/target/exhibitor-1.6.0-shaded.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44.624s
[INFO] Finished at: Tue Jan 08 11:28:59 SGT 2019
[INFO] Final Memory: 15M/94M
[INFO] ------------------------------------------------------------------------

cp target/exhibitor-1.6.0.jar /tmp

# Once built, Exhibitor is completely self-contained and can be run from the command line:
java -jar <path>/exhibitor-xxx.jar -c file

[root@localhost ~]# java -jar exhibitor-1.6.0.jar -c file
v1.6.0
INFO  com.netflix.exhibitor.core.activity.ActivityLog  Exhibitor started [main]
Jan 08, 2019 11:32:38 AM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
INFO  org.mortbay.log  Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog [main]
INFO  org.mortbay.log  jetty-1.6.0 [main]
Jan 08, 2019 11:32:38 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.18.3 12/01/2014 08:23 AM'
INFO  org.mortbay.log  Started SocketConnector@0.0.0.0:8080 [main]
Jan 08, 2019 11:33:00 AM java.util.prefs.FileSystemPreferences$6 run
WARNING: Prefs file removed in background /root/.java/.userPrefs/prefs.xml
INFO  com.netflix.exhibitor.core.activity.ActivityLog  State: latent [ActivityQueue-0]

# You can test that it’s running correctly by going to this URL in a browser: 
http://localhost:8080/exhibitor/v1/ui/index.html
http://192.168.56.103:8080/exhibitor/v1/ui/index.html

管理zookeeper集群

若是须要经过 Exhibitor 管理 zookeeper 集群须要在集群的每一个机器上安装 Exhibitor

https://github.com/soabase/ex...

重点提一下Ensemble中的Servers配置:

  • 服务器IP之间用逗号分隔,有两种类型:S表示标准类型,O表示Observer观察者
  • S:1:192.168.56.101,S:2:192.168.56.102,S:3:192.168.56.103