prometheus服务发现-consul部署研究

最近在看prometheus-book时，在服务发现这一章节，了解到“基于consul的服务发现”，之前一直都是用python代码生成json推送到服务器上，这种方式很麻烦，以前还想着编写工具程序部署到prometheus所在节点进行target的自动发现。

最近几天详细研究如何利用consul进行prometheus服务发现，首先进行consul部署的研究。

开发模式

使用命令consul agent -dev便可，而后能够经过 http://127.0.0.1:8500 访问管理界面

因为目前处于验证阶段，生产环境确定以多节点集群的方式部署，要在本机模拟集群状况，则能够经过指定不一样的端口以及搭建3台虚拟机的方式来完成。

单节点集群部署

单节点集群部署须要给每一个server指定不一样的端口，涉及的端口类型以下：

1. http-port web服务端口，默认为8500，须要经过浏览器访问管理端则要加入 -ui 或 ui:true 这个配置项

2. dns-port dns端口，默认为8600

3. serf-lan-port lan端口，默认8301

4. serf-wan-port wan端口，默认为8302

5. server-port 服务端口，默认为8300

命令行传参

1. server1

consul agent -server -bind 10.106.169.121 -node server1 -data-dir /opt/consul/server1 -ui -client 0.0.0.0 -bootstrap-expect=1 -server-port=18300 -http-port=18500 -dns-port=18600  -serf-lan-port=18301 -serf-wan-port=18302

2. server2

consul agent -server -bind 10.106.169.121 -node server2 -data-dir /opt/consul/server2 -ui -client 0.0.0.0 -server-port=28300 -http-port=28500 -dns-port=28600  -serf-lan-port=28301 -serf-wan-port=28302 -join 10.106.169.121:18301

3. server3

consul agent -server -bind 10.106.169.121 -node server3 -data-dir /opt/consul/server3 -ui -client 0.0.0.0 -server-port=38300 -http-port=38500 -dns-port=38600  -serf-lan-port=38301 -serf-wan-port=38302 -join 10.106.169.121:18301

配置文件传参

1. server1

consul agent -config-file=/opt/consul/server1.json

{
  "datacenter": "prometheus",
  "bootstrap_expect":1,
  "bind_addr":"10.106.169.121",
  "data_dir": "/opt/consul/server1",
  "log_level": "INFO",
  "node_name": "server1",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "ports": {
     "http": 18500,
     "dns": 18600,
     "serf_lan":18301,
     "serf_wan":18302,
     "server":18300
  }
}

2. server2

consul agent -config-file=/opt/consul/server2.json

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "data_dir": "/opt/consul/server2",
  "log_level": "INFO",
  "node_name": "server2",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 28500,
     "dns": 28600,
     "serf_lan":28301,
     "serf_wan":28302,
     "server":28300
  }
}

3. server3

consul agent -config-file=/opt/consul/server3.json

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "data_dir": "/opt/consul/server3",
  "log_level": "INFO",
  "node_name": "server3",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 38500,
     "dns": 38600,
     "serf_lan":38301,
     "serf_wan":38302,
     "server":38300
  }
}

容器部署

镜像的entrypoint是：/usr/local/bin/docker-entrypoint.sh，里面定义了以下两个环境变量

CONSUL_DATA_DIR=/consul/data
CONSUL_CONFIG_DIR=/consul/config

1. server1

docker run --net=host -v /opt/consul_docker/server1/config:/consul/config -v /opt/consul_docker/server1/data:/consul/data consul:1.4.1

{
  "datacenter": "prometheus",
  "bootstrap_expect":1,
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"09d82408-bc4f-49e0-4208-61ef1d4842f7",
  "node_name": "server1",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "ports": {
     "http": 18500,
     "dns": 18600,
     "serf_lan":18301,
     "serf_wan":18302,
     "server":18300,
     "grpc":-1
  }
}

2. server2

docker run --net=host -v /opt/consul_docker/server2/config:/consul/config -v /opt/consul_docker/server2/data:/consul/data consul:1.4.1

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d",
  "node_name": "server2",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 28500,
     "dns": 28600,
     "serf_lan":28301,
     "serf_wan":28302,
     "server":28300,
     "grpc":-1
  }
}

3. server3

docker run --net=host -v /opt/consul_docker/server3/config:/consul/config -v /opt/consul_docker/server3/data:/consul/data consul:1.4.1

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e",
  "node_name": "server3",
  "server": true,
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301"],
  "ports": {
     "http": 38500,
     "dns": 38600,
     "serf_lan":38301,
     "serf_wan":38302,
     "server":38300,
     "grpc":-1
  }
}

多节点部署

1. server1

vm : 10.20.0.3

consul agent -server -bind 10.20.0.3 -node server1 -data-dir /root/consul/server1 -ui -client 0.0.0.0 -bootstrap-expect=1

2. server2

vm : 10.20.0.4

consul agent -server -bind 10.20.0.4 -node server2 -data-dir /root/consul/server2 -join 10.20.0.3 -client 0.0.0.0

3. server3

vm : 10.20.0.5

consul agent -server -bind 10.20.0.5 -node server3 -data-dir /root/consul/server3 -join 10.20.0.3 -client 0.0.0.0

部分配置项说明

1. bind_addr，绑定ip，有多IP时须要指定

2. server ，是否开启server模式

3. client_addr ， 监听的ip

4. retry_join，非主节点启动时指定加入的主节点，官方推荐使用retry_join而不是join

5. node_id，可指定节点的uuid，格式：d8a09ffd-7ccb-84bd-7231-8d8b7a01951e，不指定的话为随机生成，使用集群的话最好能指定该id，保证惟一性

6. node_name，节点名称

7. config_file，指定配置文件