prometheus服务发现-consul生产环境服务部署

模拟生产环境多节点部署，使用linux服务方式部署consul集群，保证服务高可用即异常挂掉或主机重启后能自动回复，同时启用agent通讯加密和接口认证。

前期准备

1. 安装

将下载解压获得二进制文件consul拷贝到/usr/local/bin目录

sudo chown root:root /usr/local/bin/consul
consul --version # 验证

2. 自动补全

consul -autocomplete-install
complete -C /usr/local/bin/consul consul

3. 准备目录

建立/opt/consul目录，并在其下准备以下3个server的目录

/opt/consul$ tree
.
├── server1
│   ├── config
│   └── data
├── server2
│   ├── config
│   └── data
└── server3
    ├── config
    └── data

4. 生成秘钥

用于集群间通讯加密，须要保证集群中全部节点都配置该秘钥

$ consul keygen
mz8Con27P34D9fiPG1bjHA==

配置

server1

service unit

准备文件：/lib/systemd/system/consul-server1.service，内容以下：

[Unit]
Description="consul server1"
Requires=network-online.target
After=network-online.target
[Service]
ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server1/config
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

配置

准备配置文件：/opt/consul/server1/config/config.json，内容以下：

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"09d82408-bc4f-49e0-4208-61ef1d4842f7",
  "node_name": "server1",
  "data_dir":"/opt/consul/server1/data",
  "server": true,
  "bootstrap_expect":3,
  "encrypt": "mz8Con27P34D9fiPG1bjHA==",
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
  "ports": {
     "http": 18500,
     "dns": 18600,
     "serf_lan":18301,
     "serf_wan":18302,
     "server":18300,
     "grpc":-1
  },
  "acl": {
    "enabled": true,
    "default_policy": "deny",
    "down_policy": "extend-cache",
    "tokens":{
        "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
        "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
    }
  }
}

server2

service unit

准备文件：/lib/systemd/system/consul-server2.service，内容以下：

[Unit]
Description="consul server2"
Requires=network-online.target
After=network-online.target
[Service]
ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server2/config
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

配置

准备配置文件：/opt/consul/server2/config/config.json，内容以下：

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d",
  "node_name": "server2",
  "data_dir":"/opt/consul/server2/data",
  "server": true,
  "bootstrap_expect":3,
  "encrypt": "mz8Con27P34D9fiPG1bjHA==",
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
  "ports": {
     "http": 28500,
     "dns": 28600,
     "serf_lan":28301,
     "serf_wan":28302,
     "server":28300,
     "grpc":-1
  },
  "acl": {
    "enabled": true,
    "default_policy": "deny",
    "down_policy": "extend-cache",
    "tokens":{
        "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
        "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
    }
  }
}

server3

service unit

准备文件：/lib/systemd/system/consul-server3.service，内容以下：

[Unit]
Description="consul server3"
Requires=network-online.target
After=network-online.target
[Service]
ExecStart=/usr/local/bin/consul agent -config-dir=/opt/consul/server3/config
ExecReload=/usr/local/bin/consul reload
KillMode=process
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

配置

准备配置文件：/opt/consul/server3/config/config.json，内容以下：

{
  "datacenter": "prometheus",
  "bind_addr":"10.106.169.121",
  "log_level": "INFO",
  "node_id":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e",
  "node_name": "server3",
  "data_dir":"/opt/consul/server3/data",
  "server": true,
  "bootstrap_expect":3,
  "encrypt": "mz8Con27P34D9fiPG1bjHA==",
  "ui":true,
  "client_addr":"0.0.0.0",
  "retry_join":["10.106.169.121:18301","10.106.169.121:28301","10.106.169.121:38301"],
  "ports": {
     "http": 38500,
     "dns": 38600,
     "serf_lan":38301,
     "serf_wan":38302,
     "server":38300,
     "grpc":-1
  },
  "acl": {
    "enabled": true,
    "default_policy": "deny",
    "down_policy": "extend-cache",
    "tokens":{
        "master":"47eca91b-a5e7-e82d-6424-dba7637e0737",
        "agent":"47eca91b-a5e7-e82d-6424-dba7637e0737"
    }
  }
}

启动

经过命令 ： sudo systemctl enable consul-server1 consul-server2 consul-server3将服务设置为开机自动启动，再使用命令启动服务 ： sudo systemctl restart consul-server1 consul-server2 consul-server3

验证

UI

在页面http://127.0.0.1:18500/ui/prometheus/acls/tokens输入配置中的master token，再刷新界面能够在services和nodes中查看到信息

API

$ curl http://127.0.0.1:18500/v1/catalog/nodes #未带token，返回空的节点列表
[]
$ curl http://127.0.0.1:18500/v1/catalog/nodes -H 'x-consul-token: 47eca91b-a5e7-e82d-6424-dba7637e0737' # 经过在header中增长x-consul-token则可返回节点列表
[{"ID":"09d82408-bc4f-49e0-4208-61ef1d4842f7","Node":"server1","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":9,"ModifyIndex":9},{"ID":"613ccd6e-68d1-3bbd-b2a4-3cbc450f019d","Node":"server2","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":7,"ModifyIndex":7},{"ID":"d8a09ffd-7ccb-84bd-7231-8d8b7a01951e","Node":"server3","Address":"10.106.169.121","Datacenter":"prometheus","TaggedAddresses":null,"Meta":null,"CreateIndex":8,"ModifyIndex":8}]

参考

https://learn.hashicorp.com/consul/advanced/day-1-operations/deployment-guide [官方部署方式]

https://www.consul.io/docs/agent/acl-system.html [acl介绍]

https://learn.hashicorp.com/consul/advanced/day-1-operations/acl-guide [acl配置]

https://learn.hashicorp.com/consul/advanced/day-1-operations/agent-encryption [agent通讯加密]