Consul 搭建集群
Consul 搭建集群
Consul 是一套开源的分布式服务发现和配置管理系统，由 HashiCorp 公司用 Go 语言开发。
它具备不少优势。包括： 基于raft协议，简洁,支持健康检查, 同时支持HTTP和DNS协议,支持跨数据中心的WAN集群,提供图形界面,跨平台支持 Linux、Mac、Windows。
快速入门
环境准备
首先须要准备好三台虚拟机,由三台虚机组成一个consul集群。node
| ip | |
|---|---|
| s1 | 172.20.20.20 |
| s2 | 172.20.20.21 |
| s3 | 172.20.20.22 |
基于Vagrant技术，能够快速的配置出三台虚机。linux
在命令行输入web
» mkdir ms » cd ms » vagrant init centos/7
编辑Vagrantfile并配置出三台虚机。shell
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
config.vm.box = "centos/7"
config.vm.define "s1" do |s1|
s1.vm.hostname = "s1"
s1.vm.network "private_network", ip: "172.20.20.20"
end
config.vm.define "s2" do |s2|
s2.vm.hostname = "s2"
s2.vm.network "private_network", ip: "172.20.20.21"
end
config.vm.define "s3" do |s3|
s3.vm.hostname = "s3"
s3.vm.network "private_network", ip: "172.20.20.22"
end
end
启动虚机json
» vagrant up
Bringing machine 's1' up with 'virtualbox' provider...
Bringing machine 's2' up with 'virtualbox' provider...
Bringing machine 's3' up with 'virtualbox' provider...
==> s1: Importing base box 'centos/7'...
==> s1: Matching MAC address for NAT networking...
==> s1: Setting the name of the VM: ms_s1_1528794737477_2031
==> s1: Clearing any previously set network interfaces...
==> s1: Preparing network interfaces based on configuration...
s1: Adapter 1: nat
s1: Adapter 2: hostonly
==> s1: Forwarding ports...
s1: 22 (guest) => 2222 (host) (adapter 1)
==> s1: Booting VM...
==> s1: Waiting for machine to boot. This may take a few minutes...
s1: SSH address: 127.0.0.1:2222
s1: SSH username: vagrant
s1: SSH auth method: private key
s1:
s1: Vagrant insecure key detected. Vagrant will automatically replace
s1: this with a newly generated keypair for better security.
s1:
s1: Inserting generated public key within guest...
s1: Removing insecure key from the guest if it's present...
s1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> s1: Machine booted and ready!
==> s1: Checking for guest additions in VM...
s1: No guest additions were detected on the base box for this VM! Guest
s1: additions are required for forwarded ports, shared folders, host only
s1: networking, and more. If SSH fails on this machine, please install
s1: the guest additions and repackage the box to continue.
s1:
s1: This is not an error message; everything may continue to work properly,
s1: in which case you may ignore this message.
==> s1: Setting hostname...
==> s1: Configuring and enabling network interfaces...
s1: SSH address: 127.0.0.1:2222
s1: SSH username: vagrant
s1: SSH auth method: private key
==> s1: Rsyncing folder: /work/training/vagrant/ms/ => /vagrant
==> s2: Importing base box 'centos/7'...
==> s2: Matching MAC address for NAT networking...
==> s2: Setting the name of the VM: ms_s2_1528794795606_2999
==> s2: Fixed port collision for 22 => 2222. Now on port 2200.
==> s2: Clearing any previously set network interfaces...
==> s2: Preparing network interfaces based on configuration...
s2: Adapter 1: nat
s2: Adapter 2: hostonly
==> s2: Forwarding ports...
s2: 22 (guest) => 2200 (host) (adapter 1)
==> s2: Booting VM...
==> s2: Waiting for machine to boot. This may take a few minutes...
s2: SSH address: 127.0.0.1:2200
s2: SSH username: vagrant
s2: SSH auth method: private key
s2:
s2: Vagrant insecure key detected. Vagrant will automatically replace
s2: this with a newly generated keypair for better security.
s2:
s2: Inserting generated public key within guest...
s2: Removing insecure key from the guest if it's present...
s2: Key inserted! Disconnecting and reconnecting using new SSH key...
==> s2: Machine booted and ready!
==> s2: Checking for guest additions in VM...
s2: No guest additions were detected on the base box for this VM! Guest
s2: additions are required for forwarded ports, shared folders, host only
s2: networking, and more. If SSH fails on this machine, please install
s2: the guest additions and repackage the box to continue.
s2:
s2: This is not an error message; everything may continue to work properly,
s2: in which case you may ignore this message.
==> s2: Setting hostname...
==> s2: Configuring and enabling network interfaces...
s2: SSH address: 127.0.0.1:2200
s2: SSH username: vagrant
s2: SSH auth method: private key
==> s2: Rsyncing folder: /work/training/vagrant/ms/ => /vagrant
==> s3: Importing base box 'centos/7'...
==> s3: Matching MAC address for NAT networking...
==> s3: Setting the name of the VM: ms_s3_1528794863986_43122
==> s3: Fixed port collision for 22 => 2222. Now on port 2201.
==> s3: Clearing any previously set network interfaces...
==> s3: Preparing network interfaces based on configuration...
s3: Adapter 1: nat
s3: Adapter 2: hostonly
==> s3: Forwarding ports...
s3: 22 (guest) => 2201 (host) (adapter 1)
==> s3: Booting VM...
==> s3: Waiting for machine to boot. This may take a few minutes...
s3: SSH address: 127.0.0.1:2201
s3: SSH username: vagrant
s3: SSH auth method: private key
s3:
s3: Vagrant insecure key detected. Vagrant will automatically replace
s3: this with a newly generated keypair for better security.
s3:
s3: Inserting generated public key within guest...
s3: Removing insecure key from the guest if it's present...
s3: Key inserted! Disconnecting and reconnecting using new SSH key...
==> s3: Machine booted and ready!
==> s3: Checking for guest additions in VM...
s3: No guest additions were detected on the base box for this VM! Guest
s3: additions are required for forwarded ports, shared folders, host only
s3: networking, and more. If SSH fails on this machine, please install
s3: the guest additions and repackage the box to continue.
s3:
s3: This is not an error message; everything may continue to work properly,
s3: in which case you may ignore this message.
==> s3: Setting hostname...
==> s3: Configuring and enabling network interfaces...
s3: SSH address: 127.0.0.1:2201
s3: SSH username: vagrant
s3: SSH auth method: private key
==> s3: Rsyncing folder: /work/training/vagrant/ms/ => /vagrant
至此3台待测试的虚机就准备完成了bootstrap
单机安装
登陆到虚机s1,切换用户到rootcentos
» vagrant ssh s1 [vagrant@s1 ~]$ su Password: [root@s1 vagrant]#
安装一些依赖的工具api
[root@s1 vagrant]# yum install -y epel-release [root@s1 vagrant]# yum install -y jq [root@s1 vagrant]# yum install -y unzip
下载1.1.0版本到/tmp目录下浏览器
[root@s1 vagrant]# cd /tmp/ [root@s1 tmp]# curl -s https://releases.hashicorp.com/consul/1.1.0/consul_1.1.0_linux_amd64.zip -o consul.zip
解压，并赋予consul可执行权限，最后把consul移动到/usr/bin/下ruby
[root@s1 tmp]unzip consul.zip [root@s1 tmp]chmod +x consul [root@s1 tmp]mv consul /usr/bin/consul
检查consul是否安装成功
[root@s1 tmp]# consul
Usage: consul [--version] [--help] <command> [<args>]
Available commands are:
agent Runs a Consul agent
catalog Interact with the catalog
event Fire a new event
exec Executes a command on Consul nodes
force-leave Forces a member of the cluster to enter the "left" state
info Provides debugging information for operators.
join Tell Consul agent to join cluster
keygen Generates a new encryption key
keyring Manages gossip layer encryption keys
kv Interact with the key-value store
leave Gracefully leaves the Consul cluster and shuts down
lock Execute a command holding a lock
maint Controls node or service maintenance mode
members Lists the members of a Consul cluster
monitor Stream logs from a Consul agent
operator Provides cluster-level tools for Consul operators
reload Triggers the agent to reload configuration files
rtt Estimates network round trip time between nodes
snapshot Saves, restores and inspects snapshots of Consul server state
validate Validate config files/directories
version Prints the Consul version
watch Watch for changes in Consul
出现如上所示表明安装成功。
批量安装
这里只是s1虚机安装完成，还有s2和s3须要再次的安装，比较费事。还好 Vagrant 提供了一个 shell 脚本的支持。
稍微对Vagrantfile文件作的修改，在第4行增长script脚本的定义
$script = <<SCRIPT
echo "Installing dependencies ..."
yum install -y epel-release
yum install -y jq
yum install -y unzip
echo "Determining Consul version to install ..."
CHECKPOINT_URL="https://checkpoint-api.hashicorp.com/v1/check"
if [ -z "$CONSUL_DEMO_VERSION" ]; then
CONSUL_DEMO_VERSION=$(curl -s "${CHECKPOINT_URL}"/consul | jq .current_version | tr -d '"')
fi
echo "Fetching Consul version ${CONSUL_DEMO_VERSION} ..."
cd /tmp/
curl -s https://releases.hashicorp.com/consul/${CONSUL_DEMO_VERSION}/consul_${CONSUL_DEMO_VERSION}_linux_amd64.zip -o consul.zip
echo "Installing Consul version ${CONSUL_DEMO_VERSION} ..."
unzip consul.zip
sudo chmod +x consul
sudo mv consul /usr/bin/consul
SCRIPT
在指定box的下一行 增长执行script脚本
config.vm.box = "centos/7" config.vm.provision "shell", inline: $script
销毁虚机并从新初始化启动
» vagrant destroy
s3: Are you sure you want to destroy the 's3' VM? [y/N] y
==> s3: Forcing shutdown of VM...
==> s3: Destroying VM and associated drives...
s2: Are you sure you want to destroy the 's2' VM? [y/N] y
==> s2: Forcing shutdown of VM...
==> s2: Destroying VM and associated drives...
s1: Are you sure you want to destroy the 's1' VM? [y/N] y
==> s1: Forcing shutdown of VM...
==> s1: Destroying VM and associated drives...
» vagrant up
...
安装过程略长，等待脚本执行完成后登陆s1,s2,s3 执行consul验证是否安装成功
启动 Agent
这里须要解释consul一些基本概念和启动参数
基本概念
- agent 组成 consul 集群的每一个成员上都要运行一个 agent，能够经过 consul agent 命令来启动。agent 能够运行在 server 状态或者 client 状态。天然的，运行在 server 状态的节点被称为 server 节点；运行在 client 状态的节点被称为 client 节点。
- client consul的client模式，就是客户端模式。是consul节点的一种模式，这种模式下，全部注册到当前节点的服务会被转发到server，自己是不持久化这些信息。
- server 表示consul的server模式，代表这个consul是个server，这种模式下，功能和client都同样，惟一不一样的是，它会把全部的信息持久化的本地，这样遇到故障，信息是能够被保留的。
启动参数
- bootstrap-expect 集群指望的节点数，只有节点数量达到这个值才会选举leader
- server 运行在server模式
- data-dir 指定数据目录，其余的节点对于这个目录必须有读的权限
- node 指定节点的名称
- bind 为该节点绑定一个地址
- config-dir 指定配置文件，定义服务的，默认全部一.json结尾的文件都会读
- enable-script-checks=true 设置检查服务为可用
- datacenter 数据中心名称，
- join 加入到已有的集群中
- ui 使用自带的ui
- client 指定web ui、的监听地址，默认127.0.0.1只能本机访问，改成0.0.0.0可外网访问
先配置一个单机节点启动,切换用户为root
[root@s1 vagrant]# consul agent -server -bootstrap-expect 1 -data-dir /etc/consul.d -node=node1 -bind=172.20.20.20 -ui -client 0.0.0.0
打开浏览器输入http://172.20.20.20:8500/显示web界面表明启动成功
搭建服务集群
登陆虚机s1，切换用户为root，启动consul 设置节点数为3
[root@s1 vagrant]# consul agent -server -bootstrap-expect 3 -data-dir /etc/consul.d -node=node1 -bind=172.20.20.20 -ui -client 0.0.0.0
登陆虚机s2，切换用户为root，启动consul 设置节点数为3 并加入到s1中,注意 node名称不能重复
[root@s2 vagrant]# consul agent -server -bootstrap-expect 3 -data-dir /etc/consul.d -node=node2 -bind=172.20.20.21 -ui -client 0.0.0.0 -join 172.20.20.20
登陆虚机s3，重复在s2中的操做
[root@s3 vagrant]# consul agent -server -bootstrap-expect 3 -data-dir /etc/consul.d -node=node3 -bind=172.20.20.22 -ui -client 0.0.0.0 -join 172.20.20.20
这时再刷新web界面就能看到三个节点。
至此，集群搭建完成。
进阶操做
集群成员
在虚机s1的另外一个终端上运行consul members，能够看到Consul集群的成员。
[root@s1 vagrant]# consul members Node Address Status Type Build Protocol DC Segment node1 172.20.20.20:8301 alive server 1.1.0 2 dc1 <all> node2 172.20.20.21:8301 alive server 1.1.0 2 dc1 <all> node3 172.20.20.22:8301 alive server 1.1.0 2 dc1 <all>
查询节点
安装dig工具
[root@s1 vagrant]# yum install -y bind-utils
[root@s1 vagrant]# dig @172.20.20.20 -p 8600 node2.node.consul ; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7 <<>> @172.20.20.20 -p 8600 node2.node.consul ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38194 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;node2.node.consul. IN A ;; ANSWER SECTION: node2.node.consul. 0 IN A 172.20.20.21 ;; Query time: 33 msec ;; SERVER: 172.20.20.20#8600(172.20.20.20) ;; WHEN: Tue Jun 12 15:49:53 UTC 2018 ;; MSG SIZE rcvd: 62
离开集群
要离开集群，能够正常退出代理（使用Ctrl-C）。
在虚机s2上按Ctrl-C，在虚机s1上再次查询集群成员。
[root@s1 vagrant]# consul members Node Address Status Type Build Protocol DC Segment node1 172.20.20.20:8301 alive server 1.1.0 2 dc1 <all> node2 172.20.20.21:8301 failed server 1.1.0 2 dc1 <all> node3 172.20.20.22:8301 alive server 1.1.0 2 dc1 <all>
能够看到node2的状态为failed。
从新在虚机s2上启动Agent。在s2的另外一个终端上输入
[root@s2 vagrant]# consul leave Graceful leave complete
在虚机s1上查询集群成员
[root@s1 vagrant]# consul members Node Address Status Type Build Protocol DC Segment node1 172.20.20.20:8301 alive server 1.1.0 2 dc1 <all> node2 172.20.20.21:8301 left server 1.1.0 2 dc1 <all> node3 172.20.20.22:8301 alive server 1.1.0 2 dc1 <all>
node2的状态为left，离开集群成功！