[转] etcd 搭建与使用

[From] https://blog.csdn.net/ShouTouDeXingFu/article/details/81167302

 

 

etcd

一、下载etcd二进制文件包
         https://github.com/coreos/etcd/releases/download/v3.1.3/etcd-v3.1.3-linux-amd64.tar.gz

二、解压缩etcd-v3.1.3-linux-amd64.tar.gz，将其中的etcd和etcdctl两个可执行文件复制到各节点的/usr/bin目录
三、在各节点建立etcd的数据目录
         mkdir -p /data/etcd

         mkdir /etc/etcd

四、在每一个节点上建立etcd的systemd unit文件/usr/lib/systemd/system/etcd.service 和  /etc/etcd/etcd.conf  
  

etcd.service 

[Unit]

Description=Etcd Server

After=network.target

After=network-online.target

Wants=network-online.target

Documentation=https://github.com/coreos

[Service]

Type=notify

WorkingDirectory=/var/lib/etcd/

EnvironmentFile=-/etc/etcd/etcd.conf

ExecStart=/usr/bin/etcd \

--name ${ETCD_NAME} \

--cert-file=${ETCD_CERT_FILE} \

--key-file=${ETCD_KEY_FILE} \

--peer-cert-file=${ETCD_PEER_CERT_FILE} \

--peer-key-file=${ETCD_PEER_KEY_FILE} \

--trusted-ca-file=${ETCD_TRUSTED_CA_FILE} \

--peer-trusted-ca-file=${ETCD_PEER_TRUSTED_CA_FILE} \

--initial-advertise-peer-urls ${ETCD_INITIAL_ADVERTISE_PEER_URLS} \

--listen-peer-urls ${ETCD_LISTEN_PEER_URLS} \

--listen-client-urls ${ETCD_LISTEN_CLIENT_URLS} \

--advertise-client-urls ${ETCD_ADVERTISE_CLIENT_URLS} \

--initial-cluster-token ${ETCD_INITIAL_CLUSTER_TOKEN} \

--initial-cluster ${ETCD_INITIAL_CLUSTER} \

--initial-cluster-state ${ETCD_INITIAL_CLUSTER_STATE} \

--data-dir=${ETCD_DATA_DIR}

Restart=on-failure

RestartSec=5

LimitNOFILE=65536

[Install]

WantedBy=multi-user.target

 

etcd.conf    etcd集群中配置文件标红的配置不同，拷贝出来修改便可

ETCD_NAME="etcd1"

#20180108 by lys
#ETCD_CERT_FILE="/etc/k8s/kubernetes.pem"
#ETCD_KEY_FILE="/etc/k8s/kubernetes-key.pem"
#ETCD_PEER_CERT_FILE="/etc/k8s/kubernetes.pem"
#ETCD_PEER_KEY_FILE="/etc/k8s/kubernetes-key.pem"
#ETCD_TRUSTED_CA_FILE="/etc/k8s/ca.pem"
#ETCD_PEER_TRUSTED_CA_FILE="/etc/k8s/ca.pem"

ETCD_INITIAL_ADVERTISE_PEER_URLS="http://etcd1.server.biz:2380"
ETCD_LISTEN_PEER_URLS="http://etcd1.server.biz:2380"
ETCD_LISTEN_CLIENT_URLS="http://etcd1.server.biz:2379,http://127.0.0.1:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://etcd1.server.biz:2379"

ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-0"
ETCD_INITIAL_CLUSTER="etcd1=http://etcd1.server.biz:2380,etcd2=http://etcd2.server.biz:2380,etcd3=http://etcd3.server.biz:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_DATA_DIR="/data/etcd"

 

配置信息说明
上面在启动参数中指定了etcd的工做目录和数据目录是/data/etcd
--cert-file和--key-file分别指定etcd的公钥证书和私钥
--peer-cert-file和--peer-key-file分别指定了etcd的Peers通讯的公钥证书和私钥。
--trusted-ca-file指定了客户端的CA证书
--peer-trusted-ca-file指定了Peers的CA证书
--initial-cluster-state new表示这是新初始化集群，--name指定的参数值必须在--initial-cluster中
 

 

五、启动etcd
    systemctl daemon-reload

    systemctl enable etcd

    systemctl start etcd

    systemctl status etcd

 

六、检查集群是否健康，在任一节点执行
              export ETCDCTL_API=3         

etcdctl --endpoints=http://etcd2.server.biz:2379,http://etcd1.server.biz:2379,http://etcd3.server.biz:2379   member list

       

etcdctl --endpoints=http://etcd2.server.biz:2379,http://etcd1.server.biz:2379,http://etcd3.server.biz:2379  endpoint status  -w table

 

etcdctl --endpoints=http://etcd2.server.biz:2379,http://etcd1.server.biz:2379,http://etcd3.server.biz:2379  endpoint health  -w table

          

    sudo journalctl -u etcd.service -l --no-pager|less   查看日志

 

七、etcd集群成员扩容增长
etcdctl --endpoints=http://etcd2.server.biz:2379,http://etcd1.server.biz:2379,http://etcd3.server.biz:2379 member add etcd4   --peer-urls=http://etcd4.server.biz:2380 
Member 5b316fb0b94e59f8 added to cluster f0364a493c8924dc

ETCD_NAME="etcd4"
ETCD_INITIAL_CLUSTER="etcd1=http://etcd1.server.biz:2380,etcd3=http://etcd3.server.biz:2380,etcd2=http://etcd2.server.biz:2380,etcd4=http://etcd4.server.biz:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"

          此时，集群会为目标节点生成一个惟一的member ID  

注意:新节点加入集群是做为一个全新的节点加入，若是data-dir有数据，etcd启动时会读取己经存在的数据，启动时仍然用的老member ID,也会形成，集群不没法加入，因此必定要清空新节点的data-dir   
新扩容的etcd机器的etcd.conf中配置修改成 ETCD_INITIAL_CLUSTER_STATE="existing"
删除 rm-rf  /data/etcd/*
启动etcd服务便可
 

 

常见问题
1) systemd: Failed at step CHDIR spawning /usr/bin/etcd: No such file or directory
    经过查看ll /usr/bin/etcd  启动文件是存在的，后来发现etcd.service里面指定了WorkingDirectory=/var/lib/etcd/  没有建立对应目录致使，建立了/var/lib/etcd/ 目录应用启动成功
 

2)etcd 添加集群成员报错Jul 23 16:58:25 bjzw_82_137 etcd: failed to process raft message (raft: stopped)Jul 23 16:58:25 bjzw_82_137 etcd: failed to find member c79c567db2d16966 in cluster f0364a493c8924dcJul 23 16:58:25 bjzw_82_137 etcd: failed to find member c79c567db2d16966 in cluster f0364a493c8924dcJul 23 16:58:25 bjzw_82_137 etcd: failed to find member b13dd19d11c8d25d in cluster f0364a493c8924dcJul 23 16:58:25 bjzw_82_137 etcd: failed to find member b13dd19d11c8d25d in cluster f0364a493c8924dc须要将etcd.conf中的配置修改成