Kubernetes + Docker 集群部署

时间 2019-11-11

标签 kubernetes docker 集群部署栏目 Docker 繁體版

原文原文链接

环境准备：node

　　master：192.168.1.118 系统：CentOS 7.3linux

　　node1：192.168.1.155 系统：CentOS 7.3git

　　node2：192.168.1.156 　系统：CentOS 7.3github

　　（1）借助于NTP服务设定各节点时间精确同步；若节点可直接访问互联网，直接启动chronyd系统服务，并设定随系统引导而启动docker

　　（2）经过DNS完成各节点的主机名称解析，测试环境主机数量较少时也可使用hosts文件进行；bootstrap

　　（3）关闭各节点的iptables或firewalld服务，确保它们被禁止随系统引导过程启动；vim

　　（4）各节点禁用SELinuxcentos

　　（5）各节点禁用全部Swap设备；api

　　（6）若要使用ipvs模型的proxy，各节点还须要载入ipvs相关的各模块；网络

[Master + Node基础配置]

#解析主机名

　　192.168.1.118 master

　　192.168.1.155 node1

　　192.168.1.156 node2

#部署集群时，kubeadm默认会预先检查当前主机是否禁用了Swap设备，并在未禁用时强制终止部署过程；所以，在主机内存资源充裕的条件下，须要禁用全部Swap设备，不然，就须要在后文kubeadm init及kubeadm join命令执行时额外使用相关的选项忽略检查错误。

关闭Swap设备，首先关闭当前已启用的全部Swap设备：

　　 ~]#swapoff -a

然后编辑/etc/fstab配置文件，注释用于挂载Swap设备的全部行

#下载阿里云docker-ce源

~]# cd /etc/yum.repos.d/

　　wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

#安装docker和kubernetes服务

~]# yum install docker-ce -y

#若要经过默认的k8s.gcr.io镜像残酷获取Kubernetes系统组件的相关镜像，须要配置docker Unit File (/usr/lib/systemd/system/docker.service文件)中的Environment变量，为其定义合用的HTTPS_PROXY

~]# vim /usr/lib/systemd/system/docker.service

　　.......

　　[Service]
　　Type=notify
　　# the default is not to use systemd for cgroups because the delegate issues still
　　# exists and systemd currently does not support the cgroup feature set required
　　# for containers run by docker
　　Environment="HTTPS_PROXY=http://www.ik8s.io:10070"
　　Environment="NO_PROXY=127.0.0.0/8,192.168.1.0/24"         #本地IP访问无需代理
　　ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
　　ExerStartPost=/usr/bin/iptables -P FORWARD ACCEPT           #docker自1.13版本起会自动设置iptables的FORWARD默认策略为DROP，这可能会影响Kubernetes集群依赖的报文转发功能，所以须要在docker服务启动后，从新将FORWARD链的默认策略设备为ACCEPT，在“ExecStart=/usr/bin/dockerd”添加这一段
　　ExecReload=/bin/kill -s HUP $MAINPID
　　TimeoutSec=0
　　RestartSec=2
　　Restart=always

　　.......

~]# systemctl daemon-reload #重载配置文件
~]# systemctl start docker #启动docker

#跟桥接设置的两个参数必须为1

~]# sysctl -a | grep bridge

~]# vim /etc/sysctl.d/k8s.conf

　　net.bridge.bridge-nf-call-ip6tables = 1

　　net.bridge.bridge-nf-call-iptables = 1

~]# sysctl -p /etc/sysctl.d/k8s.conf #让系统重读一下配置

#建立kubernetes阿里云源

~]# cd /etc/yum.repos.d/

yum.repos.d]# vim kubernetes.repo

　　[kubernetes]
　　name=Kubernetes Repository
　　baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
　　gpgcheck=1
　　gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg

~]# yum install kubelet kubectl kubeadm -y

~]# systemctl enable docker kubelet #将docker和kubelet设置开机自启动

#开始下载kubernetes组件的镜像文件，若是无法直接链接到k8s官网则从国内下载镜像文件而后打标签便可

~]# rpm -qa|grep kubeadm #下载镜像前要先确认系统中kubeadm的版本，版本要匹配不然会报错
kubeadm-1.13.4-0.x86_64

~]# kubeadm config images pull
[config/images] Pulled k8s.gcr.io/kube-apiserver:v1.13.4
[config/images] Pulled k8s.gcr.io/kube-controller-manager:v1.13.4
[config/images] Pulled k8s.gcr.io/kube-scheduler:v1.13.4
[config/images] Pulled k8s.gcr.io/kube-proxy:v1.13.4
[config/images] Pulled k8s.gcr.io/pause:3.1
[config/images] Pulled k8s.gcr.io/etcd:3.2.24
[config/images] Pulled k8s.gcr.io/coredns:1.2.6

#确认镜像文件是否下载完成

[Master 配置]

#若未禁用Swap设备，则须要编辑kubelet的配置文件/etc/sysconfig/kubelet，设置忽略Swap启用的状态错误，内容以下：

~]# vim /etc/sysconfig/kubelet

　　KUBELET_EXTRA_ARGS="--fail-swap-on=false"

#初始化kubernetes，10.244.0.0/16是flannel默认的网段因此咱们这里不作更改，

~]# kubeadm init --kubernetes-version="v1.13.4" --pod-network-cidr="10.244.0.0/16" --ignore-preflight-errors=Swap

　 ....... 　　

　　Your Kubernetes master has initialized successfully!

　　To start using your cluster, you need to run the following as a regular user:

　　 mkdir -p $HOME/.kube
　　 sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
　　 sudo chown $(id -u):$(id -g) $HOME/.kube/config

　　You should now deploy a pod network to the cluster.
　　Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
　　 https://kubernetes.io/docs/concepts/cluster-administration/addons/

　　You can now join any number of machines by running the following on each node
　　as root:

　　kubeadm join 192.168.1.118:6443 --token gx1knl.wts9qo4ebghwk242 --discovery-token-ca-cert-hash sha256:bd7bb24b445dc95f0571c501bdc4e82aa23fdc8a7194a571790923b7d4b10468 #这段要记录起来，用来给node加入到集群中用

#确认6443端口是否已经启用了

#要开始使用集群，须要以常规用户身份运行如下命令（为了方便这里就用root用户执行了）

~]# mkdir -p $HOME/.kube

~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

#开始安装网络插件flannel，Kubernetes版本大于1.7+的能够直接执行下面这条命令下载

~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

clusterrole.rbac.authorization.k8s.io/flannel created

clusterrolebinding.rbac.authorization.k8s.io/flannel created

serviceaccount/flannel created configmap/kube-flannel-cfg created

daemonset.extensions/kube-flannel-ds-amd64 created

daemonset.extensions/kube-flannel-ds-arm64 created

daemonset.extensions/kube-flannel-ds-arm created

daemonset.extensions/kube-flannel-ds-ppc64le created

daemonset.extensions/kube-flannel-ds-s390x created

#flannel彻底下载完后，查看kube-system组件的运行状态，全部状态都显示Running表示正常

[Node 配置]

#忽略Swap错误

~]# vim /etc/sysconfig/kubelet

　　KUBELET_EXTRA_ARGS="--fail-swap-on=false"

#两个Node节点加入到kubernetes集群中

~]# kubeadm join 192.168.1.118:6443 --token gx1knl.wts9qo4ebghwk242 --discovery-token-ca-cert-hash sha256:bd7bb24b445dc95f0571c501bdc4e82aa23fdc8a7194a571790923b7d4b10468 --ignore-preflight-errors=Swap

　　[preflight] Running pre-flight checks
   　　 [WARNING Swap]: running with swap on is not supported. Please disable swap
   　　[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 18.09.3. Latest validated version: 18.06
　　[discovery] Trying to connect to API Server "192.168.1.118:6443"
　　[discovery] Created cluster-info discovery client, requesting info from "https://192.168.1.118:6443"
　　[discovery] Requesting info from "https://192.168.1.118:6443" again to validate TLS against the pinned public key
　　[discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "192.168.1.118:6443"
　　[discovery] Successfully established connection with API Server "192.168.1.118:6443"
　　[join] Reading configuration from the cluster...
　　[join] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
　　[kubelet] Downloading configuration for the kubelet from the "kubelet-config-1.13" ConfigMap in the kube-system namespace
　　[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
　　[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
　　[kubelet-start] Activating the kubelet service
　　[tlsbootstrap] Waiting for the kubelet to perform the TLS Bootstrap...
　　[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "n2" as an annotation

　　This node has joined the cluster:            #看到这个信息表示这个节点已经加入到集群中了
　　* Certificate signing request was sent to apiserver and a response was received.
　　* The Kubelet was informed of the new secure connection details.

　　Run 'kubectl get nodes' on the master to see this node join the cluster.

~]# mkdir -p $HOME/.kube

#全部 Node节点都加入到集群之后，在Mstar主机上就能看到全部节点状态已经为Ready就绪状态

#Master传输认证文件到Node节点，Node链接集群必需要须要证书认证

~]# scp /etc/kubernetes/admin.conf root@192.168.1.155:/root/.kube/config

~]# scp /etc/kubernetes/admin.conf root@192.168.1.156:/root/.kube/config

Kubernets集群搭建完毕