Ubuntu18使用kubeadm安装kubernetes1.12

安装前准备

virtualbox安装三个Ubuntu虚拟机

k8s-master 和 k8s-slave一、k8s-slave2 为了上网，都是双网卡（NAT和Host-only）

关闭防火墙

sudo ufw disable

关闭swap

sudo swapoff -a

安装Docker

卸载旧docker

sudo apt-get remove docker docker-engine docker.io

完全删除docker-ce和docker-cli

sudo apt-get autoremove docker-ce

安装依赖

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

添加docker的gpg key

为了使用https访问，使用阿里云加速

curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -

设置docker镜像源

sudo add-apt-repository \
   "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

更新源

sudo apt update && sudo apt-get update

安装docker 18.06

查看docker-ce版本

apt-cache madison docker-ce

安装指定版本

sudo apt-get install -y docker-ce=18.06.3~ce~3-0~ubuntu

若是由于libltdl版本低，能够安装新版本 wget http://launchpadlibrarian.net/236916213/libltdl7_2.4.6-0.1_amd64.deb && sudo dkpg -i libltdl7_2.4.6-0.1_amd64.deb

默认启动

sudo systemctl enable docker && sudo systemctl start docker

修改docker启动参数，进行加速

$ sudo tee /etc/docker/daemon.json <<-'EOF'
{
 "registry-mirrors": ["https://xxxxxxxx.mirror.aliyuncs.com"]
}
EOF

将xxxxxx替换为在阿里云申请的路径

安装kubernetes

更新apt源

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

sudo curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -

sudo tee /etc/apt/sources.list.d/kubernetes.list <<-'EOF'
deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main
EOF

sudo apt-get update

安装kubectl kubeadm kubelet

在master和node都安装

sudo apt-get install -y kubelet=1.12.8-00 kubeadm=1.12.8-00 kubectl=1.12.8-00
sudo apt-mark hold kubelet=1.12.8-00 kubeadm=1.12.8-00 kubectl=1.12.8-00

启动kubelet

sudo systemctl enable kubelet && sudo systemctl start kubelet

初始化集群

sudo kubeadm init --apiserver-advertise-address 192.168.56.112 --pod-network-cidr=10.244.0.0/16 --kubernetes-version=1.12.8

安装网络插件

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

加入node

sudo kubeadm join 192.168.56.112:6443 --token xxxxxx --discovery-token-ca-cert-hash sha256:xxxxxxxx

将以前kubeadm init后，输出的加入node指令拷贝到node执行。

遇到的问题

没法启动kubelet

journalctl -u kubelet 看到报错： failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/ku 这个不影响，不用关系，启动了kubernetes后就会正常

kubectl没法连上服务

执行kubectl看到报错 The connection to the server localhost:8080 was refused - did you specify the right host or port? 这是因为kubernetes默认不开放http 8080访问。

能够在/etc/kubernetes/manifests/kube-apiserver.yaml看到开启了https 6443访问。 也能够经过sudo netstat -apn | grep tcp | grep LISTEN看到开的的端口

能够使用sudo kubectl --kubeconfig=/etc/kubernetes/admin.conf api-versions访问，admin.conf中有端口以及key

使用阿里云apt源致使没法安装

备份/etc/apt/sources.list后替换为如下内容

deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

执行sudo apt update && sudo apt-get update更新apt源。可是致使了 package xxx has no installation candidate，使用Ubuntu官方源就OK

pod状态异常

安装完集群后，生成一个pod，pod一直是pending状态。查看pod的描述

sudo kubectl describe pod helloworld

pod描述信息

.......
Events:
  Type     Reason            Age                From               Message
  ----     ------            ----               ----               -------
  Warning  FailedScheduling  32s (x2 over 32s)  default-scheduler  0/3 nodes are available: 3 node(s) had taints that the pod didn't tolerate.

执行kubectl taint nodes --all node-role.kubernetes.io/master-，node正常使用参与调度

node状态异常

service没法访问，通查看node状态

sudo kubectl get pod -o yaml

存储node网络异常状况

......
conditions:
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has sufficient disk space available
      reason: KubeletHasSufficientDisk
      status: "False"
      type: OutOfDisk
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has sufficient memory available
      reason: KubeletHasSufficientMemory
      status: "False"
      type: MemoryPressure
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has no disk pressure
      reason: KubeletHasNoDiskPressure
      status: "False"
      type: DiskPressure
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: kubelet has sufficient PID available
      reason: KubeletHasSufficientPID
      status: "False"
      type: PIDPressure
    - lastHeartbeatTime: 2019-05-21T14:39:22Z
      lastTransitionTime: 2019-05-21T13:08:37Z
      message: 'runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
        message:docker: network plugin is not ready: cni config uninitialized'
      reason: KubeletNotReady
      status: "False"
      type: Ready
......

第一次出现问题是由于没有安装flannel网络插件。

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

安装完毕网络插件后，提示cni config uninitialized。

先关闭node上的kubelet； 而后使用kubeadm reset|| kubeadm join ..., 将node从新加入集群

系统宕机后重启服务

1. 关闭防火墙
2. 关闭swap
3. 重启docker
4. 重启kubelet
5. 而后使用docker先重启pause容器，而后在重启其余容器
6. 也能够使用kubeadm reset重置kubernetes，而后从新初始化集群
7. 注意使用journal查看docker和kubelet日志，另外要主要docker和kubelet的systemd启动参数

参考资料

基于 Ubuntu 18.04 安装部署K8s集群 Ubuntu-18.04使用kubeadm安装kubernetes-1.12.0 kubeadm安装k8s集群 apt-get 软件源文件格式以及ubuntu 18.04更换阿里源 [cni config uninitialized解决方案](cni config uninitialized "cni config uninitialized解决方案") cni config uninitialized