Centos7.6 使用kubeadm部署Kubernetes1.15集群

1、环境准备

172.16.8.101    k8s-master01
172.16.8.102    k8s-master02
172.16.8.103    k8s-master03
172.16.8.104    k8s-master04
172.16.8.105    k8s-master05
172.16.8.106    k8s-master06
#harbor
172.16.8.201    k8s-node01

一、升级内核

CentOS 7.x 系统自带的 3.10.x 内核存在一些 Bugs，致使运行的 Docker、Kubernetes 不稳定，例如：
高版本的 docker(1.13 之后) 启用了 3.10 kernel 实验支持的 kernel memory account 功能(没法关闭)，当节点压力大如频繁启动和中止容器时会致使 cgroup memory leak；
网络设备引用计数泄漏，会致使相似于报错："kernel:unregister_netdevice: waiting for eth0 to become free. Usage count = 1";
https://blog.51cto.com/10880347/2420861
https://blog.csdn.net/xiegh2014/article/details/96192233

二、设置/etc/hosts

cat >> /etc/hosts<<EOF
172.16.8.101    k8s-master01
172.16.8.102    k8s-master02
172.16.8.103    k8s-master03
172.16.8.104    k8s-master04
172.16.8.105    k8s-master05
172.16.8.106    k8s-master06
#harbor
172.16.8.201    k8s-node01
EOF

三、关闭防火墙及SELINUX

https://devops51.blog.csdn.net/article/details/53031781

四、关闭swap分区

swapoff -a
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

五、Kernel性能调优

必须关闭 tcp_tw_recycle，不然和NAT 冲突，会致使服务不通；
关闭 IPV6，防止触发 docker BUG；

cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0 
vm.overcommit_memory=1 
vm.panic_on_oom=0 
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF

sysctl -p /etc/sysctl.d/k8s.conf

六、加载内核模块

modprobe ip_vs_rr
modprobe br_netfilter

七、设置系统时区

调整系统 TimeZone
timedatectl set-timezone Asia/Shanghai

将当前的 UTC 时间写入硬件时钟
timedatectl set-local-rtc 0

重启依赖于系统时间的服务
systemctl restart rsyslog 
systemctl restart crond

yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget
yum install -y yum-utils device-mapper-persistent-data lvm2

使用阿里云Docker Yum源
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

选择安装的版本

yum list docker-ce --showduplicates | sort -r

已加载插件：fastestmirror, langpacks
可安装的软件包
Loading mirror speeds from cached hostfile

• epel: mirror01.idc.hinet.net
• elrepo: mirror.rackspace.com
  docker-ce.x86_64 3:19.03.1-3.el7 docker-ce-stable
  docker-ce.x86_64 3:19.03.0-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.8-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.7-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.6-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.5-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.4-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.3-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.2-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable
  docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.3.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.2.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.06.0.ce-3.el7 docker-ce-stable
  docker-ce.x86_64 18.03.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 18.03.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.12.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.12.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.09.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.09.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.06.2.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.06.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.06.0.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.03.3.ce-1.el7 docker-ce-stable
  docker-ce.x86_64 17.03.2.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.03.1.ce-1.el7.centos docker-ce-stable
  docker-ce.x86_64 17.03.0.ce-1.el7.centos docker-ce-stable

yum install docker-ce-18.09.8-3.el7 -y

启动DOcker
systemctl start docker
systemctl enable docker

cat > /etc/docker/daemon.json<<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"registry-mirrors" : [
"https://ot2k4d59.mirror.aliyuncs.com/"
]
}
EOF
配置Docker镜像仓库必须重启
systemctl restart docker

查看Docker信息
[root@k8s-master01 ~]# docker info
Client:
Debug Mode: false

Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.09.8
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.4.183-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.936GiB
Name: k8s-master01
ID: YZ5Q:2IUX:THW4:Z7DF:GK43:QG4O:D2M2:DVUL:3YI3:G3Y3:J4QE:5A64
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://ot2k4d59.mirror.aliyuncs.com/
Live Restore Enabled: false
Product License: Community Engine

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes