K8S二进制部署node节点

本文介绍node节点的二进制部署过程。

1、软件包下载地址
Node包：https://dl.k8s.io/v1.9.6/kubernetes-node-linux-amd64.tar.gz

# tar -zxvpf kubernetes-node-linux-amd64.tar.gz 
# mv kubernetes/node/bin/kube* /usr/local/sbin/

2、在master上建立kubelet-bootstrap 用户并绑定system:node-bootstrapper 角色
kubelet 启动时向 kube-apiserver 发送 TLS bootstrapping 请求，须要先将 bootstrap token 文件中的 kubelet-bootstrap 用户赋予 system:node-bootstrapper 角色，而后 kubelet 才有权限建立认证请求(certificatesigningrequests)。-user=kubelet-bootstrap 是前文master节点上文件 /etc/kubernetes/token.csv 中指定的用户名

在master节点上运行：

# kubectl create clusterrolebinding kubelet-bootstrap \
--clusterrole=system:node-bootstrapper \
--user=kubelet-bootstrap
clusterrolebinding "kubelet-bootstrap" created

3、node节点上设置kubelet
一、设置集群参数

# kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/ssl/etcd/ca.pem \
  --embed-certs=true \
  --server=https://192.168.115.5:6443 \
  --kubeconfig=bootstrap.kubeconfig

二、设置客户端认证参数，token一样来自前文master节点上文件 /etc/kubernetes/token.csv 

# kubectl config set-credentials kubelet-bootstrap \
  --token=3e6916ba861192f279c67d827952ea30 \
  --kubeconfig=bootstrap.kubeconfig

三、设置上下文参数

# kubectl config set-context default \
  --cluster=kubernetes \
  --user=kubelet-bootstrap \
  --kubeconfig=bootstrap.kubeconfig

四、设置默认上下文

# kubectl config use-context default --kubeconfig=bootstrap.kubeconfig
# mv bootstrap.kubeconfig /etc/kubernetes/

五、建立工做目录

# mkdir /var/lib/kubelet

六、配置kubelet启动脚本

# cat /usr/lib/systemd/system/kubelet.service 
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
ExecStart=/usr/local/sbin/kubelet \
  --address=192.168.115.6 \
  --hostname-override=192.168.115.6 \
  --pod-infra-container-image=gcr.io/google_containers/pause-amd64:3.0 \
  --experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig \
  --kubeconfig=/etc/kubernetes/kubelet.kubeconfig \
  --cert-dir=/etc/ssl/kubernetes \
  --cluster-dns=10.254.0.2\
  --cluster-domain=cluster.local. \
  --hairpin-mode promiscuous-bridge \
  --allow-privileged=true \
  --serialize-image-pulls=false \
  --logtostderr=true \
  --v=2
ExecStartPost=/sbin/iptables -A INPUT -s 10.0.0.0/8 -p tcp --dport 4194 -j ACCEPT
ExecStartPost=/sbin/iptables -A INPUT -s 172.16.0.0/12 -p tcp --dport 4194 -j ACCEPT
ExecStartPost=/sbin/iptables -A INPUT -s 192.168.0.0/16 -p tcp --dport 4194 -j ACCEPT
ExecStartPost=/sbin/iptables -A INPUT -p tcp --dport 4194 -j DROP
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

七、启动测试

# systemctl daemon-reload
# systemctl start kubelet
# systemctl status kubelet

八、在master上对node节点的csr进行受权

# kubectl get nodes  
# kubectl get csr 
# kubectl certificate approve node-csr-s6NbHbQp8M3fxKbRTO9AW6_L6KNi89gQdGByxm6sGn8

九、在master上进行角色绑定

# kubectl get nodes  
# kubectl describe clusterrolebindings system:node 
# kubectl create clusterrolebinding kubelet-node-clusterbinding \
--clusterrole=system:node --user=system:node:192.168.115.6

# kubectl describe clusterrolebindings kubelet-node-clusterbinding

也能够将在整个集群范围内将 system:node ClusterRole 授予组”system:nodes”：

# kubectl create clusterrolebinding kubelet-node-clusterbinding \
> --clusterrole=system:node --group=system:nodes  
clusterrolebinding "kubelet-node-clusterbinding" created

常见错误：
error: failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained:

解决方案：

# swapoff -a

4、node节点上设置kube-proxy
一、建立 kube-proxy 证书签名请求

# cat kube-proxy-csr.json
{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "FuZhou",
      "L": "FuZhou",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

# cfssl gencert -ca=/etc/ssl/etcd/ca.pem \
  -ca-key=/etc/ssl/etcd/ca-key.pem \
  -config=/etc/ssl/etcd/ca-config.json \
  -profile=kubernetes  kube-proxy-csr.json | cfssljson -bare kube-proxy
# mv kube-proxy*.pem /etc/ssl/kubernetes/

# rsync /etc/ssl/kubernetes/* vm2:/etc/ssl/kubernetes/
# rsync /etc/ssl/kubernetes/* vm3:/etc/ssl/kubernetes/

二、在node节点上建立 kube-proxy kubeconfig 文件
设置集群参数

# kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/ssl/etcd/ca.pem \
  --embed-certs=true \
  --server=https://192.168.115.5:6443 \
  --kubeconfig=kube-proxy.kubeconfig

设置客户端参数

# kubectl config set-credentials kube-proxy \
  --client-certificate=/etc/ssl/kubernetes/kube-proxy.pem \
  --client-key=/etc/ssl/kubernetes/kube-proxy-key.pem \
  --embed-certs=true \
  --kubeconfig=kube-proxy.kubeconfig

设置上下文参数

# kubectl config set-context default \
  --cluster=kubernetes \
  --user=kube-proxy \
  --kubeconfig=kube-proxy.kubeconfig

设置默认上下文

# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig
# mv kube-proxy.kubeconfig  /etc/kubernetes/

三、建立kube-proxy工做目录

# mkdir -p /var/lib/kube-proxy

四、配置kube-proxy启动脚本

# cat /usr/lib/systemd/system/kube-proxy.service 
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
WorkingDirectory=/var/lib/kube-proxy
ExecStart=/usr/local/sbin/kube-proxy \
  --bind-address=192.168.115.6 \
  --hostname-override=192.168.115.6\
  --cluster-cidr=172.30.0.0/16 \
  --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig \
  --logtostderr=true \
  --v=2
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

五、启动测试

# systemctl daemon-reload
# systemctl start kube-proxy
# netstat -ntpl |grep kube

按照一样的方法部署另外一台node主机vm3


5、建立pod测试

# cat nginx-rc.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  replicas: 2
  selector:
    name: nginx
  template:
    metadata:
      labels: 
       name: nginx
    spec:
      containers:
      - name: nginx-test
        image: docker.io/nginx
        ports:
        - containerPort: 80

# cat nginx-svc.yaml   
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels: 
   name: nginx
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
    name: http
    nodePort: 8401
  selector:
name: nginx

访问测试