咱们使用的是 AWS 的 EC2 来搭建咱们的集群,安装方式使用 kubeadm 来进行安装,若是使用二进制安装,能够参考我相关文档。node
名称 | 配置 | 内网IP | 外网IP |
---|---|---|---|
k8s-master | 2核4GB | 172.31.20.184 | 54.226.118.74 |
k8s-node1 | 2核4GB | 172.31.27.69 | 52.90.221.230 |
k8s-node2 | 2核4GB | 172.31.30.9 | 3.85.219.119 |
安装源你们能够参照官方文档 https://docs.docker.com/install/linux/docker-ce/ubuntu/ ,我这里再也不进行演示,如没有特殊说明,操做将在三台集群上面都要执行。linux
apt-get install docker-ce=18.06.3~ce~3-0~ubuntu systemctl enable docker
安装源文档请参考官方文档 https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm ,详细步骤我这里进行省略。nginx
sudo apt-get update && sudo apt-get install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF sudo apt-get update sudo apt-get install -y kubelet kubeadm kubectl sudo apt-mark hold kubelet kubeadm kubectl
请参考文档 https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/ ,在 master 节点运行。git
kubeadm init --kubernetes-version=v1.17.1 \ --pod-network-cidr=10.244.0.0/16 \ --service-cidr=10.96.0.0/12
在 node 节点上运行添加集群的命令。github
kubeadm join 172.31.20.184:6443 --token w3fu9a.rs8eknt079n2e8r8 \ --discovery-token-ca-cert-hash sha256:7392d6f6576b3c9ba5f78d1c54d9a0b1369f77bd498da8104a096b62c6b14c06
之后的 kubectl 都是在 master 节点上进行操做,添加 cni 插件,咱们这里选择 flannel。web
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml
到目前为止,咱们的集群已经建立完成,参照官方文档执行,过程很简单。docker
root@ip-172-31-20-184:~# kubectl get no NAME STATUS ROLES AGE VERSION ip-172-31-20-184 Ready master 21m v1.17.1 ip-172-31-27-69 Ready <none> 16m v1.17.1 ip-172-31-30-9 Ready <none> 16m v1.17.1
整个 CI/CD 过程当中咱们用到了不少工具,好比 gitlab,jenkins,harbor,在生产环境,建议你们把 gitlab 和 harbor 放在独立的机器上面,我这里为了简便,直接放在 k8s 集群中。ubuntu
请参见官方文档 https://helm.sh/docs/intro/install/ 。api
wget https://get.helm.sh/helm-v3.0.2-linux-amd64.tar.gz tar xf helm-v3.0.2-linux-amd64.tar.gz mv linux-amd64/helm /usr/local/bin/ chmod +x /usr/local/bin/helm
参见官方文档 https://docs.gitlab.com/charts/installation/ 。bash
helm repo add gitlab https://charts.gitlab.io/
后来发现 gitlab 目前还不支持 helm3,那我就选择了使用 EC2 来进行建立。
https://about.gitlab.com/install/#ubuntu?version=ce
参见官方文档 https://github.com/goharbor/harbor-helm/blob/master/README.md
helm repo add harbor https://helm.goharbor.io helm install my-release harbor/harbor
安装过程当中仍是出现了一些问题,是由于没有 pv,懒得去设置了,后来仍是选择了单机进行安装。
参照下面文档安装 https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md 。
咱们以云原生的方式,将jenkins master,jenkins slave所有部署于kubernetes之上,从而打造一个高可用,弹性伸缩的CI/CD管道。
由于 master 须要持久存储,咱们这里就简单的选择 nfs ,咱们在 master 节点上进行建立服务。
建立过程请参照文档 https://blog.csdn.net/qq_37860012/article/details/86717891 。
root@ip-172-31-20-184:/home/nfs# showmount -e 172.31.20.184 Export list for 172.31.20.184: /home/nfs/jenkins *
master 节点目前授予权限。
chown ubuntu.ubuntu -R /home/nfs/jenkins
参考文档 https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/
前面所须要的东西都已经配置好了,那咱们开始安装 jenkins,请应用我下面的 yaml 文件
apiVersion: v1 kind: ServiceAccount metadata: name: jenkins --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: jenkins rules: - apiGroups: [""] resources: ["pods"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/exec"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/log"] verbs: ["get","list","watch"] - apiGroups: [""] resources: ["secrets"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: jenkins roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: jenkins subjects: - kind: ServiceAccount name: jenkins --- apiVersion: v1 kind: Service metadata: name: jenkins spec: selector: app: jenkins ports: - name: http port: 8080 targetPort: 8080 protocol: TCP - name: agent port: 50000 protocol: TCP targetPort: 50000 --- apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: name-virtual-host-ingress spec: rules: - host: jenkins.wzlinux.com http: paths: - backend: serviceName: jenkins servicePort: 8080 --- apiVersion: apps/v1 kind: Deployment metadata: name: jenkins spec: replicas: 1 selector: matchLabels: app: jenkins strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 maxUnavailable: 0 template: metadata: labels: app: jenkins spec: securityContext: fsGroup: 1000 serviceAccountName: jenkins containers: - name: jenkins image: jenkins/jenkins:lts-alpine imagePullPolicy: IfNotPresent ports: - containerPort: 8080 name: web protocol: TCP - containerPort: 50000 name: agent protocol: TCP volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home env: - name: JAVA_OPTS value: "-Xms1G -Xmx1G -XX:PermSize=512m -XX:MaxPermSize=1024m -Duser.timezone=Asia/Shanghai" - name: TRY_UPGRADE_IF_NO_MARKER value: "true" volumes: - name: jenkins-home # hostPath: # path: "/home/jenkins" nfs: server: 172.31.20.184 path: "/home/nfs/jenkins/"