jx是云原生CICD,devops的一个最佳实践之一,目前在快速的发展成熟中。最近调研了JX,这里为第3篇,介绍下如何安装jenkins x。html
前置条件
- 安装K8S
- 安装ceph集群(jx须要storage class建立pv)
- 申请一个域名(可选),能够修改hosts实现
- helm
- git私服
安装ceph集群
这里的服务器为centos 7。mysql
使用官方的ceph-deploy安装便可,先安装ceph-deploy,而后每台机器安装运行环境linux
pip install ceph-deploy export CEPH_DEPLOY_REPO_URL=http://mirrors.ustc.edu.cn/ceph/rpm-jewel/el7 export CEPH_DEPLOY_GPG_URL=http://mirrors.ustc.edu.cn/ceph/keys/release.asc ceph-deploy install docker86-156 docker86-155 docker86-154
而后安装集群nginx
ceph-deploy new docker86-156 docker86-155 docker86-154
修改配置文件git
cat <<EOF >>ceph.conf #osd_journal_size = 10000 public network = 192.168.86.0/24 osd_pool_default_size = 2 osd_pool_default_min_size = 1 osd_crush_chooseleaf_type = 1 osd_crush_update_on_start = true max_open_files = 131072 osd pool default pg num = 128 osd pool default pgp num = 128 mon_pg_warn_max_per_osd = 0 mon clock drift allowed = 2 mon clock drift warn backoff = 30 mon_pg_warn_max_per_osd = 300 EOF
分发配置文件:github
ceph-deploy --overwrite-conf config push docker86-155 docker86-154 docker86-156
安装服务web
ceph-deploy mon create-initial ceph-deploy admin docker86-156 docker86-155 docker86-154
安装osdsql
ceph-deploy disk zap docker86-156:sdb docker86-155:sdb docker86-154:sdb ceph-deploy osd prepare docker86-156:sdb docker86-155:sdb docker86-154:sdb ceph-deploy osd activate docker86-156:sdb1 docker86-154:sdb1
添加pool
ceph osd pool create k8smeta 128 ceph osd pool create k8sdata 128 ceph fs new k8s k8smeta k8sdata ceph osd pool ls detail
K8S使用ceph
生成 Ceph secretdocker
grep key /etc/ceph/ceph.client.admin.keyring |awk '{printf "%s", $NF}'|base64
假如获得: $SECRET==shell
在k8s建立Secret
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Secret metadata: name: ceph-secret namespace: default type: "kubernetes.io/rbd" data: key: $SECRET== EOF
建立StorageClass
cat <<EOF | kubectl apply -f - apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ceph-web provisioner: kubernetes.io/rbd parameters: monitors: 192.168.86.156,192.168.86.155,192.168.86.154 adminId: admin adminSecretName: ceph-secret adminSecretNamespace: default pool: rbd userId: admin userSecretName: ceph-secret EOF
能够将ceph设为默认的storage class:
kubectl patch storageclass ceph-web -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
安装git私服gitea(可选)
若是已经有git的,或者直接使用GitHub的跳过。
建立PV:
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: PersistentVolume metadata: name: cephfs-github-pv namespace: gitea labels: name: cephfs-github-pv spec: capacity: storage: 200Gi accessModes: - ReadWriteMany cephfs: monitors: - 192.168.86.156:6789 path: /github user: admin secretRef: name: ceph-secret readOnly: false persistentVolumeReclaimPolicy: Retain EOF
PVC
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cephfs-github-pvc namespace: gitea spec: accessModes: - ReadWriteMany storageClassName: "" resources: requests: storage: 200Gi selector: matchLabels: name: cephfs-github-pv EOF
gitea部署:
cat <<EOF | kubectl apply -f - apiVersion: extensions/v1beta1 kind: Deployment metadata: name: giteamysql namespace: gitea spec: strategy: type: Recreate template: metadata: labels: app: giteamysql spec: containers: - image: gitea/gitea:latest imagePullPolicy: IfNotPresent name: gitea resources: {} volumeMounts: - name: ceph mountPath: /data volumes: - name: ceph persistentVolumeClaim: claimName: cephfs-github-pvc EOF
建立服务:
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Service metadata: name: giteamysql-service namespace: gitea labels: app: charts spec: ports: - port: 80 targetPort: 3000 selector: app: giteamysql type: NodePort EOF
建立ingress
cat <<EOF | kubectl apply -f - apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: nginx.ingress.kubernetes.io/proxy-body-size: "0" nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600" kubernetes.io/tls-acme: 'true' name: giteamysql-ingress namespace: gitea spec: rules: - host: github.youdomain.com http: paths: - backend: serviceName: giteamysql-service servicePort: 80 path: / EOF
一切正常的话,打开github.youdomain.com,按提示进行安装,设置管理员密码便可。
安装好后建立一个token, $git_access_token
域名与tls
将域名的通配符,a记录到k8s集群。
申请TLS证书,使用certbot
$ yum -y install yum-utils $ yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional $ sudo yum install certbot
而后申请
certbot certonly --manual -d *.domain.com --email youmail@domain.com
这个会要求建立一个A记录,按提示建立便可。
一些OK的话,服务器/etc/letsencrypt/live/domain.com/ 会生成tls证书。
$ ll /etc/letsencrypt/live/iflyresearch.com/ total 4 lrwxrwxrwx. 1 root root 40 Oct 17 15:11 cert.pem -> ../../archive/iflyresearch.com/cert1.pem drwxr-xr-x 2 root root 78 Nov 14 09:33 certs lrwxrwxrwx. 1 root root 41 Oct 17 15:11 chain.pem -> ../../archive/iflyresearch.com/chain1.pem lrwxrwxrwx. 1 root root 45 Oct 17 15:11 fullchain.pem -> ../../archive/iflyresearch.com/fullchain1.pem lrwxrwxrwx. 1 root root 43 Oct 17 15:11 privkey.pem -> ../../archive/iflyresearch.com/privkey1.pem
要在k8s使用,须要建立secret:
kubectl create secret tls research-tls-secret --cert=cert.pem --key=./privkey.pem -n=kube-system
安装helm
jx依赖helm,须要先安装,能够参见本系列第一篇
安装jenkins X
先建立一个namespace:incubation
写入 ceph-secret:
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Secret metadata: name: ceph-secret namespace: incubation type: "kubernetes.io/rbd" data: key: $SECRET== EOF
下载jx执行文件:
wget https://github.com/jenkins-x/jx/releases/download/v1.3.380/jx-linux-amd64.tar.gz tar xzv jx-linux-amd64.tar.gz -C ~/.jx/bin export PATH=$PATH:~/.jx/bin echo 'export PATH=$PATH:~/.jx/bin' >> ~/.bashrc
而后执行安装命令:
jx install --external-ip=192.168.86.214 --namespace='incubation' --git-provider-url='http://github.iflyresearch.com' --git-username='jqpeng' --git-api-token=' $git_access_token' --domain='iflyresearch.com' --provider=kubernetes
- $git_access_token 替换为你的token
- external-ip 填写k8s的虚拟ip
而后按提示,输出jenkins等access_token等参数。
注意:
- 安装过程,会依赖一些gcr.io的镜像,请参见《google gcr.io、k8s.gcr.io 国内镜像》解决
- 若是使用gitea,安装修改下vim gitAuth.yaml,修改kind为gitea
做者:Jadepeng 出处:jqpeng的技术记事本--http://www.cnblogs.com/xiaoqi 您的支持是对博主最大的鼓励,感谢您的认真阅读。 本文版权归做者全部,欢迎转载,但未经做者赞成必须保留此段声明,且在文章页面明显位置给出原文链接,不然保留追究法律责任的权利。