Jenkins 使用 Kubernetes Plugin 完成持续构建与发布
介绍
基于Kubernetes和Jenkins来实现CI/CD。 全部须要跑任务的jenkins slave(pod)经过模版动态建立，当任务执行结束自动删除。node
系统总体架构
job流程
环境
kubernetspython
jenkins配置
jenkins-deployment.yaml
apiVersion: "apps/v1beta1"
kind: "Deployment"
metadata:
name: "jenkins"
labels:
name: "jenkins"
spec:
replicas: 1
template:
metadata:
name: "jenkins"
labels:
name: "jenkins"
spec:
containers:
- name: jenkins
image: jenkinsci/jenkins:2.154
imagePullPolicy: IfNotPresent
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
env:
- name: TZ
value: Asia/Shanghai
ports:
- containerPort: 8080
name: web
- containerPort: 50000
name: agent
volumes:
- name: jenkins-home
nfs:
path: "/nfs/jenkins/data"
server: "cpu029.hogpu.cc"
terminationGracePeriodSeconds: 10
serviceAccountName: jenkins
jenkins-account.yaml
--- apiVersion: v1 kind: ServiceAccount metadata: name: jenkins --- kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: jenkins rules: - apiGroups: [""] resources: ["pods"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/exec"] verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""] resources: ["pods/log"] verbs: ["get","list","watch"] - apiGroups: [""] resources: ["secrets"] verbs: ["get"] - apiGroups: [""] resources: ["configmap"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: jenkins roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: jenkins subjects: - kind: ServiceAccount name: jenkins
jenkins-service.yaml
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: jenkins
name: jenkins
spec:
type: NodePort
ports:
- port: 8080
name: web
targetPort: 8080
- port: 50000
name: agent
targetPort: 50000
selector:
name: jenkins
说明
说明一下：这里 Service 咱们暴漏了端口 8080 和 50000，8080 为访问 Jenkins Server 页面端口，50000 为建立的 Jenkins Slave 与 Master 创建链接进行通讯的默认端口，若是不暴露的话，Slave 没法跟 Master 创建链接。这里使用 NodePort 方式暴漏端口，并未指定其端口号，由 Kubernetes 系统默认分配，固然也能够指定不重复的端口号（范围在 30000~32767）git
建立jenkins
接下来，经过 kubectl 命令行执行建立 Jenkins Service。 $ kubectl create namespace kubernetes-plugin $ kubectl config set-context $(kubectl config current-context) --namespace=kubernetes-plugin $ kubectl create -f jenkins-deployment.yaml $ kubectl create -f jenkins-account.yaml $ kubectl create -f jenkins-service.yaml
ps：github
建立一个新的 namespace 为 kubernetes-plugin，而且将当前 context 设置为 kubernetes-plugin namespace 这样就会自动切换到该空间下。
查看状态
jianyu.tian@yz-gpu-k8s004 ~]$ kubectl get deployment,svc,pods NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deploy/jenkins 1 1 1 1 1h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc/jenkins NodePort 10.106.235.91 <none> 8080:31051/TCP,50000:30545/TCP 2h NAME READY STATUS RESTARTS AGE po/jenkins-64564fc5c9-pzlpb 1/1 Running 0 1h
ps：web
Jenkins Master 服务已经启动起来了，而且将端口暴漏到 8080:31051，50000:30545，此时能够经过浏览器打开 http://<Cluster_IP>:30645 访问 Jenkins 页面了。
jenkins web界面初始化
1.主要对jenkins-plugin插件作说明docker
安装完毕后，点击 “系统管理” —> “系统设置” —> “新增一个云” —> 选择 “Kubernetes”，而后填写 Kubernetes 和 Jenkins 配置信息。centos
ps：
Name 处默认为 kubernetes，也能够修改成其余名称，若是这里修改了，下边在执行 Job 时指定 podTemplate() 参数 cloud 为其对应名称，不然会找不到，cloud 默认值取：kubernetes
Kubernetes URL 处我填写了 https://kubernetes.default.sv... 这里我填写了 Kubernetes Service 对应的 DNS 记录，经过该 DNS 记录能够解析成该 Service 的 Cluster IP，或者直接填写外部 Kubernetes 的地址 https://<ClusterIP>:<Ports>。
Jenkins URL 处我填写了 http://jenkins.kubernetes-plugin:8080，跟上边相似，也是使用 Jenkins Service 对应的 DNS 记录，不过要指定为 8080 端口，由于咱们设置暴漏 8080 端口。同时也能够用 http://<ClusterIP>:<Node_Port>api
配置完毕，能够点击 “Test Connection” 按钮测试是否可以链接的到 Kubernetes，若是显示 Connection test successful 则表示链接成功，配置没有问题。浏览器
测试
建立一个 Pipeline 类型 Job：bash
pipeline {
agent any
//并行操做
stages {
stage("test_all") {
parallel {
stage("python3-cuda9.2") {
agent {
kubernetes {
label 'mxnet-python3-cuda9'
yaml """
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
name: "mxnet-python3-cuda9"
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: hobot.workas
operator: In
values:
- gpu
- key: kubernetes.io/nvidia-gpu-name
operator: In
values:
- TITAN_V
containers:
- name: mxnetone
image: docker.hobot.cc/dlp/mxnetci:runtime-py3.6-cudnn7.3-cuda9.2-centos7
imagePullPolicy: Always
resources:
limits:
nvidia.com/gpu: 1
"""
}
}
stages {
stage("拉取代码") {
steps {
container("mxnetone") {
checkout(
[
$class: 'GitSCM',
branches: [[name: 'nnvm']],
browser: [$class: 'Phabricator', repo: 'rMXNET', repoUrl: ''],
doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true, recursiveSubmodules: true, reference: '', trackingSubmodules: false]],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'zhaoming_private', url: '']]
]
)
}
}
}
stage("编译") {
steps {
container("mxnetone") {
sh """
nvidia-smi
source /root/.bashrc
make deps
echo -e "USE_PROFILER=1\nUSE_GLOG=0\nUSE_HDFS=0" >> ./make/config.mk
sed -i "s#USE_CUDA_PATH = /usr/local/cuda-8.0#USE_CUDA_PATH = /usr/local/cuda-9.2#g" ./make/config.mk
make lint
make -j 12
ln -s /home/data ./
make test | tee unittest.log
"""
}
}
}
stage("单元测试") {
steps {
container("mxnetone") {
sh """
cp -rf python/mxnet ./
cp -f lib/libmxnet.so mxnet/
echo "-------Running tests under Python3-------"
python3 -V
python3 `which nosetests` tests/python/train
python3 `which nosetests` -v -d tests/python/unittest
"""
}
}
}
}
}
stage("python2-cuda9.2") {
agent {
kubernetes {
label 'mxnet-python2-cuda9'
yaml """
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
name: "mxnet-python2-cuda9"
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: hobot.workas
operator: In
values:
- gpu
- key: kubernetes.io/nvidia-gpu-name
operator: In
values:
- TITAN_V
containers:
- name: mxnettwo
image: docker.hobot.cc/dlp/mxnetci:runtime-cudnn7.3-cuda9.2-centos7
imagePullPolicy: Always
resources:
limits:
nvidia.com/gpu: 1
"""
}
}
stages {
stage("拉取代码") {
steps {
container("mxnettwo") {
checkout(
[
$class: 'GitSCM',
branches: [[name: 'nnvm']],
browser: [$class: 'Phabricator', repo: 'rMXNET', repoUrl: ''],
doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: true, recursiveSubmodules: true, reference: '', trackingSubmodules: false]],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'zhaoming_private', url: '']]
]
)
}
}
}
stage("编译") {
steps {
container("mxnettwo") {
sh """
nvidia-smi
pip2 install numpy==1.14.3 -i https://mirrors.aliyun.com/pypi/simple/
source /root/.bashrc
make deps
echo -e "USE_PROFILER=1\nUSE_GLOG=0\nUSE_HDFS=0" >> ./make/config.mk
sed -i "s#USE_CUDA_PATH = /usr/local/cuda-8.0#USE_CUDA_PATH = /usr/local/cuda-9.2#g" ./make/config.mk
make lint
make -j 12
ln -s /home/data ./
make test | tee unittest.log
"""
}
}
}
stage("单元测试") {
steps {
container("mxnettwo") {
sh """
cp -rf python/mxnet ./
cp -f lib/libmxnet.so mxnet/
echo "-------Running tests under Python2-------"
python2 -V
python2 `which nosetests` tests/python/train
python2 `which nosetests` -v -d tests/python/unittest
"""
}
}
}
}
}
}
}
}
}
文档
相关文章
- 1. 初试 Jenkins 使用 Kubernetes Plugin 完成持续构建与发布
- 2. 使用Github+Jenkins完成持续集成与持续部署（一）持续集成
- 3. 构建持续集成Jenkins
- 4. 使用Jenkins进行持续构建与发布应用到Kubernetes集群中
- 5. 使用Jenkins进行持续集成与发布
- 6. Jenkins 持续构建的使用
- 7. K8S构建Jenkins持续集成平台
- 8. Jenkins 与 Kubernetes 持续集成 Springboot + Helm
- 9. GitLab push自动触发Jenkins构建与持续集成
- 10. Jenkins+SVN+tomcat持续集成发布
- 更多相关文章...
- • 使用Redis和SSM（Spring+Spring MVC+MyBatis）搭建抢红包开发环境和超发现象完整实例 - 红包项目实战
- • ASP.NET Web Pages - 发布网站 - ASP.NET 教程
- • Composer 安装与使用
- • JDK13 GA发布：5大特性解读
相关标签/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
欢迎关注本站公众号,获取更多信息
相关文章
- 1. 初试 Jenkins 使用 Kubernetes Plugin 完成持续构建与发布
- 2. 使用Github+Jenkins完成持续集成与持续部署（一）持续集成
- 3. 构建持续集成Jenkins
- 4. 使用Jenkins进行持续构建与发布应用到Kubernetes集群中
- 5. 使用Jenkins进行持续集成与发布
- 6. Jenkins 持续构建的使用
- 7. K8S构建Jenkins持续集成平台
- 8. Jenkins 与 Kubernetes 持续集成 Springboot + Helm
- 9. GitLab push自动触发Jenkins构建与持续集成
- 10. Jenkins+SVN+tomcat持续集成发布