快速部署OpenShift应用

时间  2019-11-11
标签 快速 部署 openshift 应用 繁體版
原文   原文链接

本文介绍了使用Service Catalog和OC命令部署OpenShift应用、部署基本概念和流程、扩展存储、清理OpenShift对象等。以Angular 6集成Spring Boot 2,Spring Security,JWT和CORS中的Spring Boot和Angular项目为例，详细讲解了S2I和Pipeline两种部署方式。javascript

OKD版本3.11，Spring Boot项目源码heroes-api，Angular项目源码heroes-web。php

初识OpenShift部署

Service Catalog

快速部署OpenShift应用
OpenShift初始安装中含有一些样例APP供你们学习使用。其中有Apache HTTP Server和Apache HTTP Server(httpd)，这二者有什么区别？分别点击进入能够发现：
快速部署OpenShift应用
Apache HTTP Server使用template(template名字为httpd-example)部署方式。
快速部署OpenShift应用
Apache HTTP Server(httpd)使用builder image(image stream名字为httpd)部署方式。
Service Catalog样例使用了template和builder image(image+source)两种部署方式。进入Application Console中的openshift项目可查看template和image。
查看template，点击Resources -> Other Resources -> Template：
快速部署OpenShift应用
查看Image Stream，点击Builds -> Images：
快速部署OpenShift应用html

其余部署方式
在Service Catalog中，除从Catalog直接选择Item外，还提供了其余三种方式：
快速部署OpenShift应用
Deploy Image能够直接从image或image stream部署应用：
快速部署OpenShift应用
Import YAML / JSON 用来从YAML或JSON建立资源，好比image stream、template：
快速部署OpenShift应用
Select from Project 从指定的Project中选择template来部署应用：
快速部署OpenShift应用java

部署Apache HTTP Server

Apache HTTP Server的两种部署方式本质上是相同的，Build策略均为S2I(Source-to-Image)，使用S2I构建的Docker镜像来部署应用。Source均使用Apache HTTP Server (httpd) S2I Sample Application，Docker基础镜像(builder image)均使用Apache HTTP Server Container Image。httpd-example template定义了总体部署流程并实现了参数化。
如下是httpd-example template中BuildConfig部分的定义：node

- apiVersion: v1
  kind: BuildConfig
  metadata:
    annotations:
      description: Defines how to build the application
      template.alpha.openshift.io/wait-for-ready: 'true'
    name: '${NAME}'
  spec:
    output:
      to:
        kind: ImageStreamTag
        name: '${NAME}:latest'
    source:
      contextDir: '${CONTEXT_DIR}'
      git:
        ref: '${SOURCE_REPOSITORY_REF}'
        uri: '${SOURCE_REPOSITORY_URL}'
      type: Git
    strategy:
      sourceStrategy:
        from:
          kind: ImageStreamTag
          name: 'httpd:2.4'
          namespace: '${NAMESPACE}'
      type: Source
    triggers:
      - type: ImageChange
      - type: ConfigChange
      - github:
          secret: '${GITHUB_WEBHOOK_SECRET}'
        type: GitHub
      - generic:
          secret: '${GENERIC_WEBHOOK_SECRET}'
        type: Generic

参数定义及默认值：python

parameters:
  - description: The name assigned to all of the frontend objects defined in this template.
    displayName: Name
    name: NAME
    required: true
    value: httpd-example
  - description: The OpenShift Namespace where the ImageStream resides.
    displayName: Namespace
    name: NAMESPACE
    required: true
    value: openshift
  - description: Maximum amount of memory the container can use.
    displayName: Memory Limit
    name: MEMORY_LIMIT
    required: true
    value: 512Mi
  - description: The URL of the repository with your application source code.
    displayName: Git Repository URL
    name: SOURCE_REPOSITORY_URL
    required: true
    value: 'https://github.com/openshift/httpd-ex.git'
  - description: >-
      Set this to a branch name, tag or other ref of your repository if you are
      not using the default branch.
    displayName: Git Reference
    name: SOURCE_REPOSITORY_REF
  - description: >-
      Set this to the relative path to your project if it is not in the root of
      your repository.
    displayName: Context Directory
    name: CONTEXT_DIR
  - description: >-
      The exposed hostname that will route to the httpd service, if left blank a
      value will be defaulted.
    displayName: Application Hostname
    name: APPLICATION_DOMAIN
...

咱们先使用builder image方式部署Apache，来了解一下部署的总体流程：
快速部署OpenShift应用
点击"advanced options"，能够设置git branch、context、secret，自定义Route、Build Configuration、Deployment Configuration、Resource Limits等。此处填完基本内容后直接点击Create，建立App，而后从成功页面进入Project Overview：
快速部署OpenShift应用
部署过程当中自动建立Service、Route、Build、Deployment、Image。进入Application Console的Applications和Builds能够查看详细信息，其中会建立3个pod：httpd-1-build、http-1-deploy、httpd-1-xxxxx，部署完毕后http-1-deploy会自动删除。
部署成功后，测试访问Apache Server(Route定义的Hostname)，页面以下：
快速部署OpenShift应用
下面解释一下涉及到的基本概念。git

基本概念

Service(Kubernetes Service)
内部load balancer，用在OpenShift内部网络中，可以使用Service ClusterIP或Hostname访问。angularjs

apiVersion: v1
kind: Service
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftWebConsole
  creationTimestamp: '2019-03-26T02:12:50Z'
  labels:
    app: httpd
  name: httpd
  namespace: my-project
  resourceVersion: '3004428'
  selfLink: /api/v1/namespaces/my-project/services/httpd
  uid: a81c759f-4f6c-11e9-9a7d-02fa2ffc40e6
spec:
  clusterIP: 172.30.225.159
  ports:
    - name: 8080-tcp
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    deploymentconfig: httpd
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

其中，selector定义了查找container(pod)进行负载均衡的标签。
Route
定义一个hostname来公开Service，以便外部客户能够访问Service，默认hostname为：[app-name]-[project-name].[openshift_master_default_subdomain]。
Build
构建App Image，使用S2I时即从builder image和Source Code来构建App Image。默认builder image和build配置变化时会从新build。github

查看Builds -> httpd -> #1 的YAML文本，能够了解Build流程为FetchInputs -> Assemble -> CommitContainer -> PushImage：golang

...
status:
  completionTimestamp: '2019-03-26T02:13:30Z'
  config:
    kind: BuildConfig
    name: httpd
    namespace: my-project
  duration: 40000000000
  output:
    to:
      imageDigest: 'sha256:5c1f20f20baaa796f4518d11ded13c6fac33e7a377774cfec77aa1e6e6a7cbb2'
  outputDockerImageReference: 'docker-registry.default.svc:5000/my-project/httpd:latest'
  phase: Complete
  stages:
    - durationMilliseconds: 3434
      name: FetchInputs
      startTime: '2019-03-26T02:12:56Z'
      steps:
        - durationMilliseconds: 3434
          name: FetchGitSource
          startTime: '2019-03-26T02:12:56Z'
    - durationMilliseconds: 2127
      name: CommitContainer
      startTime: '2019-03-26T02:13:11Z'
      steps:
        - durationMilliseconds: 2127
          name: CommitContainer
          startTime: '2019-03-26T02:13:11Z'
    - durationMilliseconds: 3426
      name: Assemble
      startTime: '2019-03-26T02:13:10Z'
      steps:
        - durationMilliseconds: 3426
          name: AssembleBuildScripts
          startTime: '2019-03-26T02:13:10Z'
    - durationMilliseconds: 16143
      name: PushImage
      startTime: '2019-03-26T02:13:14Z'
      steps:
        - durationMilliseconds: 16143
          name: PushImage
          startTime: '2019-03-26T02:13:14Z'
  startTimestamp: '2019-03-26T02:12:50Z'

Build Strategy
OpenShift支持Source-to-Image、Docker、Pipeline、Custom四种Build Strategy。

strategy:
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "builder-image:latest"
strategy:
  dockerStrategy:
    from:
      kind: "ImageStreamTag"
      name: "debian:latest"
spec:
  source:
    git:
      uri: "https://github.com/openshift/ruby-hello-world"
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfilePath: some/repo/dir/filename
strategy:
  customStrategy:
    from:
      kind: "DockerImage"
      name: "openshift/sti-image-builder"

Deployment
部署App Image，Deployment包含三种对象：DeploymentConfig、ReplicationController、Pod。DeploymentConfig包含部署策略、image配置、环境变量等，ReplicationController包含复制相关信息。App Image和deployment配置变化时会自动从新Deploy。

进入Deployments -> httpd -> #1，编辑Replicas或调节pods数能够增删pod：
快速部署OpenShift应用
Deployment Strategy
修改或升级App，即从新部署应用时的部署方式。部署配置(DeploymentConfig)支持三种策略：Rolling、Recreate、Custom。经过修改Route能够实现蓝/绿部署、A/B部署。

  • Rolling 默认策略，当新版本Pod状态变为Ready后才scale down老版本Pod，可能同时存在新老版本的Pod
  • Recreate 先终止全部Pod(Scale down the previous deployment to zero)再部署新Pod
  • Custom 自定义部署行为

ImageStream
OpenShift管理容器镜像的方式，其中定义了dockerImageReference，ImageStream tag定义了同docker image各版本的映射关系。Build成功后会自动建立ImageStream。

apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftWebConsole
  creationTimestamp: '2019-03-26T02:12:50Z'
  generation: 1
  labels:
    app: httpd
  name: httpd
  namespace: my-project
  resourceVersion: '3004571'
  selfLink: /apis/image.openshift.io/v1/namespaces/my-project/imagestreams/httpd
  uid: a81b14bf-4f6c-11e9-9a7d-02fa2ffc40e6
spec:
  lookupPolicy:
    local: false
status:
  dockerImageRepository: 'docker-registry.default.svc:5000/my-project/httpd'
  tags:
    - items:
        - created: '2019-03-26T02:13:30Z'
          dockerImageReference: >-
            docker-registry.default.svc:5000/my-project/httpd@sha256:5c1f20f20baaa796f4518d11ded13c6fac33e7a377774cfec77aa1e6e6a7cbb2
          generation: 1
          image: >-
            sha256:5c1f20f20baaa796f4518d11ded13c6fac33e7a377774cfec77aa1e6e6a7cbb2
      tag: latest

Template
定义总体部署流程并实现参数化，包含Service、Route、ImageStream、BuildConfig、DeploymentConfig、parameters等部分。

了解了以上基本概念就很容易理解httpd-example template了，您能够本身部署测试，此处再也不赘述。

OC Tool

使用oc new-app部署应用
继续以前，先将之前建立的测试project删除或新建一个project。

$ oc delete project my-project
$ oc new-project my-project

在Service Catalog一节咱们提到了部署应用的三种方式：template、builder image(image+source)、image，对应的命令以下：

$ oc new-app httpd-example -p APPLICATION_DOMAIN=httpd-example.apps.itrunner.org
$ oc new-app openshift/httpd:2.4~https://github.com/openshift/httpd-ex.git --name=httpd-ex
$ oc new-app my-project/httpd-ex --name=httpd

说明：

  1. image+source的语法为[image]~[source]
  2. 第三种方式使用的image为第二种方式中生成的
  3. 后面两种方式不会自动建立Route，须要手工建立：
$ oc expose service httpd-ex --name httpd-ex --hostname=httpd-ex.apps.itrunner.org
$ oc expose service httpd --name httpd --hostname=httpd.apps.itrunner.org

从JSON/YAML建立资源：

$ oc create -f <filename> -n <project>

使用oc命令还能够直接从source code建立应用，可使用本地或远程source code：

$ oc new-app /path/to/source/code
$ oc new-app https://github.com/sclorg/cakephp-ex

能够指定子目录：

$ oc new-app https://github.com/sclorg/s2i-ruby-container.git --context-dir=2.0/test/puma-test-app

能够指定branch：

$ oc new-app https://github.com/openshift/ruby-hello-world.git#beta4

OpenShift自动检测代码根目录或指定目录，若是存在Dockerfile则使用Docker build策略，若是存在Jenkinsfile则使用Pipeline build策略，不然使用Source build策略(S2I)。

下面的例子使用了Source build策略：

$ oc new-app https://github.com/sclorg/cakephp-ex

使用Source build策略时，new-app经过检测根目录或指定目录的文件来肯定language builder：

Language Files
jee pom.xml
nodejs app.json, package .json
perl cpanfile, index.pl
php composer.json, index.php
python requirements.txt, setup.py
ruby Gemfile, Rakefile, config.ru
scala build.sbt
golang Godeps, main.go

而后根据语言，从OpenShift Server或Docker Hub Registry中查找与语言匹配的image。

也能够指定策略，以下：

$ oc new-app /home/user/code/myapp --strategy=docker

查看template和image stream
查看全部template和image stream：

$ oc new-app --list

单独查看template或image stream：

$ oc get templates -n openshift
$ oc get imagestreams -n openshift

查看httpd-example template详细信息：

$ oc describe template httpd-example -n openshift

查看httpd image stream详细信息：

$ oc describe imagestream httpd -n openshift

查看httpd-example template的YAML定义：

$ oc new-app --search --template=httpd-example --output=yaml

从全部template、image stream、docker image中查找"httpd"：

$  oc new-app --search httpd

再谈Route

前面的例子Route使用的协议均为http，如何启用https呢?
使用Web Console时，编辑route启用Secure route便可：
快速部署OpenShift应用
TLS Termination有三种类型：edge、passthrough、reencrypt

  • edge 访问route使用https协议，route到内部网络为非加密的，如未配置证书则使用默认证书。
  • reencrypt 所有访问路径均是加密的
  • passthrough 加密通讯直接发送到目标，route不需提供TLS Termination。

使用oc命令建立route：

$ oc create route edge httpd-ex --service httpd-ex --hostname httpd-ex.apps.itrunner.org --path / --insecure-policy Redirect -n my-project

S2I

Source-to-Image (S2I)是一个框架，能够容易地将应用程序源代码做为输入生成一个新的docker image。

使用S2I构建image，在装配过程当中能够执行大量复杂的操做，全部操做仅建立一个新的layer，加速了处理过程。S2I使得软件开发工程师没必要关心docker image的制做，仅负责编写assemble、run等脚本，也能够防止开发工程师在image构建过程当中执行任意yum安装等不适宜的操做。S2I简化了docker image的制做。

S2I须要如下三个基本要素：

  • builder image S2I以此image为基础来构建新的image
  • Sources
  • S2I Scripts

在构建过程当中，S2I先获取sources和scripts，将其打包为tar文件后放入builder image中。在执行assemble script前，S2I解压tar文件到io.openshift.s2i.destination指定的目录，默认目录为/tmp（分别解压到/tmp/src、/tmp/scripts目录）。
快速部署OpenShift应用

S2I Scripts

S2I Scripts能够位于如下位置，优先级从高到低：

  1. 在BuildConfig中指定位置
strategy:
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "builder-image:latest"
    scripts: "http://somehost.com/scripts_directory"
  1. 应用程序源码的.s2i/bin目录
  2. 在builder image的Dockerfile中定义位置（在样例Apache HTTP Server中使用了这种方式 ）
LABEL io.openshift.s2i.scripts-url="image:///usr/libexec/s2i"

io.openshift.s2i.scripts-url和BuildConfig定义位置都可采用如下形式：

  • image:///path_to_scripts_dir - image的绝对路径
  • file:///path_to_scripts_dir - host绝对或相对路径
  • http(s)://path_to_scripts_dir - URL

S2I Scripts：

Script Description
assemble (required) 获取源码、编译、打包。在增量编译时，若是定义了save-artifacts，先恢复artifact
run (required) 运行应用
save-artifacts (optional) 收集依赖以便加速后续编译，好比.m2
usage (optional) 显示image使用帮助信息
test/run (optional) 检查image可否正常工做

Example assemble script

#!/bin/bash

# restore build artifacts
if [ "$(ls /tmp/artifacts/ 2>/dev/null)" ]; then
    mv /tmp/artifacts/* $HOME/.
fi

# move the application source
mv /tmp/s2i/src $HOME/src

# build application artifacts
pushd ${HOME}
make all

# install the artifacts
make install
popd

Example run script

#!/bin/bash

# run the application
/opt/application/run.sh

Example save-artifacts script

#!/bin/bash

# Besides the tar command, all other output to standard out must 
# be surpressed.  Otherwise, the tar stream will be corrupted.
pushd ${HOME} >/dev/null
if [ -d deps ]; then
    # all deps contents to tar stream
    tar cf - deps
fi
popd >/dev/null

注意：save-artifacts只能有tar stream输出，不能含有其它任何输出。
Example usage script

#!/bin/bash

# inform the user how to use the image
cat <<EOF
This is a S2I sample builder image, to use it, install
https://github.com/openshift/source-to-image
EOF

S2I Tool

仅为学习S2I的基本知识和S2I Tool的使用，部署OpenShift应用时是没必要安装的。

安装S2I Tool
下载S2I后解压安装：

# tar -xvf release.tar.gz .
# cp /path/to/s2i /usr/local/bin

S2I命令

create       建立生成builder image的基础目录结构
build         构建新的image
rebuild      重建image
usage       显示image usage信息
version     显示s2i version
completion  Generate completion for the s2i command (bash or zsh)

示例

  1. 使用s2i create建立目录结构

s2i create语法：

s2i create <imageName> <destination> [flags]
$ s2i create ruby-centos7 ruby-centos7

执行以上命令生成的目录结构以下：

ruby-centos7
├── Dockerfile
├── Makefile
├── README.md
├── s2i
│   └── bin
│       ├── assemble
│       ├── run
│       ├── save-artifacts
│       └── usage
└── test
    ├── run
    └── test-app
        └── index.html
  1. 构建builder image
$ cd ruby-centos7
$ make build
  1. 构建App image

s2i build语法：

s2i build <source> <image> [<tag>] [flags]
# cd ruby-centos7
# s2i build test/test-app/ ruby-centos7 ruby-app
  1. 运行image
# docker run --rm -d -p 8080:8080 --name ruby-app ruby-app

增量构建与save-artifacts

Maven、Angular等项目编译时须要下载其余依赖，为提升编译速度，避免重复下载，S2I支持增量build。增量build依赖之前构建的image，image名字必须相同，且image中必须含有save-artifacts script。

增量build流程以下：

  1. S2I从之前构建的app image建立一个新的docker container
  2. S2I运行container中的save-artifacts将依赖打包为tar文件输出到stdout
  3. S2I下载source与artifacts一块儿打包做为新image的输入
  4. S2I启动新container、运行assemble ...

artifacts默认解压到/tmp/artifacts目录中。

启用增量构建

  • 使用s2i命令时增长参数--incremental=true：
# s2i build test/test-app/ ruby-centos7 ruby-app --incremental=true
  • 在OpenShift Web Console中修改BuildConfig，增长incremental: true，而后点击Start Build。
strategy:
  type: "Source"
  sourceStrategy:
    from:
      kind: "ImageStreamTag"
      name: "incremental-image:latest" 
    incremental: true

部署Spring Boot App

Builder Image

Dockerfile

# heroes-api-centos7
FROM centos:latest

RUN yum -y update && yum clean all

# Set the labels that are used for OpenShift to describe the builder image.
LABEL maintainer="Sun Jingchuan <jason@163.com>" \
      io.k8s.description="Heroes API" \
      io.k8s.display-name="Heroes API" \
      io.openshift.expose-services="8080:http" \
      io.openshift.tags="spring-boot,heroes-api" \
      # this label tells s2i where to find its mandatory scripts(run, assemble, save-artifacts)
      # io.openshift.s2i.scripts-url="image:///usr/libexec/s2i" \
      io.openshift.s2i.scripts-url="image:///tmp/scripts" \
      io.openshift.s2i.destination="/tmp"

ENV JAVA_HOME=/usr/lib/jdk1.8.0_202 \
    MAVEN_HOME=/usr/lib/apache-maven-3.6.0 \
    APP_ROOT=/opt/heroes
ENV PATH=${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${APP_ROOT}/bin:${PATH} HOME=${APP_ROOT}

# Include jdk and maven in lib
COPY lib /usr/lib
COPY bin ${APP_ROOT}/bin
# Copy the S2I scripts to /usr/libexec/s2i
# COPY .s2i/bin /usr/libexec/s2i

RUN chmod -R u+x ${APP_ROOT}/bin && \
    chgrp -R 0 ${APP_ROOT} && \
    chmod -R g=u ${APP_ROOT} /etc/passwd

USER 10001
WORKDIR ${APP_ROOT}

ENTRYPOINT [ "uid_entrypoint" ]

EXPOSE 8080

# Inform the user how to run this image.
# CMD ["/usr/libexec/s2i/usage"]

说明：

  1. lib目录中存放了JDK和Maven
  2. 使用源码中的S2I Scripts，builder image中不提供，但需定义io.openshift.s2i.scripts-url
  3. 默认，为了安全，OpenShift中只能使用User ID，不能使用用户名，不能使用docker默认用户root，所以要授予目录适当权限。
  4. 源码bin目录中存放了用户建立脚本uid_entrypoint，内容以下：
#!/bin/bash

if ! whoami &> /dev/null; then
  if [ -w /etc/passwd ]; then
    echo "${USER_NAME:-default}:x:$(id -u):0:${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd
  fi
fi

exec "$@"

编译builder image并上传到Registry

# docker build -t heroes-api-centos7:v1.0.0 .
# docker tag heroes-api-centos7:v1.0.0 registry.itrunner.org/heroes-api-centos7:v1.0.0
# docker push registry.itrunner.org/heroes-api-centos7:v1.0.0

import image

$ oc import-image heroes-api-centos7:v1.0.0 -n heroes --confirm --insecure --from='registry.itrunner.org/heroes-api-centos7:v1.0.0'

导入命令中指定了参数-n heroes，image会导入heroes项目中。成功导入后可在项目的Builds -> Images中查看image，但在Service Catalog中仍不可见，须要修改Image Stream定义，增长annotations：

apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  annotations:
    openshift.io/image.dockerRepositoryCheck: '2019-03-27T08:40:27Z'
  creationTimestamp: '2019-03-27T08:39:50Z'
  generation: 1
  name: heroes-api-centos7
  namespace: heroes
  resourceVersion: '3337267'
  selfLink: /apis/image.openshift.io/v1/namespaces/heroes/imagestreams/heroes-api-centos7
  uid: e280929e-506b-11e9-a2ec-0288bf58ecc2
spec:
  lookupPolicy:
    local: false
  tags:
    - annotations:
        description: build heroes-api on CentOS 7
        iconClass: icon-spring
        openshift.io/display-name: Heroes API
        openshift.io/provider-display-name: itrunner
        sampleRepo: 'https://github.com/sunjc/heroes-api.git'
        supports: itrunner
        tags: 'builder,java'
        version: '1.0.0'
      from:
        kind: DockerImage
        name: 'registry.itrunner.org/heroes-api-centos7:v1.0.0'
...

为了在Service Catalog中显示，tags中必须含有“builder”。只有导入openshift项目的image才是全局可见的，不然仅在本项目Catalog可见。如未显示请刷新页面。

S2I Scripts

  1. assemble
#!/bin/bash -e

# restore build artifacts
if [ -d /tmp/artifacts/.m2 ]; then
    echo "restore build artifacts"
    mv /tmp/artifacts/.m2 $HOME/.
fi

# move the application source
mv /tmp/src $HOME/src

# build the application artifacts
pushd $HOME/src
mvn clean package -Pdev -Dmaven.test.skip=true
popd

# move the artifacts
mv $HOME/src/target/heroes-api-1.0.0.jar $HOME/
rm -rf $HOME/src
  1. run
#!/bin/bash

java -jar $HOME/heroes-api-1.0.0.jar
  1. save-artifacts
#!/bin/bash

# Besides the tar command, all other output to standard out must be surpressed.  Otherwise, the tar stream will be corrupted.
pushd ${HOME} >/dev/null
if [ -d .m2 ]; then
    # all .m2 contents to tar stream
    tar cf - .m2
fi
popd >/dev/null
  1. usage
#!/bin/bash

# inform the user how to use the image
cat <<EOF
This is a S2I sample builder image, to use it, install
https://github.com/openshift/source-to-image
EOF

部署

$ oc new-app heroes/heroes-api-centos7:v1.0.0~https://github.com/sunjc/heroes-api.git --name=heroes-api
$ oc expose service heroes-api --name heroes-api --hostname heroes.apps.itrunner.org --path /api -n heroes
或
$ oc create route edge heroes-api --service heroes-api --hostname heroes.apps.itrunner.org --path /api --insecure-policy Redirect -n heroes

说明：

  1. 若是使用私有git仓库，new-app须要增长参数--source-secret=yoursecret。Secret可在Resources -> Secrets中配置。
  2. route的path配置为/api，不是根目录，这样能够防止外部用户访问api-docs、swagger-ui
  3. 为了内部用户能够访问api-docs、swagger-ui，给service增长externalIPs参数，指定一个或多个OpenShift Node地址，以下：
spec:
  clusterIP: 172.30.80.170
  externalIPs:
    - 10.188.12.116
  ports:
    - name: 8080-tcp
      port: 8080
      protocol: TCP
      targetPort: 8080

部署Angular App

Builder Image

Dockerfile

# heroes-web-centos7
FROM centos/httpd:latest

RUN yum -y update && \
    curl -sL https://rpm.nodesource.com/setup_10.x | bash - && yum -y install nodejs && \
    yum clean all && npm install -g @angular/cli@latest

# Set the labels that are used for OpenShift to describe the builder image.
LABEL maintainer="Sun Jingchuan <jason@163.com>" \
      io.k8s.description="Heroes Web" \
      io.k8s.display-name="Heroes Web" \
      io.openshift.expose-services="8080:http" \
      io.openshift.tags="angular,heroes-web" \
      # this label tells s2i where to find its mandatory scripts(run, assemble, save-artifacts)
      # io.openshift.s2i.scripts-url="image:///usr/libexec/s2i" \
      io.openshift.s2i.scripts-url="image:///tmp/scripts" \
      io.openshift.s2i.destination="/tmp"

ENV APP_ROOT=/opt/heroes
ENV PATH=${APP_ROOT}/bin:${PATH} HOME=${APP_ROOT} HTTPD_MAIN_CONF_PATH=/etc/httpd/conf

COPY bin ${APP_ROOT}/bin
# Copy the S2I scripts to /usr/libexec/s2i
# COPY .s2i/bin /usr/libexec/s2i

RUN chmod -R u+x ${APP_ROOT}/bin && \
    chgrp -R 0 ${APP_ROOT} && \
    chmod -R g=u ${APP_ROOT} /etc/passwd /var/www/html /run/httpd && \
    chown -R root:root /run/httpd /etc/httpd && \
    sed -i -e "s/^User apache/User default/" ${HTTPD_MAIN_CONF_PATH}/httpd.conf && \
    sed -i -e "s/^Group apache/Group root/" ${HTTPD_MAIN_CONF_PATH}/httpd.conf && \
    sed -i -e "s/^Listen 80/Listen 8080/" ${HTTPD_MAIN_CONF_PATH}/httpd.conf && \
    sed -ri " s!^(\s*CustomLog)\s+\S+!\1 |/usr/bin/cat!g; s!^(\s*ErrorLog)\s+\S+!\1 |/usr/bin/cat!g;" ${HTTPD_MAIN_CONF_PATH}/httpd.conf

USER 10001
WORKDIR ${APP_ROOT}

ENTRYPOINT [ "uid_entrypoint" ]

EXPOSE 8080

# Inform the user how to run this image.
# CMD ["/usr/libexec/s2i/usage"]

说明：

  1. 须要安装nodejs、angular/cli
  2. httpd相关目录要授予适当权限，须要修改httpd用户、组、日志输出、监听端口

修改用户后，不能再使用80端口，不然会报错：permission denied: ah00072: make_sock: could not bind to address
编译builder image并上传到Registry

# docker build -t heroes-web-centos7:v1.0.0 .
# docker tag heroes-web-centos7:v1.0.0 registry.itrunner.org/heroes-web-centos7:v1.0.0
# docker push registry.itrunner.org/heroes-web-centos7:v1.0.0

import image

$ oc import-image heroes-web-centos7:v1.0.0 -n heroes --confirm --insecure --from='registry.itrunner.org/heroes-web-centos7:v1.0.0'

编辑ImageStream

$ oc edit is/heroes-web-centos7

增长以下annotations:

tags:
- annotations:
    description: build heroes-web on CentOS 7
    iconClass: icon-angularjs
    openshift.io/display-name: Heroes Web
    openshift.io/provider-display-name: itrunner
    sampleRepo: 'https://github.com/sunjc/heroes-web.git'
    supports: itrunner
    tags: builder,javascript
    version: 1.0.0
  from:
    kind: DockerImage
    name: 'registry.itrunner.org/heroes-web-centos7:v1.0.0'

S2I Scripts

  1. assemble
#!/bin/bash -e

# move the application source
mv /tmp/src $HOME/src

# restore build artifacts
if [ "$(ls /tmp/artifacts/ 2>/dev/null)" ]; then
    mv /tmp/artifacts/* $HOME/src
fi

# build the application artifacts
pushd $HOME/src
npm install
ng build --prod --base-href=/heroes/

# Install the artifacts
mv dist /var/www/html/heroes
mv node_modules $HOME/node_modules
popd

rm -rf $HOME/src
  1. run
#!/bin/bash

# run the application
exec httpd -D FOREGROUND $@
  1. save-artifacts
#!/bin/bash

# Besides the tar command, all other output to standard out must be surpressed.  Otherwise, the tar stream will be corrupted.
pushd ${HOME} >/dev/null
if [ -d node_modules ]; then
    # all node_modules contents to tar stream
    tar cf - node_modules
fi
popd >/dev/null
  1. usage
#!/bin/bash

# inform the user how to use the image
cat <<EOF
This is a S2I sample builder image, to use it, install
https://github.com/openshift/source-to-image
EOF

部署

$ oc new-app heroes/heroes-web-centos7:v1.0.0~https://github.com/sunjc/heroes-web.git --name=heroes-web
$ oc expose service heroes-web --name heroes-web --hostname heroes.apps.itrunner.org --path /heroes --port 8080-tcp -n heroes
或
$ oc create route edge heroes-web --service heroes-web --hostname heroes.apps.itrunner.org --path /heroes \
   --insecure-policy Redirect --port 8080-tcp -n heroes

Image管理

Internal Registry

前面的例子，咱们使用了私有Docker Registry "registry.itrunner.org"，先将base image上传到私有Registry，而后再导入到OpenShift中。咱们也能够直接使用OpenShift内部的Registry(安装在default项目中)，执行push、pull等操做。

为了访问Internal Registry必须先执行docker login登陆Registry，需使用openshift用户名(或email)，使用有效的openshift token做为密码。

  1. 登陆openshift获取token
$ oc login https://openshift.itrunner.org:8443 -u jason --certificate-authority=/path/to/cert.crt
$ oc whoami -t
  1. 登陆Internal Registry
# docker login -u <user_name> -e <email_address> -p <token_value> <registry_server>:<port>

如在openshift节点内访问Internal Registry，registry_server可使用service hostname "docker-registry.default.svc"，port为5000；如在外部访问，registry_server则需使用route hostname，好比"docker-registry-default.apps.itrunner.org" ，docker-registry route的TLS Termination需配置为Re-encrypt。

# docker login -u jason -p xxxxxxxxxxx docker-registry-default.apps.itrunner.org
  1. Push Image
# docker tag heroes-web-centos7:v1.0.0 docker-registry-default.apps.itrunner.org/heroes/heroes-web-centos7:latest
# docker push docker-registry-default.apps.itrunner.org/heroes/heroes-web-centos7:latest

注意：tag格式必须为project/name
push成功后，openshift将自动建立image stream。

  1. Pull Image
# docker pull docker-registry-default.apps.itrunner.org/heroes/heroes-web-centos7:latest
  1. 查看Repository
$  curl -u jason:xxxxxxxxxx -kv https://docker-registry-default.apps.itrunner.org/v2/_catalog?n=100

用户必须有list权限：

$ oc adm policy add-cluster-role-to-user registry-viewer user
  1. 查看image stream
$ oc get is -n heroes
NAME             DOCKER REPO                                                       TAGS            UPDATED
heroes-api           docker-registry.default.svc:5000/heroes/heroes-api           latest          7 days ago
heroes-api-centos7   docker-registry.default.svc:5000/heroes/heroes-api-centos7   v1.0.0          7 days ago
heroes-web           docker-registry.default.svc:5000/heroes/heroes-web           latest          7 days ago
heroes-web-centos7   docker-registry.default.svc:5000/heroes/heroes-web-centos7   latest,v1.0.0   19 hours ago

Image Stream

Image Stream是OpenShift管理容器镜像的方式，其中定义了dockerImageReference，利用tag定义了同docker image各版本的映射关系。Image Stream自己不包含image data，image stream元数据与其余集群信息一块儿存储在etcd实例中。
image stream更新后能够触发build或deployment，如docker image已更新，但未更新image stream则不会触发build或deployment。
image能够源自OpenShift Internal Registry、外部Registry(好比registry.redhat.io、docker.io、私有Registry)、OpenShift集群中的image stream。

apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  annotations:
    openshift.io/image.dockerRepositoryCheck: '2019-04-09T05:59:17Z'
  creationTimestamp: '2019-04-01T08:02:37Z'
  generation: 2
  name: heroes-web-centos7
  namespace: heroes
  resourceVersion: '2145022'
  selfLink: >-
    /apis/image.openshift.io/v1/namespaces/heroes/imagestreams/heroes-web-centos7
  uid: 83deac8c-5454-11e9-af2b-02f23e935364
spec:
  lookupPolicy:
    local: false
  tags:
    - annotations:
        description: >-
          build heroes-web on CentOS 7. WARNING: By selecting this tag, your
          application will automatically update to use the latest version.
        iconClass: icon-angularjs
        openshift.io/display-name: Heroes Web (Latest)
        openshift.io/provider-display-name: itrunner
        sampleRepo: 'https://github.com/sunjc/heroes-web.git'
        supports: itrunner
        tags: 'builder,javascript'
      from:
        kind: DockerImage
        name: 'docker-registry.default.svc:5000/heroes/heroes-web-centos7:latest'
      generation: 2
      importPolicy: {}
      name: latest
      referencePolicy:
        type: Source
    - annotations:
        description: build heroes-web on CentOS 7
        iconClass: icon-angularjs
        openshift.io/display-name: Heroes Web 1.0.0
        openshift.io/provider-display-name: itrunner
        sampleRepo: 'https://github.com/sunjc/heroes-web.git'
        supports: itrunner
        tags: 'builder,javascript'
        version: 1.0.0
      from:
        kind: DockerImage
        name: 'registry.itrunner.org/heroes-web-centos7:v1.0.0'
      generation: 1
      importPolicy:
        insecure: true
      name: v1.0.0
      referencePolicy:
        type: Source
status:
  dockerImageRepository: 'docker-registry.default.svc:5000/heroes/heroes-web-centos7'
  tags:
    - items:
        - created: '2019-04-08T07:37:11Z'
          dockerImageReference: >-
            docker-registry.default.svc:5000/heroes/heroes-web-centos7@sha256:7e4126ec9ec0d4158d962936a38f255806731d33d6fe03b29d95d82759823fcd
          generation: 2
          image: >-
            sha256:7e4126ec9ec0d4158d962936a38f255806731d33d6fe03b29d95d82759823fcd
      tag: latest
    - items:
        - created: '2019-04-01T08:02:37Z'
          dockerImageReference: >-
            registry.itrunner.org/heroes-web-centos7@sha256:7e4126ec9ec0d4158d962936a38f255806731d33d6fe03b29d95d82759823fcd
          generation: 1
          image: >-
            sha256:7e4126ec9ec0d4158d962936a38f255806731d33d6fe03b29d95d82759823fcd
      tag: v1.0.0

Reference Policy
当使用从外部Registry导入的image时，"引用策略" 容许指定从何处提取image。有两个选项：Local和Source：

  • Local 从OpenShift Internal Registry提取image
  • Source 直接从外部Registry提取image

查询Image Stream信息

$ oc describe is/heroes-web-centos7
$ oc describe istag/heroes-web-centos7:latest

为外部Image添加tag

$ oc tag docker.io/openshift/base-centos7:latest base-centos7:latest

为Image Stream附加tag
给现有tag附加一个tag：

$ oc tag heroes-api-centos7:v1.0.0 heroes-api-centos7:latest

更新tag

$ oc tag heroes-api-centos7:v1.0.1 heroes-api-centos7:latest

说明：与附加tag不一样，两个tag都应存在
删除tag

$ oc tag -d heroes-api-centos7:v1.0.0

或

$ oc delete istag/heroes-api-centos7:v1.0.0

删除Image Stream

$ oc delete is base-centos7

按期更新tag
可使用--scheduled：

$ oc tag docker.io/python:3.6.0 python:3.6 --scheduled

也能够在tag定义中设置importPolicy.scheduled为true：

apiVersion: v1
kind: ImageStream
metadata:
  name: python
spec:
  tags:
  - from:
      kind: DockerImage
      name: docker.io/python:3.6.0
    name: latest
    importPolicy:
      scheduled: true

周期默认为15分钟。

Binary Build

Binary Build主要应用于测试和Jenkins pipeline场景。开发人员提交源码前若是想先测试一下，可以使用本地源码来构建App Image。Binary Build不能自动触发，只能手动执行。
Binary Build使用oc start-build命令，须要提供BuildConfig或存在的build，支持从几种如下source来构建App Image：

  • --from-file 好比Dockerfile
  • --from-dir 本地目录，start-build打包此目录并上传到openshift
  • --from-archive tar、tar.gz、zip等
  • --from-repo 本地源码目录，start-build打包最近commit的代码并上传到openshift

From a directory

$ oc start-build heroes-web --from-dir="." --follow
或
$ oc start-build --from-build=heroes-web-1 --from-dir="." --follow

From a Git repository

$ git commit -m "My changes"
$ oc start-build heroes-web --from-repo="." --follow

Jenkins Pipeline

...
stage("Build Image") {
  steps {
    dir('heroes-web/dist') {
      sh 'oc start-build heroes-web --from-dir . --follow'
    }
  }
}
...

Pipeline部署

Jenkins是普遍应用的CI工具，大多数工程师都有使用经验，更习惯使用Jenkins部署应用。使用Jenkins Pipeline部署OpenShift应用，不影响原有流程，能够正常执行测试、代码质量检查、编译打包等操做，只需在部署时调用oc start-build。

安装Jenkins

OpenShift提供了两个Jenkins Template：jenkins-ephemeral、jenkins-persistent，一种使用瞬时存储，一种使用持久存储，二者均使用jenkins image stream(docker.io/openshift/jenkins-2-centos7:v3.11)。jenkins image安装了OpenShift Client、OpenShift Login、OpenShift Sync、Kubernetes、Kubernetes Credentialst等插件。安装后既能够在OpenShift中运行Job，也能够在Jenkins中运行Job。

jenkins-persistent采用动态存储配置，PVC默认名称为jenkins(JENKINS_SERVICE_NAME)：

- apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: '${JENKINS_SERVICE_NAME}'
  spec:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: '${VOLUME_CAPACITY}'
    storageClassName: glusterfs-storage-block
- apiVersion: v1
  kind: DeploymentConfig
  metadata:
    annotations:
      template.alpha.openshift.io/wait-for-ready: 'true'
    name: '${JENKINS_SERVICE_NAME}'
  spec:
    ...
    template:
      metadata:
        labels:
          name: '${JENKINS_SERVICE_NAME}'
      spec:
        containers:
          - capabilities: {}
            ...
            volumeMounts:
              - mountPath: /var/lib/jenkins
                name: '${JENKINS_SERVICE_NAME}-data'
        ...
        volumes:
          - name: '${JENKINS_SERVICE_NAME}-data'
            persistentVolumeClaim:
              claimName: '${JENKINS_SERVICE_NAME}'

如使用GlusterFS须要先修改Template，指定storageClassName：

- apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: '${JENKINS_SERVICE_NAME}'
  spec:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: '${VOLUME_CAPACITY}'
    storageClassName: glusterfs-storage-block

下面使用jenkins-persistent来安装，命令以下：

$ oc project heroes
$ oc new-app jenkins-persistent -p VOLUME_CAPACITY=2Gi  -p MEMORY_LIMIT=2Gi
--> Deploying template "openshift/jenkins-persistent" to project heroes

     Jenkins
     ---------
     Jenkins service, with persistent storage.

     NOTE: You must have persistent volumes available in your cluster to use this template.

     A Jenkins service has been created in your project.  Log into Jenkins with your OpenShift account.  The tutorial at https://github.com/openshift/origin/blob/master/examples/jenkins/README.md contains more information about using this template.

     * With parameters:
        * Jenkins Service Name=jenkins
        * Jenkins JNLP Service Name=jenkins-jnlp
        * Enable OAuth in Jenkins=true
        * Memory Limit=2Gi
        * Volume Capacity=2Gi
        * Jenkins ImageStream Namespace=openshift
        * Disable memory intensive administrative monitors=false
        * Jenkins ImageStreamTag=jenkins:2
        * Fatal Error Log File=false

--> Creating resources ...
    route.route.openshift.io "jenkins" created
    persistentvolumeclaim "jenkins" created
    deploymentconfig.apps.openshift.io "jenkins" created
    serviceaccount "jenkins" created
    rolebinding.authorization.openshift.io "jenkins_edit" created
    service "jenkins-jnlp" created
    service "jenkins" created
--> Success
    Access your application via route 'jenkins-heroes.apps.itrunner.org'
    Run 'oc status' to view your app.

注意：如MEMORY_LIMIT配置低，则Jenkins Master节点架构为Linux (i386)。
查看存储
安装成功后，能够从安装Jenkins的node查看存储：

$  oc get pods -o wide
NAME              READY     STATUS      RESTARTS   AGE       IP            NODE                              NOMINATED NODE
jenkins-1-hw5q5   1/1       Running     0          5m        10.131.0.26   app2.itrunner.org   <none>
$ oc get pvc
NAME          STATUS        VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS              AGE
jenkins       Bound         pvc-bf3ff63d-6f1c-11e9-9dd9-02ef509f23d0   2Gi        RWO            glusterfs-storage   5m
# mount | grep pvc-bf3ff63d-6f1c-11e9-9dd9-02ef509f23d0
10.188.12.116:vol_0e157791c95b65a94011aed789d2037b on /var/lib/origin/openshift.local.volumes/pods/c12d7625-6f1c-11e9-ad9d-02499a450338/volumes/kubernetes.io~glusterfs/63d-6f1c-11e9-9dd9-02ef509f23d0 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)

# cd /var/lib/origin/openshift.local.volumes/pods/c12d7625-6f1c-11e9-ad9d-02499a450338/volumes/kubernetes.io~glusterfs/63d-6f1c-11e9-9dd9-02ef509f23d0

扩展存储
当原配置的存储容量不知足需求时，能够扩展存储。

  1. StorageClass的属性allowVolumeExpansion必须设为true
$ oc edit sc/glusterfs-storage

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: 2019-04-28T02:58:06Z
  name: glusterfs-storage
  resourceVersion: "1903911"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/glusterfs-storage
  uid: 723320e6-6961-11e9-b13d-02947d98b66e
parameters:
  resturl: http://heketi-storage.app-storage.svc:8080
  restuser: admin
  secretName: heketi-storage-admin-secret
  secretNamespace: app-storage
provisioner: kubernetes.io/glusterfs
reclaimPolicy: Delete
volumeBindingMode: Immediate

查看glusterfs-storage：

$ oc describe sc glusterfs-storage
Name:                  glusterfs-storage
IsDefaultClass:        No
Annotations:           <none>
Provisioner:           kubernetes.io/glusterfs
Parameters:            resturl=http://heketi-storage.app-storage.svc:8080,restuser=admin,secretName=heketi-storage-admin-secret,secretNamespace=app-storage
AllowVolumeExpansion:  True
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>
  1. 更新PVC spec.resources.requests.storage
$ oc edit pvc/jenkins

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftNewApp
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/glusterfs
  creationTimestamp: 2019-05-05T10:01:26Z
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app: jenkins-persistent
    template: jenkins-persistent-template
  name: jenkins
  namespace: heroes
  resourceVersion: "1904277"
  selfLink: /api/v1/namespaces/heroes/persistentvolumeclaims/jenkins
  uid: bf3ff63d-6f1c-11e9-9dd9-02ef509f23d0
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
  storageClassName: glusterfs-storage
  volumeName: pvc-bf3ff63d-6f1c-11e9-9dd9-02ef509f23d0
...

安装/更新插件
使用openshift用户登陆jenkins。
快速部署OpenShift应用
进入系统管理 -> 插件管理，安装/更新插件。为了编译Angular须要安装NodeJS插件（另外一种选择，可使用node agent）。
配置全局工具

  1. JDK

快速部署OpenShift应用

  1. Maven

快速部署OpenShift应用

  1. NodeJS

快速部署OpenShift应用

首次调用时，Jenkins会自动安装全局工具，pipeline配置以下：

tools {
  jdk 'jdk8'
  maven 'maven-3.6'
}
tools {
  nodejs 'nodejs-10.15'
}

建立配置文件
快速部署OpenShift应用

下面仍以部署Spring Boot和Angular工程为例介绍Pipeline的使用，开始以前请重建heroes project。

部署Spring Boot App

  1. 构建Builder Image

Jenkins负责编译、打包，所以Builder Image再也不须要Maven，修改以下：
Dockerfile

# jdk8-centos7
FROM centos:latest

RUN yum -y update && yum clean all

# Set the labels that are used for OpenShift to describe the builder image.
LABEL maintainer="Sun Jingchuan <jason@163.com>" \
      io.k8s.description="Oracle JDK 1.8.0_202 based on CentOS 7" \
      io.k8s.display-name="Oracle JDK 1.8.0_202" \
      io.openshift.expose-services="8080:http" \
      io.openshift.tags="jdk8"

ENV JAVA_HOME=/usr/lib/jdk1.8.0_202 \
    APP_ROOT=/opt/app-root
ENV PATH=${JAVA_HOME}/bin:${APP_ROOT}/bin:${PATH} HOME=${APP_ROOT}

COPY lib/jdk1.8.0_202 ${JAVA_HOME}
COPY bin ${APP_ROOT}/bin

RUN chmod -R u+x ${APP_ROOT}/bin && \
    chgrp -R 0 ${APP_ROOT} && \
    chmod -R g=u ${APP_ROOT} /etc/passwd

USER 10001
WORKDIR ${APP_ROOT}

ENTRYPOINT [ "uid_entrypoint" ]

EXPOSE 8080

编译、上传builder image
此次咱们上传到OpenShift Docker Registry，注意push前要先执行docker login。

# docker build -t jdk8-centos7 .
# docker tag jdk8-centos7:latest docker-registry-default.apps.itrunner.org/heroes/jdk8-centos7:latest
# docker push docker-registry-default.apps.itrunner.org/heroes/jdk8-centos7:latest
  1. 建立App Image Dockerfile

App Image以builder image为基础，仅需拷贝Jenkins编译好的jar，内容以下：
Dockerfile.app

# heroes-api
FROM heroes/jdk8-centos7:latest

COPY heroes-api-1.0.0.jar $HOME

CMD java -jar $HOME/heroes-api-1.0.0.jar
  1. 从Dockerfile建立BuildConfig
$ cat Dockerfile.app | oc new-build -D - --name heroes-api

执行后生成BuildConfig的部份内容以下：

spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: 'heroes-api:latest'
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    dockerfile: "FROM heroes/jdk8-centos7:latest\r\n\r\nCOPY heroes-api-1.0.0.jar $HOME\r\n\r\nCMD java -jar $HOME/heroes-api-1.0.0.jar"
    type: Dockerfile
  strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'jdk8-centos7:latest'
    type: Docker

会自启动build，但此时没有jar致使build失败，pipeline调用时才会传入jar。

  1. 建立Pipeline BuildConfig

pipeline.yml

apiVersion: v1
kind: BuildConfig
metadata:
  name: heroes-api-pipeline
spec:
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfile: |-
        pipeline {
          agent any
          tools {
            jdk 'jdk8'
            maven 'maven-3.6'
          }
          stages {
            stage("Clone Source") {
              steps {
                checkout([$class: 'GitSCM',
                  branches: [[name: '*/master']],
                  extensions: [
                    [$class: 'RelativeTargetDirectory', relativeTargetDir: 'heroes-api']
                  ],
                  userRemoteConfigs: [[url: 'https://github.com/sunjc/heroes-api.git']]
                ])
              }
            }
            stage("Build Backend") {
              steps {
                dir('heroes-api') {
                  sh 'mvn clean package -Pdev -Dmaven.test.skip=true'
                }
              }
            }
            stage("Build Image") {
              steps {
                dir('heroes-api/target') {
                  sh 'oc start-build heroes-api --from-dir . --follow'
                }
              }
            }
          }
        }
    type: JenkinsPipeline

Pipeline能够内嵌在BuildConfig内，也能够引用git中的Jenkinsfile（推荐）:

apiVersion: v1
kind: BuildConfig
metadata:
  name: heroes-api-pipeline
spec:
  source:
    git:
      uri: "https://github.com/sunjc/heroes-api"
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfilePath: Jenkinsfile
    type: JenkinsPipeline

建立pipeline：

$ oc create -f ./pipeline.yml

若是在项目中没有安装Jenkins，建立pipeline后将自动部署jenkins-ephemeral。

  1. 启动pipeline
$ oc start-build heroes-api-pipeline

也能够在jenkins或Application Console -> Builds -> Pipelines中启动。

  1. 部署App
$ oc new-app heroes-api
$ oc create route edge heroes-api --service heroes-api --hostname heroes.apps.itrunner.org --path /api --insecure-policy Redirect

部署Angular App

  1. 构建Builder Image

Dockerfile

# httpd-centos7
FROM centos/httpd:latest

RUN yum -y update && yum clean all

LABEL maintainer="Sun Jingchuan <jason@163.com>" \
      io.k8s.description="Apache Httpd 2.4" \
      io.k8s.display-name="Apache Httpd 2.4" \
      io.openshift.expose-services="8080:http" \
      io.openshift.tags="httpd"

ENV APP_ROOT=/opt/app-root
ENV PATH=${APP_ROOT}/bin:${PATH} HOME=${APP_ROOT} HTTPD_MAIN_CONF_PATH=/etc/httpd/conf

COPY bin ${APP_ROOT}/bin
COPY .s2i/bin/run ${APP_ROOT}/bin/run

RUN chmod -R u+x ${APP_ROOT}/bin && \
    chgrp -R 0 ${APP_ROOT} && \
    chmod -R g=u ${APP_ROOT} /etc/passwd /var/www/html /run/httpd && \
    chown -R root:root /run/httpd /etc/httpd && \
    sed -i -e "s/^User apache/User default/" ${HTTPD_MAIN_CONF_PATH}/httpd.conf && \
    sed -i -e "s/^Group apache/Group root/" ${HTTPD_MAIN_CONF_PATH}/httpd.conf && \
    sed -i -e "s/^Listen 80/Listen 8080/" ${HTTPD_MAIN_CONF_PATH}/httpd.conf && \
    sed -ri " s!^(\s*CustomLog)\s+\S+!\1 |/usr/bin/cat!g; s!^(\s*ErrorLog)\s+\S+!\1 |/usr/bin/cat!g;" ${HTTPD_MAIN_CONF_PATH}/httpd.conf

USER 10001
WORKDIR ${APP_ROOT}

ENTRYPOINT [ "uid_entrypoint" ]

EXPOSE 8080

编译、上传Builder Image

# docker build -t httpd-centos7 .
# docker tag httpd-centos7:latest docker-registry-default.apps.itrunner.org/heroes/httpd-centos7:latest
# docker push docker-registry-default.apps.itrunner.org/heroes/httpd-centos7:latest
  1. 建立App Image Dockerfile

Dockerfile.app

# heroes-web
FROM heroes/httpd-centos7:latest

COPY heroes /var/www/html/heroes

CMD $HOME/bin/run
  1. 从Dockerfile建立BuildConfig
$ cat Dockerfile.app | oc new-build -D - --name heroes-web

执行后将启动build，此时没有传入内容致使build失败。

  1. 建立Pipeline BuildConfig

pipeline.yml

apiVersion: v1
kind: BuildConfig
metadata:
  name: heroes-web-pipeline
spec:
  source:
    git:
      uri: "https://github.com/sunjc/heroes-web"
  strategy:
    jenkinsPipelineStrategy:
      jenkinsfilePath: Jenkinsfile
    type: JenkinsPipeline

Jenkinsfile

pipeline {
  agent any
  tools {
    nodejs 'nodejs-10.15'
  }
  stages {
    stage("Clone Source") {
      steps {
        checkout([$class: 'GitSCM',
          branches: [[name: '*/master']],
          extensions: [
            [$class: 'RelativeTargetDirectory', relativeTargetDir: 'heroes-web']
          ],
          userRemoteConfigs: [[url: 'https://github.com/sunjc/heroes-web']]
        ])
      }
    }
    stage("Build Angular") {
      steps {
        dir('heroes-web') {
          sh 'npm install'
          sh 'ng config -g cli.warnings.versionMismatch false'
          sh 'ng build --prod --base-href=/heroes/'
        }
      }
    }
    stage("Build Image") {
      steps {
        dir('heroes-web/dist') {
          sh 'oc start-build heroes-web --from-dir . --follow'
        }
      }
    }
  }
}

建立pipeline：

$ oc create -f ./pipeline.yml
  1. 启动pipeline
$ oc start-build heroes-web-pipeline
  1. 部署App
$ oc new-app heroes-web
$  oc create route edge heroes-web --service heroes-web --hostname heroes.apps.itrunner.org --path /heroes \
   --insecure-policy Redirect --port 8080-tcp -n heroes

OpenShift DSL

OpenShift Jenkins image安装了OpenShift Jenkins Client插件，支持使用OpenShift Domain Specific Language(DSL)定义pipeline。

def templatePath = 'https://raw.githubusercontent.com/openshift/nodejs-ex/master/openshift/templates/nodejs-mongodb.json' 
def templateName = 'nodejs-mongodb-example' 
pipeline {
  agent {
    node {
      label 'nodejs' 
    }
  }
  options {
    timeout(time: 20, unit: 'MINUTES') 
  }
  stages {
    stage('preamble') {
        steps {
            script {
                openshift.withCluster() {
                    openshift.withProject() {
                        echo "Using project: ${openshift.project()}"
                    }
                }
            }
        }
    }
    stage('cleanup') {
      steps {
        script {
            openshift.withCluster() {
                openshift.withProject() {
                  openshift.selector("all", [ template : templateName ]).delete() 
                  if (openshift.selector("secrets", templateName).exists()) { 
                    openshift.selector("secrets", templateName).delete()
                  }
                }
            }
        }
      }
    }
    stage('create') {
      steps {
        script {
            openshift.withCluster() {
                openshift.withProject() {
                  openshift.newApp(templatePath) 
                }
            }
        }
      }
    }
    stage('build') {
      steps {
        script {
            openshift.withCluster() {
                openshift.withProject() {
                  def builds = openshift.selector("bc", templateName).related('builds')
                  timeout(5) { 
                    builds.untilEach(1) {
                      return (it.object().status.phase == "Complete")
                    }
                  }
                }
            }
        }
      }
    }
    stage('deploy') {
      steps {
        script {
            openshift.withCluster() {
                openshift.withProject() {
                  def rm = openshift.selector("dc", templateName).rollout().latest()
                  timeout(5) { 
                    openshift.selector("dc", templateName).related('pods').untilEach(1) {
                      return (it.object().status.phase == "Running")
                    }
                  }
                }
            }
        }
      }
    }
    stage('tag') {
      steps {
        script {
            openshift.withCluster() {
                openshift.withProject() {
                  openshift.tag("${templateName}:latest", "${templateName}-staging:latest") 
                }
            }
        }
      }
    }
  }
}

具体请参考OpenShift Pipeline Builds

清理对象

随着不断的构建、部署应用，build、deployment、image等对象会逐渐增多，管理员应按期清理再也不须要的对象。
OpenShift提供了oc adm prune命令来清理对象，支持auth、groups、builds、deployments、images等类型。

$ oc adm prune <object_type> <options>

Pruning builds

$ oc adm prune builds [<options>]
Option Description
--confirm Indicate that pruning should occur, instead of performing a dry-run
--orphans Prune all builds whose build config no longer exists, status is complete, failed, error, or canceled
--keep-complete=N Per build config, keep the last N builds whose status is complete. (default 5)
--keep-failed=N Per build config, keep the last N builds whose status is failed, error, or canceled (default 1)
--keep-younger-than=duration Do not prune any object that is younger than duration relative to the current time. (default 60m) 
$ oc adm prune builds --orphans --keep-complete=5 --keep-failed=1 --keep-younger-than=60m --confirm

Pruning deployments

$ oc adm prune deployments [<options>]
Option Description
--confirm Indicate that pruning should occur, instead of performing a dry-run
--orphans Prune all deployments whose deployment config no longer exists, status is complete or failed, and replica count is zero
--keep-complete=N Per deployment config, keep the last N deployments whose status is complete and replica count is zero. (default 5)
--keep-failed=N Per deployment config, keep the last N deployments whose status is failed and replica count is zero. (default 1)
--keep-younger-than=duration Do not prune any object that is younger than duration relative to the current time. (default 60m) Valid units of measurement include nanoseconds (ns), microseconds (us), milliseconds (ms), seconds (s), minutes (m), and hours (h) 
$ oc adm prune deployments --orphans --keep-complete=5 --keep-failed=1 --keep-younger-than=60m --confirm

Pruning images

$ oc adm prune images [<options>]

system:admin用户不能执行清理image的操做，必须使用普通用户登陆，且用户必须有system:image-pruner角色。

$ oc login https://openshift.itrunner.org:8443 --token=xxxx
Option Description
--all Include images that were not pushed to the registry, but have been mirrored by pullthrough. This is on by default. To limit the pruning to images that were pushed to the integrated registry, pass --all=false
--certificate-authority The path to a certificate authority file to use when communicating with the OKD-managed registries. Defaults to the certificate authority data from the current user’s configuration file. If provided, secure connection will be initiated
--confirm Indicate that pruning should occur, instead of performing a dry-run. This requires a valid route to the integrated container image registry. If this command is run outside of the cluster network, the route needs to be provided using --registry-url
--force-insecure Use caution with this option. Allow an insecure connection to the Docker registry that is hosted via HTTP or has an invalid HTTPS certificate
--keep-tag-revisions=N For each image stream, keep up to at most N image revisions per tag. (default 3)
--keep-younger-than=duration Do not prune any image that is younger than duration relative to the current time. Do not prune any image that is referenced by any other object that is younger than duration relative to the current time. (default 60m)
--prune-over-size-limit Prune each image that exceeds the smallest limit defined in the same project. This flag cannot be combined with --keep-tag-revisions nor --keep-younger-than
--registry-url The address to use when contacting the registry. The command will attempt to use a cluster-internal URL determined from managed images and image streams. In case it fails (the registry cannot be resolved or reached), an alternative route that works needs to be provided using this flag. The registry host name may be prefixed by https:// or http:// which will enforce particular connection protocol
--prune-registry In conjunction with the conditions stipulated by the other options, this option controls whether the data in the registry corresponding to the OKD Image API Objects is pruned. By default, image pruning processes both the Image API Objects and corresponding data in the registry. This options is useful when you are only concerned with removing etcd content, possibly to reduce the number of image objects, but are not concerned with cleaning up registry storage; or intend to do that separately by Hard Pruning the Registry, possibly during an appropriate maintenance window for the registry

使用--keep-younger-than清理image，不会清理如下状况的image：

  • 建立时间在--keep-younger-than内的全部Pod
  • 建立时间在--keep-younger-than内的全部ImageStream
  • 正在运行的Pod
  • 状态为pending的Pod
  • 全部replication controllers
  • 全部deployment configurations
  • 全部build configurations
  • 全部builds

使用--prune-over-size-limit清理超过指定Limit的image，不会清理如下状况的image：

  • 正在运行的Pod
  • 状态为pending的Pod
  • 全部replication controllers
  • 全部deployment configurations
  • 全部build configurations
  • 全部builds

示例：

$ oc adm prune images --keep-tag-revisions=3 --keep-younger-than=60m --confirm

$ oc adm prune images --prune-over-size-limit --confirm

清理image后不会更新registry cache，为了清理cache能够从新部署registry：

$ oc rollout latest dc/docker-registry -n default

docker prune

docker相关prune命令：

# docker container prune
# docker image prune
# docker volume prune

慎用，会删除openshift/origin-docker-builder、openshift/origin-deployer。
快速部署OpenShift应用

推荐文档

潘晓华Michael - OpenShift

参考文档

OKD Latest Documentation
source-to-image

相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程