apollo须要使用数据库,若是是mysql,须要版本在5.6以上:
本次环境mysql部署在10.4.7.11上,使用mysql5.7,为测试简单起见,各环境数据库使用同一个,不作隔离node
rpm -Uvh https://dev.mysql.com/get/mysql80-community-release-el7-1.noarch.rpm yum -y install yum-utils yum-config-manager --disable mysql80-community yum-config-manager --enable mysql57-community yum install mysql-server -y
cat >/etc/my.cnf <<'EOF' [mysqld] character_set_server = utf8mb4 collation_server = utf8mb4_general_ci init_connect = "SET NAMES 'utf8mb4'" [mysql] default-character-set = utf8mb4 EOF
systemctl start mysqld systemctl enable mysqld mysql -u root -p`grep password /var/log/messages|awk '{print $NF}'` # 修改密码 > set global validate_password_policy=0; > set global validate_password_length=1; > set password=password('123456'); > flush privileges; # 检查字符集:须要四个都是utf8mb4 > \s
configdb初始化脚本
portal初始化脚本mysql
wget -O apolloconfig.sql https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql # 导入sql文件 mysql -uroot -p123456 < apolloconfig.sql # 检查是否导入成功 mysql -uroot -p123456 -e "show databases;"|grep ApolloConfigDB
mysql -uroot -p123456 > grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigDB.* to 'apollo'@'10.4.7.%' identified by "123456"; # 修改数据 > use ApolloConfigDB > update ServerConfig set Value='http://apollo-config.zq.com/eureka' where Id=1;
vi /var/named/zq.com.zone mysql A 10.4.7.11 apollo-config A 10.4.7.10 apollo-admin A 10.4.7.10 apollo-portal A 10.4.7.10 # 重启并验证 systemctl restart named dig -t A apollo-config.zq.com @10.4.7.11 +short
因为portal使用的是另外一个portaldb,咱们须要在数据库中新建portdb,并初始化git
wget -O apollo-portal.sql https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/portaldb/V1.0.0__initialization.sql # 导入sql文件 mysql -uroot -p123456 < apollo-portal.sql # 检查是否导入成功 mysql -uroot -p123456 -e "show databases;"|grep ApolloPortalDB
都使用apollo
用户来管理数据库是为了方便,若是有相关的安全考虑能够给config和portal分别使用不一样的数据库帐号github
mysql -uroot -p123456 > grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apollo"@"10.4.7.%" identified by "123456"; # 更新部门名 > update ApolloPortalDB.ServerConfig set Value='[{"orgId":"zq01","orgName":"研发部"},{"orgId":"zq02","orgName":"运维部"}]' where Id=2;
操做在7.200
上完成web
wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip mkdir /data/dockerfile/apollo-configservice unzip -o apollo-configservice-1.5.1-github.zip -d /data/dockerfile/apollo-configservice/
cd /data/dockerfile/apollo-configservice/config # 修改数据库链接地址 sed -i 's#fill-in-the-correct-server#mysql.zq.com#g' application-github.properties # 修改数据库链接用户和密码 sed -i 's#FillInCorrectUser#apollo#g' application-github.properties sed -i 's#FillInCorrectPassword#123456#g' application-github.properties # 查看结果 config]# egrep -v "^#|$^" application-github.properties spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456
程序中自带的start.sh
启动脚本时不适用与K8S运行,所以须要专门下载他们提供的K8S内使用的脚本spring
# 1.从官网下载启动脚本 cd /data/dockerfile/apollo-configservice/scripts/ wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/apollo-on-kubernetes/apollo-config-server/scripts/startup-kubernetes.sh # 2. 添加一行使用主机名的变量 sed -i '5i APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)' startup-kubernetes.sh # 3.根据须要修改下jvm限制
cd .. cat >Dockerfile <<'EOF' FROM harbor.zq.com/base/jre8:8u112 ENV VERSION 1.5.1 RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\ echo "Asia/Shanghai" > /etc/timezone ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar ADD config/ /apollo-configservice/config ADD scripts/ /apollo-configservice/scripts CMD ["sh","/apollo-configservice/scripts/startup-kubernetes.sh"] EOF
docker build . -t harbor.zq.com/infra/apollo-configservice:v1.5.1 docker push harbor.zq.com/infra/apollo-configservice:v1.5.1
mkdir /data/k8s-yaml/apollo-configservice cd /data/k8s-yaml/apollo-configservice
给configservice建立cm资源的清单的目的是方便修改
其实里面的内容就是前面修改的application-github.properties
文件
若是肯定不会修改,能够不建立此cm,直接写死配置到docker镜像中docker
cat >cm.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: apollo-configservice-cm namespace: infra data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456 eureka.service.url = http://apollo-config.zq.com/eureka app.properties: | appId=100003171 EOF
在同一个configmap资源中,能够添加多个配置文件,上述配置就有两个,分别是:
application-github.properties
和app.properties
数据库
cat >dp.yaml <<'EOF' kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: infra labels: name: apollo-configservice spec: replicas: 1 selector: matchLabels: name: apollo-configservice template: metadata: labels: app: apollo-configservice name: apollo-configservice spec: volumes: - name: configmap-volume configMap: name: apollo-configservice-cm containers: - name: apollo-configservice image: harbor.zq.com/infra/apollo-configservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-configservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 EOF
cat >svc.yaml <<'EOF' kind: Service apiVersion: v1 metadata: name: apollo-configservice namespace: infra spec: ports: - protocol: TCP port: 8080 targetPort: 8080 selector: app: apollo-configservice EOF
cat >ingress.yaml <<'EOF' kind: Ingress apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: infra spec: rules: - host: apollo-config.zq.com http: paths: - path: / backend: serviceName: apollo-configservice servicePort: 8080 EOF
service中不必定必须暴露8080,分配的clusterIP中全部的端口均可以
但ingress中的servicePort必定要与service中暴露的端口匹配
kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/cm.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/dp.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/svc.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-configservice/ingress.yaml
kubectl -n infra get pod|grep apollo-config # 检查命令 kubectl -n infra logs apollo-configservice-64fc749978-9nz5h --tail=4
须要等到eureka启动之后才能够,接下来使用浏览器访问apollo-config.zq.com
操做在7.200
上完成
wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-adminservice-1.5.1-github.zip mkdir /data/dockerfile/apollo-adminservice unzip -o apollo-adminservice-1.5.1-github.zip -d /data/dockerfile/apollo-adminservice/
因为使用了configmap资源将配置文件挂载出来了,因此不在修改配置文件,如需修改配置文件,请参考部署apollo-configservice时候的修改方法:
程序中自带的start.sh
启动脚本时不适用与K8S运行,所以须要专门下载他们提供的K8S内使用的脚本
# 1.从官网下载启动脚本 cd /data/dockerfile/apollo-adminservice/scripts/ wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/apollo-on-kubernetes/apollo-admin-server/scripts/startup-kubernetes.sh # 2. 添加一行使用主机名的变量 sed -i '5i APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)' startup-kubernetes.sh # 3.修改端口为8080 sed -i 's#8090#8080#g' startup-kubernetes.sh
官方配置文件端口改成8090的目的是虚拟机部署的时候端口不冲突
但咱们用K8S部署,会给他单独的clusterIP,因此不用担忧端口重复
cd .. cat >Dockerfile <<'EOF' FROM stanleyws/jre8:8u112 ENV VERSION 1.5.1 RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\ echo "Asia/Shanghai" > /etc/timezone ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar ADD config/ /apollo-adminservice/config ADD scripts/ /apollo-adminservice/scripts CMD ["/bin/bash","/apollo-adminservice/scripts/startup-kubernetes.sh"] EOF
因为要使用cm配置资源,所以就不改config中的配置了
docker build . -t harbor.zq.com/infra/apollo-adminservice:v1.5.1 docker push harbor.zq.com/infra/apollo-adminservice:v1.5.1
adminservice向注册中心注册服务,不直接对外提供服务,所以不须要暴露端口,只须要cm资源和dp资源
mkdir /data/k8s-yaml/apollo-adminservice cd /data/k8s-yaml/apollo-adminservice
cat >cm.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: apollo-adminservice-cm namespace: infra data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloConfigDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456 eureka.service.url = http://apollo-config.zq.com/eureka app.properties: | appId=100003172 EOF
注意每一个服务的appId都不会同样哦
cat >dp.yaml <<'EOF' kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-adminservice namespace: infra labels: name: apollo-adminservice spec: replicas: 1 selector: matchLabels: name: apollo-adminservice template: metadata: labels: app: apollo-adminservice name: apollo-adminservice spec: volumes: - name: configmap-volume configMap: name: apollo-adminservice-cm containers: - name: apollo-adminservice image: harbor.zq.com/infra/apollo-adminservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-adminservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 EOF
kubectl create -f http://k8s-yaml.zq.com/apollo-adminservice/cm.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-adminservice/dp.yaml
~]# kubectl -n infra get pod|grep apollo-admin apollo-adminservice-6cd4fcfdc8-2drnq 1/1 Running 0 9s # 检查命令 kubectl -n infra logs apollo-configservice-6cd4fcfdc8-2drnq --tail=4
经过 apollo-config.zq.com 检查是否注册到了eureka:
已经顺利的注册到了注册中心中。
wget https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-portal-1.5.1-github.zip mkdir /data/dockerfile/apollo-portal unzip -o apollo-portal-1.5.1-github.zip -d /data/dockerfile/apollo-portal/
因为使用concigmap资源,故不在这里修改
注意若是要修改的话,要分别修改两个文件
apollo-env.properties
修改数据库配置apollo-env.properties
修改支持的环境列表# 1.从官网下载启动脚本 cd /data/dockerfile/apollo-portal/scripts/ wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/apollo-on-kubernetes/apollo-portal-server/scripts/startup-kubernetes.sh # 2. 添加一行使用主机名的变量 sed -i '5i APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)' startup-kubernetes.sh # 3.修改端口为8080 sed -i 's#8070#8080#g' startup-kubernetes.sh
cd /data/dockerfile/apollo-portal/ cat >Dockerfile <<'EOF' FROM stanleyws/jre8:8u112 ENV VERSION 1.5.1 RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\ echo "Asia/Shanghai" > /etc/timezone ADD apollo-portal-${VERSION}.jar /apollo-portal/apollo-portal.jar ADD config/ /apollo-portal/config ADD scripts/ /apollo-portal/scripts CMD ["/bin/bash","/apollo-portal/scripts/startup-kubernetes.sh"] EOF
docker build . -t harbor.zq.com/infra/apollo-portal:v1.5.1 docker push harbor.zq.com/infra/apollo-portal:v1.5.1
mkdir /data/k8s-yaml/apollo-portal cd /data/k8s-yaml/apollo-portal
cat >cm.yaml <<'EOF' apiVersion: v1 kind: ConfigMap metadata: name: apollo-portal-cm namespace: infra data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.zq.com:3306/ApolloPortalDB?characterEncoding=utf8 spring.datasource.username = apollo spring.datasource.password = 123456 app.properties: | appId=100003173 apollo-env.properties: | dev.meta=http://apollo-config.zq.com EOF
这里暂时只管理一个环境,等跑通了之后,再演示多环境问题
cat >dp.yaml <<'EOF' kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-portal namespace: infra labels: name: apollo-portal spec: replicas: 1 selector: matchLabels: name: apollo-portal template: metadata: labels: app: apollo-portal name: apollo-portal spec: volumes: - name: configmap-volume configMap: name: apollo-portal-cm containers: - name: apollo-portal image: harbor.zq.com/infra/apollo-portal:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-portal/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 EOF
cat >svc.yaml <<'EOF' kind: Service apiVersion: v1 metadata: name: apollo-portal namespace: infra spec: ports: - protocol: TCP port: 8080 targetPort: 8080 selector: app: apollo-portal EOF
cat >ingress.yaml <<'EOF' kind: Ingress apiVersion: extensions/v1beta1 metadata: name: apollo-portal namespace: infra spec: rules: - host: apollo-portal.zq.com http: paths: - path: / backend: serviceName: apollo-portal servicePort: 8080 EOF
kubectl create -f http://k8s-yaml.zq.com/apollo-portal/cm.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-portal/dp.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-portal/svc.yaml kubectl create -f http://k8s-yaml.zq.com/apollo-portal/ingress.yaml
因为前面已经一块儿添加了域名解析,所以portal建立好后不须要在添加域名解析,直接浏览器登陆验证
网页:apollo-portal.zq.com
默认用户名:apollo
默认密码:admin
登陆成功后,立马修改密码为apollo123
到此,apollo的三个组件都已经交付到k8s里了。
使用配置中心,须要开发对代码进行调整,将一些配置,经过变量的形式配置到apollo中,服务经过配置中心来获取具体的配置
项目属性:
AppId:dubbo-demo-service
应用名称:dubbo服务提供者
部门:研发部
为新项目添加配置以下:
key | value | 备注 |
---|---|---|
dubbo.registry | zookeeper://zk1.zq.com:2181 | 注册中心地址 |
dubbo.port | 20880 | dubbo服务监听端口 |
发布后效果图以下:
仍是使用以前的流水线,可是使用分支为apollo的代码进行打包,参数以下:
参数名 | 参数值 |
---|---|
app_name | dubbo-demo-service |
image_name | app/dubbo-demo-service |
git_repo | https://gitee.com/noah-luo/dubbo-demo-service.git |
git_ver | apollo |
add_tag | 200512_0746 |
mvn_dir | ./ |
target_dir | ./dubbo-server/target |
mvn_cmd | mvn clean package -Dmaven.test.skip=true |
base_image | base/jre8:8u112 |
maven | 3.6.1 |
修改dp.yaml资源配置清单
C_OPTS
,以便指定配置中心vim /data/k8s-yaml/dubbo-server/dp.yaml #----------原内容---------- spec: containers: - name: dubbo-demo-service image: harbor.zq.com/app/dubbo-demo-service:master_200509_0800 ports: - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-server.jar #----------新内容---------- spec: containers: - name: dubbo-demo-service image: harbor.zq.com/app/dubbo-demo-service:apollo_200512_0746 ports: - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-server.jar - name: C_OPTS value: -Denv=dev -Dapollo.meta=http://apollo-config.zq.com
应用资源配置清单:
kubectl apply -f http://k8s-yaml.zq.com/dubbo-server/dp.yaml
项目属性:
AppId:dubbo-demo-web
应用名称:dubbo服务消费者
部门:运维部
为新项目添加配置以下:
key | value | 备注 |
---|---|---|
dubbo.registry | zookeeper://zk1.zq.com:2181 | 注册中心地址 |
发布后效果图以下:
略
仍是使用以前的流水线,可是使用分支为apollo的代码进行打包,参数以下:
参数名 | 参数值 |
---|---|
app_name | dubbo-demo-consumer |
image_name | app/dubbo-demo-consumer |
git_repo | git@gitee.com:noah-luo/dubbo-demo-web.git |
git_ver | apollo |
add_tag | 200512_0801 |
mvn_dir | ./ |
target_dir | ./dubbo-client/target |
mvn_cmd | mvn clean package -Dmaven.test.skip=true |
base_image | base/jre8:8u112 |
maven | 3.6.1 |
构建完成后,修改资源配置清单并应用:
修改dp.yaml资源配置清单
C_OPTS
,以便指定配置中心vim /data/k8s-yaml/dubbo-consumer/dp.yaml #----------原内容---------- spec: containers: - name: dubbo-demo-consumer image: harbor.zq.com/app/dubbo-demo-consumer:master_200506_1430 ports: - containerPort: 8080 protocol: TCP - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-client.jar #----------新内容---------- spec: containers: - name: dubbo-demo-consumer image: harbor.zq.com/app/dubbo-demo-consumer:apollo_200512_0801 ports: - containerPort: 8080 protocol: TCP - containerPort: 20880 protocol: TCP env: - name: JAR_BALL value: dubbo-client.jar - name: C_OPTS value: -Denv=dev -Dapollo.meta=http://apollo-config.zq.com
应用资源配置清单:
kubectl apply -f http://k8s-yaml.zq.com/dubbo-consumer/dp.yaml
管理机上,修改dubbo-monitor的dp资源的使用的cm资源
set -i 's#dubbo-monitor-cm-pro#dubbo-monitor-cm#g' /data/k8s-yaml/dubbo-monitor/dp-cm.yaml
任意node节点应用资源
kubectl apply -f http://k8s-yaml.zq.com/dubbo-monitor/dp.yaml
登陆dubbo-monitor查看
访问http://dubbo-monitor.zq.com/
浏览器查看
访问http://dubbo-demo.zq.com/hello?name=lg
apollo中看实例列表