kubernetes ingress(三)： traefik: 多域名及证书配置

时间 2019-12-10

标签 kubernetes ingress traefik 域名证书配置繁體版

原文原文链接

目标：

部署三个服务traefik-ui，grafana，prometheus，并经过traefik 反向代理。node

service	namespaces	domain name	https
traefik-ui	traefik	traefik.qyd.com	Y
grafana	kube-system	grafana.dfb.com	N
prometheus	kube-system	prometheus.qyd.com	Y

步骤：

一、部署traefik

相关资源ymlgit

建立traefik 这个命名空间，使用configmap 挂载配置。github

kubectl create cm -n traefik  traefik-config --from-file=traefik.toml

apiVersion: v1
items:
- apiVersion: v1
  data:
    traefik.toml: |
      graceTimeOut = 10
      traefikLogsFile = "/log/traefik.log"
      accessLogsFile = "/log/access.log"
      logLevel = "INFO"
      MaxIdleConnsPerHost = 60
      InsecureSkipVerify = true
      defaultEntryPoints = ["https","http"]
      [entryPoints]
        [entryPoints.http]
        address = ":80"
              [entryPoints.http.redirect]
              regex = "^http://(.*).qyd.com/(.*)"
              replacement = "https://$1.qyd.com/$2"

      [entryPoints.https]
      address = ":443"
        [entryPoints.https.tls]
          [[entryPoints.https.tls.certificates]]
          certFile = "/ssl/qyd/tls.crt"
          keyFile = "/ssl/qyd/tls.key"
          [[entryPoints.https.tls.certificates]]
          certFile = "/ssl/dfb/tls.crt"
          keyFile = "/ssl/dfb/tls.key"
      [metrics]
        [metrics.prometheus]
          entryPoint = "traefik"


  kind: ConfigMap
  metadata:
    name: traefik-config
    namespace: traefik
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

获取 qyd.com 和dfb.com 两个域名的证书，并建立secret。web

kubectl create secret generic dfb-tls-cert --from-file=dfb/tls.crt --from-file=dfb/tls.key -n traefik
kubectl create secret generic qyd-tls-cert --from-file=qyd/tls.crt --from-file=qyd/tls.key -n traefik

部署traefik-ingreess-controllerdocker

kubectl app -f rbac.yml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - extensions
    resources:
    - ingresses/status
    verbs:
    - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: traefik
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: traefik

kubectl apply -f deployment.yml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    k8s-app: traefik-ingress-lb
  name: traefik-ingress-controller
  namespace: traefik
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      containers:
      - args:
        - --configFile=/etc/traefik/traefik.yml
        - --api
        - --kubernetes
        image: itanony.com/repository/docker-hosted/test/treafik:v1.7.10
        imagePullPolicy: IfNotPresent
        name: traefik-ingress-lb
        ports:
        - containerPort: 80
          hostPort: 80
          name: http
          protocol: TCP
        - containerPort: 8080
          hostPort: 8080
          name: admin
          protocol: TCP
        - containerPort: 443
          hostPort: 443
          name: https
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/traefik/
          name: config
        - mountPath: /ssl/qyd/
          name: qyd-cert
        - mountPath: /ssl/dfb/
          name: dfb-cert
        - mountPath: /log/
          name: logs
      dnsPolicy: ClusterFirst
      hostNetwork: true
      nodeSelector:
        cpu: high
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: traefik-ingress-controller
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      volumes:
      - name: qyd-cert
        secret:
          defaultMode: 420
          secretName: qyd-tls-cert
      - name: dfb-cert
        secret:
          defaultMode: 420
          secretName: dfb-tls-cert
      - configMap:
          defaultMode: 420
          name: traefik-config
        name: config
      - hostPath:
          path: /var/log/traefik
          type: ""
        name: logs

注意deployment.yml 中修改images地址。另外由于是测试，故采用nodeselector 只部署到一台固定的node节点，采用宿主机网络模式。ingress controller 的高可用留在之后研究。
查看pod 状态api

kubectl get pods -n traefik

traefik 启动后会监控一个8080 的端口提供一个管理的web-ui，能够查看frontend 和backend 的对应关系，及一些基本的监控数据
咱们建立一个ClusterIP 的service，并建立ingress，经过traefik 使用traefik.qyd.com 域名来反向代理浏览器

kubectl apply -f traefik-web-ui.yml

apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: traefik
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: traefik
spec:
  rules:
  - host: traefik.qyd.com
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

在本机hosts中添加 traefik.qyd.com 的hosts 记录解析到traefik 部署的node节点。
经过浏览器访问。页面正常显示，而且使用http 访问时会自动跳转到https。网络

部署prometheus 和grafana 代理

这里只讨论经过traefik-ingres 代理prometheus 和grafan。部署过程请Google。app

建立prometheus 和 grafana 的ingress 。经过traefik 分别使用 prometheus.yd.com 和grafana.dfb.com 反向代理。frontend

注意yml 中namespace，serviceName，servicePort 与本身集群中服务的名称一致。

kubectl apply -f grafana-ingress.yml
kubectl apply -f prometheus-ingress.yml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: grafana
  namespace: kube-system
spec:
  rules:
  - host: grafana.dfb.com
    http:
      paths:
      - backend:
          serviceName: monitoring-grafana
          servicePort: 80
        path: /

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: prometheus
  namespace: kube-system
spec:
  rules:
  - host: prometheus.qyd.com
    http:
      paths:
      - backend:
          serviceName: prometheus
          servicePort: prometheus
        path: /

一样在本机hosts 中添加两个域名的解析记录。经过浏览器访问正常，prometheus.qyd.com访问http 会rewrite到https，grafana.dfb.com不会作rewrite。至此部署部分结束

配置解析

多域名配置https，咱们不须要对每个域名指定证书，只须要在entrypoints 中指定证书路径。traefik 会自动根据请求中的主机头和证书中的CN进行匹配。
生产中可能遇到同一个反向代理下。有的域名须要启用https 的强制rewrite。有些则不能作强制rewrite。traefik 提供entryPoints.http.redirect 经过正则来对须要rewrite 的域名进行正则匹配。这里感受有点不灵活。也可能还有更好的方式。