Kebernetes 学习总结(5) Ingress

时间 2019-11-06

标签 kebernetes 学习总结 ingress 繁體版

原文原文链接

虽然kubernetes集群内部署的pod、server都有本身的IP，可是却没法提供外网访问，之前能够经过监听NodePort的方式暴露服务，可是这种方式并不灵活，生产环境也不建议使用。Ingresss是kubernetes集群中的一个API资源对象，扮演边缘路由器(edge router)的角色，也能够理解为集群防火墙、集群网关，能够自定义路由规则来转发、管理、暴露服务(一组pod)，很是灵活，生产环境建议使用这种方式。另外LoadBlancer也能够暴露服务，不过这种方式须要向云平台申请负债均衡器；虽然目前不少云平台都支持，可是这种方式深度耦合了云平台。
Ingress相关组件：
1) Ingress controller
kubernetes中的controller有不少，好比CronJob、DeamonSet、Deployment、ReplicationSet、StatefulSet等，它们的做用就是监控集群的变化，使集群始终保持指望的最终状态(yml文件)。同理Ingress controller的做用就是实时感知Ingress路由规则集合的变化，再与Api Server交互，获取Service、Pod在集群中的IP等信息，而后发送给反向代理web服务器(ingress pods)，刷新其路由配置信息，这就是它的服务发现机制。
2) 反向代理web服务器(ingress pods) 负责转发外部请求到后端service，好比Nginx、Apache、traefik等等
3) Ingress 定义路由规则集合
4) NodePort类型的service 负责将外部流量引入kubernetes cluster，而后由反向代理web服务器处理。node

本篇总结中的示例大体以下：首先部署ingress controller及 ingress pods；而后利用NodePort类型的service将外部流最引入kubernetes cluster中；而后再定义后端deployment及后端service；而后定义ingress 转发规则集合；最后访问测试。
1) 部署ingress podsnginx

[root@docker79 ingress]# mkdir example
[root@docker79 ingress]# cd example/
[root@docker79 example]#  for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml default-backend.yaml udp-services-configmap.yaml ; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file ; done
[root@docker79 example]# kubectl apply -f .
configmap/nginx-configuration created
deployment.extensions/default-http-backend created
service/default-http-backend created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
namespace/ingress-nginx configured
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
configmap/tcp-services created
configmap/udp-services created
deployment.extensions/nginx-ingress-controller created
[root@docker79 example]#
[root@docker79 example]# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-6586bc58b6-8rxtp       1/1       Running   0          59m
nginx-ingress-controller-6bd7c597cb-xdrcm   1/1       Running   0          59m
[root@docker79 example]#

注：较新版本已经再也不支持上述文件，新版本(2019/07)已经把众多文件合并为一个mandatory.yaml文件，操做以下所示：git

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
kubectl apply -f mandatory.yaml

2) 在ingress-nginx 名称空间定义nodePort类型的service 引入外部流量，过程以下所示：github

[root@docker79 ingress]# cat nginx-frontend-ingress-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-frontend-ingress-svc
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30080
  - name: https
    port: 443
    targetPort: 443
    protocol: TCP
    nodePort: 30443
  selector:
    app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f nginx-frontend-ingress-svc.yaml
service/nginx-frontend-ingress-svc created
[root@docker79 ingress]# kubectl get svc -n ingress-nginx
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
default-http-backend   ClusterIP   10.102.71.36   <none>        80/TCP                       1h
nginx-frontend-ingress-svc     NodePort    10.103.52.44   <none>        80:30080/TCP,443:30443/TCP   11s
[root@docker79 ingress]#

3) 下例中部署两个后端nginx-backend-deploy、tomcat-backend-deploy。
首先部署nginx-backend-deploy及nginx-backend-svc，过程以下：web

[root@docker79 ingress]# vim backend-nginx.yaml
[root@docker79 ingress]# cat backend-nginx.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-backend-svc
  namespace: default
spec:
  selector:
    app: nginx-backend
    release: canary
  ports:
  - name: http
    targetPort: 80
    port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-backend-deploy
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-backend
      release: canary
  template:
    metadata:
      labels:
        app: nginx-backend
        release: canary
    spec:
      containers:
      - name: nginx-container
        image: nginx:1.15-alpine
        ports:
        - name: http
          containerPort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f backend-nginx.yaml
service/nginx-backend-svc created
deployment.apps/nginx-backend-deploy created
[root@docker79 ingress]# kubectl get pods
NAME                                   READY     STATUS    RESTARTS   AGE
nginx-backend-deploy-9c9b85bf4-6587j   1/1       Running   0          8s
nginx-backend-deploy-9c9b85bf4-k64tv   1/1       Running   0          8s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl get svc
NAME                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes          ClusterIP   10.96.0.1        <none>        443/TCP   3d
nginx-backend-svc   ClusterIP   10.110.150.187   <none>        80/TCP    1m
[root@docker79 ingress]#

而后部署tomcat-backend-deploy及tomcat-backend-svc，过程以下：docker

[root@docker79 ingress]# vim backend-tomcat.yaml
[root@docker79 ingress]# cat backend-tomcat.yaml
apiVersion: v1
kind: Service
metadata:
  name: tomcat-backend-svc
  namespace: default
spec:
  selector:
    app: tomcat-backend
    release: canary
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: ajp
    port: 8009
    targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat-backend
      release: canary
  template:
    metadata:
      labels:
        app: tomcat-backend
        release: canary
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5.32-jre8-alpine
        ports:
        - name: http
          containerPort: 8080
        - name: ajp
          containerPort: 8009
[root@docker79 ingress]# kubectl apply -f backend-tomcat.yaml
service/tomcat-backend-svc created
deployment.apps/tomcat-deploy created
[root@docker79 ingress]# kubectl get pods
NAME                                   READY     STATUS    RESTARTS   AGE
nginx-backend-deploy-9c9b85bf4-6587j   1/1       Running   0          24m
nginx-backend-deploy-9c9b85bf4-k64tv   1/1       Running   0          24m
tomcat-deploy-7b4c97ff9f-fv8rx         1/1       Running   0          2m
tomcat-deploy-7b4c97ff9f-rj94r         1/1       Running   0          2m
[root@docker79 ingress]# kubectl get svc
NAME                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
kubernetes           ClusterIP   10.96.0.1        <none>        443/TCP             3d
nginx-backend-svc    ClusterIP   10.110.150.187   <none>        80/TCP              24m
tomcat-backend-svc   ClusterIP   10.105.189.191   <none>        8080/TCP,8009/TCP   2m
[root@docker79 ingress]#

4) 部署 ingress 转发规则集合
首先部署nginx-backend-svc的ingress 转发规则集合，过程以下所示:shell

[root@docker79 ingress]# vim ingress-rules-ngx.yaml
[root@docker79 ingress]# cat ingress-rules-ngx.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-ngx
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: nginx.inspiry.com
    http:
      paths:
      - path:
        backend:
          serviceName: nginx-backend-svc
          servicePort: 80
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl apply -f ingress-rules.yaml
ingress.extensions/ingress-rule-ngx created
[root@docker79 ingress]# kubectl get ingress
NAME               HOSTS               ADDRESS   PORTS     AGE
ingress-rule-ngx   nginx.inspiry.com             80        9s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress  ingress-rule-ngx
Name:             ingress-rule-ngx
Namespace:        default
Address:
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  nginx.inspiry.com
                        nginx-backend-svc:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-ngx","namespace":"default"},"spec":{"rules":[{"host":"nginx.inspiry.com","http":{"paths":[{"backend":{"serviceName":"nginx-backend-svc","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
[root@docker79 ingress]#

而后再部署tomcat-backend-svc的ingress 转发规则集合，过程以下所示：json

[root@docker79 ingress]# vim ingress-rules-tomcat.yaml
[root@docker79 ingress]# cat ingress-rules-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-tomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: tomcat.inspiry.cn
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat-backend-svc
          servicePort: 8080
[root@docker79 ingress]# kubectl apply -f ingress-rules-tomcat.yaml
ingress.extensions/ingress-rule-tomcat created
[root@docker79 ingress]# kubectl get ingress
NAME                  HOSTS               ADDRESS   PORTS     AGE
ingress-rule-ngx      nginx.inspiry.com             80        11m
ingress-rule-tomcat   tomcat.inspiry.cn             80        6s
[root@docker79 ingress]#
[root@docker79 ingress]# kubectl describe ingress ingress-rule-tomcat
Name:             ingress-rule-tomcat
Namespace:        default
Address:
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  tomcat.inspiry.cn
                        tomcat-backend-svc:8080 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-rule-tomcat","namespace":"default"},"spec":{"rules":[{"host":"tomcat.inspiry.cn","http":{"paths":[{"backend":{"serviceName":"tomcat-backend-svc","servicePort":8080},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>
[root@docker79 ingress]#

5) 测试
在kubernetes cluster 外的任何一台主机上使用 curl http://nginx.inspiry.com:30080/ 或 curl http://tomcat.inspiry.cn:30080/ 便可访问相应nginx-backend 、tomcat-backend 的pods。
生产环境中还能够在k8s cluster外再建立一层 LB，将接收到的 nginx.inspiry.com、tomcat.inspiry.cn 域名分别转发到192.168.20.7九、192.168.20.7八、192.168.20.77为后端的upstream上。vim

观察ingress controller的内容以下：后端

[root@docker79 ~]# kubectl get pods -n ingress-nginx
NAME                                        READY     STATUS    RESTARTS   AGE
default-http-backend-6586bc58b6-8rxtp       1/1       Running   0          1h
nginx-ingress-controller-6bd7c597cb-xdrcm   1/1       Running   0          1h
[root@docker79 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-6bd7c597cb-xdrcm -- /bin/sh
$ ls
fastcgi.conf        geoip    mime.types      nginx.conf         scgi_params      uwsgi_params.default
fastcgi.conf.default    koi-utf  mime.types.default  nginx.conf.default     scgi_params.default  win-utf
fastcgi_params      koi-win  modsecurity         opentracing.json       template
fastcgi_params.default  lua  modules         owasp-modsecurity-crs  uwsgi_params
$ cat nginx.conf
...... (可仔细观察该文件的内容，发现相应的转发规则是随着 ingress rules的变化而变化的)

补充: ingress 的https流量

# kubectl create secret tls domain-name-crt --cert=server.pem --key=server.key
# cat ingress-rule-appapi-svc.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-rule-appapi
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - appapi-devel.domain.cn
    secretName: domain-name-crt
  rules:
  - host: appapi-devel.domain.cn
    http:
      paths:
      - path:
        backend:
          serviceName: appapi-svc
          servicePort: 21900
# kubectl apply  -f   ingress-rule-appapi-svc.yaml

<完>