在Kuberbetes中除了使用NodePort,LoadBalancer, Port Proxy(hostPort)等实现外部访问入口以外,还可使用Kubernetes 自带的Ingress来实现服务的负载均衡和策略路由的工做,其本质至关于一个Nginx代理服务器,能够对请求(http)实现精细的控制。php
Ingress 是能够利用 Nginx、Haproxy 等负载均衡器暴露集群内服务的工具,html
使用Ingress能够配置一个用于外部访问的URL地址,同时实现流量负载,SSL,基于名称的虚拟主机(例如Nginx的upstream)等。用户经过发送POST请求到API server,来申请Ingress资源, Ingress控制器负责完成Ingress转发工做,一般会使用负载均衡,也能够配置边缘路由器或其余前端以帮助以HA方式处理流量。前端
在当前版本中,Ingress处于测试版本,在使用Ingress时,须要先建立Ingress Contronller.node
参考连接nginx
在Kubernetes中,Ingress Controller将以Pod的形式运行,监控apiserver的 ingress接口后端的backend services,若是service发生变化,则Ingress Controller 应自动更新其转发规则。
Ingress Controller 须要实现基于不一样HTTP URL向后转发的负载均衡分发规则,并可灵活设置7层的负载分发策略。若是公有云服务商能提供该类型的HTTP路由LoadBlanacer,也能够设置其为Ingress Controller。git
在新的版本中Ingress Controller和具备负载均衡的软件已经捆绑到一块儿,目前支持多种类型, 如nginx ,haproxy 等。 github
使用以下的定义的文件,能够直接从官方下载,须要修改谷歌的镜像地址(https://kubernetes.github.io/ingress-nginx/deploy/):web
--- apiVersion: v1 kind: Namespace metadata: name: ingress-nginx --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: default-http-backend labels: app: default-http-backend namespace: ingress-nginx spec: replicas: 1 selector: matchLabels: app: default-http-backend template: metadata: labels: app: default-http-backend spec: terminationGracePeriodSeconds: 60 containers: - name: default-http-backend # Any image is permissible as long as: # 1. It serves a 404 page at / # 2. It serves 200 on a /healthz endpoint image: mirrorgooglecontainers/defaultbackend:1.4 livenessProbe: httpGet: path: /healthz port: 8080 scheme: HTTP initialDelaySeconds: 30 timeoutSeconds: 5 ports: - containerPort: 8080 resources: limits: cpu: 10m memory: 20Mi requests: cpu: 10m memory: 20Mi --- apiVersion: v1 kind: Service metadata: name: default-http-backend namespace: ingress-nginx labels: app: default-http-backend spec: ports: - port: 80 targetPort: 8080 selector: app: default-http-backend --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "extensions" resources: - ingresses verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx spec: replicas: 1 selector: matchLabels: app: ingress-nginx template: metadata: labels: app: ingress-nginx annotations: prometheus.io/port: '10254' prometheus.io/scrape: 'true' spec: serviceAccountName: nginx-ingress-serviceaccount containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.16.2 args: - /nginx-ingress-controller - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io # - --report-node-internal-ip-address=true securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 33 runAsUser: 33 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 # hostPort: 80 若是不使用NodePort,可使用hostPort的方式直接映射到宿主端口 - name: https containerPort: 443 # hostPort: 443 livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1
使用以下文件建立 Ingress的services:apache
apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx spec: type: NodePort ports: - name: http port: 80 targetPort: 80 protocol: TCP - name: https port: 443 targetPort: 443 protocol: TCP selector: app: ingress-nginx
这里使用的nodeport的方式,对于其余云平台可使用官方提供的对应yaml文件建立。后端
查看配置是否成功:
# kubectl get pods --all-namespaces -l app=ingress-nginx --watch NAMESPACE NAME READY STATUS RESTARTS AGE ingress-nginx nginx-ingress-controller-6c9fcdf8d9-fvn57 1/1 Running 0 38m # kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE default-http-backend-7fb45cbc-kv24l 1/1 Running 0 39m nginx-ingress-controller-6c9fcdf8d9-fvn57 1/1 Running 0 39m # kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default-http-backend ClusterIP 10.1.74.56 <none> 80/TCP 40m ingress-nginx NodePort 10.1.28.135 <none> 80:34596/TCP,443:28411/TCP 24m
在安装有kube-proxy的节点上会显示Nodeport映射的端口:
[root@node-2 ~]# netstat -lntp|egrep "28411|34596" tcp6 0 0 :::28411 :::* LISTEN 1164/kube-proxy tcp6 0 0 :::34596 :::* LISTEN 1164/kube-proxy [root@node-3 ~]# netstat -lntp|egrep "28411|34596" tcp6 0 0 :::28411 :::* LISTEN 1164/kube-proxy tcp6 0 0 :::34596 :::* LISTEN 1164/kube-proxy
访问任意kube-proxy节点的端口,会返回404(还未配置Ingress)说明Ingress安装完成:
# curl 10.0.0.3:34596 default backend - 404
固然,在指定NodePort时,也能够指定端口:
apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx spec: type: NodePort ports: - name: http port: 80 targetPort: 80 nodePort: 20080 protocol: TCP - name: https port: 443 targetPort: 443 nodePort: 20443 protocol: TCP selector: app: ingress-nginx
使用Ingress能够实现多种访问策略:
定义以下Ingress配置文件:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test1 spec: rules: - host: test.com http: paths: - path: /demo backend: serviceName: php-apache servicePort: 80 # curl test.com:20080/demo/index.html demo page!
定义以下文件:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test2 spec: rules: - host: test.com http: paths: - path: /tomcat backend: serviceName: tomcat-service servicePort: 8080 - path: /apache backend: serviceName: php-apache servicePort: 80
这里会将路径带入到后端服务中,若是后端的服务没有此路径,将会找不到服务,从而返回404。
这里使用示例定义一个Ingress 用于访问集群中的 tomcat和apache,当前的服务以下:
# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE php-apache ClusterIP 10.1.175.166 <none> 80/TCP 4d tomcat-service ClusterIP 10.1.228.9 <none> 8080/TCP 1h # kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE php-apache-56b5765b95-rhvg2 1/1 Running 2 5d 10.2.15.6 10.0.0.2 tomcat-deployment-65799d5fc4-5dx4h 1/1 Running 0 3h 10.2.22.7 10.0.0.3
建立一个Ingress:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test spec: rules: - host: tomcat.com http: paths: - backend: serviceName: tomcat-service servicePort: 8080 - host: apache.com http: paths: - path: /demo/index.html backend: serviceName: php-apache servicePort: 80
建立成功后,会显示以下信息:
# kubectl get ing -o wide NAME HOSTS ADDRESS PORTS AGE test tomcat.com,apache.com 80 2h
在不使用kubernetes支持的公有云平台下,默认的 ADDRESS地址会显示为空,若是须要显示节点的IP,能够添加
--report-node-internal-ip-address=true
的参数,或者修改flags.go z中的源码,对应的useNodeInternalIP = flags.Bool("report-node-internal-ip-address", false,
中 false 修改成 true。
在本主机host 文件中配置上域名和映射20080端口的主机节点,经过不一样的域名访问此节点:
[root@node-1 ingress]# kubectl get nodes NAME STATUS ROLES AGE VERSION 10.0.0.2 Ready <none> 25d v1.10.4 10.0.0.3 Ready <none> 24d v1.10.4 [root@node-1 ingress]# echo "10.0.0.2 tomcat.com apache.com" >> /etc/hosts [root@node-1 ~]# curl apache.com:20080 OK! [root@node-1 ~]# curl tomcat.com:20080 tomcat demo! [root@node-1 ~]# curl 10.0.0.2:20080 default backend - 404 # 当找不到对应的后端Service时,会去访问指定的默认backend。
查看代理信息:
[root@node-1 ~]# curl --head tomcat.com:20080 HTTP/1.1 200 Server: nginx/1.13.12 Date: Tue, 03 Jul 2018 11:47:31 GMT Content-Type: text/html;charset=UTF-8 Connection: keep-alive Vary: Accept-Encoding [root@node-1 ~]# curl --head apache.com:20080 HTTP/1.1 200 OK Server: nginx/1.13.12 Date: Tue, 03 Jul 2018 11:47:41 GMT Content-Type: text/html; charset=UTF-8 Connection: keep-alive Vary: Accept-Encoding X-Powered-By: PHP/5.6.14
登陆Ingress的Pod,咱们能够看到已经添加了相应的转发规则:
# kubectl exec -it nginx-ingress-controller-6c9fcdf8d9-fvn57 -n ingress-nginx -- sh $ cat nginx.conf ... upstream default-php-apache-80 { least_conn; keepalive 32; server 10.2.15.6:80 max_fails=0 fail_timeout=0; } upstream default-tomcat-service-8080 { least_conn; keepalive 32; server 10.2.22.7:8080 max_fails=0 fail_timeout=0; } ... server { server_name apache.com ; listen 80; listen [::]:80; set $proxy_upstream_name "-"; ... ## start server tomcat.com server { server_name tomcat.com ; listen 80; listen [::]:80; set $proxy_upstream_name "-";
对于使用https设置TLS的安全证书方面,Ingress 也能够支持。
经过如下步骤进行设置:
目前,Ingress仅支持单个TLS端口443,并假定TLS termination。 若是Ingress中的TLS配置部分指定了不一样的主机,则它们将根据经过SNI TLS扩展指定的主机名(假如Ingress controller支持SNI)在多个相同端口上进行复用。TLS secret中必须包含名为tls.crt和tls.key的密钥,这里面包含了用于TLS的证书和私钥,其格式以下:
apiVersion: v1 data: tls.crt: base64 encoded cert # cert 文件内容 tls.key: base64 encoded key # key 文件内容 kind: Secret metadata: name: testsecret namespace: default type: Opaque
在Ingress中引用 Secret:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: no-rules-map spec: tls: - secretName: testsecret backend: serviceName: s1 servicePort: 80
首先生成证书:
# openssl req -x509 -nodes -days 5000 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=myweb.com"
利用生成的key 和 crt文件建立Secret:
# kubectl create secret tls myweb-ingress-secret --key tls.key --cert tls.crt secret "myweb-ingress-secret" created # kubectl get secret NAME TYPE DATA AGE default-token-hmvnc kubernetes.io/service-account-token 3 26d myweb-ingress-secret kubernetes.io/tls 2 13s
咱们也可使用YAML文件的方式建立,其格式和下面的相似:
# kubectl get secret myweb-ingress-secret -o yaml apiVersion: v1 data: tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMr ... Gh0UlJJUkV5bUJpZjZNdmRlOERmUVRiT0x5OUF5Y0xVb2gyL2RlRnhOST0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo= tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2d0lCQURBTkJna3Foa2lHOXcwQ ... MzekE9PQotLS0tLUVORCBQUklWQVRFIEtFWS0tLS0tCg== kind: Secret metadata: name: myweb-ingress-secret namespace: default type: kubernetes.io/tls
TLS的证书建立到Secret以后,就能够利用他来建立Ingress对象了,使用以下文件:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myweb-ingress spec: tls: - secretName: myweb-ingress-secret rules: - host: myweb.com http: paths: - path: /demo backend: serviceName: php-apache servicePort: 80
使用默认的443端口访问(本地使用的NodePort,映射到Ingress的443端口):
# curl -k https://myweb.com:20443/demo/index.html demo test https page!
使用网页能够访问网页内容: https://myweb.com:20443/demo