kubernetes之配置Metrics Server

Kubernetes 1.8 关于资源使用状况的 metrics，能够经过 Metrics API 获取到， Kubernetes 1.11 已经废弃 heapster。这里咱们基于 Kubernetes 1.14.1 版本安装 Metrics Server。

首先，先说明下集群环境：

[root@node-01]# kubectl get nodes NAME STATUS ROLES AGE VERSION node-01   Ready    master   2d1h   v1.14.1 node-02   Ready    master   2d1h   v1.14.1 node-03   Ready    master   2d1h   v1.14.1 node-04   Ready    <none>   2d1h   v1.14.1 node-05   Ready    <none>   2d1h   v1.14.1 node-06   Ready    <none>   2d1h   v1.14.1

当整个集群部署完成后，kubectl top 命令不会返回任何内容，由于 Heapster 和 metrics-server 都没有安装，可是自 Kubernetes 1.11版本后 heapster已经被废弃了，取而代之的是更丰富的 metrics-server。

配置 /etc/kubernetes/manifests/kube-controller-manager.yaml

--horizontal-pod-autoscaler-use-rest-clients=true

kubedam 建立的集群，修改配置文件后会自动加载。若是手动建立的集群，须要重启kube-controller-manager服务。

准备部署 Metrics Server 的 yaml文件

[root@node-01]# git clone https://github.com/kubernetes-incubator/metrics-server

下载完成后还须要对 metrics-server/deploy/1.8+/resource-reader.yaml文件进行修改，须要修改的内容以下：

[root@node-01 1.8+]# cat resource-reader.yaml --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-server rules: - apiGroups: - "" resources: - pods - nodes - namespaces # 增长此行 - nodes/stats verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system

修改 metrics-server/deploy/1.8+/metrics-server-deployment.yaml文件：

[root@node-01 1.8+]# cat metrics-server-deployment.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: metrics-server namespace: kube-system --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers - name: tmp-dir emptyDir: {} containers: - name: metrics-server image: k8s.gcr.io/metrics-server-amd64:v0.3.2 command: - /metrics-server - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP # 若是不配置此项，会报错找不到node imagePullPolicy: Always volumeMounts: - name: tmp-dir mountPath: /tmp

上面若是报错是由于 node-01 和 node-02 是一个独立的 Kubernetes 演示环境，只是修改了这两个节点系统的 /etc/hosts文件，而并无内网的 DNS 服务器，因此 metrics-server 中不认识 node-01 和 node-02 的名字。

修改完成就能够正式部署了：

[root@node-01 1.8+]# kubectl apply -f . clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created serviceaccount/metrics-server created deployment.extensions/metrics-server created service/metrics-server created clusterrole.rbac.authorization.k8s.io/system:metrics-server created clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created

Metrics Server 相关 pod 、service 默认部署在 kube-system的 NAMESPACE 下：

[root@node-01 1.8+]# kubectl get pods -n kube-system | grep metrics metrics-server-5845cc8fd4-kkq6b         1/1     Running   0 18m [root@node-01 1.8+]# kubectl get svc -n kube-system | grep metrics metrics-server            ClusterIP   10.245.141.103   <none>        443/TCP                   20m

部署完成后使用以下命令查看node相关指标,须要等30s左右的时间：

[root@node-01 1.8+]# kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes" {"kind":"NodeMetricsList","apiVersion":"metrics.k8s.io/v1beta1","metadata":{"selfLink":"/apis/metrics.k8s.io/v1beta1/nodes"},"items":[
{"metadata":{"name":"node-02","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/node-02","creationTimestamp":"2019-05-08T08:17:11Z"},"timestamp":"2019-05-08T08:17:01Z","window":"30s","usage":{"cpu":"221367011n","memory":"1914616Ki"}},
{"metadata":{"name":"node-03","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/node-03","creationTimestamp":"2019-05-08T08:17:11Z"},"timestamp":"2019-05-08T08:17:08Z","window":"30s","usage":{"cpu":"198021879n","memory":"1809160Ki"}},
{"metadata":{"name":"node-04","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/node-04","creationTimestamp":"2019-05-08T08:17:11Z"},"timestamp":"2019-05-08T08:17:03Z","window":"30s","usage":{"cpu":"55570780n","memory":"719012Ki"}},
{"metadata":{"name":"node-05","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/node-05","creationTimestamp":"2019-05-08T08:17:11Z"},"timestamp":"2019-05-08T08:17:01Z","window":"30s","usage":{"cpu":"60116633n","memory":"851180Ki"}},
{"metadata":{"name":"node-06","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/node-06","creationTimestamp":"2019-05-08T08:17:11Z"},"timestamp":"2019-05-08T08:16:59Z","window":"30s","usage":{"cpu":"51157291n","memory":"677532Ki"}},
{"metadata":{"name":"node-01","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/node-01","creationTimestamp":"2019-05-08T08:17:11Z"},"timestamp":"2019-05-08T08:17:02Z","window":"30s","usage":{"cpu":"263183209n","memory":"2460972Ki"}}]}

Metrics API

Metrics Server 从 Kubernetes 集群中每一个 Node 上 kubelet 的 API 收集 metrics 数据。经过 Metrics API 能够获取Kubernetes 资源的 Metrics 指标，Metrics API 挂载/apis/metrics.k8s.io/下。 能够使用kubectl top命令访问 Metrics API，例如:

[root@node-01 ~]# kubectl top pods NAME CPU(cores) MEMORY(bytes) my-nginx-6785b88976-7rrll 0m 1Mi nginx-deployment-6d6fdc59f7-pfcfj 1m 1Mi nginx-deployment-6d6fdc59f7-vcclz 1m 1Mi [root@node-01 ~]# kubectl top nodes NAME CPU(cores) CPU%   MEMORY(bytes)   MEMORY% node-01   276m         6%     2403Mi          31% node-02   245m         6%     1868Mi          24% node-03   206m         5%     1766Mi          22% node-04   74m          1%     703Mi           9% node-05   77m          1%     832Mi           10% node-06   56m          1%     661Mi           8%

至此，Kubernetes 集群中的 Metrics Server 就配置完成了。可是在dashboard中看不到内存和CPU信息，而若是使用heapster则能看到。

全部yaml文件以下

# cat aggregated-metrics-reader.yaml kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: system:aggregated-metrics-reader labels: rbac.authorization.k8s.io/aggregate-to-view: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-admin: "true" rules: - apiGroups: ["metrics.k8s.io"] resources: ["pods"] verbs: ["get", "list", "watch”]
 # cat auth-delegator.yaml --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: metrics-server:system:auth-delegator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegator subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system # cat auth-reader.yaml --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: metrics-server-auth-reader namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-reader subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system ————— # cat metrics-apiservice.yaml --- apiVersion: apiregistration.k8s.io/v1beta1 kind: APIService metadata: name: v1beta1.metrics.k8s.io spec: service: name: metrics-server namespace: kube-system group: metrics.k8s.io version: v1beta1 insecureSkipTLSVerify: true groupPriorityMinimum: 100 versionPriority: 100 # cat metrics-server-deployment.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: metrics-server namespace: kube-system --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers - name: tmp-dir emptyDir: {} containers: - name: metrics-server image: k8s.gcr.io/metrics-server-amd64:v0.3.2 command: - /metrics-server - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP imagePullPolicy: Always volumeMounts: - name: tmp-dir mountPath: /tmp # cat metrics-server-service.yaml --- apiVersion: v1 kind: Service metadata: name: metrics-server namespace: kube-system labels: kubernetes.io/name: "Metrics-server" kubernetes.io/cluster-service: "true" spec: selector: k8s-app: metrics-server ports: - port: 443 protocol: TCP targetPort: 443 # cat resource-reader.yaml --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-server rules: - apiGroups: - "" resources: - pods - nodes - namespaces # 增长此行 - nodes/stats verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system