Kubernetes监控——Heapster+InfluxDB+Grafana

    Kubernetes & Docker监控之最后一章，前三篇以下：
        一、kubernetes+docker监控之简介

        二、 kubernetes+docker监控之Docker监控——cadvisor

        三、 Docker监控——Cadvisor+InfluxDB+Grafana搭建过程

1）、heapster下载和启动：

（1）、heapster镜像下载：

docker pull index.tenxcloud.com/google_containers/heapster:v1.1.0

（2）、heapster单独做为容器启动：

docker run -d index.tenxcloud.com/google_containers/heapster:v1.1.0 "--source=kubernetes:http://192.168.16.100:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth=" "--sink=influxdb:http://192.168.16.234:8086"

 

参数说明：

--source ：指定kube-apiserver，这里使用https，须要注意下 --sink ：指定influxdb，heapster自动建立数据库【k8s】 inClusterConfig ：Use kube config in service accounts associated with heapster's                       namesapce. (default: true) kubeletPort ：kubelet port to use (default: 10255) kubeletHttps  ： whether to use https to connect to kubelets (default: false) apiVersion ： API version to use to talk to Kubernetes. Defaults to the version                        in kubeConfig. insecure     ：whether to trust kubernetes certificates (default: false) auth     ：client auth file to use. Set auth if the service accounts are not usable. useServiceAccount     ：whether to use the service account token if one is mounted                       at /var/run/secrets/kubernetes.io/serviceaccount/token (default: false)

        heapster容器，启动过程：

[root@localhost kube-1.2]# docker run -d index.tenxcloud.com/google_containers/heapster:v1.1.0 "--source=kubernetes:http://192.168.16.100:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth=" "--sink=influxdb:http://192.168.16.234:8086" b86aaaaa88ccf648120611254bfeb0078a8b9dd853d4b5a2d6a0abb97743bf1d   [root@localhost kube-1.2]# docker logs b86aaaaa8    I0714 07:05:18.116787       1 heapster.go:65] /heapster --source=kubernetes:http://192.168.16.100:8080?inClusterConfig=false&kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&auth= --sink=influxdb:http://192.168.16.234:8086 I0714 07:05:18.117158       1 heapster.go:66] Heapster version 1.1.0 I0714 07:05:18.239220       1 configs.go:60] Using Kubernetes client with master "http://192.168.16.100:8080" and version "v1" I0714 07:05:18.239270       1 configs.go:61] Using kubelet port 10250 I0714 07:05:31.255132       1 influxdb.go:223] created influxdb sink with options: host:192.168.16.234:8086 user:root db:k8s I0714 07:05:31.255170       1 heapster.go:92] Starting with InfluxDB Sink I0714 07:05:31.255177       1 heapster.go:92] Starting with Metric Sink I0714 07:05:31.271159       1 heapster.go:171] Starting heapster on port 8082 I0714 07:06:05.000403       1 manager.go:79] Scraping metrics start: 2016-07-14 07:05:00 +0000 UTC, end: 2016-07-14 07:06:00 +0000 UTC E0714 07:06:05.000504       1 kubelet.go:270] No nodes received from APIserver. I0714 07:06:05.000520       1 manager.go:152] ScrapeMetrics: time: 5.756µs size: 0 I0714 07:06:05.365486       1 influxdb.go:201] Created database "k8s" on influxDB server at "192.168.16.234:8086" [root@localhost kube-1.2]# [root@localhost kube-1.2]# docker ps |grep heapster b86aaaaa88cc        index.tenxcloud.com/google_containers/heapster:v1.1.0   "/heapster --source=k"   2 minutes ago     Up 2 minutes                            furious_mahavira [root@localhost kube-1.2]#

        看到influxdb已经有k8s的数据库和表了，剩下的就是grafana搜索和显示了

[root@localhost kube-1.2]# influx Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring. Connected to http://localhost:8086 version 0.9.6 InfluxDB shell 0.9.6 > > > show databases name: databases --------------- name _internal cadvisor k8s > > use k8s Using database k8s > > SHOW MEASUREMENTS name: measurements ------------------ name cpu/node_reservation cpu/node_utilization cpu/usage cpu/usage_rate filesystem/limit filesystem/usage memory/major_page_faults memory/major_page_faults_rate memory/node_reservation memory/node_utilization memory/page_faults memory/page_faults_rate memory/usage memory/working_set network/rx network/rx_errors network/rx_errors_rate network/rx_rate network/tx network/tx_errors network/tx_errors_rate network/tx_rate uptime

 

2）、InfluxDB安装与配置：

        请参考以前Cadvisor+InfluxDB+Grafana的内容，http://my.oschina.net/fufangchun/blog/718382

3）、Grafana安装与配置：

（1）、Grafana安装：

        请参考以前Cadvisor+InfluxDB+Grafana的内容，http://my.oschina.net/fufangchun/blog/718382

（2）、Grafana配置：

Ⅰ、配置influxdb数据源：

 

Ⅱ、新建展现页：

        展现页，用来区分不一样的influxdb数据源

 

 

Ⅲ、新建数据项：

        根据特定的条件（sql），从influxdb查询相关的数据，并展现。

        Grafana会自动显示k8s库的字段，能够根据需求进行选择，每一个where后面，加上：【pod_name =~ /^\.*/】，按照pod名称分类

SELECT last(value) FROM k8s."default"."network/tx" WHERE pod_name =~ /^\.*/ AND time > now() - 30m GROUP BY time(10s), pod_name

 

Ⅳ、完整的grafana展现：

    这里上传可能看不清，能够去百度盘下载，我这里是有些数据是分别作了，按照pod和namespace进行分组的，有多租户的状况，比较实用，能够根据状况，自行定义。

       百度盘地址：    http://pan.baidu.com/s/1qXQIqPA

     Kubernetes & Docker监控 ，至此完结，欢迎分享讨论！！！