Python性能监控Graphite
1、简介python
Graphite 是一个Python写的web应用，采用django框架，Graphite用来进行收集服务器全部的及时状态，用户请求信息，Memcached命中率，RabbitMQ消息服务器的状态，Unix操做系统的负载状态，Graphite服务器大约每分钟须要有4800次更新操做，Graphite采用简单的文本协议和绘图功能能够方便地使用在任何操做系统上。mysql
graphite有三个组件：ios
graphite-web：web接口c++
carbon：至关于network interfacegit
whisper：至关于rrdtoolgithub
graphite官方文档：web
http://graphite.wikidot.com/documentationsql
http://graphite.readthedocs.org/en/latest/数据库
2、安装graphiteapache
一、安装epel源
rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm sed -i 's@^#@@' /etc/yum.repos.d/epel.repo sed -i 's@mirrorlist@#mirrorlist@' /etc/yum.repos.d/epel.repo
二、安装适应版本的Django软件包，版本太高会出现bug
yum install python-simplejson wget https://kojipkgs.fedoraproject.org//packages/Django14/1.4.14/1.el6/noarch/Django14-1.4.14-1.el6.noarch.rpm rpm -ivh Django14-1.4.14-1.el6.noarch.rpm
三、安装graphite
yum install graphite-web python-carbon python-whisper
四、安装MySQL数据库
yum install mysql mysql-server MySQL-python service mysqld start chkconfig mysqld on mysqladmin -uroot password 123456 mysql -uroot -p123456 -e 'create database graphite;'
五、修改graphite配置文件
# cat >> /etc/graphite-web/local_settings.py << EOF
SECRET_KEY = '123qwe'
ALLOWED_HOSTS = [ '*' ]
TIME_ZONE = 'Asia/Shanghai'
DEBUG = True
DATABASES = {
'default': {
'NAME': 'graphite',
'ENGINE': 'django.db.backends.mysql',
'USER': 'root',
'PASSWORD': '123456',
'HOST': '127.0.0.1',
'PORT': '3306'
}
}
from graphite.app_settings import *
EOF
六、同步数据库
mkdir -p /opt/graphite/storage cd /etc/graphite-web/ django-admin syncdb --settings=local_settings --pythonpath=. yes root zhengys@allentuns.com 123456 123456
七、修改graphite数据目录
chown -R apache.apache /opt/graphite/storage
八、启动服务
/etc/init.d/carbon-cache start chkconfig carbon-cache on /etc/init.d/httpd start chkconfig httpd on
3、访问展现graphite
一、Chrome浏览器访问Ghipte的地址：
二、提供监控网卡流量的脚本
[root@Allentuns ~]# cat network_traffic.py
#!/usr/bin/env python
from subprocess import Popen,PIPE
import socket
import shlex
import time
import sys
import os
def get_traffic(f):
p = Popen(shlex.split(f),stdout=PIPE,stderr=PIPE)
result = p.stdout.read()
d = [i for i in result.split('\n')[3:] if i]
dic_traffic = {}
for i in d:
devname = i.split(':')[0].strip()
Receive = i.split(':')[1].split()[0].strip()
Transmit = i.split(':')[1].split()[8].strip()
dic_traffic[devname] = {'in':Receive,'out':Transmit}
return dic_traffic
if __name__ == '__main__':
try:
HOST = '127.0.0.1'
PORT = 2003
s = socket.socket()
s.connect((HOST,PORT))
except:
print "Couldn't connect to %(server)s on port %(port)d, is carbon-agent.py running?" % {'server':HOST,'port':PORT}
sys.exit(1)
while True:
cur_traffic = get_traffic('cat /proc/net/dev')
time.sleep(5)
five_s_traffic = get_traffic('cat /proc/net/dev')
diff_dic = {}
for k in cur_traffic:
traffic_in = int(five_s_traffic[k]['in']) - int(cur_traffic[k]['in'])
traffic_out = int(five_s_traffic[k]['out']) - int(cur_traffic[k]['out'])
diff_dic[k] = {'in':traffic_in,'out':traffic_out}
now = int(time.time())
for k,v in diff_dic.items():
net_in = 'network.%s_in %s %s\n' % (k,v['in'],now)
net_out = 'network.%s_out %s %s\n' % (k,v['out'],now)
s.sendall(net_in)
s.sendall(net_out)
time.sleep(5)
三、后台方式运行监控网卡流量脚本
[root@Allentuns ~]# python network_traffic.py &
4、安装Diamond
diamond ：搜集器、用于搜集数据
diamond的github官方站点：https://github.com/python-diamond/Diamond/wiki
一、安装Diamond
yum install gcc gcc-c++ python-configobj python-pip python-devel pip install diamond==3.4.421 (有时候会安装不成功) 若是下载安装不成功可使用如下方式进行 wget https://pypi.python.org/packages/source/d/diamond/diamond-3.4.421.tar.gz#md5=080ab9f52a154d81f16a4fd27d11093a tar xf diamond-3.4.421.tar.gz cd diamond-3.4.421 python setup.py install
二、配置
cd /etc/diamond/ cp diamond.conf.example diamond.conf 主要修改三个配置文件： [root@Allentuns diamond]# vim /etc/diamond/diamond.conf `GraphiteHandler` //59行 host = localhost `default` //173行 interval = 10 //时间搜集一次 [root@Allentuns diamond]# vim /etc/diamond/handlers/ArchiveHandler.conf #log_file = ./storage/archive.log //注释此行 [root@Allentuns diamond]# vim /etc/diamond/handlers/GraphiteHandler.conf host = localhost
三、启动diamond服务
chmod +x /etc/init.d/diamond /etc/init.d/diamond start chkconfig diamond on
5、继续访问展现diamond自动采集信息
一、Chrome浏览器访问Ghipte的地址：
你会发如今Graphite下多了一个servers的目录，这个目录就是diamond自动采集的信息
二、在这里提供了两个python脚本，用来搜集web站点的httpcode，是基于diamond的方式
[root@Allentuns ~]# cd /usr/share/diamond/collectors [root@Allentuns collectors]# mkdir httpcode && cd $_ [root@Allentuns httpcode]# ll 总用量 8 -rwxr-xr-x 1 root root 1356 3月 31 11:12 filerev.py -rwxr-xr-x 1 root root 3737 3月 31 11:12 httpcode.py
三、运行搜集httpcode的脚本
首先删除原来diamond生成的servers目录
[root@Allentuns httpcode]# rm -rf /var/lib/carbon/whisper/servers/
而后手动运行diamond的httpcode脚本
[root@Allentuns httpcode]# diamond -f -l -r ./httpcode.py -c /etc/diamond/diamond.conf
ERROR: Pidfile exists. Server already running? #须要手动中止diamond服务
[root@Allentuns httpcode]# /etc/init.d/diamond stop
Stopping diamond: [肯定]
[root@Allentuns httpcode]# diamond -f -l -r ./httpcode.py -c /etc/diamond/diamond.conf
[2015-03-31 11:13:56,198] [MainThread] Changed UID: 0 () GID: 0 ().
[2015-03-31 11:13:56,198] [MainThread] Loaded Handler: diamond.handler.graphite.GraphiteHandler
[2015-03-31 11:13:56,201] [MainThread] GraphiteHandler: Established connection to graphite server localhost:2003.
[2015-03-31 11:13:56,202] [MainThread] Loaded Handler: diamond.handler.archive.ArchiveHandler
[2015-03-31 11:13:56,206] [MainThread] Loading Collectors from: .
[2015-03-31 11:13:56,209] [MainThread] Loaded Module: httpcode
[2015-03-31 11:13:56,209] [MainThread] Loaded Collector: httpcode.HttpCodeCollector
[2015-03-31 11:13:56,209] [MainThread] Initialized Collector: HttpCodeCollector
[2015-03-31 11:13:56,210] [MainThread] Skipped loading disabled Collector: HttpCodeCollector
[2015-03-31 11:13:56,210] [MainThread] Started task scheduler.
[2015-03-31 11:13:57,211] [MainThread] Stopping task scheduler.
[2015-03-31 11:14:01,217] [MainThread] Stopped task scheduler.
[2015-03-31 11:14:01,217] [MainThread] Exiting.
若是没有报错，则查看浏览器会发现多了一个servers目录；可是当时目录就是没有生成，我还真纳闷了。原来在配置文件中没有启动此配置
[root@Allentuns httpcode]# vim httpcode.py
......
config = super(HttpCodeCollector, self).get_default_config()
config.update({
'path': 'weblog',
'enabled': 'True' #开启此选项
})
若是用diamond来搜集，则无需此选项，由于diamond有针对类的配置文件，在配置文件中开启会比在脚本中开启看起来更统一
四、在脚本中关闭，在diamond中的配置文件中自动启用此选项
# cd /etc/diamond/collectors/ # cp CPUCollector.conf HttpCodeCollector.conf # cat HttpCodeCollector.conf byte_unit = byte, enabled = true
五、浏览器查看
Chrome刷新Ghipte的web页面，查看
Ghipte -> servers -> ec2-54-201-82-69 -> weblog(自定义) -> http 会出现如下监控曲线图
咱们可使用ab -c 100 -n 100 http://localhost/ 产生200状态码
使用刷新Ghipte的浏览器页面产生304的状态码
另外补充一个截图
目前主流的开源监控有Cacti、Nagios、Zabbix等等，社区活跃，功能强大
Graphite虽然在功能上和社区在没法与此对比，可是在灵活度上仍是值得一提的，轻量级的监控程序，更为重要的是Graphite是Python编写的，因此在问题排查，脚本编写等都会很是顺手
一样也很是感谢更多Python开源者的贡献！！！
相关文章
- 1. Python性能监控Graphite
- 2. Graphite监控scrapy
- 3. graphite 监控---安装
- 4. graphite+grafana监控openstack和ceph
- 5. graphite+nginx+grafana监控部署
- 6. python性能监控初试
- 7. 性能监控
- 8. 安装Collectd+Statsd + Graphite 监控openstack swift
- 9. 性能监控 -- 中间件性能监控【Weblogic控制台】
- 10. Weblogic性能监控-中间件性能监控【控制台】
- 更多相关文章...
- • SQLite - Python - SQLite教程
- • Redis watch命令——监控事务 - Redis教程
- • JDK13 GA发布：5大特性解读
- • SpringBoot中properties文件不能自动提示解决方法
相关标签/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
欢迎关注本站公众号,获取更多信息
