gmond dead but subsys locked

时间 2019-11-07

标签 gmond dead subsys locked 繁體版

原文原文链接

在作ganglia的puppet的统一管理，同步时发现服务没法正常启动，网络

# puppetd --test
info: Retrieving plugin
info: Loading facts in /var/lib/puppet/lib/facter/haproxyrunning.rb
info: Loading facts in /var/lib/puppet/lib/facter/kernel_mod_ip_conntrack.rb
info: Loading facts in /var/lib/puppet/lib/facter/kernel_mod_nf_conntrack.rb
info: Caching catalog for taurus-bj20.internal.gexing.com
info: Applying configuration version '1377502166'
notice: /Stage[main]/Common/Exec[find /var/lib/puppet/clientbucket -name paths -execdir cat {} \; -execdir pwd \; -execdir date -r {} +"%F %T" \; -exec echo \; > /var/log/puppet/clientbucket.log]/returns: executed successfully
notice: /Stage[main]/Ganglia::Service/Service[gmond]/ensure: ensure changed 'stopped' to 'running'
notice: Finished catalog run in 9.40 seconds

同步是成功的，回馈的信息现实服务已经启动起来了，但实际上却没有相关进程，查看状态发现
ide

service gmond status 时显示：

gmond dead but subsys locked

查看日志spa

# tail -f /var/log/messages 
Aug 26 15:29:27 taurus-bj20 puppet-agent[22030]: Applying configuration version '1377502166'
Aug 26 15:29:33 taurus-bj20 puppet-agent[22030]: (/Stage[main]/Common/Exec[find /var/lib/puppet/clientbucket -name paths -execdir cat {} \; -execdir pwd \; -execdir date -r {} +"F T" \; -exec echo \; > /var/log/puppet/clientbucket.log]/returns) executed successfully
Aug 26 15:29:36 taurus-bj20 /usr/sbin/gmond[22620]: Error creating UDP server on port 8649 bind=taurus-bj20. Exiting.#012
Aug 26 15:29:36 taurus-bj20 puppet-agent[22030]: (/Stage[main]/Ganglia::Service/Service[gmond]/ensure) ensure changed 'stopped' to 'running'
Aug 26 15:29:37 taurus-bj20 puppet-agent[22030]: Finished catalog run in 9.40 seconds
Aug 26 15:30:03 taurus-bj20 /usr/sbin/gmond[22684]: Error creating UDP server on port 8649 bind=taurus-bj20. Exiting.#012
Aug 26 15:30:30 taurus-bj20 /usr/sbin/gmond[22755]: Error creating UDP server on port 8649 bind=taurus-bj20. Exiting.#012
Aug 26 15:30:31 taurus-bj20 /usr/sbin/gmond[22774]: Error creating UDP server on port 8649 bind=taurus-bj20. Exiting.#012
Aug 26 15:30:32 taurus-bj20 /usr/sbin/gmond[22793]: Error creating UDP server on port 8649 bind=taurus-bj20. Exiting.#012
Aug 26 15:30:33 taurus-bj20 /usr/sbin/gmond[22812]: Error creating UDP server on port 8649 bind=taurus-bj20. Exiting.#012

查看端口，8649并没在用，那颇有多是本地的网络问题，由于我定义节点是采用的puppet的facter取的hostname日志

udp_recv_channel {
port = 8649
bind = taurus-bj20
}

而hosts文件server

172.16.2.19 taurus-bj19
172.16.2.20 taurus-bj19 #太粗心了，这居然写错了
进程

问题找到了，原来是这个主机名没法被识别，修改主机信息后，再次同步

进程启动起来了

总结进程启动不起来的缘由是由于gmond在建立本地的8649端口监控时的没法链接问题形成的！
ip