Jenkins工做相关

Jenkins比较消耗资源，bug也比较多，公司后期会转向其余系统，暂时有些东东记录在这里。

轮询没法检测新提交的问题

现象是同时选中“Build when a change is pushed to GitHub”和 “Poll SCM", git有提交时，不会触发构建。

一般这种状况使用hook便可，在代码目录的hooks子目录配置post-commit便可。可是我这边不能彻底控制git，因此只能用轮询的方法，Jenkins的轮询比较低效，常常出现内存溢出等不稳定状况，也须要外来脚本进行干预，为此用python脚本：

#!/usr/bin/python
# -*- coding: utf-8 -*-

import logging
import logging.handlers
import time
import re
import os.path
import subprocess32
import sys
sys.path.append('../lib/')


from requests.auth import HTTPDigestAuth

dial_logger = logging.getLogger('MyLogger')
dial_logger.setLevel(logging.DEBUG)
fh = logging.handlers.RotatingFileHandler(
    '../logs/dial_log_server.log',
    maxBytes=10000000,
    backupCount=5)
fh.setLevel(logging.DEBUG)
formatter = logging.Formatter(u'%(asctime)s [%(levelname)s] %(message)s')
fh.setFormatter(formatter)
dial_logger.addHandler(fh)
dial_logger.info("Application starting! \n")


while True:

    for line in open("../conf/url.csv"):
        
        try:
        
            now = time.strftime("%Y-%m-%d_%H:%M:%S", time.gmtime())
            print(now)
    
            name, git_url, branch, jenkins_url = line.strip().split(',')
            # print(name, git_url, branch, jenkins_url)
            dial_logger.info("Beginning Scan {0}! \n".format(name))
            dial_logger.info("{0} {1} {2}! \n".format(git_url, branch, jenkins_url))
            cmd = "git ls-remote -h " + git_url
            result = subprocess32.check_output(cmd,shell=True)
            #print(result)
            version = re.findall(r'(\S+)\s+{0}'.format(branch), result, re.MULTILINE)[0]
            dial_logger.info("current version: {0}! \n".format(version))
            file_name = '/tmp/{0}'.format(name)
    
            if os.path.exists(file_name):
                old_version = open(file_name).read().strip()
                # print(old_version, version)
                dial_logger.info("old version: {0}! \n".format(old_version))
                if old_version == version:
                    dial_logger.info("Don not call {0} \n".format(name))
                    print("Don not call {0} \n".format(name))
                    continue
    
            f = open(file_name,'w')
            f.write(version)
            f.close()
            
            dial_logger.info("Call {0} \n".format(jenkins_url))
            cmd = "curl --user xurongzhong:123456 -s {0} &".format(jenkins_url)
            # print(cmd)
            dial_logger.info("command: {0}! \n".format(cmd))
            subprocess32.call(cmd, shell=True)
            time.sleep(1.5)
            
        except Exception as e:
            dial_logger.error(str(e), exc_info=True)        

    dial_logger.info("End Scan! \n")
    time.sleep(3 * 60)

注意点：

* curl后面的接的地址不要有换行。

* subprocess32是为了适应centos 6上面的python2.6.6

使用说明：

1，脚本dial_log_server.py放置在bin目录，同时创建同级目录：conf和logs。

2，bin目录中增长启停脚本：

shutdown.sh

#!/bin/sh

APP_MAIN=dial_log_server.py
PID=0

getPID(){
    pythonps=`ps aux | grep $APP_MAIN | grep -v grep`
    if [ -n "$pythonps" ]; then
        PID=`echo $pythonps | awk '{print $2}'`
    else
        PID=0
    fi
}

shutdown(){
    getPID
    echo "================================================================================================================"
    if [ $PID -ne 0 ]; then
        echo -n "Stopping $APP_MAIN(PID=$PID)..."
        kill -9 $PID
        if [ $? -eq 0 ]; then
            echo "[Success]"
            echo "================================================================================================================"
        else
            echo "[Failed]"
            echo "================================================================================================================"
        fi
        getPID
        if [ $PID -ne 0 ]; then
            shutdown
        fi
    else
        echo "$APP_MAIN is not running"
        echo "================================================================================================================"
    fi
}

shutdown
exit 0

startup.sh

#!/bin/sh

APP_MAIN=dial_log_server.py
APP_LOG=logs
PID=0

getPID(){
    pythonps=`ps aux | grep $APP_MAIN | grep -v grep`
    if [ -n "$pythonps" ]; then
        PID=`echo $pythonps | awk '{print $2}'`
    else
        PID=0
    fi
}


startup(){
    getPID
    echo "================================================================================================================"
    if [ $PID -ne 0 ]; then
        echo "$APP_MAIN already started(PID=$PID)"
        echo "================================================================================================================"
    else
        echo -n "Starting $APP_MAIN"
         if [ ! -d "../$APP_LOG" ]; then
            mkdir "../$APP_LOG"
         fi
        nohup python $APP_MAIN > ../$APP_LOG/nohup.log 2>&1 &
        sleep 2
        getPID
        if [ $PID -ne 0 ]; then
            echo "(PID=$PID)...[Success]"
            echo "================================================================================================================"
        else
            echo "[Failed]"
            echo "================================================================================================================"
        fi
    fi
}

startup

3，conf目录生成配置文件：

Advertisement_Manage,ssh://root@172.17.100.19/srv/repos/Advertisement_Manage,refs/heads/master,http://172.17.100.18:8080/job/Advertisement_Web_New/build?token=123456
ads-ad-local,ssh://root@172.17.100.19/srv/repos/ads-ad,refs/heads/master,http://172.17.100.18:8080/job/ads-ad-local/build?token=123457
ads-ad-client-local,ssh://root@172.17.100.19/srv/repos/ads-ad,refs/heads/master,http://172.17.100.18:8080/job/ads-ad-client-local/build?token=123458

4，Jenkins对应项目"触发远程构建"的“身份验证令牌”。

5，后续规划：

* 基于ConfigParser的配置文件。

* 增长Jenkins无反应时重启。

* 其余辅助功能。

Jenkins动辄内存不足

修改/etc/sysconfig/jenkins,  设置以下或者更高的值：

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Xmx4096m -XX:MaxPermSize=2048m"

jenkins的系统配置中，全局属性，环境变量 MAVEN_OPTS

-Xmx4096m -XX:MaxPermSize=2048m

Maven项目配置，全局MAVEN_OPTS

-Xmx4096m -XX:MaxPermSize=2048m

/etc/profile

export JAVA_OPTS="-XX:MaxPermSize=2048m -Xms4096m -Xmx4096m"
export GRADLE_OPTS="-XX:MaxPermSize=2048m"
export SONAR_RUNNER_OPTS="-Xmx4096m -XX:MaxPermSize=2048m"

增长Findbugs静态分析

在pom.xml的build部分增长以下内容：

                                <plugin>
                                    <groupId>org.codehaus.mojo</groupId>
                                    <artifactId>findbugs-maven-plugin</artifactId>
                                    <version>3.0.1</version>
                                    <configuration>
                                        <findbugsXmlOutput>true</findbugsXmlOutput>
                                        <findbugsXmlWithMessages>true</findbugsXmlWithMessages>
                                        <xmlOutput>true</xmlOutput>
                                        <onlyAnalyze>nl.berg.packt.FindBugs_all.candles.*</onlyAnalyze>
                                        <effort>Max</effort>
                                   </configuration>
                                </plugin>

注意：

* Findbugs分析很消耗内存，须要配置好jvm参数。

* findbugsXmlOutput等标签所有要为小写，一些文档有坑，部分为大写，会有以下报错：

[ERROR] No plugin found for prefix 'findBugs' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/var/lib/jenkins/.m2/repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]

* Jenkins的上面选中”Publish FindBugs analysis results“，编译参数增长：" findbugs:findbugs "

好比：

clean package findbugs:findbugs cobertura:cobertura -Denv=dev -Ddubbo.protocol.port=28999

Jenkins 基于Cobertura的代码覆盖率支持

在pom.xml的build部分增长以下内容：

                                <plugin>
                                        <groupId>org.codehaus.mojo</groupId>
                                        <artifactId>cobertura-maven-plugin</artifactId>
                                        <version>2.6</version>
                                        <configuration>
                                        <formats>
                                                <format>xml</format>
                                                <format>html</format>
                                        </formats>
                                        </configuration>
                                </plugin>

注意Jenkins不能在页面上展现覆盖率，须要下载html文件。好比：/var/lib/jenkins/workspace/ads-ad-local/ads-ad/target/site/cobertura目录。

展现结果仍是挺漂亮的，好比：