批量并发执行工具PDO,主要是解决批量执行的繁锁,更安全便捷的操做工具.
自己是解决公司内部的一些问题,而且有不少特定环境的一些使用,如今抽离出其中均可以使用的部分.linux
先获取依赖的第三方库:git
go get github.com/cihub/seelog go get github.com/robfig/config
安装go 环境.github
go build pdo.go
配置文件目录,默认在~/.pdo.若是不在此处指定.web
获取机器列表和相对应的路径有三种途径.(这里去掉了数据库这种特定的)redis
若是列表名称是这样的结构,xxx.yyy 那么过滤的就是yyy,若是没有这个须要,能够忽略.数据库
配置文件中:macos
[IDC] JX:yf01,cq01,dbl01,ai01 TC:cq02,tc,m1,db01
日志主要记录使用者,使用过的命令,保证多人操做的时候能够查看到.centos
日志配置文件查看github.com/cihub/seelog
主配置文件格式查看github.com/robfig/config缓存
会有主机和命令和单台执行确认.安全
[PDO] logconf:/home/work/.pdo/log.xml [IDC] JX:yf01,cq01,dbl01,ai01,jx,cp01 TC:cq02,tc,m1,db01,st01 [TEMPLATE] container : /home/work/.pdo/template/container.sh startbykill : /home/work/.pdo/template/startbykill.sh [CMD] restart: bash bin/xxxControl.sh N%%N%%N%%restart findLog: find xxx00* -name "debug" -type d findCount: ls log | wc -l
第一列必定是host,hostname或者ip均可以,第二列可选是命令工做的路径.
cat godir/1.list yf-xxx-app01.yf01 /home/work/xxx001 yf-xxx-app02.yf01 /home/work/xxx004 yf-xxx-app03.yf01 /home/work/xxx002
cat 1.list | pdo -r 2 "pwd" >>>> Welcome ajian... yf-xxx-pre01.vm -/home/work/xxx001 yf-xxx-app01.yf01 -/home/work/xxx001 yf-xxx-app02.yf01 -/home/work/xxx001 yf-xxx-app03.yf01 -/home/work/xxx001 yf-xxx-app04.yf01 -/home/work/xxx001 1-xxx-app17.m1 -/home/work/xxx001 m1-xxx-app25.m1 -/home/work/xxx001 m1-xxx-app0220.m1 -/home/work/xxx001 m1-xxx-app0154.m1 -/home/work/xxx004 cq01-xxx-app0242.cq01 -/home/work/xxx003 ai-xxx-app01.ai01 -/home/work/xxx004 db-xxx-app17.db01 -/home/work/xxx003 db-xxx-app63.db01 -/home/work/xxx001 #--Total--# 13 #---CMD---# pwd //每一次确认 Continue (y/n):y go on ... [1/13] yf-xxx-app01.yf01 [SUCCESS]. /home/work/xxx001 Continue (y/n):[1/13] yf-xxx-pre01.vm [SUCCESS]. /home/work/xxx001 //单台执行完 第二次确认 Continue (y/n): //后面就是按2并发执行.
这个主要是解决一些重复执行的繁锁的单行命令.看下面的一个重启命令非常麻烦,但经过转换以后就输入很方便了.
-cmd为缩写命令= bash bin/xxxControl.sh N%%N%%N%%restart
$ pdo -f 1.list -cmd restart >>>> Welcome ajian... yf-xxx-app01.yf01 -/home/work/xxx001 yf-xxx-app02.yf01 -/home/work/xxx001 yf-xxx-app03.yf01 -/home/work/xxx001 yf-xxx-app04.yf01 -/home/work/xxx001 yf-xxx-app00.yf01 -/home/work/xxx001 yf-xxx-app0148.yf01 -/home/work/xxx004 dbl-xxx-app0109.dbl01 -/home/work/xxx003 m1-xxx-app17.m1 -/home/work/xxx001 m1-xxx-app25.m1 -/home/work/xxx001 m1-xxx-app0220.m1 -/home/work/xxx001 m1-xxx-app0154.m1 -/home/work/xxx004 cq01-xxx-app0242.cq01 -/home/work/xxx003 cq01-xxx-app0179.cq01 -/home/work/xxx001 cq01-xxx-app0131.cq01 -/home/work/xxx005 st01-xxx-app03.st01 -/home/work/xxx001 st01-xxx-app04.st01 -/home/work/xxx001 st01-xxx-app02.st01 -/home/work/xxx001 st01-xxx-app00.st01 -/home/work/xxx001 st01-xxx-app05.st01 -/home/work/xxx001 cq02-xxx-app0258.cq02 -/home/work/xxx001 cq02-xxx-app0287.cq02 -/home/work/xxx001 jx-xxx-app17.jx -/home/work/xxx001 ai-xxx-app10.ai01 -/home/work/xxx001 db-xxx-app17.db01 -/home/work/xxx003 #--Total--# 24 #---CMD---# bash bin/xxxControl.sh N%%N%%N%%restart Continue (y/n):
使用带-o 指定输出目录,将不会再打印在屏幕上,主要是对grep日志这种需求使用.速度要比屏幕打印快不少,是实时写入.
$ cat 1.list | pdo -o xxxout "pwd" >>>> Welcome ajian... yf-xxx-app01.yf01 -/home/work/xxx001 yf-xxx-app02.yf01 -/home/work/xxx001 yf-xxx-app03.yf01 -/home/work/xxx001 yf-xxx-app04.yf01 -/home/work/xxx001 yf-xxx-app00.yf01 -/home/work/xxx001 yf-xxx-app0148.yf01 -/home/work/xxx004 dbl-xxx-app0109.dbl01 -/home/work/xxx003 m1-xxx-app17.m1 -/home/work/xxx001 m1-xxx-app25.m1 -/home/work/xxx001 m1-xxx-app0220.m1 -/home/work/xxx001 m1-xxx-app0154.m1 -/home/work/xxx004 cq01-xxx-app0242.cq01 -/home/work/xxx003 cq01-xxx-app0179.cq01 -/home/work/xxx001 cq01-xxx-app0131.cq01 -/home/work/xxx005 st01-xxx-app03.st01 -/home/work/xxx001 st01-xxx-app04.st01 -/home/work/xxx001 st01-xxx-app02.st01 -/home/work/xxx001 st01-xxx-app00.st01 -/home/work/xxx001 st01-xxx-app05.st01 -/home/work/xxx001 cq02-xxx-app0258.cq02 -/home/work/xxx001 cq02-xxx-app0287.cq02 -/home/work/xxx001 cq02-xxx-app0212.cq02 -/home/work/xxx001 jx-xxx-app17.jx -/home/work/xxx001 ai-xxx-app10.ai01 -/home/work/xxx001 db-xxx-app17.db01 -/home/work/xxx003 #--Total--# 25 #---CMD---# pwd Continue (y/n):y go on ... [1/25] yf-xxx-app01.yf01 [SUCCESS]. Continue (y/n):y go on ... [2/25] yf-xxx-app02.yf01 [SUCCESS]. [3/25] yf-xxx-app03.yf01 [SUCCESS]. [4/25] yf-xxx-app04.yf01 [SUCCESS]. [5/25] yf-xxx-app00.yf01 [SUCCESS]. [6/25] yf-xxx-app0148.yf01 [SUCCESS]. [7/25] dbl-xxx-app0109.dbl01 [SUCCESS]. [8/25] m1-xxx-app17.m1 [SUCCESS]. [9/25] m1-xxx-app25.m1 [SUCCESS]. [10/25] m1-xxx-app0220.m1 [SUCCESS]. [11/25] m1-xxx-app0154.m1 [SUCCESS]. [12/25] cq01-xxx-app0242.cq01 [SUCCESS]. [13/25] cq01-xxx-app0179.cq01 [SUCCESS]. [14/25] cq01-xxx-app0131.cq01 [SUCCESS]. [15/25] st01-xxx-app03.st01 [SUCCESS]. [16/25] st01-xxx-app04.st01 [SUCCESS]. [17/25] st01-xxx-app02.st01 [SUCCESS]. [18/25] st01-xxx-app00.st01 [SUCCESS]. [19/25] st01-xxx-app05.st01 [SUCCESS]. [20/25] cq02-xxx-app0258.cq02 [SUCCESS]. [21/25] cq02-xxx-app0287.cq02 [SUCCESS]. [22/25] cq02-xxx-app0212.cq02 [SUCCESS]. [23/25] jx-xxx-app17.jx [SUCCESS]. [24/25] ai-xxx-app10.ai01 [SUCCESS]. [25/25] db-xxx-app17.db01 [SUCCESS].
时间都带单位,如1秒 1s , 1分钟 1m , 1小时 1h .
这里的1.log是一个大文件.
$ pdo -f 1.list -t 1s -o out/ -r 3 "cat 1.log" >>>> Welcome ajian... yf-xxx-app01.yf01 -/home/work/xxx001 yf-xxx-app02.yf01 -/home/work/xxx001 yf-xxx-app03.yf01 -/home/work/xxx001 yf-xxx-app04.yf01 -/home/work/xxx001 yf-xxx-app00.yf01 -/home/work/xxx001 yf-xxx-app0148.yf01 -/home/work/xxx004 dbl-xxx-app0109.dbl01 -/home/work/xxx003 m1-xxx-app17.m1 -/home/work/xxx001 m1-xxx-app25.m1 -/home/work/xxx001 m1-xxx-app0220.m1 -/home/work/xxx001 m1-xxx-app0154.m1 -/home/work/xxx004 cq01-xxx-app0242.cq01 -/home/work/xxx003 cq01-xxx-app0179.cq01 -/home/work/xxx001 cq01-xxx-app0131.cq01 -/home/work/xxx005 st01-xxx-app03.st01 -/home/work/xxx001 st01-xxx-app04.st01 -/home/work/xxx001 st01-xxx-app02.st01 -/home/work/xxx001 st01-xxx-app00.st01 -/home/work/xxx001 st01-xxx-app05.st01 -/home/work/xxx001 cq02-xxx-app0258.cq02 -/home/work/xxx001 cq02-xxx-app0287.cq02 -/home/work/xxx001 cq02-xxx-app0211.cq02 -/home/work/xxx001 cq02-xxx-app0212.cq02 -/home/work/xxx001 jx-xxx-app17.jx -/home/work/xxx001 ai-xxx-app10.ai01 -/home/work/xxx001 db-xxx-app17.db01 -/home/work/xxx003 #--Total--# 26 #---CMD---# cat log/ral-zoo.log Continue (y/n):y go on ... [1/26] yf-xxx-app01.yf01 [Time Over KILLED]. Continue (y/n):y go on ... [2/26] yf-xxx-app04.yf01 [Time Over KILLED]. [3/26] yf-xxx-app03.yf01 [Time Over KILLED]. [4/26] yf-xxx-app02.yf01 [Time Over KILLED]. [5/26] yf-xxx-app0148.yf01 [SUCCESS]. [6/26] dbl-xxx-app0109.dbl01 [SUCCESS]. [7/26] yf-xxx-app00.yf01 [Time Over KILLED]. [8/26] m1-xxx-app0220.m1 [SUCCESS]. [9/26] m1-xxx-app25.m1 [Time Over KILLED]. [10/26] m1-xxx-app17.m1 [Time Over KILLED]. [11/26] m1-xxx-app0154.m1 [SUCCESS]. [12/26] cq01-xxx-app0242.cq01 [SUCCESS]. [13/26] cq01-xxx-app0179.cq01 [Time Over KILLED]. [14/26] cq01-xxx-app0131.cq01 [SUCCESS]. [15/26] st01-xxx-app03.st01 [Time Over KILLED]. [16/26] st01-xxx-app04.st01 [Time Over KILLED]. [17/26] st01-xxx-app02.st01 [Time Over KILLED]. [18/26] st01-xxx-app00.st01 [Time Over KILLED]. [19/26] st01-xxx-app05.st01 [Time Over KILLED]. [20/26] cq02-xxx-app0211.cq02 [SUCCESS]. [21/26] cq02-xxx-app0258.cq02 [SUCCESS]. [22/26] cq02-xxx-app0287.cq02 [SUCCESS]. [23/26] ai-xxx-app10.ai01 [SUCCESS]. [24/26] cq02-xxx-app0212.cq02 [SUCCESS]. [25/26] jx-xxx-app17.jx [Time Over KILLED]. [26/26] db-xxx-app17.db01 [SUCCESS].
copy文件实际上是能够copy目录的,只要远端的目录是存在的就不会报错.
$ cat 1.host | pdo -c get.sh /tmp/ >>>> Welcome ajian... yf-xxx-upload05.yf01 -/home/work yf-xxx-upload01.yf01 -/home/work yf-xxx-upload02.yf01 -/home/work #--Total--# 3 #---CMD---# get.sh --> /tmp/ Continue (y/n):y go on ... [1/3] yf-xxx-upload05.yf01 [SUCCESS]. Continue (y/n):y go on ... [2/3] yf-xxx-upload01.yf01 [SUCCESS]. [3/3] yf-xxx-upload02.yf01 [SUCCESS]. //检查下文件 $ cat 1.host | pdo "ls /tmp/get.sh" >>>> Welcome ajian... yf-xxx-upload05.yf01 -/home/work yf-xxx-upload01.yf01 -/home/work yf-xxx-upload02.yf01 -/home/work #--Total--# 3 #---CMD---# ls /tmp/get.sh Continue (y/n):y go on ... [1/3] yf-xxx-upload05.yf01 [SUCCESS]. /tmp/get.sh Continue (y/n):y go on ... [2/3] yf-xxx-upload01.yf01 [SUCCESS]. /tmp/get.sh [3/3] yf-xxx-upload02.yf01 [SUCCESS]. /tmp/get.sh
-R 就是至关于第四种列表来源,当执行错误,或者ctrl+c的时候就可使用上,避免列表反复执行某些命令.
此次多加两台服务器,有两台是没有这个上面脚本文件的.因此新加的服务器会报错.
$ cat 2.list | pdo "ls /tmp/get.sh" >>>> Welcome ajian... yf-xxx-upload05.yf01 -/home/work yf-xxx-upload01.yf01 -/home/work yf-xxx-upload02.yf01 -/home/work yf-xxx-upload03.yf01 -/home/work yf-xxx-upload04.yf01 -/home/work #--Total--# 5 #---CMD---# ls /tmp/get.sh Continue (y/n):y go on ... [1/5] yf-xxx-upload05.yf01 [SUCCESS]. /tmp/get.sh Continue (y/n):y go on ... [2/5] yf-xxx-upload01.yf01 [SUCCESS]. /tmp/get.sh [3/5] yf-xxx-upload02.yf01 [SUCCESS]. /tmp/get.sh [4/5] yf-xxx-upload03.yf01 [FAILED]. ls: /tmp/get.sh: No such file or directory [5/5] yf-xxx-upload04.yf01 [FAILED]. ls: /tmp/get.sh: No such file or directory //使用-R 就能够直接拿到上一次执行失败的列表. $pdo -R "ls /tmp/get.sh" >>>> Welcome ajian... yf-xxx-upload03.yf01 -/home/work yf-xxx-upload04.yf01 -/home/work #--Total--# 2 #---CMD---# ls /tmp/get.sh Continue (y/n):y go on ... [1/2] yf-xxx-upload03.yf01 [FAILED]. ls: /tmp/get.sh: No such file or directory //若是是使用的ctrl+C中断了列表,-R会记录未执行完(包括已经执行但失败的列表) $ cat 1.host | pdo -T 10s "ls /tmp/get.sh" >>>> Welcome ajian... yf-xxx-upload05.yf01 -/home/work yf-xxx-upload01.yf01 -/home/work yf-xxx-upload02.yf01 -/home/work yf-xxx-upload03.yf01 -/home/work yf-xxx-upload04.yf01 -/home/work #--Total--# 5 #---CMD---# ls /tmp/get.sh Continue (y/n):y go on ... [1/5] yf-xxx-upload05.yf01 [SUCCESS]. /tmp/get.sh Continue (y/n):y go on ... [2/5] yf-xxx-upload01.yf01 [SUCCESS]. /tmp/get.sh ^C$ pdo -R "ls /tmp/get.sh" >>>> Welcome ajian... yf-xxx-upload02.yf01 -/home/work yf-xxx-upload03.yf01 -/home/work yf-xxx-upload04.yf01 -/home/work #--Total--# 3 #---CMD---# ls /tmp/get.sh Continue (y/n):y go on ... [1/3] yf-xxx-upload02.yf01 [SUCCESS]. /tmp/get.sh Continue (y/n):
### -e脚本执行功能
$ cat t.sh #!/bin/bash cd /tmp/ && pwd echo "test" touch /tmp/t.log $ cat 1.host | pdo -e t.sh >>>> Welcome ajian... yf-xxx-upload05.yf01 -/home/work yf-xxx-upload01.yf01 -/home/work yf-xxx-upload02.yf01 -/home/work #--Total--# 3 #---CMD---# Script: t.sh Continue (y/n):y go on ... [1/3] yf-xxx-upload05.yf01 [SUCCESS]. /tmp test
模板功能主要是解决重复的脚本修改动做,能够固化成一些模板,直接使用.
配置中能够本身添加模板
$ cat ~/.pdo/pdo.conf [TEMPLATE] container : /home/work/.pdo/template/container.sh
模板内容,这个模版主要是在一台服务器上的xxxxxx目录里面进行操做. {{.CMD}} 就是会被替换的位置.
$ cat /home/work/.pdo/template/container.sh #!/bin/bash grep -l "^appName:" /home/work/xxx[0-9][0-9][0-9]/xxx.conf | while read file ; do eval $(awk '{if($1 ~ /xxxPath/){printf "apppath=%s\n",$2};if($1 ~ /appName/){printf "appName=%s",$2}}' $file) echo $appName if [ -d "$apppath" ];then cd $apppath {{.CMD}} fi done
使用嵌入命令
$ pdo -a xxxtest -temp container "pwd" >>>> Welcome ajian... yf-xxx-app02.yf01 -/home/work/xxx001 yf-xxx-app03.yf01 -/home/work/xxx001 yf-xxx-app00.yf01 -/home/work/xxx001 yf-xxx-app0148.yf01 -/home/work/xxx004 dbl-xxx-app0109.dbl01 -/home/work/xxx003 m1-xxx-app0220.m1 -/home/work/xxx001 m1-xxx-app0154.m1 -/home/work/xxx004 cq01-xxx-app0242.cq01 -/home/work/xxx003 cq01-xxx-app0179.cq01 -/home/work/xxx001 cq02-xxx-app0258.cq02 -/home/work/xxx001 cq02-xxx-app0287.cq02 -/home/work/xxx001 cq02-xxx-app0211.cq02 -/home/work/xxx001 jx-xxx-app17.jx -/home/work/xxx001 db-xxx-app17.db01 -/home/work/xxx003 #--Total--# 14 #---CMD---# pwd Continue (y/n):y go on ... [1/14] yf-xxx-app02.yf01 [SUCCESS]. xxxtest /home/work/xxx001 jingyan /home/work/xxx002 pc_anti /home/work/xxx003 bakan /home/work/xxx004 smallapp /home/work/xxx006 appui /home/work/xxx008 Continue (y/n):n exit ...
还能够嵌入脚本
//脚本内容 $ cat 1.sh echo "1.sh" pwd //嵌入脚本使用-b $ pdo -a xxxtest -temp container -b 1.sh >>>> Welcome ajian... yf-xxx-app02.yf01 -/home/work/xxx001 yf-xxx-app03.yf01 -/home/work/xxx001 yf-xxx-app00.yf01 -/home/work/xxx001 yf-xxx-app0148.yf01 -/home/work/xxx004 dbl-xxx-app0109.dbl01 -/home/work/xxx003 m1-xxx-app0220.m1 -/home/work/xxx001 m1-xxx-app0154.m1 -/home/work/xxx004 cq01-xxx-app0242.cq01 -/home/work/xxx003 cq01-xxx-app0179.cq01 -/home/work/xxx001 cq02-xxx-app0258.cq02 -/home/work/xxx001 cq02-xxx-app0287.cq02 -/home/work/xxx001 cq02-xxx-app0211.cq02 -/home/work/xxx001 jx-xxx-app17.jx -/home/work/xxx001 db-xxx-app17.db01 -/home/work/xxx003 #--Total--# 14 #---CMD---# Continue (y/n):y go on ... [1/14] yf-xxx-app02.yf01 [SUCCESS]. xxxtest 1.sh /home/work/xxx001 jingyan 1.sh /home/work/xxx002 pc_anti 1.sh /home/work/xxx003 bakan 1.sh /home/work/xxx004 smallapp 1.sh /home/work/xxx006 appui 1.sh /home/work/xxx008
这个功能有两种使用场景:
因此这种显示方式取决于时间的前后顺序,交错输出.
拿redis的迁移过程为例子:
redis迁移至少有原来的一主一从,新主和新从.在迁移的过程当中须要同时观察四台服务器的变化.若是是每次ssh四台服务器tail 日志是很麻烦并且容易出错.
如今使用pdo命令:
//操做的主机列表1.list tc-yyy-redis40.tc /home/yyy/redis-shard3 //old master cq02-yyy-redis80.cq02 /home/yyy/redis-shard3 //new master yf-yyy-redis40.yf01 /home/yyy/redis-shard3 //old slave jx-yyy-redis80.jx /home/yyy/redis-shard3 //new slave 第一步操做: yf-yyy-redis40.yf01为主 --> cq02-yyy-redis80.cq02 #命令 #cat 1.list | pdo -r 5 -y -show row -match "success" "tail -f log/redis.log" > yf-yyy-redis40.yf01 >> [11523] 06 Jan 13:56:51 * Slave ask for new-synchronization //被要求同步 > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 13:56:58 * (non critical): Master does not understand REPLCONF listening-port: Reading from master: Connection timed out > yf-yyy-redis40.yf01 >> [11523] 06 Jan 13:56:58 * Slave ask for synchronization > yf-yyy-redis40.yf01 >> [11523] 06 Jan 13:56:58 * Starting BGSAVE for SYNC > yf-yyy-redis40.yf01 >> [11523] 06 Jan 13:56:58 * Background saving started by pid 22855 > yf-yyy-redis40.yf01 >> [22855] 06 Jan 13:58:31 * DB saved on disk //dump到磁盘 > yf-yyy-redis40.yf01 >> [11523] 06 Jan 13:58:31 * Background saving terminated with success > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 13:58:31 * MASTER <-> SLAVE sync: receiving 1868940396 bytes from master //从接收到主的文件 > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 13:58:47 * MASTER <-> SLAVE sync: Loading DB in memory //将接收到的文件加载到内存 > yf-yyy-redis40.yf01 >> [11523] 06 Jan 13:58:47 * Synchronization with slave succeeded //文件同步成功 > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 14:01:21 # Update masterstarttime[1382324097] after loading db > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 14:01:21 * AA: see masterstarttime: ip[10.36.114.56], port[9973], timestamp[1382324097] > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 14:01:21 * Write aof_global_offset[92961804447] to new aof_file[46] success > cq02-yyy-redis80.cq02 >> [14752] 06 Jan 14:01:21 * MASTER <-> SLAVE sync: Finished with success //slave完成主从同步,说明第一步已经结束.
说明:
如下是一个测试脚本:随机打印数字 1.sh
#!/bin/bash for x in `seq 1 10` ; do echo $x sleep $[ ( $RANDOM % 4 ) + 1 ]s done //可使用以下命令: # cat 1.list | pdo -r 5 -y -show row -match "5" -e 1.sh
还有更多的组合哦.