复习一下Linux命令awk的使用

好久不用linux命令，好多都忘记了，只是经常使用的记得比较比如如tail、 grep、 top、 touch、 ps-ef、sed等，用的少的日久失修，因此人脑不像计算机，计算机磁盘满了手动清除才能存下新的文件。可是人脑毕竟不是计算机，遗忘是天然界赋予人的脑的自动清除算法。。。 下面回忆一下awk的统计日志用法:

1.下面我有一个rockmq的日志

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@1990a65e
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@150c158
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@4524411f
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@401e7803
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@1990a65e
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@150c158
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@4524411f
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set into: sun.nio.ch.KQueueSelectorImpl@401e7803
复制代码

统计@后面字符串出现的数量：
好比:

2 | 401e7803
2 | 4524411f
复制代码

下面是执行命令:

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log | awk -F '(into:|@)' '{print $3}'| sort | uniq -c
   2 150c158
   2 1990a65e
   2 401e7803
   2 4524411f
hxr-mac:rocketmqlogs houxiurong$   
复制代码

总结

命令说明:
1.从每一行取出须要的字符串并分割 awk -F '(into:|@)' '{print $1}' 按照into: 和 @符号分割，输出结果

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $1}'
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set
hxr-mac:rocketmqlogs houxiurong$
复制代码

2.分割后第二个字段值

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $2}'
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
 sun.nio.ch.KQueueSelectorImpl
hxr-mac:rocketmqlogs houxiurong$
复制代码

3.分割后第三个字段值,这个字段才是须要的字段

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $3}'
1990a65e
150c158
4524411f
401e7803
1990a65e
150c158
4524411f
401e7803
hxr-mac:rocketmqlogs houxiurong$
复制代码

4.第三个字段可是里面有重复的值，要怎么处理了？使用管道| sort排序

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $3}' | sort
150c158
150c158
1990a65e
1990a65e
401e7803
401e7803
4524411f
4524411f
hxr-mac:rocketmqlogs houxiurong$
复制代码

经过==分割三个字段的值并展现:

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print "=="$1,"=="$2,"=="$3}' | sort |uniq -c
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==150c158
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==1990a65e
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==401e7803
   1 ==2019-09-27 17:51:49 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==4524411f
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==150c158
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==1990a65e
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==401e7803
   1 ==2019-09-27 17:53:02 INFO main - instrumented a special java.util.Set  == sun.nio.ch.KQueueSelectorImpl ==4524411f
复制代码

sort 的 -n：依照数值的大小排序；-r 按照相反顺序排列

排序后还不是我须要的值，须要统计这个值出现的次数，能够使用
uniq经常使用命令字段:
-c 在输出行前面加上每行在输入文件中出现的次数。 -d 仅显示重复行。 -u 仅显示不重复的行。

5.将结果输出到文件中

hxr-mac:rocketmqlogs houxiurong$ cat namesrv_default.log |awk -F '(into:|@)' '{print $3}' | sort |uniq -c | sort -nr > testfile
hxr-mac:rocketmqlogs houxiurong$ ls
namesrv_default.log	testfile
hxr-mac:rocketmqlogs houxiurong$ cat testfile
   2 4524411f
   2 401e7803
   2 1990a65e
   2 150c158
复制代码

复习awk命令结束。。。