sed进阶

时间 2019-12-12

标签 sed 进阶栏目 Linux 繁體版

原文原文链接

下面这些命令未必常常会用到，但当须要时，知道这些确定是件好事。shell

1、多行命令

sed命令一般是对一行数据进行处理，而后下一行重复处理。ubuntu

sed编辑器包含了三个可用来处理多行文本的特殊命令bash

N：将数据流中的下一行加进来建立一个多行组来处理
D：删除多行组中的一行
P：打印多行组中的一行

1.1 next命令

两种删除匹配的下一行的办法：app

cat data1.txt 
This is the header line.

This is a data line.

This is the last line.

sed '/^$/d' data1.txt 
This is the header line.
This is a data line.
This is the last line.

sed '/header/{n ; d}' data1.txt 
This is the header line.
This is a data line.

This is the last line.

空行处理

1.2 合并文本行

用大写N，能够一次性读取两行。区别与n是移动到下一行。编辑器

$ sed '/first/{ N ; s/\n/ / }' data2.txt 
This is the header line.
This is the first data line. This is the second data line.
This is the last line.

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

$ cat data3.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.
Thank you for your attendance.

$ sed 'N ; s/System.Administrator/Desktop User/' data3.txt
On Tuesday, the Linux Desktop User's group meeting will be held.
All Desktop Users should attend.
Thank you for your attendance.

N使用例子

在N后面最好匹配多行命令，而单行命令则能够放在N前面，如：ide

$ sed 'N
> s/System\nAdministrator/Desktop\nUser/
> s/System Administrator/Desktop User/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All System Administrators should attend.

$ cat data4.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.

$ sed '
> s/System Administrator/Desktop User/
> N
> s/System\nAdministrator/Desktop\nUser/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All Desktop Users should attend.

先后写的例子

1.3 删除多行命令

若是在N后面用d，就会把多行都一块儿删除。可是若是用D，就会删除到\n为止。工具

$ sed 'N ; /System\nAdministrator/d' data4.txt 
All System Administrators should attend.
$ cat data4.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.
$ sed 'N ; /System\nAdministrator/D' data4.txt 
Administrator's group meeting will be held.
All System Administrators should attend.

d和D的对比

而后有删除header前的空白行测试

$ sed '/^$/{N ; /header/D}' data5.txt 
This is the header line.
This is a date line.

This is the last line.

$ sed '{N ; /header/D}' data5.txt 
This is a date line.

This is the last line.
$ sed '{N ; /header/d}' data5.txt 
This is a date line.

This is the last line.
$ cat data5.txt 

This is the header line.
This is a date line.

This is the last line.

删除header前的空行

相似的和D类似的，P也是会和N配合输出第一行this

$ cat data3.txt 
On Tuesday, the Linux System
Administrator's group meeting will be held.
All System Administrators should attend.
Thank you for your attendance.
$ sed -n 'N ; /System\nAdministrator/P' data3.txt
On Tuesday, the Linux System

P的使用

2、保持空间

sed编辑器有一块称做保持空间的缓冲区域。在处理某些行时，能够用保持空间来临时保存一些行。spa

h：将模式空间那关键复制到保持空间
H：将模式空间附加到保持空间
g：将保持空间复制到模式空间
G：将保持空间附加到模式空间
x：交换模式空间和保持空间的内容

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.
$ sed -n '/first/ {h ; p; n ; p ; g ; p }' data2.txt 
This is the first data line.
This is the second data line.
This is the first data line.

$ sed -n '/first/ {h ; n ; p ; g ; p }' data2.txt 
This is the second data line.
This is the first data line.

保持空间

3、排除命令

感叹号命令（！）用来排除命令

$ sed -n '/header/!p' data2.txt 
This is the first data line.
This is the second data line.
This is the last line.
$ sed 'N;
> s/System\nAdministrator/Desktop\nUser/
> s/System Administrator/Desktop User/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All System Administrators should attend.
$ sed '$!N;
> s/System\nAdministrator/Desktop\nUser/
> s/System Administrator/Desktop User/
> ' data4.txt
On Tuesday, the Linux Desktop
User's group meeting will be held.
All Desktop Users should attend.

！例子

而后是将文件反转的例子：

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.
$ sed -n '{1!G ; h ; $p }' data2.txt 
This is the last line.
This is the second data line.
This is the first data line.
This is the header line.

反转例子

4、改变流

4.1 分支指令

基于地址、地址模式或地址区间排除一整块命令，这容许你只对数据流中的特定行执行一组命令。

分支命令b格式：[address]b [label]

address参数：决定了那些行的数据会触发分支明林。

label参数：定义了要跳转到的位置。

$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

$ sed '{2,3b ; s/This is/Is this/ ; s/line./test?/}' data2.txt
Is this the header test?
This is the first data line.
This is the second data line.
Is this the last test?

命令b使用例子

上面的例子里，分支跳过了2,3两行。若是不想直接跳转到脚本结尾，可为分支命令定义一个要跳转到的标签。

标签最多长7个字符，例子：

chen@ubuntu:~/shell/ch21$ sed '{/first/b jump1 ; s/This is the/No jump on/
> :jump1
> s/This is the/Jump here on/}' data2.txt
No jump on header line.
Jump here on first data line.
No jump on second data line.
No jump on last line.

chen@ubuntu:~/shell/ch21$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

命令b跳转例子

若是文中出现first，程序就跳转到jump1脚本行。若是没有匹配，sed则会继续执行脚本中的命令，包括分支标签后的命令。

chen@ubuntu:~/shell/ch21$ echo "This, is, a, test, to, remove, commas." | sed -n '{
> :start
> s/,//1p
> b start
> }'
This is, a, test, to, remove, commas.
This is a, test, to, remove, commas.
This is a test, to, remove, commas.
This is a test to, remove, commas.
This is a test to remove, commas.
This is a test to remove commas.
^C

命令b用做循环

要能中止循环，只要加上模式匹配在命令b前面就行了

chen@ubuntu:~/shell/ch21$ echo "This, is, a, test, to, remove, commas." | sed -n '{
> :start
> s/,//1p
> /,/b start
> }'
This is, a, test, to, remove, commas.
This is a, test, to, remove, commas.
This is a test, to, remove, commas.
This is a test to, remove, commas.
This is a test to remove, commas.
This is a test to remove commas.

改进例子

4.2 测试指令

相似于分支命令，测试test命令（t）也能够用来改变sed编辑器脚本执行流程。

若是天幻命令成功匹配并替换了一个模式，测试命令就会跳转到指定标签。若是替换命令未能匹配指定模式，测试命令就不会跳转。

测试命令格式与分支命令格式相同：[address]t [label]

和分支命令同样，若是没有标签的状况下，若是测试成功，sed会跳转到脚本结尾。

测试命令基本上就是一个if-then

chen@ubuntu:~/shell/ch21$ sed '{
> s/first/matched/
> t
> s/This is the/No match on/
> }' data2.txt
No match on header line.
This is the matched data line.
No match on second data line.
No match on last line.
chen@ubuntu:~/shell/ch21$ cat data2.txt 
This is the header line.
This is the first data line.
This is the second data line.
This is the last line.

测试命令例子

测试命令的循环方法：

chen@ubuntu:~/shell/ch21$ echo "This, is, a, test, to, remove, commas." | sed -n '{
> :start
> s/,//1p
> t start
> }'
This is, a, test, to, remove, commas.
This is a, test, to, remove, commas.
This is a test, to, remove, commas.
This is a test to, remove, commas.
This is a test to remove, commas.
This is a test to remove commas.

测试命令循环方法

5、模式替代

在使用通配符时，很难知道到底哪些文本会匹配模式。

chen@ubuntu:~/shell/ch21$ echo "The cat sleeps in his hat." | sed 's/cat/"cat"/'The "cat" sleeps in his hat.

chen@ubuntu:~/shell/ch21$ echo "The cat sleeps in his hat." | sed 's/.at/".at"/g'
The ".at" sleeps in his ".at".

通配符没法再引号中使用

5.1 &符号

用&符号能够用来表明替换命令中的匹配的模式。无论匹配出来什么样的文本，均可以使用&符号，来使用这段文本。

myfly2@ubuntu:~/shell/ch21$ echo "The cat sleeps in his hat." | sed 's/.at/"&"/g'
The "cat" sleeps in his "hat".

&符号例子

5.2 替代单独的单词

有时候咱们不须要整个字符串，只想提取字符串的一部分。

sed编辑器用圆括号来定义替换模式中的子模式，而后用特殊字符来引用每一个子模式。

chen@ubuntu:~/shell/ch21$ echo "The System Administrator manual" | sed '
> s/\(System\) Administrator/\1 User/'
The System User manual

chen@ubuntu:~/shell/ch21$ echo "The System Administrator manual" | sed '
s/System \(Administrator\)/\1 User/'
The Administrator User manual

chen@ubuntu:~/shell/ch21$ echo "That furry cat is pretty" | sed 's/furry \(.at\)/\1/'
That cat is pretty
chen@ubuntu:~/shell/ch21$ echo "That furry hat is pretty" | sed 's/furry \(.at\)/\1/'
That hat is pretty
#当须要在两个或多个子模式间插入文本时，这个特性尤为有用
chen@ubuntu:~/shell/ch21$ echo "1234567" | sed '{
> :start
> s/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/
> t start
> }'
1,234,567
#分红两部分：
#.*[0-9]
#[0-9]{3}
#第一个子模式是以数字结尾的任意长度的字符。
#第二个子模式是若干组三位数字

子模式例子

6、在脚本中使用sed

使用包装脚本，sed脚本过程繁琐，若是脚本很长的话。能够将sed编辑器命令放到shell包装脚本中。

这样能够不用每次都键入脚本。

chen@ubuntu:~/shell/ch21$ cat reverse.sh
#!/bin/bash
# Shell wrapper for sed editor script.
#                               to reverse text file lines.
#
sed -n '{ 1!G ; h ; $p }' $1
#
chen@ubuntu:~/shell/ch21$ chmod +x reverse.sh
chen@ubuntu:~/shell/ch21$ ls
data2.txt  data4.txt  reverse.sh
chen@ubuntu:~/shell/ch21$ ./reverse.sh data2.txt
This is the last line.
This is the second data line.
This is the first data line.
This is the header line.

写成文件形式的sed

6.2 重定向sed的输出

chen@ubuntu:~/shell/ch21$ cat fact.sh
#!/bin/bash
# Add commas to number in factorial answer
#
factorial=1
counter=1
number=$1
#
while [ $counter -le $number ]
do
        factorial=$[ $factorial * $counter ]
        counter=$[ $counter + 1 ]
done
#
result=$(echo $factorial | sed '{
:start
s/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/
t start
}')
#
echo "The result is $result"
#
chen@ubuntu:~/shell/ch21$ ./fact.sh 20
The result is 2,432,902,008,176,640,000

能够发sed结果保存

7、建立sed实用工具

7.1 加倍行间距

chen@ubuntu:~/shell/ch21$ sed 'G' data2.txt
This is the header line.

This is the first data line.

This is the second data line.

This is the last line.

chen@ubuntu:~/shell/ch21$ sed '$!G' data2.txt
This is the header line.

This is the first data line.

This is the second data line.

This is the last line.

加倍行间距

7.2 对可能还有空白行的文件加倍行间距

#先删除原来空行，而后再加空行
chen@ubuntu:~/shell/ch21$ sed '/^$/d;$!G' data6.txt
This is line one.

This is line two.

This is line three.

This is line four.
chen@ubuntu:~/shell/ch21$

加倍行间距例子

7.3 给文件中的行编行

chen@ubuntu:~/shell/ch21$ sed '=' data2.txt
1
This is the header line.
2
This is the first data line.
3
This is the second data line.
4
This is the last line.
chen@ubuntu:~/shell/ch21$ sed '=' data2.txt | sed 'N; s/\n/ /'
1 This is the header line.
2 This is the first data line.
3 This is the second data line.
4 This is the last line.
chen@ubuntu:~/shell/ch21$


#=号命令增长行号
#N命令将两行合并成一行
#而后把换行符换成空格

增长行号的脚本

7.4 打印末尾行

chen@ubuntu:~/shell/ch21$ sed '{
:start
$q ; N ; 11,$D
b start
}' data7.txt
This is line 6.
This is line 7.
This is line 8.
This is line 9.
This is line 10.
This is line 11.
This is line 12.
This is line 13.
This is line 14.
This is line 15.
chen@ubuntu:~/shell/ch21$ cat data7.txt
This is line 1.
This is line 2.
This is line 3.
This is line 4.
This is line 5.
This is line 6.
This is line 7.
This is line 8.
This is line 9.
This is line 10.
This is line 11.
This is line 12.
This is line 13.
This is line 14.
This is line 15.

打印行尾

7.5 删除行

删除不须要的空白行，若是有选择的删除空白行，须要一点创造力。

7.5.1 删除连续的空白行

chen@ubuntu:~/shell/ch21$ sed '/./,/^$/!d' data8.txt
This is line one.

This is line two.

This is line three.

This is line four.
chen@ubuntu:~/shell/ch21$ cat data8.txt
This is line one.

This is line two.

This is line three.


This is line four.

删除连续的空白行例子

7.5.2 删除开头的空白行

chen@ubuntu:~/shell/ch21$ cat data9.txt


This is line one.

This is line two.
chen@ubuntu:~/shell/ch21$ sed '/./,$!d' data9.txt
This is line one.

This is line two.

脚本例子

7.5.3 删除结尾的空白行

1 chen@ubuntu:~/shell/ch21$ sed '{
2 > :start
3 > /^\n*$/{$d ; N ; b start}
4 > }' data10.txt
5 This is the first line.
6 This is the seconde line.
7 chen@ubuntu:~/shell/ch21$ cat data10.txt
8 This is the first line.
9 This is the seconde line.

例子

7.6 删除HTML标签