Linux下解析和格式化输出JSON的命令行工具 - jq

 

JSON 是一种轻量级且与语言无关的数据存储格式，易于与大多数编程语言集成，也易于理解 。虽然它以 JavaScript 开头，并且主要用于在服务器和浏览器之间交换数据，但如今正在用于许多领域，包括嵌入式系统。JSON是前端编程常常用到的格式，对于PHP或者Python，解析JSON都不是什么大事，尤为是PHP的json_encode和json_decode,干的至关的漂亮。Linux下也有处理JSON的神器：jq。对于JSON格式而言，jq就像sed/awk/grep这些神器同样的方便，jq没有乱七八糟的依赖，只须要一个binary文件jq就能够了。Linux 上使用命令行工具jq来解析并格式化打印 JSON，它对于在 shell 脚本中处理大型 JSON 数据或在 shell 脚本中处理 JSON 数据很是有用。

1）安装jq工具

centos7能够直接yum安装jq
[root@k8s-master01 ~]# yum install -y jq
[root@k8s-master01 ~]# jq --version
jq-1.5

2) 使用jq
简单的来讲，jq是linux下一个解析json格式数据的实用工具，能够将在linux下一团糟的json数据按格式输出，还能够直接选中关键字打印出它的值。

JSON 数据可能放在一行上令人难以解读，所以为了使其具备必定的可读性，JSON 格式化输出就可用于此目的的。以下示例：

[root@k8s-master01 ~]# wget -cq http://jsonip.com/ -O -
{"ip":"114.242.193.201","about":"/about","Pro!":"http://getjsonip.com","reject-fascism":"Pro abortion is pro life and pro rights"}

如今使用jq格式化输出它们：
[root@k8s-master01 ~]# wget -cq http://jsonip.com/ -O - | jq '.'
{
  "ip": "114.242.193.201",
  "about": "/about",
  "Pro!": "http://getjsonip.com",
  "reject-fascism": "Women make their own choices. Support abortion rights."
}

jq 的基本过滤和标识符功能
jq 能够从 STDIN 或文件中读取 JSON 数据。你能够根据状况使用。
单个符号 . 是最基本的过滤器。这些过滤器也称为对象标识符-索引。jq 使用单个 . 过滤器基本上至关将输入的 JSON 文件格式化输出。
◈ 单引号：没必要始终使用单引号。可是若是你在一行中组合几个过滤器，那么你必须使用它们。
◈ 双引号：你必须用两个双引号括起任何特殊字符，如 @、＃、$，例如 jq .foo.”@bar”。
◈ 原始数据打印：无论出于任何缘由，若是你只须要最终解析的数据（不包含在双引号内），请使用带有 -r 标志的 jq 命令，以下所示：jq -r .foo.bar。

jq能够解析特定数据
要过滤出 JSON 的特定部分，你须要了解格式化输出的 JSON 文件的数据层次结构。以下示例

[root@k8s-master01 ~]# cat test.json
[{"hostCompany":"Beijing Autelan Technology","hostModel":"CS-VIC-2000-C","hostsn":"01010730b12014A00477","mac":"00:1F:64:CE:F3:8E","cpuModel":"MIPS 74Kc V4.12","cpuSN":"000000","memoryModel":"abcdefg","memorySN":"000000","boardSN":"01010730b12014A00477","networkCardMac":"00:1F:64:CE:F3:8F","lowFreModel":"AR9344","lowFreSN":"000000","hignFreModel":"AR9582","hignFreSN":"000000","gpsModel":"abcdefg","gpsSN":"000000","MEID_3g":"A000004E123ABD2","Company_3g":"ZTEMT INCORPORATED","modelOf3g":"MC271X","snOf3g":"A000004E123ABD2","iccid":"89860314400200885980","Operators":"CTCC","hardVersion":"1.20","firmwareVersion":"1.0.6.29"}]


[root@k8s-master01 ~]# jq . test.json
[
  {
    "hostCompany": "Beijing Autelan Technology",
    "hostModel": "CS-VIC-2000-C",
    "hostsn": "01010730b12014A00477",
    "mac": "00:1F:64:CE:F3:8E",
    "cpuModel": "MIPS 74Kc V4.12",
    "cpuSN": "000000",
    "memoryModel": "abcdefg",
    "memorySN": "000000",
    "boardSN": "01010730b12014A00477",
    "networkCardMac": "00:1F:64:CE:F3:8F",
    "lowFreModel": "AR9344",
    "lowFreSN": "000000",
    "hignFreModel": "AR9582",
    "hignFreSN": "000000",
    "gpsModel": "abcdefg",
    "gpsSN": "000000",
    "MEID_3g": "A000004E123ABD2",
    "Company_3g": "ZTEMT INCORPORATED",
    "modelOf3g": "MC271X",
    "snOf3g": "A000004E123ABD2",
    "iccid": "89860314400200885980",
    "Operators": "CTCC",
    "hardVersion": "1.20",
    "firmwareVersion": "1.0.6.29"
  }
]


[root@k8s-master01 ~]# jq .[] test.json
{
  "hostCompany": "Beijing Autelan Technology",
  "hostModel": "CS-VIC-2000-C",
  "hostsn": "01010730b12014A00477",
  "mac": "00:1F:64:CE:F3:8E",
  "cpuModel": "MIPS 74Kc V4.12",
  "cpuSN": "000000",
  "memoryModel": "abcdefg",
  "memorySN": "000000",
  "boardSN": "01010730b12014A00477",
  "networkCardMac": "00:1F:64:CE:F3:8F",
  "lowFreModel": "AR9344",
  "lowFreSN": "000000",
  "hignFreModel": "AR9582",
  "hignFreSN": "000000",
  "gpsModel": "abcdefg",
  "gpsSN": "000000",
  "MEID_3g": "A000004E123ABD2",
  "Company_3g": "ZTEMT INCORPORATED",
  "modelOf3g": "MC271X",
  "snOf3g": "A000004E123ABD2",
  "iccid": "89860314400200885980",
  "Operators": "CTCC",
  "hardVersion": "1.20",
  "firmwareVersion": "1.0.6.29"
}


[root@k8s-master01 ~]# jq -r .[].mac test.json
00:1F:64:CE:F3:8E

[root@k8s-master01 ~]# jq -r '.[] | .mac' test.json
00:1F:64:CE:F3:8E

[root@k8s-master01 ~]# jq -r '.[] |.mac, .gpsSN' test.json
00:1F:64:CE:F3:8E
000000

[root@k8s-master01 ~]# jq -r '.[].mac, .[].gpsSN' test.json
00:1F:64:CE:F3:8E
000000