我已经为此苦苦挣扎了几个小时，我很确定我遗漏了一些东西。
鉴于此 JSON：
[
{
"LAST_JOB_POD":"gitlab-web-65-gwwwh",
"STARTED_AT":"31-05-2018-18:18:48",
"FINISHED":"false",
"FIRST_INDEXED":"0",
"LAST_INDEXED":"3143",
"failed_projects":{
"1082": "4:Deadline Exceeded, trace",
"1273": "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-5.0.3/lib/elasticsearch/transport/transport/base.rb:201:in `__raise_transport_error'",
"2492": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"3060": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)"
}
},
{
"LAST_JOB_POD":"gitlab-web-65-gwwwh",
"STARTED_AT":"31-05-2018-18:18:48",
"FINISHED":"false",
"FIRST_INDEXED":"0",
"LAST_INDEXED":"3143",
"failed_projects":{
"5570": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6103": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6188": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6695": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6721": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6728": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6747": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)"
}
},
{
"LAST_JOB_POD":"gitlab-web-65-gwwwh",
"STARTED_AT":"31-05-2018-18:18:48",
"FINISHED":"false",
"FIRST_INDEXED":"0",
"LAST_INDEXED":"3143",
"failed_projects":{
"6760": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6939": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6941": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6942": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"6947": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)",
"7201": "/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/elasticsearch-transport-5.0.3/lib/elasticsearch/transport/transport/base.rb:201:in `__raise_transport_error'",
"7707": ", trace - [\"/opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/elastic/indexer.rb:64:in `run_indexer!'\"",
"7787": "/opt/gitlab/embedded/lib/ruby/2.3.0/net/protocol.rb:176:in `rbuf_fill': Net::ReadTimeout (Faraday::TimeoutError)"
}
}
]
我目前正在使用jq提取failed_projects条目，但与
[] | select(.failed_projects != null) | . as $object | {"failed_projects"}[]
我得到了不同组的结果：
{
"1082": "...",
...
}
{
"5570": "...",
...
}
{
"6760": "...",
...
}
我想要完成的是将具有相同异常的 ID 分组。例如：
[{
"Exception": "ReadTimeout",
[{
"ID": 2492,
"ID": 3060
}]
},
{
"Exception": "Deadline Exceeded",
[{
"ID": 1082
}]
}]
说明性输出作为 JSON 无效并且具有重复键的对象，这可能不是您真正想要的，但以下 jq 程序将产生符合一般问题描述的输出。由于您似乎没有指定精确的分组标准，因此我将最后一个“：”之后的错误消息文本作为分组标准。（例如，如果您想考虑第一个“:”之后的文本，请使用“^[^:]*: *”作为正则表达式。）
第一步收集.failed_projects并应用，to_entries以便我们可以轻松访问 ID 和错误消息文本：
[.[] | .failed_projects | to_entries[]]
接下来我们提取分组标准，并使用它来形成组：
| map(.value |= sub("^.*: *";""))
| group_by(.value)
最后，我们将组转换为以下形式的 JSON 对象：{GROUP: ARRAY_OF_IDs}：
| map( .[0].value as $key
| [.[] | .key] as $value
| {($key): $value} )
将上述片段放在一个文件 program.jq 中，并使用调用：
jq -f program.jq input.json
产生如下所示的输出。您显然希望修改分组标准。您可能还希望将 ID 字符串转换为 JSON 数字，这可以tonumber通过(tonumber? // .).
要理解 program.jq，您可能希望从第一个片段开始，然后依次添加其他每个片段。
[
{
"Deadline Exceeded, trace": [
"1082"
]
},
{
"TimeoutError)": [
"6728",
"6747",
"6939",
"5570",
"6103",
"6188",
"6695",
"6721",
"2492",
"6760",
"3060",
"6941",
"6942",
"6947",
"7787"
]
},
{
"in `__raise_transport_error'": [
"1273",
"7201"
]
},
{
"in `run_indexer!'\"": [
"7707"
]
}
]
本文收集自互联网，转载请注明来源。
如有侵权，请联系 [email protected] 删除。
我来说两句