how to capture the properties tag from json file

Question

this is very short list of my json file:

  "slider-client" : {
    "properties_attributes" : { },
    "properties" : { }
  }
},
{
  "spark2-hive-site-override" : {
    "properties_attributes" : { },
    "properties" : {
      "hive.metastore.client.connect.retry.delay" : "5",
      "hive.server2.enable.doAs" : "false",
      "hive.server2.thrift.port" : "10016",
      "hive.server2.transport.mode" : "binary",
      "hive.metastore.client.socket.timeout" : "1800"
    }
  }
},
{
  "tez-env" : {
    "properties_attributes" : { },
    "properties" : {
      "heap_dump_location" : "/tmp",
      "content" : "\n# Tez specific configuration\nexport TEZ_CONF_DIR={{config_dir}}\n\n# Set HADOOP_HOME to point to a specific hadoop install directory\nexport HADOOP_HOME=${HADOOP_HOME:-{{hadoop_home}}}\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}",
      "enable_heap_dump" : "false",
      "tez_user" : "tez"
    }
  }
},

how we can capture only the lines that start with:

"properties" : {

and ended with

example of expected output

    "properties" : { }
    "properties" : {
      "hive.metastore.client.connect.retry.delay" : "5",
      "hive.server2.enable.doAs" : "false",
      "hive.server2.thrift.port" : "10016",
      "hive.server2.transport.mode" : "binary",
      "hive.metastore.client.socket.timeout" : "1800"
    }

    "properties" : {
      "heap_dump_location" : "/tmp",
      "content" : "\n# Tez specific configuration\nexport TEZ_CONF_DIR={{config_dir}}\n\n# Set HADOOP_HOME to point to a specific hadoop install directory\nexport HADOOP_HOME=${HADOOP_HOME:-{{hadoop_home}}}\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}",
      "enable_heap_dump" : "false",
      "tez_user" : "tez"
    }

what I did until now is this syntax , is it the best approach ?

 awk '/"properties" : {/,/^[[:blank:]]*}$/'  file.json

Your JSON excerpt is not valid JSON, important parts are missing. The simplest way to do what you want is use to use `jq`, but without knowing the exact JSON format, we can't make a ready-to-use answer. Have you tried reading the `jq` man page, and attempting it yourself? — dirkt, Aug 16 '17 at 07:21
yes I read the jq but after few tests I cant to find the right way — jango, Aug 16 '17 at 07:27
There are plenty of questions here about parsing a JSON file and sensible answers will always be based on a JSON parser, not on `sed`/`awk`. Unless of course you have full control on the JSON output and thus can avoid all kind of side-effects, which you didn't state in your question. — xhienne, Aug 17 '17 at 12:21

score 1 · Answer 1 · answered Aug 16 '17 at 08:16

1

To be a valid JSON your input should be considered as an array of objects.

jq solution:

jq '.[] | to_entries[] | .value | if has("properties") then {"properties": .properties} else empty end' yourfile

The output (corrected):

{
  "properties": {}
}
{
  "properties": {
    "hive.metastore.client.connect.retry.delay": "5",
    "hive.server2.enable.doAs": "false",
    "hive.server2.thrift.port": "10016",
    "hive.server2.transport.mode": "binary",
    "hive.metastore.client.socket.timeout": "1800"
  }
}
{
  "properties": {
    "heap_dump_location": "/tmp",
    "content": "\n# Tez specific configuration\nexport TEZ_CONF_DIR={{config_dir}}\n\n# Set HADOOP_HOME to point to a specific hadoop install directory\nexport HADOOP_HOME=${HADOOP_HOME:-{{hadoop_home}}}\n\n# The java implementation to use.\nexport JAVA_HOME={{java64_home}}",
    "enable_heap_dump": "false",
    "tez_user": "tez"
  }
}

answered Aug 16 '17 at 08:16

RomanPerekhrest

29,703
3
43
67

I run it but we get this "jq: error: Cannot check whether string has a string key" – jango Aug 16 '17 at 08:20
1

@jango, I repeat (as I wrote it in my answer): **To be a valid JSON your input should be considered as an array of objects** – RomanPerekhrest Aug 16 '17 at 08:21
sorry but seems my json not include array of objects so what we can do with this? – jango Aug 16 '17 at 08:26
1

@jango, *what we can do*? that means that your input is not a valid json or not a json at all. Do restructure your input to make it valid json, or don't call it *json* – RomanPerekhrest Aug 16 '17 at 08:28
what you think about this - awk '/"properties" : {/,/^[[:blank:]]*}$/' file.json – jango Aug 16 '17 at 08:29
1

@jango: Please, as has been mentioned multiple times [edit](https://unix.stackexchange.com/posts/386350/edit) your question and include a valid and representative example. Also read and follow [these](https://stackoverflow.com/help/mcve) instructions – Thor Aug 16 '17 at 08:30
see my update in the question , I write my solution but not sure that this is the right syntax – jango Aug 16 '17 at 08:32
what you think about ny solution ? – jango Aug 16 '17 at 08:34
@Thor do you have opinion about my solution ? – jango Aug 16 '17 at 08:39
1

@jango: You do not listen, so no opinion – Thor Aug 16 '17 at 08:40
but I give here awk solution , why awk not good for this? – jango Aug 16 '17 at 08:43
@RomanPerekhrest any opinion about my solution ? – jango Aug 16 '17 at 08:44

score -1 · Answer 2 · answered Aug 16 '17 at 08:41

-1

to capture the properties until "}" we can do:

awk '/"properties" : {/,/^[[:blank:]]*}$/'  file.json

answered Aug 16 '17 at 08:41

jango

403
2
13
18

I will happy to get any comments ? – jango Aug 16 '17 at 08:42
awk/sed are not the right tools to work with json at all – RomanPerekhrest Aug 16 '17 at 08:55
yes but this awk do the job very well – jango Aug 16 '17 at 08:56
2

it can be called an arbitrary happy case, but not robust solution. It should not be advised to other people/community – RomanPerekhrest Aug 16 '17 at 08:57
Does it handle well properties with value `{ }`? – xhienne Aug 17 '17 at 12:18

how to capture the properties tag from json file

2 Answers2