60

I'm familiar with "jq" for parsing json.

I work with one service that produces a json response where one of the properties is itself a json string. How do I convert that quoted value to a valid json string so I can then process it with jq?

For instance, if I just view the plain pretty-printed json from "jq .", here's a short excerpt of the output:

"someJsonString": "{\"date\":\"2018-01-08\", ...

I can use jq to get the value of that property, but I need to convert the quoted string to valid json by "unescaping" it.

I suppose I could pipe it into sed, removing the opening and ending double quotes, and removing all backslashes ("sed -e 's/^"//' -e 's/"$//' -e 's/\\//g'"). That seems to work, but that doesn't seem like the most robust solution.

Update:

Just to be a little clearer on what I'm doing, here are a couple of elided samples that show what I've tried:

% curl -s -q -L 'http://.../1524.json' | jq '.results[0].someJsonString' | jq .
"{\"date\":\"2018-01-08\",...
% echo $(curl -s -q -L 'http:/.../1524.json' | jq '.results[0].someJsonString') | jq .
"{\"date\":\"2018-01-08\",...

Update:

Here's a completely standalone example:

% cat stuff.json | jq .
{
  "stuff": "{\"date\":\"2018-01-08\"}"
}
% cat stuff.json | jq '.stuff'
"{\"date\":\"2018-01-08\"}"
% cat stuff.json | jq '.stuff' | jq .
"{\"date\":\"2018-01-08\"}"

Update:

If I tried to process that last output with a real jq expression, it does something like this:

% cat stuff.json | jq '.stuff' | jq '.date'
assertion "cb == jq_util_input_next_input_cb" failed: file "/usr/src/ports/jq/jq-1.5-3.x86_64/src/jq-1.5/util.c", line 371, function: jq_util_input_get_position
Aborted (core dumped)
David M. Karr
  • 891
  • 1
  • 7
  • 16

2 Answers2

87

With jq's fromjson function:

Sample stuff.json contents:

{
  "stuff": "{\"date\":\"2018-01-08\"}"
}

jq -c '.stuff | fromjson' stuff.json

The output:

{"date":"2018-01-08"}
RomanPerekhrest
  • 29,703
  • 3
  • 43
  • 67
  • This seems unecssary. Provided alternative answer – OneCricketeer Apr 27 '18 at 18:33
  • 6
    it is actually nice because you can do directly things like `jq -c '.stuff | fromjson | .date' stuff.json` for example; which you cannot do without the `fromjson` function – Tom Oct 05 '20 at 13:11
  • very nice indeed, it works even for nasty nested cases: `jq -c '.some.where | fromjson | .even.deeper | fromjson' stuff.json | jq . > interesting-part.json` – ojdo Mar 10 '22 at 16:34
44

There is a raw flag for this

    -r      output raw strings, not JSON texts;


jq -rc .stuff stuff.json

Output

{"date":"2018-01-08"}
OneCricketeer
  • 1,229
  • 1
  • 11
  • 14
  • 1
    The difference is that with Roman's answer, you are guaranteed to get valid JSON output, or error messages if it's not valid JSON. – Kusalananda Apr 27 '18 at 18:48
  • Valid point, but if this is being used in automation, I think it would be unusual to suddenly not have valid json output. The most convenient form will be perfectly fine almost all the time. It's still good to know about more precise methods, however. – David M. Karr Apr 28 '18 at 22:30
  • @DavidM.Karr "unusual to suddenly not have valid json output" HA! Riiiight. Error handling in automation? Errors will **never** happen! Why bother! – Bruno Bronosky Feb 06 '19 at 18:21
  • 1
    This requires piping to another `jq` for further JSON processing, whereas with Roman's approach you can continue the same `jq` expression. – Raman Mar 04 '19 at 16:53
  • @Raman I don't think so. You can do `'.stuff.anotherField'` – OneCricketeer Mar 05 '19 at 01:31
  • 3
    @cricket_007: tried it with jq 1.5, and confirmed it doesn't work: `jq -rc '.stuff.date'` produces `jq: error (at :0): Cannot index string with string "date"`. However: `.stuff | fromjson | .date` works fine. – Raman Mar 05 '19 at 07:51
  • @OneCricketeer - `parsejson` isn't "unnecessary", in fact it allows you to *avoid* an unnecessary additional invocation of jq to further process the parsed object. But you can certainly combine both approaches for a useful effect: `jq -r '.stuff | fromjson | .date' – drwatsoncode Jan 21 '21 at 02:29