How to refer to matched groups in jq gsub?

Question

Using jq, I want to search for a pattern via regex, and wrap the matched string with something like <div> tags

$ echo "\"This is a valid json file"\" | jq '. | gsub("valid";"how_to_refer_to_matches";"i") -

How to refer to the matched results in the second argument of gsub?
What if there are more than 1 matches?

Kusalananda · Accepted Answer · 2021-07-09T19:50:38.610

8

$ jq -n '"some string" | gsub("(?<a>[[:lower:]]+)"; "<div>\(.a)</div>")'
"<div>some</div> <div>string</div>"

or

$ jq -n '"some string" | gsub("(?<a>[[:lower:]]+)"; "<div>" + .a + "</div>")'
"<div>some</div> <div>string</div>"

This replaces each run of lowercase letters with the same letters, flanked by <div> and </div>. The capture is done with (?<a>RE) where a is a key name that you can refer to in the replacement using .a (it does not need to be a single letter), and where RE is some expression. You may capture multiple groups with different key names at the same time.

gsub() works like e.g. gsub() in awk and like s///g in Perl, sed, and ed, so it will perform the substitution once for each non-overlapping match of the expression. This is illustrated in my example above where there are two matches of the pattern in the string.

Just to use a more similar example as you:

$ jq -n '"this is a valid JSON string" | gsub("(?<match>valid)"; "<div>" + .match + "</div>")'
"this is a <div>valid</div> JSON string"

edited Jul 09 '21 at 19:50

answered Jul 09 '21 at 19:36

Kusalananda

320,670
36
633
936

Thanks! Can you point me where in the docs this notation : (?RE) is mentioned? https://stedolan.github.io/jq/manual/#RegularexpressionsPCRE – Zeta.Investigator Jul 09 '21 at 19:45
1

@Zeta.Investigator Look at the documentation for `gsub`. It refers to the documentation of `sub` (which only handles the _first_ match on a line), which in turn mentions the `capture` function. What you are looking for is in the documentation of `sub` and `capture`. – Kusalananda Jul 09 '21 at 19:47
Thanks https://stedolan.github.io/jq/manual/#example82 – Zeta.Investigator Jul 09 '21 at 19:48
What would be the difference between `gsub` and `sub(;;"g")` (with global switch) ? Should I open another question? – Zeta.Investigator Jul 09 '21 at 19:50
1

@Zeta.Investigator Functionally, it would be no difference at all. It would just be slightly more obfuscated code which would be harder to understand. – Kusalananda Jul 09 '21 at 19:52
One more thing: `\(.a)` is the whole match. How can I get the groups? Due to the docs, it should be key `captures` of the match. So `\(.a.captures[].string)`. But this doesn't work. For example `gsub("(?(123.*321))";...)` How can I get only `.*` excluding 123 and 321 around it? – Zeta.Investigator Jul 09 '21 at 20:40
1

@Zeta.Investigator No, `.a` is only the group that you matched with `(?RE)`. If the expression RE covers the whole match, then `.a` is the whole match. For another example, try `(?.).(?.)` and replace with e.g. `\(.b)\(.a)`. This would transform `test string` into `ststitng`, just like in `echo 'test string' | sed 's/\(.\).\(.\)/\2\1/g'`. – Kusalananda Jul 09 '21 at 20:51
1

@Zeta.Investigator Using your example from your last comment (which I didn't see at first, sorry), you would write the expression as `123(?.*)321`. – Kusalananda Jul 09 '21 at 20:52

How to refer to matched groups in jq gsub?

1 Answers1