parse colon separated value pairs, with escaped colon characters in field(s)

Question

I'm trying to process output from nmcli command (colon separated values), but failing to handle / ignore escaped colons in MAC or BSSID. I've tried awk, jq, sed, regex but just not getting it...

piped command:

nmcli -f SSID,BSSID,CHAN,FREQ,SIGNAL,SECURITY -t dev wifi

example output:

WiFi:00\:0A\:97\:6F\:7B\:AC:165:5660 MHz:15:

If I move BSSID to the end of the line / field's list (and then just cut the last field) it'll work, but now it's a matter a pride. Please help!

I did find this existing stackoverflow thread, which really went about it nicely, but I'm ultimatly running into the same problem of how to ignore the escaped colon in the MAC!@%$ address: parse colon separated value pairs

Most recent (readable) attempt:

printf '%s' "$(nmcli -f SSID,BSSID,CHAN,FREQ,SIGNAL,SECURITY -t dev wifi)" | jq -sR 'split("\n") | map(split(":")) | map({"SSID": .[0], "BSSID": .[1], "CHAN": .[2], "RATE": .[3], "FREQ": .[4], "SIGNAL": .[5], "SECURITY": .[6]})'

and problem output:

...
{
"SSID": "WiFi",
"BSSID": "00\\",
"CHAN": "0A\\",
"RATE": "97\\",
"FREQ": "6F\\",
"SIGNAL": "7B\\",
"SECURITY": "AC"
},
...

Would also need JQ to leave colons in place and not escape as shown above, if at all possible, ... please don't just say python...

`read` will do it, as will `cut`. split on colons, then ask for fields `1, 2-7, 8,9,10, and 11`. — Jeff Schaller, Mar 21 '18 at 00:43
Please don't make more work for other people by vandalizing your posts. By posting on the Stack Exchange (SE) network, you've granted a non-revocable right, under the [CC BY-SA 3.0 license](https://creativecommons.org/licenses/by-sa/3.0) for SE to distribute that content. By SE policy, any vandalism will be reverted. If you want to know more about deleting a post, consider taking a look at: [How does deleting work?](https://meta.stackexchange.com/q/5221) — iBooot, May 12 '19 at 20:25

fearphage · Answer 1 · 2018-04-12T13:23:22.987

A few notes here:

You can prevent nmcli from escaping the fields with --escape/-e no
You don't have to quote json field names (without special characters)

SSID and Security don't always get returned so I added a placeholder (--)

printf '%s' "$(nmcli -f SSID,BSSID,CHAN,FREQ,SIGNAL,SECURITY -e no -t dev wifi)" |
  jq -sR 'split("\n") |
    map(split(":") |
    {SSID: (if .[0] == "" then "--" else .[0] end), 
      BSSID: .[1:6] | join(":"),
      CHAN: .[7],
      FREQ: .[8],
      SIGNAL: .[9],
      SECURITY: (if .[10]  == "" then "--" else .[10] end)})'

steeldriver · Answer 2 · 2018-03-21T01:06:00.397

2

Perl allows you to construct a field separator with lookbehind - and also handily provides a JSON module. So you could do something like

nmcli -f SSID,BSSID,CHAN,FREQ,SIGNAL,SECURITY -t dev wifi | 
  perl -F'(?<!\\):' -MJSON=encode_json -lne 'print encode_json([map { s/\\:/:/g ; $_ } @F])'

Testing with your example nmcli output:

printf 'WiFi:00\:0A\:97\:6F\:7B\:AC:165:5660 MHz:15:' | 
  perl -F'(?<!\\):' -MJSON=encode_json -lne 'print encode_json([map { s/\\:/:/g ; $_ } @F])'
["WiFi","00:0A:97:6F:7B:AC","165","5660 MHz","15"]

edited Mar 21 '18 at 01:06

answered Mar 21 '18 at 00:55

steeldriver

78,509
12
109
152

1

@LordVance FWIW on Ubuntu, I simply had to install the `libjson-perl` package – steeldriver Mar 21 '18 at 01:07

agc · Answer 3 · 2018-04-23T07:22:06.537

2

Using sed to clean the data, and mlr to turn it into JSON:

h='SSID,BSSID,CHAN,FREQ,SIGNAL,SECURITY'
nmcli -f $h -t dev wifi | 
sed 's/^:/"",/;s/\([^\]\):/\1,/g;s/[\]:/:/g' | 
mlr --c2j --jvstack  --jlistwrap  label $h

Output example:

{
  "SSID": "FooBarBaz",
  "BSSID": "13:F6:8C:56:9B:3B",
  "CHAN": 36,
  "FREQ": "5180 MHz",
  "SIGNAL": 20,
  "SECURITY": "WPA1 WPA2"
},

edited Apr 23 '18 at 07:22

answered Apr 23 '18 at 04:46

agc

7,045
3
23
53

score 1 · Answer 4 · answered Mar 21 '18 at 00:42

GNU awk can handle fields as sequences of either escaped characters or non-colon-nor-backslash characters (see this answer), so the following should work. Not very efficient, but at least a solution...

#!/bin/bash
line="WiFi:00\:0A\:97\:6F\:7B\:AC:165:5660 MHz:15:"
# line=`nmcli -f SSID,BSSID,CHAN,FREQ,SIGNAL,SECURITY -t dev wifi`
for i in 1 2 3 4 5; do
    echo $line | gawk -v FPAT='(\\\\.|[^\\\\:])*' '{print $'$i'}' 
done

parse colon separated value pairs, with escaped colon characters in field(s)

4 Answers4