Why does awk concatenate strings from different lines

Question

Good day.

I'm trying to run a ping6 on IPv6 addresses pulled from /etc/resolv.conf.dnsph. It seems though, as if awk is stringing the IPv6 addresses onto one line.

$ grep ^nameserver /etc/resolv.conf.dnsph |awk '{print $2}'
2001:1890:1001:2224::1
2001:1890:1001:2424::1
$ ping6 $(grep ^nameserver /etc/resolv.conf.dnsph |awk '{print $2}')
ping6: Source routing is deprecated by RFC5095.
Usage: ping6 [-aAbBdDfhLnOqrRUvV] [-c count] [-i interval] [-I interface]
             [-l preload] [-m mark] [-M pmtudisc_option]
             [-N nodeinfo_option] [-p pattern] [-Q tclass] [-s packetsize]
             [-S sndbuf] [-t ttl] [-T timestamp_option] [-w deadline]
             [-W timeout] destination
$ echo $(grep ^nameserver /etc/resolv.conf.dnsph |awk '{print $2}')
2001:1890:1001:2224::1 2001:1890:1001:2424::1

Adding "END {printf "\n"}" doesn't make any difference:

$ ping6 $(awk '/^nameserver/ {print $2}; END {printf "\n"}' /etc/resolv.conf.dnsph)
ping6: Source routing is deprecated by RFC5095.
Usage: ping6 [-aAbBdDfhLnOqrRUvV] [-c count] [-i interval] [-I interface]
             [-l preload] [-m mark] [-M pmtudisc_option]
             [-N nodeinfo_option] [-p pattern] [-Q tclass] [-s packetsize]
             [-S sndbuf] [-t ttl] [-T timestamp_option] [-w deadline]
             [-W timeout] destination
$ echo $(awk '/^nameserver/ {print $2}; END {printf "\n"}' /etc/resolv.conf.dnsph)
2001:1890:1001:2224::1 2001:1890:1001:2424::1

Please advise.

Bjoern

See [Why does my shell script choke on whitespace or other special characters?](https://unix.stackexchange.com/q/131766) and [Security implications of forgetting to quote a variable in bash/POSIX shells](https://unix.stackexchange.com/q/171346). Stephen gave you the answer, but you wouldn't have had the question if you had quoted your variable. Try with `echo "$(grep ^nameserver /etc/resolv.conf.dnsph |awk '{print $2}')"` to see. — terdon, May 11 '23 at 18:43

Stephen Kitt · Accepted Answer · 2023-05-11T18:49:49.777

4

When it’s given multiple addresses, ping6 interprets them as hops to use to reach the target (the last address). In your case, you need to loop over the output:

for ip in $(awk '/^nameserver/ { print $2 }' /etc/resolv.conf.dnsph); do
    ping6 "$ip"
done

AWK correctly outputs each IP address on a single line (as can be seen in your first command), but unquoted command substitution causes the newlines to be effectively removed — they act as separators between words.

edited May 11 '23 at 18:49

answered May 11 '23 at 18:36

Stephen Kitt

411,918
54
1,065
1,164

Thank you Stephen! Works like a charm. I thought I had previously seen someone mention that awk does not print a newline, but can be done by appending `END {print "\n"}` – Bjoern May 11 '23 at 20:36
1

@Bjoern: awk `print` does add newline by default; to be exact, it _always_ adds ORS (Output Record Separator) and ORS _defaults_ to newline. `printf` does not add anything (only what you specify in the format string plus argument(s)). Either you misunderstood 'someone' or 'someone' was wrong. – dave_thompson_085 May 12 '23 at 01:01

score 0 · Answer 2 · answered May 12 '23 at 06:03

The $(...) construct is intended to provide command line parameters to commands. Command line parameters are separated by spaces, not new lines, so the shell "optimizes" the output. However you can quote $(...), like this:

> echo $(echo 1;echo 2)
1 2
> echo "$(echo 1;echo 2)"
1
2

Still the problem has nothing to do with awk concatenating lines (i.e.: it does not), but the use of $(...). Actually awk can also do the grep job using awk '/^nameserver/ {print $2}' /etc/resolv.conf.dnsph, and you can also avoid the shell for loop by calling ping from within awk:

awk '/^nameserver/ { system("ping6 " $2); }' /etc/resolv.conf.dnsph

word splitting isn't specific to command substitution though. But also, quoting the command substitution here would give as an argument a single string that contains a newline in the middle, and that's also likely not too useful for `ping6` (or many other tools) — ilkkachu, May 12 '23 at 07:13

Why does awk concatenate strings from different lines

2 Answers2