I am reading the AWS eic_harvest_hostkeys script and I don't understand this line:
key=$(/usr/bin/awk '{$1=$1};1' < "${file}")
What is the of benefit awk?
Isn't key=$(/bin/cat "${file}") better?
Asked
Active
Viewed 1,894 times
7
K-attila-
- 624
- 2
- 13
-
1See: [How do I trim leading and trailing whitespace from each line of some output?](//unix.stackexchange.com/q/102008) – Stéphane Chazelas Feb 11 '22 at 10:53
-
It's not that there's a **benefit** of one over the other, they just do different things. – Ed Morton Feb 11 '22 at 19:13
-
Use of cat: `key=$(< "${file}")` does the same thing as `key=$(/bin/cat "${file}")` – Wastrel Feb 12 '22 at 15:19
-
3It seems a bit *awk*ward. – LeopardShark Feb 12 '22 at 15:55
-
2@Wastrel not quite. When `file=-x`, i.e. it's referencing a filename that begins with a dash, the `key=$(<"$file")` works correctly whereas `key=$(cat "$file")` will produce an error such as `cat: unknown option -- x` with many (all?) implementations of `cat` – roaima Feb 12 '22 at 18:40
-
1@roaima Ah, so that's why I don't use cat! – Wastrel Feb 13 '22 at 14:07
3 Answers
17
key=$(/bin/cat "${file}") sets key to the content of the file¹, without its trailing newlines. (I'm assuming a text file, without null bytes.)
key=$(/usr/bin/awk '{$1=$1};1' < "${file}") normalizes horizontal whitespace: it removes spaces and tabs at the beginning and at the end of each line, and it compressed internal sequences of spaces and tabs to a single space. While $1=$1 looks like a no-op, it actually causes awk to rebuild the line based on the updated field, and rebuilding the line erases the information of exactly what whitespace was around fields. It then also removes trailing newlines.
¹ assuming $file doesn't start with a - which would cause cat to treat it as an option argument. Using key=$(cat < "$file") would be better (key=$(cat -- "$file") would also be an improvement but file='-' would still be a problem).
Stéphane Chazelas
- 522,931
- 91
- 1,010
- 1,501
Gilles 'SO- stop being evil'
- 807,993
- 194
- 1,674
- 2,175
-
1@Wastrel - tell that to the asker. Gilles is just explaining what was presented in the question. – Toby Speight Feb 12 '22 at 15:17
-
-
The `cat` version does not strip or convert newlines. Of course, that goes along with the general thrust of this answer, that the `awk` version modifies data passing through, whereas the `cat` version does not. – John Bollinger Feb 13 '22 at 16:39
-
1@JohnBollinger The command substitution strips trailing newlines. Both commands preserve other newlines. – Gilles 'SO- stop being evil' Feb 13 '22 at 19:42
12
The assignment to $1 forces awk to rewrite the input line to a canonical format.
{ echo 'one two three'; echo ' indented with trailing '; } | cat
one two three
indented with trailing
{ echo 'one two three'; echo ' indented with trailing '; } | awk '{$1=$1}; 1'
one two three
indented with trailing
Furthermore, when file=-x, i.e. it's referencing a filename that begins with a dash, using key=$(<"$file") will work correctly whereas key=$(cat "$file") will produce an error such as cat: unknown option -- x with many (all?) implementations of cat.
roaima
- 107,089
- 14
- 139
- 261
-
6Note that the `;` in between `{$1=$1}` and `1` which (contrary to the OP) you omitted is required in the standard `awk` syntax (a newline would also do), though in practice, it's not needed in any `awk` implementations that I know (still, the maintainer of gawk objected to relaxing that requirement in the standard). – Stéphane Chazelas Feb 11 '22 at 11:01
-
@StéphaneChazelas thank you. I wasn't aware of the `;` being required by the standard – roaima Feb 11 '22 at 11:23
0
even that's too verbose already. just do :
[m/n/g]awk NF=NF
don't even need any quotation at shell level,
since the first NF on left side prevents the shell from attempting to do anything with that equal = sign
RARE Kpop Manifesto
- 88
- 6