gawk inplace and stdout

Question

Is it possible to use gawk's -i inplace option and also print things to stdout?

For example, if I wanted to update a file, and if there are any changes print the name of the file and the changed lines to stderr I could do something like

find -type f -name 'myfiles' -exec gawk -i inplace '{if(gsub(/pat/, "repl")) { print FILENAME > "/proc/self/fd/2" ; print > "/proc/self/fd/2"; } print;}' {} +

but is there a way to use stdout instead, or a cleaner way to print that block to the alternate stream?

In this particular case you might want to run something like `find -type f -name 'myfiles' -exec grep -q 'pattern' {} \; -print -exec gawk -i inplace '{do_your_sub_and_print_to_dev/stderr_too}' {} \;` that way you will only use awk to edit the files that actually contain lines matching that pattern. — don_crissti, Nov 08 '16 at 01:14
@don_crissti I often forget how much quicker `grep` can be. My first instinct is to avoid having to "process" the file twice, but I wouldn't be at all surprised if it is indeed faster nonetheless. It just goes to show why I can trust my gut on profiling tasks — Eric Renouf, Nov 08 '16 at 01:19
Yes, it's quite fast and keep in mind that: 1) if there's no match, you only process the file once (the part with `-print -exec gawk` is not executed anymore) and 2) it will stop at 1st match so unless the 1st match is on the last line you're still not processing the whole file twice (it's more like 1.X times) . Also, if `gawk -i inplace` works like `sed -i` it will edit the file in-place anyway - i.e. updating timestamps, inode etc even if there was nothing to edit... — don_crissti, Nov 08 '16 at 01:27
Beware [using gawk -i inplace like that introduces a security vulnerability](/q/749645) unless you modify `$AWKPATH` not to include `.` or make sure you run that command from within a working directory where nobody could create a file called `inplace` or `inplace.awk`. — Stéphane Chazelas, Jun 24 '23 at 21:14

Stéphane Chazelas · Accepted Answer · 2023-06-23T09:13:31.713

You should use /dev/stderr or /dev/fd/2 instead of /proc/self/fd/2. gawk handles /dev/fd/x and /dev/stderr by itself (regardless of whether the system has those files or not).

When you do a:

print "x" > "/dev/fd/2"

gawk does a write(2, "x\n"), while when you do:

print "x" > "/proc/self/fd/2"

since it doesn't treat /proc/self/fd/x specially, it does a:

fd = open("/proc/self/fd/2", O_WRONLY|O_CREAT|O_TRUNC);
write(fd, "x\n");

First /proc/self/fd is Linux specific and on Linux they are problematic. The two versions above are not equivalent when stderr is to a regular or other seekable file or to a socket (for which the latter would fail) (not to mention that it wastes a file descriptor).

That being said, if you need to write to the original stdout, you need to save it away in another fd like:

gawk -i /usr/share/awk/inplace.awk '{
   print "goes into the-file"
   print "to stdout" > "/dev/fd/3"}' the-file 3>&1

gawk does redirect stdout with in-place to the file. It's needed because for instance, you'd want:

awk -i /usr/share/awk/inplace.awk '{system("uname")}' file

to store the uname output into the file.

In any case, do not use -i inplace as gawk tries to load the inplace extension (as inplace or inplace.awk) from the current working directory first, where someone could have planted malware. The path of the inplace extension supplied with gawk may vary with the system, see the output of gawk 'BEGIN{print ENVIRON["AWKPATH"]}'

score 2 · Answer 2 · answered Nov 08 '16 at 03:20

Just for fun, an alternate approach using only POSIX features of find, sh, grep and (most notably) ex:

find . -type f -name 'myfiles' -exec sh -c '
  for f do grep -q "pat" "$f" &&
    { printf "%s\n" "$f";
      printf "%s\n" "%s/pat/repl/g" x | ex "$f";
  }; done' find-sh {} +

(All line breaks in the above command are optional; it can be condensed to a one-liner.)

Or, arguably more legibly:

find . -type f -name 'myfiles' -exec sh -c '
  for f
  do
    if grep -q "pat" "$f"; then
      printf "%s\n" "$f"
      printf "%s\n" "%s/pat/repl/g" x | ex "$f"
    fi
  done' find-sh {} +

:)

(This command is untested; edits welcome if I made any goofs.)

gawk inplace and stdout

2 Answers2

Linked