1

I issued the following command, thinking it would take the contents of foo.txt, process it, and write it back to the original file. Instead when I opened the file, it was empty.

cat foo.txt | tr " " "\n" > foo.txt

What exactly is going on in this line, and how can I rephrase it to redirect output back to foo.txt?

Luca
  • 115
  • 4
  • You cannot read and write a file at the same time with pipes and redirections. So write to a temporary file, and as a final step copy it to the original file. Or use a tool that modifies a file "in place". – dirkt Mar 24 '22 at 18:11
  • see https://unix.stackexchange.com/a/186126/65304 – steeldriver Mar 24 '22 at 18:13
  • Your redirect truncates the output file before `tr` ever sees the file. You might try `tmp=$(mktemp); tr " " "\n" < foo.txt; mv -f $tmp foo.txt; rm -f $tmp` – doneal24 Mar 24 '22 at 18:14
  • 2
    Also, [`sponge`](https://manpages.ubuntu.com/manpages/focal/en/man1/sponge.1.html) from the [moreutils package](https://launchpad.net/ubuntu/focal/+package/moreutils): `tr " " "\n" < foo.txt | sponge foo.txt` – glenn jackman Mar 24 '22 at 21:00
  • @doneal24, a couple of things: 1) use `&& mv` instead of `; mv` -- this way, the original file does not get overwritten if the process fails; 2) $tmp no longer exists to `rm`, it has already been moved. – glenn jackman Mar 24 '22 at 21:02
  • @dirkt More accurately you **can** read and write a file at the same time with pipes and redirections - that is the problem here - but the result is unlikely to ever be what you want. Either the file you want is likely to get clobbered, or worse the content will loop round and round the pipe. Eg: `echo hello hello hello > foo` `cat < foo >> foo` – Philip Couling Apr 13 '22 at 13:55

1 Answers1

1

The redirection (>foo.txt) is done by the parent shell before the cat command runs, resetting the file to 0 bytes. Use something like

tr " " "\n" foo.txt >foo.new && \
  mv foo.new foo.txt

Also avoid the Useless Use of cat.

waltinator
  • 4,439
  • 1
  • 16
  • 21
  • 1
    Using `mktemp` for the intermediate file is safer than a fixed file name. – doneal24 Mar 24 '22 at 18:19
  • @doneal24 `mktemp` would keep evil eyes out, but considering the threat environment, and the resulting data (same file, with spaces translated to newlines), is it worth the added complexity? I think not. – waltinator Mar 24 '22 at 22:34
  • You killed an Almost Necessary Use Of Cat. `tr` reads stdin, writes stdout, no filename args exist. But `< foo.txt` is good. – Paul_Pedant Mar 25 '22 at 08:29
  • “Also avoid the Useless Use of cat.” Why? – Guildenstern Apr 05 '23 at 21:43