12

I can print the first line of a file using

sed -n 1p file.txt

I can delete the first line of a file using

sed -i 1d file.txt

Is there a way to print-and-delete in sed? Kind of like a FIFO pop() or Perl's shift.

Andrew Cheong
  • 673
  • 2
  • 9
  • 23

3 Answers3

11

You can use the command w to write some lines to a different output file. On Linux and many other unix variants, /dev/stdout is the program's standard output, which isn't where sed writes with the -i option. If your system doesn't have /dev/stdout or a variant such as /dev/fd/1, you can write to a named pipe.

sed -i -e '1 w /dev/stdout' -e '1d' file.txt
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • Note that it's regardless of whether the system supports `/dev/stdout` or not. GNU `sed` (which you need for `-i`) treats `/dev/stdout` specially, it doesn't open the `/dev/stdout` file. If it did and stdout went to a file, your command would overwrite that file on Linux (where `/dev/stdout` is implemented as a symlink). – Stéphane Chazelas Jan 14 '14 at 14:21
  • [`info --index-search='w (write file) command' sed`](http://www.gnu.org/software/sed/manual/html_node/Other-Commands.html#index-Write-to-a-file-151) for details – Stéphane Chazelas Jan 14 '14 at 14:27
  • Just what I was looking for too. – Sridhar Sarnobat Oct 16 '16 at 08:59
  • @Stéphane-Chazelas how do you know all of this? There is like no tool you do not know down to tge last bit. – Bananguin May 07 '18 at 19:06
  • Why bother with stdout at all. The portion '1 w /dev/stdout' can be replaced by '1 w file.txt', no need to redirect at the end. – pauljohn32 Jul 26 '20 at 02:20
  • @pauljohn32 This works if you have a hard-coded file name, but not if it's coming down in a shell variable and you can't be sure that it doesn't contain whitespace or other special characters. I prefer to show more robust code. – Gilles 'SO- stop being evil' Jul 26 '20 at 19:51
  • Thanks for the explanation. Appears to me a shell variable does work for the outfile name (and path) if you change the single quote -e to double quote. But I also confirm your suggestion to use stdout works on Ubuntu 20.04 – pauljohn32 Jul 27 '20 at 14:22
6

This answer is an extension of Gilles' answer.

Needing to write the address twice is not very DRY, and while it works fine for this example of just deleting the first line, that method becomes more problematic if you have a more complicated search pattern for lines you want to "extract" from a file. Instead of writing the address twice, you can use curly braces to do both the w and d commands on the same address. Using the original example, that would look like this:

sed -i -e '1{w /dev/stdout' -e 'd}' file.txt

or more generally:

sed -i -e '/some long regex/{w /dev/stdout' -e 'd}' file.txt
dhakimian
  • 61
  • 1
  • 4
  • You are correct about the semicolon; edited it out. However, even with the ability to reuse the last regex, I think that if you have multiple commands to run on the same address, it makes more sense conceptually to have them in a block under that address. – dhakimian May 07 '18 at 18:51
  • Can anybody explain the syntax here, where the quotes divide the parameter in halves? I don't see why second -e 'd}' works because it does not have opening squiggly brace. Or why first quoted part works without a closing '}'. – pauljohn32 Jul 26 '20 at 02:19
  • 2
    The documentation for `-e` says: " '-e SCRIPT' Add the commands in SCRIPT to the set of commands to be run while processing the input." So, they're not standalone commands that are executed sequentially but separately, they are instead all run together in the same context, just like if they were on different lines in a script file specified with `-f`, or separated with semicolons. This pattern is used with commands like `w` and `a` that can't really be terminated any other way in a one-liner than ending the expression and starting a new one. – dhakimian Jul 27 '20 at 16:37
0

Unfortunately this is not possible. Essentially, you are asking for the output from sed to go to two different places: you'd like the first line to be printed to the stdout of sed, and for the remaining lines to be placed back in the file itself. But the output of sed can only go to one place (by default it goes to stdout, and if you specify the -i option it goes back into the file).

Fortunately, I can't see any downsides from running the two commands separately.

AGWA
  • 579
  • 5
  • 3