40

I'm so confused with GNU sed, POSIX sed, and BSD sed's myriad of options. How do I replace literal \n with the newline character using these three sed types?

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
Avinash Raj
  • 3,653
  • 4
  • 20
  • 34

5 Answers5

58
sed 's/\\n/\
/g'

Notice the backslash just before hitting return in the replacement string.

unxnut
  • 5,908
  • 2
  • 19
  • 27
  • does this works for all kind of sed's? – Avinash Raj Jul 04 '14 at 13:44
  • I believe it should; I tested only on Linux using gnu version. – unxnut Jul 04 '14 at 14:07
  • 11
    @AvinashRaj. That's POSIX and worked with the original `sed` command in Unix v7 in 1979. The only place where that might not work would be non-POSIX stripped-down implementations of `sed` like some stripped-down busybox-based ones. That won't work in csh but it's a csh issue. – Stéphane Chazelas Jul 04 '14 at 14:13
  • 1
    Since `\\n` is used in the search portion, why not use `\n` in the replace ? the former seems to imply the latter exists. ie. why not this `$ echo '1\n2'|sed 's/\\n/\n/g'` –  Sep 06 '18 at 19:24
  • In this case, `\n` is a literal, implying that it appears as the two characters: abackslash followed by `n`, and not as a single newline character. Hence,two backslashes for making backslash a literal and then `n` – unxnut Sep 06 '18 at 23:23
  • This solution adds a trailing `\n` – krish7919 Feb 17 '19 at 12:06
  • For POSIX, if you need to change '\n' --> NEWLINE you should use 2 steps: cat file | sed 's/\\n/±/g' | sed $'s/±/\\\n/g' . Note '±' is random character, not used in stream. – Andzej Wal Jul 12 '22 at 15:02
  • @AndzejWal Note that `$'...'` is not POSIX. – Kusalananda Oct 04 '22 at 16:55
  • @krish7919 It adds a trailing newline if there isn't already one there. This is due to `sed` always outputting text with no non-terminated lines. – Kusalananda Oct 04 '22 at 16:57
23

Since you already have your portable sed answer, I'll just mention that sed is rarely the best tool if you need to somehow manipulate newlines. Perl is very nearly as portable as sed, is installed by default on most *nix systems and can deal with this very easily:

printf '%s\n' 'aa\nbb' | perl -pe 's/\\n/\n/g'

Another very portable choice is awk:

printf '%s\n' 'aa\nbb' | awk  '{gsub("\\\\n","\n")};1'

On Solaris, remember to use the standard awk in /usr/xpg4/bin, the one in /bin is a historical one and should not be used for new scripts.

terdon
  • 234,489
  • 66
  • 447
  • 667
  • I'd say `awk` has similar portability issues (Is my `awk` plain old `awk`, `nawk` or `gawk`?). Actually, the version in my Solaris boxes lacks `gsub()`. My vote here is for Perl (`perl -nlawe '...'` approximates the automatic behaviour of `awk`). – arielCo Jul 04 '14 at 22:07
  • 2
    @arielCo: For `awk` portability, stick with a version of `awk` that is POSIX compliant and only use those features and you should be fine. On Solaris you will find one as `/usr/xpg4/bin/awk` . – Scrutinizer Jul 05 '14 at 04:48
  • @terdon how about this [`echo "aa\nbb" | awk '{gsub(/\\n/,"\n");}1;'`](http://stackoverflow.com/a/24574563/3297613)? – Avinash Raj Jul 05 '14 at 12:05
  • @arielCo actually, awk is [more portable](http://unix.stackexchange.com/q/123144/22222) than perl. My second awk approach which does not use `gsub()` should work just about everywhere. Even busybox has an `awk`. – terdon Jul 05 '14 at 12:07
  • @AvinashRaj yes, that works too. I needed the `\\\\ ` 'cause I was quoting, not using `//`. – terdon Jul 05 '14 at 12:10
  • @terdon: I used the wrong term, intending "version independent". With `awk` I need to limit myself to POSIX, which can be exasperating; my informal "minimum" version of Perl is 5.8, which gives me all the power I need for text processing. But yes, barebones awk is practically universal. – arielCo Jul 05 '14 at 20:05
  • @Scrutinizer: POSIX awk is practically universally available, but... yuck :þ – arielCo Jul 05 '14 at 20:10
  • 1
    gsub() is POSIX (gsub, was added in SVR3.1, POSIX awk is based on SVR4 with a few additions see the gawk manual for more information), gensub() is the GNU extension. The original awk still found as /bin/awk on Solaris, did not support support gsub(), but didn't not support a multi-character FS either. – Stéphane Chazelas Nov 22 '15 at 12:34
1

If Holdspace is empty you can also do:

sed '/\\n/G;s/\\n\(.*\)\(.\)/\2\1/;P;D'

...but uxnut's answer is already both faster and more simple, so you can take it as you will.

Another extraneous possibility:

INPUT | sed -n l | while read v ; do printf "${v%?}" ; done

But beware, ^that translates all standard C-style \backslash escapes - like \backspace and \return and \00octals and whathaveyou.

Community
  • 1
mikeserv
  • 57,448
  • 9
  • 113
  • 229
1

With gnu sed the following also works:

gsed -n -e 'H;${x;s/\\n/\n/g;p;}'
Kevdog777
  • 3,194
  • 18
  • 43
  • 64
Dirk Hoffmann
  • 119
  • 3
-1

Although I like sed, I think the simplest and most portable solution is using echo:

echo -e "string with\n in it"

Or if you have a file that has strings with "\n"s in it, use command substitution:

echo -e $(filename)

Then add redirection to save to a file as needed.

Kenton Groombridge
  • 11
  • 1
    You can hardly get less portable than `echo`. See [Why is printf better than echo?](https://unix.stackexchange.com/q/65803). (you may also want to read [When is double-quoting necessary?](https://unix.stackexchange.com/q/68694)) – Stéphane Chazelas Oct 04 '22 at 16:22
  • This may print the literal string `-e string with\n in it` depending on the shell and what options are currently set. – Kusalananda Oct 04 '22 at 16:53
  • The question is using the sed command which implies that the system is Unix based. I know of no Unix based system which shell doesn't have echo as a built-in or the system has it as a command. I very much understand the purpose of double quotes and single quotes and for my example either will work just fine using the most ubiquitous shell at this time "bash". Yes printf can also be used, but almost every old shell such as bourne shell will have echo. printf may also be built-in to the shell, but not on the older shells, and the printf command may not be supported on older systems as well. – Kenton Groombridge Oct 05 '22 at 17:36