13

I want to use a here-doc for sed commands and provide the file to be read and the output file.

I've looked at Here-Documents from Advanced Bash Scripting guide but it does not mention anything about regular arguments in using a here-doc. Is it even possible?

I'd like to achieve something like the following:

#!/bin/bash
OUT=/tmp/outfile.txt
IN=/my_in_file.txt

sed $IN << SED_SCRIPT
    s/a/1/g
    s/test/full/g

SED_SCRIPT 
> $OUT;

Any help is really appreciated.

Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
Tristian
  • 1,499
  • 2
  • 13
  • 13

5 Answers5

21

You can tell GNU sed to read the script from standard input with -f -, -f meaning to read the script from a file, and - meaning standard input as is common with a lot of commands.

sed -f - "$IN" > "$OUT" << SED_SCRIPT
    s/a/1/g
    s/test/full/g
SED_SCRIPT

POSIX sed also supports -f, but the use of - for standard input is not documented. In this case, you could use /dev/stdin on Linux systems (and I seem to recall Solaris has this too, but I cannot confirm that right now)

Using <<-SED_SCRIPT (with the '-' prefix) will allow the closing SED_SCRIPT tag to be indented.

EightBitTony
  • 20,963
  • 4
  • 61
  • 62
camh
  • 38,261
  • 8
  • 74
  • 62
  • Thank you, what you said worked, how ever I'd like to point out that the closing token cannot be indented, otherwise it causes an error in bash, I assume this is because it also includes the white-space in the closing one. – Tristian Aug 16 '12 at 02:29
  • Note that Apple OS X sed does not support reading a script from standard input. (At least not in that manner or any other documented manner.) – danorton May 24 '13 at 17:00
  • @danorton: I have added a [solution](http://unix.stackexchange.com/a/170200/19702) which does not require support of `-f -` in `sed` but it requires process substitution: `<( command )`. – pabouk - Ukraine stay strong Nov 27 '14 at 02:09
  • Note: Redirection operator `<<-` in _Bash Ref. Manual_ : "[_all leading tab characters are stripped from input lines and the line containing delimiter_](https://www.gnu.org/software/bash/manual/bash.html#Here-Documents)". If the (closing) delimiter is intended with spaces rather than tabs: `bash: warning: here-document at line 21 delimited by end-of-file (wanted 'SED_SCRIPT')\bash: bash-script.sh: line 90: syntax error: unexpected end of file`. (Git Bash v2.34.1; GNU bash, version 4.3.48(1)-release (x86_64-suse-linux-gnu)). Indentation of the `sed` commands works with tabs _and_ spaces. – Gerold Broser Jun 24 '22 at 10:13
7

In case sed does not support reading of a script from stdin (using -f -), you can use process substitution (available in bash, zsh, ksh93):

sed "$IN" > "$OUT" -f <( cat << SED_SCRIPT
    s/a/1/g
    s/test/full/g
SED_SCRIPT)

The closing parenthesis ) must follow the end delimiter (SEC_SCRIPT) immediately or after a newline. In the case of process substitution you can also use echo instead of a here document:

sed "$IN" > "$OUT" -f <( echo \
"    s/a/1/g
    s/test/full/g" )
pabouk - Ukraine stay strong
  • 2,360
  • 26
  • 32
  • 1
    On Mac OS X, it didn't like the input file (`"$IN"` or even `testfile.txt`) immediately following the sed command itself — it wanted a `-e script` or `-f file`. Re-arranging @pabouk 's answer worked for me: `sed -f <( cat << ETC ... ETC) "$IN" > "$OUT"` – Stephen P Oct 04 '16 at 22:21
  • On Max OSX with bash 4.2.45 and standard BSD sed, the *closing parentheses* had to be on a new line below the closing `SED_SCRIPT`. – Alexander Klimetschek Dec 08 '17 at 20:06
0

Why not use multiple expressions with -e like this:

#!/bin/bash
OUT=/tmp/outfile.txt
IN=/my_in_file.txt

sed 
  -e "s/a/1/g" \
  -e "s/test/full/g" \
  $IN > $OUT

I find that slightly more readable for smaller inline sed scripts. BTW you could also use the in-place editing of sed:

sed -i.org
  -e "s/a/1/g" \
  -e "s/test/full/g" \
  $IN

This would save the original file under ${IN}.org and apply the changes to the input file.

tjr
  • 1
0

There are some great answers on this page. I was looking to be as 'clean' as I could and came up with this version standing on the shoulders of the suggestions above. Comments ofr improvment or fixes are always welcome.

Sampe solution:

function sed_script()
{
    cat << SED_SCRIPT
  s/a/1/g;
  s/test/full/g;
SED_SCRIPT
}

Use as follows:

cat "$IN" | sed -f <(sed_script) > "$OUT"

Worked on my use case, which had a lot of messy sed commands to run.

will
  • 480
  • 1
  • 6
  • 12
  • It makes sense to create a function that outputs a `sed` script _if you need to use this `sed` script in many different places_. However, it would probably never be useful to use it without invoking `sed`. It would therefore make more sense to also include the `sed` call in the function, so that you get `sed_script "$IN" >"$OUT"`. – Kusalananda Feb 24 '23 at 14:21
  • @Kusalananda ... Good point; although the thrus of making a function is primarily for maintainability and readability. Even when ''developing' script, one still needs to be able to quickly (human-)parse the intention and grok how it ought to apply/work-out. _imho_ – will Feb 28 '23 at 13:07
-1

Maybe a nicer approach:

OUT=/dev/stdout

IN=my_in_file.txt
sed -s 's/in/out/' < $IN > $OUT
  -or-
sed -s 's/in/out/' > $OUT < $IN

IN=my_in_text
sed -s 's/in/out/' <<< $IN > $OUT
  -or-
sed -s 's/in/out/' > $OUT <<< $IN

:)

Magnus
  • 169
  • 1
  • 5