0

I'm having a problem with sed, when using its 's' (search and replace command) with its 'e' flag, as in 

sed 's/.../.../e'

The problem is that the characters $, (, and ) in the standard input appear to confuse it.

I first ran into it here and have now boiled it down to this:

echo '$-' | sed 's/x//e'               # works when pattern is NOT matched
$-

echo '$-' | sed 's/-//e'               # fails when pattern is matched: gags on $
sh: $: command not found

echo '-' | sed 's/-//e'                # works when $ isn't present

echo '-(' | sed 's/-//e'               # Also fails when '(' or ')' are found
sh: -c: line 1: syntax error: unexpected end of file

echo '(' | sed 's/-//e'                # But works with this
(

echo '$-' | sed 's/-/echo xx/e'        # also fails
sh: xx: command not found

echo '$-' | sed 's/\(-\)/echo '\1'/e'  # also fails
sh: 1: command not found

Here's what the GNU manual says:

3.3 The s Command

...

The s command can be followed by zero or more of the following flags:

...

e

This command allows one to pipe input from a shell command into pattern space. If a substitution was made, the command that is found in pattern space is executed and pattern space is replaced with its output. A trailing newline is suppressed; results are undefined if the command to be executed contains a NUL character. This is a GNU sed extension.

bash --version
GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)

sed --version
sed (GNU sed) 4.7
Packaged by Debian

What am I missing or doing wrong?

Elliptical view
  • 3,559
  • 4
  • 25
  • 44
  • @Kamil Maciorowski, thanks, but no. What I'm trying to do is to pipe a long list of pathname lines thru a sed filter which operates on some parts of them, even calling a bash function to help rewrite them. Looks like I might have to figure out another way to do this, as it appears that the e flag doesn't exactly do what it says in the manual. It seems that the entire output is executed, not just the replacement part. – Elliptical view Feb 23 '20 at 21:12
  • OK. For future reference: to avoid such misunderstandings it's good to state your expected output. E.g. I cannot tell what you expected from `echo '$-' | sed 's/-//e'`. – Kamil Maciorowski Feb 23 '20 at 21:14
  • From your comments I deduce maybe you misinterpreted "pattern space". See [this part of the manual](https://www.gnu.org/software/sed/manual/sed.html#Execution-Cycle): `first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space. Then commands are executed`. With `echo '$-' | sed 's/-/echo xx/e'` the pattern space is not `echo xx`. The pattern space is `$-` at first, then `s` does its job and the pattern space becomes `$echo xx`. – Kamil Maciorowski Feb 23 '20 at 21:27

1 Answers1

3

Running

echo '$-' | sed 's/-//e'

removes the - from $-, leaving just $, and the result is run as a shell command, because of /e.

The result is the same is if you try to run $ from the command line:

sh: $: command not found

And similarly for all other lines.

choroba
  • 45,735
  • 7
  • 84
  • 110
  • Thanks, but in `echo '$-' | sed 's/-/echo xx/e'` I would have thought that the meaning of 'the command that is found in pattern space is executed' would be to execute `echo xx`. But I think I see what you mean: It is not executing just the `echo xx`, but rather the entire result, or something like `$$(echo xx)`. But if so then it makes it very hard to predict what will be executed, and thus this appears to make it insecure in a way too. – Elliptical view Feb 23 '20 at 21:05
  • 1
    Yes, that's why you seldom find `/e` in sed in practice (unlike a similar but different `/e` in Perl, see [perlop](https://perldoc.pl/perlop#s/PATTERN/REPLACEMENT/msixpodualngcer)). – choroba Feb 23 '20 at 22:41