Exported bash functions sometimes visible from Perl

Question

My Redhat 9, OpenBSD 4.9, FreeBSD 10, Macos X, LinuxMint 17.3, and Ubuntu 14.04.4 all print OK when running this:

myfunc() { echo OK; }
export -f myfunc
perl -e open\(\$fh,\"\|-\",\"@ARGV\"\)\;close\$fh\; /bin/bash\ -c\ myfunc\\\ a

My Ubuntu 16.04.1 gives:

bash: myfunc: command not found

But if I remove \\\ a it works.

perl -e open\(\$fh,\"\|-\",\"@ARGV\"\)\;close\$fh\; /bin/bash\ -c\ myfunc

I have the feeling something is configured wrongly on my system, but what should I look for?

Edit

thrig found a shorter version that also fails. Using that I straced on a failing and a non-failing system:

stdout strace -ff perl -e 'system @ARGV' /bin/bash\ -c\ myfunc\\\ a|grep bash

Failing:

execve("/usr/bin/perl", ["perl", "-e", "system @ARGV", "/bin/bash -c myfunc\\ a"], [/* 71 vars */]) = 0
[pid  7728] execve("/bin/sh", ["sh", "-c", "/bin/bash -c myfunc\\ a"], [/* 71 vars */]) = 0
[pid  7729] execve("/bin/bash", ["/bin/bash", "-c", "myfunc a"], [/* 70 vars */]) = 0

Non-failing:

execve("/usr/bin/perl", ["perl", "-e", "system @ARGV", "/bin/bash -c myfunc\\ a"], [/* 20 vars */]) = 0
[pid 26497] execve("/bin/sh", ["sh", "-c", "/bin/bash -c myfunc\\ a"], [/* 20 vars */]) = 0
[pid 26498] execve("/bin/bash", ["/bin/bash", "-c", "myfunc a"], [/* 20 vars */]) = 0

That looks awfully similar. Removing the \\\ a gives on both systems:

execve("/usr/bin/perl", ["perl", "-e", "system @ARGV", "/bin/bash -c myfunc"], [/* 71 vars */]) = 0
[pid  7826] execve("/bin/bash", ["/bin/bash", "-c", "myfunc"], [/* 71 vars */]) = 0

So Perl drops the sh -c if there is only a single command. Maybe sh -c eats the function on the Ubuntu 16.04?

/bin/sh is dash on both systems.

Edit2

env shows the function. This displays the function as part of the environment on both systems:

perl -e 'system @ARGV' /bin/bash\ -c\ env

Ubuntu 16.04 and one working system:

BASH_FUNC_myfunc%%=() {  echo OK
}

Other working system:

BASH_FUNC_myfunc()=() {  echo OK
}

But this shows only the definition on the working systems:

perl -e 'system @ARGV' /bin/bash\ -c\ env';true'

Edit3

Workaround:

myfunc() { echo OK; }
export -f myfunc
perl -e open\(\$fh,\"\|-\",@ARGV\)\;close\$fh\; /bin/bash -c myfunc\ a

Slightly less obfuscated (second version): `perl -e 'open($fh, "|-", @ARGV); close $fh;' "/bin/bash -c myfunc"` These both return "OK" on OS X. — Kusalananda, Aug 01 '16 at 15:19
Same behaviour on OpenBSD 6.0-beta as on Ubuntu 16.04.1, BTW. — Kusalananda, Aug 01 '16 at 15:23
`perl -e 'system @ARGV'` would be even more concise, and also removes the confusing backwhacks. — thrig, Aug 01 '16 at 15:29
@Kusalananda Can I ask you to test the workaround on OpenBSD? — Ole Tange, Aug 01 '16 at 17:12
The workaround in "Edit3" returns `OK` on OpenBSD (I had to change the path to `bash` as there's no `bash` in `/bin`). — Kusalananda, Aug 01 '16 at 17:17
[Shellshock](https://en.wikipedia.org/wiki/Shellshock_%28software_bug%29)? — Cyrus, Aug 01 '16 at 17:38
@Cyrus Nope: As you can see one of the other working systems exports the function using the %% syntax, too. — Ole Tange, Aug 01 '16 at 18:12
i'm wondering why you're escaping everything rather than just quoting? e.g. `perl -e "system ('bash','-c','declare -fp myfunc')"`. or `perl -e 'system @ARGV' -- bash -c 'declare -pF myfunc'` — cas, Aug 03 '16 at 11:49
@cas Because the escaping is done by a program (GNU Parallel). — Ole Tange, Aug 03 '16 at 12:43

score 1 · Answer 1 · 2019-04-30T04:49:04.047

The problem is that the /bin/sh from systems like Debian or Ubuntu (dash) or OpenBSD will clear from the environment any variables whose names contain fancy chars like %, and that includes those BASH_FUNC_foo%%=() { ..., which are used to encode bash's exported functions.

If the |- from the open function is followed by a single argument instead of a list of arguments, and that argument contains any shell metacharacters (the backslash is one of those), then perl will pass it as an argument to /bin/sh -c instead of using execvp(2) directly.

The same holds true for the system, exec, open2, etc functions in perl, and is documented in perldoc -f system.

A simpler example:

$ foo(){ echo foo; }; export -f foo

$ perl -e 'system shift' '/bin/bash -c foo\ a'
/bin/bash: foo: command not found
$ perl -e 'system shift' '/bin/bash -c foo'
foo
$ perl -e 'system shift' '/bin/bash -c "foo"'
/bin/bash: foo: command not found

$ perl -e 'open F, "|-", shift' '/bin/bash -c foo\ a'
/bin/bash: foo: command not found
$ perl -e 'open F, "|-", shift' '/bin/bash -c foo'
foo

The work-around is to run external commands with the user's $ENV{SHELL}, which will let them use any shell features seamlessly:

perl -e 'open F, "|-", $ENV{SHELL}, "-c", "@ARGV"' '/bin/bash -c "foo a"'

`$ENV{SHELL}` is not guaranteed to preserve those env vars though. The only shell that is guaranteed to is `bash` as that's a `bash` feature. I'd say the correct way would be to do `perl -e 'system @ARGV' bash -c 'foo a'` here. — Stéphane Chazelas, Apr 30 '19 at 07:30
Which is just right, because the user may not care about `bash`'s exported functions, but may expect to use other features of their shell transparently in the commands passed as arguments to that perl program. — , Apr 30 '19 at 08:41

J_H · Answer 2 · 2019-04-30T02:03:01.983

You're having trouble with portability of an exported bash function across systems. Yes, that makes sense.

To improve portability simply

$ touch myfunc
$ chmod a+x myfunc

and put the desired bash code in that file, starting with a #! /usr/bin/env bash shebang line. (env obeys $PATH, you see.)

Then you're relying on the ability of programs (including gnu parallel) to fork+exec, rather than on how well variants like sh or dash conform to bash behavior.

Exported bash functions sometimes visible from Perl

2 Answers2