6

All my Python and Perl scripts are simply NOT iterpreted via shebang. Never. But they work as expected when I explicitly call the binary.

I double checked my Perl and Python installations, it is just too strange: they shebang-way execution works very well in the target system chroot on a sane host but not in the actual running system.

I work on a homemade Linux system which worked just great before that problem appeared. See by yourself:

A test on the 'xscreensaver-text' Perl program, once via shebang then with the interpreter on the CLI:

$ LC_ALL=C LANG=C /usr/bin/xscreensaver-text
/usr/bin/xscreensaver-text: line 23: require: command not found
/usr/bin/xscreensaver-text: line 25: use: command not found
/usr/bin/xscreensaver-text: line 29: BEGIN: command not found
/usr/bin/xscreensaver-text: line 31: use: command not found
/usr/bin/xscreensaver-text: line 32: syntax error near unexpected token `('
/usr/bin/xscreensaver-text: line 32: `use POSIX qw(strftime);'

$ LC_ALL=C LANG=C perl /usr/bin/xscreensaver-text
poopy
Linux 3.11.1

Sat Oct  5 23:07:33 2013

up 11:35,  2 users
load average: 0.09, 0.08, 0.06

SO this for Perl programs but the same happens with Python scripts. We've been messing with encodings and terminfo's and different kernel, still no success. I've even rebuilt my entire system. It works just great in a chroot env, but once I boot it I have this problem.

Here's an strace output:

$ LC_ALL=C LANG=C strace /usr/bin/xscreensaver-text
execve("/usr/bin/xscreensaver-text", ["/usr/bin/xscreensaver-text"], [/* 50 vars */]) = -1 ENOEXEC (Exec format error)
write(2, "strace: exec: Exec format error\n", 32strace: exec: Exec format error
) = 32
exit_group(1)                           = ?
+++ exited with 1 +++

$ LC_ALL=C LANG=C strace perl /usr/bin/xscreensaver-text
execve("/usr/bin/perl", ["perl", "/usr/bin/xscreensaver-text"], [/* 50 vars */]) = 0
brk(0)                                  = 0x601000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff12e312000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=240674, ...}) = 0
mmap(NULL, 240674, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff12e2d7000
close(3)                                = 0
open("/usr/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0 \37\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1868472, ...}) = 0
mmap(NULL, 3981888, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff12dd24000
mprotect(0x7ff12dee6000, 2097152, PROT_NONE) = 0
mmap(0x7ff12e0e6000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c2000) = 0x7ff12e0e6000
mmap(0x7ff12e0ec000, 16960, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff12e0ec000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff12e2d6000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff12e2d4000
arch_prctl(ARCH_SET_FS, 0x7ff12e2d4740) = 0
mprotect(0x7ff12e0e6000, 16384, PROT_READ) = 0
mprotect(0x7ff12e313000, 4096, PROT_READ) = 0
munmap(0x7ff12e2d7000, 240674)          = 0
brk(0)                                  = 0x601000
brk(0x622000)                           = 0x622000
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7ff12e2d4a10) = 5680
wait4(5680, poopy
Linux 3.11.1

Sat Oct  5 23:11:49 2013

up 11:39,  2 users
load average: 0.08, 0.12, 0.08

[{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 5680
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=5680, si_status=0, si_utime=2, si_stime=0} ---
exit_group(0)                           = ?
+++ exited with 0 +++

Contents (only the beginning) of the script : 

$ cat /usr/bin/xscreensaver-text
#!/usr/bin/perl -w
# Copyright � 2005-2013 Jamie Zawinski 
#
#
# Created: 19-Mar-2005.

require 5;
#use diagnostics;   # Fails on some MacOS 10.5 systems
use strict;

# Some Linux systems don't install LWP by default!
# Only error out if we're actually loading a URL instead of local data.
BEGIN { eval 'use LWP::UserAgent;' }
--*snip*--
Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
appzer0
  • 63
  • 5
  • How about the contents of the script? – phemmer Oct 05 '13 at 21:19
  • What does `perl -E 'say unpack("H*", scalar <>)' /usr/bin/xscreensaver-text` print on the system where it doesn't work? – cjm Oct 05 '13 at 21:29
  • 1
    `$ perl -E 'say unpack("H*", scalar <>)' /usr/bin/xscreensaver-text 23212f7573722f62696e2f7065726c202d770a` – appzer0 Oct 05 '13 at 21:29
  • Well, that's what it should say. I'd guess there's something wrong with your kernel. It's not recognizing shebangs. If all else fails, you could fall back on the [classic Perl preamble](http://stackoverflow.com/q/2308874/8355). – cjm Oct 05 '13 at 21:37
  • Thanks very much for your answer. Though we tried with different kernels, including some that were working OK, same problem. – appzer0 Oct 05 '13 at 21:44
  • 4
    Do _all_ shebangs fail or only Perl and python? What about `#!/bin/sh` or, better `#!/usr/bin/env perl`. I assume that by home made, you mean a [LFS](http://www.linuxfromscratch.org/lfs/) install and your own compiled kernel? – terdon Oct 05 '13 at 21:50
  • What about `grep . /proc/sys/fs/binfmt_misc/*`? – Stéphane Chazelas Oct 05 '13 at 21:53
  • @StephaneChazelas that dir is empty here. – appzer0 Oct 05 '13 at 22:02
  • 1
    @terdon I tried a Ruby script and it is exactly the same behaviour. – appzer0 Oct 05 '13 at 22:03
  • 1
    Could those interpreters possibly be scripts themselves? – Stéphane Chazelas Oct 05 '13 at 22:05
  • Any selinux, apparmor or the like? – Stéphane Chazelas Oct 05 '13 at 22:09
  • @StephaneChazelas This is a standard Perl installation, no special script or wrapper of any kind. Perl is 755 (and it is OK to call the Perl binary, only the shebang-way is failing). No apparmor or selinux or security fw – appzer0 Oct 05 '13 at 22:11
  • One thing that strikes me as odd is that the error message says the `require` is on line 23, but in the script you provided it's on line 7. Where the heck is it getting those extra 16 lines from? What happens if you change your active shell (switch to another shell and then try running it)? – phemmer Oct 05 '13 at 22:21
  • @Patrick I've shortened the script actually for the needs of pasting here. This script is part of the xscreensaver package, available in any major *Linux distribution. – appzer0 Oct 05 '13 at 22:32
  • 1
    Does `modprobe -r binfmt_script` return anything else than `FATAL: Module binfmt_script is builtin`? Does `modprobe -c | grep binfmt` return anything interesting? – Stéphane Chazelas Oct 05 '13 at 22:38
  • Do you see the same if you start the system with `init=/bin/sh` as kernel cmdline parameter? – Stéphane Chazelas Oct 05 '13 at 23:07
  • 1
    @StephaneChazelas, Bingo! The CONFIG_BINFMT_SCRIPT option has been included in the kernel since 3.10. – appzer0 Oct 06 '13 at 08:44

1 Answers1

15

Your kernel was compiled without CONFIG_BINFMT_SCRIPT=y. This setting controls shebang support.

From make menuconfig:

Symbol: BINFMT_SCRIPT [=y]                                             
   Type  : tristate                                                    
   Prompt: Kernel support for scripts starting with #!                 
     Location:                                                         
   (1) -> Executable file formats / Emulations                         
     Defined at fs/Kconfig.binfmt:68

Reconfigure and recompile your kernel. (Technically, it can also be built as a module, but there's no point in doing that for something as fundamental as #! support.)

cjm
  • 26,740
  • 12
  • 88
  • 84
  • 1
    You'd think his _Though we tried with different kernels, including some that were working OK, same problem_ would rule that out but it could actually be that he did build it as a module and forgot to include the module or run the depmod. So I agree that's the best bet so far. +1 – Stéphane Chazelas Oct 05 '13 at 23:17
  • `#!/bin/sh` shouldn't work also. – Alex Oct 05 '13 at 23:18
  • 4
    @Alex, that's a different issue. If the system returns ENOEXEC on a file, most user space applications (through `execvp()` or a shell) try to run it with `/bin/sh`, that's why he gets shell syntax errors when running the perl script. And that's why you wouldn't notice non-working shebangs if they were all `#! /bin/sh` ones. – Stéphane Chazelas Oct 05 '13 at 23:21
  • Confirm, `execv*()` functions are frontend for execve() so that's no the case, but the bash at least try to fix NOEXEC running the file as a shell script if there is a `#!` – Alex Oct 06 '13 at 00:20
  • @Alex, `execlp(3)` and `execvp(3)` also execute a shell upon `execve(2)` returning ENOEXEC, as required by POSIX. Which you can verify with a `ltrace -S env ./script-with-no-shebang` 2>&1 | grep exec` – Stéphane Chazelas Oct 06 '13 at 07:25
  • I'll fill all the shebang non sh scripts of `rm -fr *`, somebody have to pay for all those `sh` around :) Thank you @StephaneChazelas – Alex Oct 06 '13 at 07:37
  • 1
    Thanks so much! BINFMT_SCRIPT was compiled as a module, thus non-loaded. – appzer0 Oct 06 '13 at 08:45
  • Wow, I had never heard of such a thing being turn-off-able! I wonder why they would do that? – tchrist Oct 06 '13 at 21:35
  • @tchrist, [it was added](https://github.com/torvalds/linux/commit/2535e0d723e4d7723b030f39fb350e436bdb983f) for embedded systems that don't run scripts. But for a normal Linux system you definitely want it turned on. – cjm Oct 07 '13 at 02:55