6

I transfered /etc/* files from one Linux server to another and now I have a strange error. Both are Debian Squeeze.

When I run a command, e.g. ssh, I get an error:

bash: /usr/bin/ssh: No such file or directory

It looks like the binary exists:

-rwxr-xr-x 1 root root 358756 Sep 21 14:30 /usr/bin/ssh

Bash also autocompletes the command when I type ss and press Tab. I tried reinstalling, purging etc but it didn't solve anything. There is probably something cached under /etc/ but I don't know what excatly.

jasonwryan
  • 71,734
  • 34
  • 193
  • 226
Lauris
  • 163
  • 1
  • 6
  • 1
    What's the output of `ldd /usr/bin/ssh`? It might depend on a library (`.so` file) that doesn't exist. – mrb Sep 28 '12 at 14:32
  • @mrb the output is "not a dynamic executable". – Lauris Sep 28 '12 at 14:33
  • How did you copy the files? What does `file /usr/bin/ssh` tell you? – Mat Sep 28 '12 at 14:38
  • @Mat Copied like this: `scp -r /etc/* user@remote:/etc/` and the output of the "file" command is `/usr/bin/ssh: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, stripped` – Lauris Sep 28 '12 at 14:42
  • If that helps I also installed the same packages using `dpkg --set-selections` & `apt-get dselect-upgrade`. – Lauris Sep 28 '12 at 14:45
  • 3
    So file says it uses shared libs, and `ldd` doesn't... something's screwed up. Try `readelf -l /usr/bin/ssh` and look for a "Requesting interpreter" line (or error). – Mat Sep 28 '12 at 14:48
  • @Mat `ldd` actually *runs* the program. So of course it doesn't work. (BTW: `ldd` is a shell script; `less \`which ldd\`` and you can see how it works) – derobert Sep 28 '12 at 14:51
  • 1
    @mrb Same comment about `ldd` actually running the program to you, too. Its important that people know this, less they try `ldd` when analyzing an untrusted binary. – derobert Sep 28 '12 at 14:53
  • @derobert: it doesn't actually run the program, it invokes the runtime linker and does pretty much all the runtime linking with debug options, but it doesn't invoke the program's main. – Mat Sep 28 '12 at 14:54
  • @Mat: `strace -f ldd \`which cat\` |& grep execve`: `… [pid 8490] execve("/bin/cat", ["cat"], [/* 46 vars */] ` – derobert Sep 28 '12 at 14:55
  • @derobert: try `ldd /usr/bin/vi` or `ldd /sbin/halt` - this won't run vi or `halt`. The runtime linker gets debug options from the environment, set up from the `ldd` script, and stops short from actually running the code. – Mat Sep 28 '12 at 14:57
  • @Mat yes, the glibc start code (still part of the executable) doesn't invoke the program's main. But the point is that we know (from the error posted) that `execve` is failing. And so its not surprising that ldd does as well (as that `strace` shows, it actually does an `execve` of the program) – derobert Sep 28 '12 at 15:00
  • @derobert: I was reacting to your statement that `ldd` _runs_ the executable, which would make `ldd` bloody dangerous. We both agree, I think, that `ldd` not managing to list dependencies and `file` saying it is dynamically linked, is a problem, right? – Mat Sep 28 '12 at 15:02
  • Is the binary the same architecture as the rest of the userspace (i386)? – jordanm Sep 28 '12 at 15:08
  • @Mat It is bloody dangerous to run on an executable you don't trust (see, e.g., http://man7.org/linux/man-pages/man1/ldd.1.html), though some versions of ldd are patched against this. The disagreement is a problem, but (for example) is exactly what is expected (I just tested this) if the executable is the wrong architecture. – derobert Sep 28 '12 at 15:09
  • You *only* transferred `/etc`? Weird that it broke ssh, but ls still works. Were the two machines the same architecture (or at least running the same, i.e., both running i386)? – derobert Sep 28 '12 at 15:15
  • 1
    ldd doesn't call `_start`/`main` as normal when called on a normal dynamically-linked executable, but an executable _can_ be crafted to run arbitrary code on being examined with some versions of ldd. – Random832 Sep 28 '12 at 16:12
  • 1
    @strangefishinthetown And what happens when you run `/usr/bin/ssh` (i.e. full path)? Is the file actually executable? – jw013 Sep 28 '12 at 16:44
  • 4
    @strangefishinthetown You might try removing `/etc/ld.so.cache` and checking the contents of `/etc/ld.so.conf` and `/etc/ld.so.conf.d`. Run `ldconfig` to rebuild the cache afterwards. – Jim Paris Sep 28 '12 at 17:36

1 Answers1

4

I think Jim Paris's comment is right on the mark. You can get “no such file or directory” even if the specified file exists, if the dynamic loader for that file does not exist. A common case is trying to run a binary from a different architecture that the kernel supports but not the user-land system — typically a 32-bit binary on a 64-bit system that doesn't have 32-bit libraries installed. See Getting "Not found" message when running a 32-bit binary on a 64-bit system for a more detailed explanation of that case.

Here, it's possible that /usr/bin/ssh uses a library that is installed in a different place on the new system. Since you copied the dynamic linker's cache file /etc/ld.so.cache, the dynamic linker is looking for that library in a place where it doesn't exist. Run ldconfig to update the cache.

It's also possible that you'd enabled prelinking. If so, remove /etc/prelink.cache.

Community
  • 1
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175