69

Will the executable of a small, extremely simple program, such as the one shown below, that is compiled on one flavor of Linux run on a different flavor? Or would it need to be recompiled?

Does machine architecture matter in a case such as this?

int main()
{
  return (99);
}
TRiG
  • 331
  • 1
  • 3
  • 18
JCDeen
  • 765
  • 1
  • 5
  • 9
  • 3
    Thanks to everyone for outstanding answers! I learned much more than I anticipated. I made the code artificially simple on purpose so that it would depend on as few libraries as possible; but I really should have stated that up front. Most of my C++ coding across platforms involved developing with a Microsoft tool such as Visual Studio & then porting the code over to a *nix system and recompiling. – JCDeen Jan 30 '18 at 15:12
  • 4
    The many facets & considerations expressed here has amazed me! I honestly wish I could choose several as THE answer. Thanks again to all! Sincerely. – JCDeen Jan 30 '18 at 15:23
  • 2
    Android is also a Linux-based operating system. Best of luck, however, running any code compiled against `glibc` there, or vice versa. Granted, it's [not entirely impossible](https://en.wikipedia.org/wiki/Hybris_(software)). – undercat Jan 31 '18 at 01:02
  • 3
    For maximum compatibility of a command-line tool, you may want to use uClibc, musl or dietlibc instead of glibc, and statically link your 32-bit executable (`gcc -m32 -static`). By doing so, any i386 or amd64 Linux will be able to run the executable. – pts Jan 31 '18 at 12:06
  • 14
    You should be returning **42**! :) – Homunculus Reticulli Jan 31 '18 at 14:33
  • 1
    @HomunculusReticulli interestingly, the question has also 42 upvotes ;) – Twometer Feb 01 '18 at 14:31
  • 1
    As you figured out, architecture is much more important than flavour. – Mast Feb 01 '18 at 23:11

6 Answers6

74

In short: If you're taking a compiled binary from one host to another using the same (or a compatible) architecture, you may be perfectly fine taking it to another distribution. However as complexity of the code increases, the likelihood of being linked against a library that is not installed; installed in another location; or installed at a different version, increases. Taking for instance your code, for which ldd reports the following dependencies when compiled with gcc -o exit-test exit-test.c on a (Debian-derived) Ubuntu Linux host:

$ ldd exit-test
    linux-gate.so.1 =>  (0xb7748000)
    libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb757b000)
    /lib/ld-linux.so.2 (0x8005a000)

Obviously this binary won't run if I kick it over to, say, a Mac (./exit-test: cannot execute binary file: Exec format error). Let's try moving it to a RHEL box:

$ ./exit-test
-bash: ./exit-test: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory

Oh dear. Why might this be?

$ ls /lib/ld-l* # reference the `ldd` output above
ls: cannot access /lib/ld-l*: No such file or directory

Even for this use-case, forklifting it failed due to missing shared libraries.

However, if I compile it with gcc -static exit-test-static exit-test.c, porting it to the system without the libraries works just fine. At the expense, of course, of disk space:

$ ls -l ./exit-test{,-static}
-rwxr-xr-x  1 username  groupname    7312 Jan 29 14:18 ./exit-test
-rwxr-xr-x  1 username  groupname  728228 Jan 29 14:27 ./exit-test-static

Another viable solution would be to install the requisite libraries on the new host.

As with many things in the U&L universe, this is a cat with many skins, two of which are outlined above.

DopeGhoti
  • 73,792
  • 8
  • 97
  • 133
  • 4
    Indeed, I forgot about static binaries. Some vendors adopt static binaries, and some malware authors too to maximize binary compatibility between Linux versions *of the same architecture* – Rui F Ribeiro Jan 29 '18 at 21:29
  • 8
    Forklifting...? – user253751 Jan 30 '18 at 04:44
  • 2
    @immibis I think it means copying data (the executable) from one environment (distro) to another, where the data is not designed for the destination environment. – wjandrea Jan 30 '18 at 06:57
  • 13
    Your Linux example is rather artificial unfortunately, and illustrates your point about architectures rather than distributions: you built a 32-bit binary on Debian and tried to run it on 64-bit RHEL; those are different architectures... Same-architecture binaries with so few library dependencies can be copied just fine. – Stephen Kitt Jan 30 '18 at 07:37
  • 2
    @StephenKitt: It's not unreasonable. On Windows, 32 bit binaries run on 64 bit systems. And visual C++'s equivalent of `libc` is redistributable. – MSalters Jan 30 '18 at 09:16
  • 7
    @MSalters I’m not saying it’s unreasonable, I’m saying it’s a bad example given the point DopeGhoti is trying to make (you can’t copy binaries from one distribution to another — which is wrong). Of course 64-bit Linux on Intel supports running 32-bit executables too, with the appropriate infrastructure. A valid example in this case IMO would be building an `amd64` binary and running it on another `amd64` distribution, or building an `i386` binary and running it on another `i386` distribution. – Stephen Kitt Jan 30 '18 at 09:28
  • @immibis When you fork the data on disk instead of forking a process, I guess. =) – jpmc26 Jan 30 '18 at 12:32
  • I wasn't saying that you _can't_ - I was explicitly showing that you _can_, but there are problems which are very likely to become manifest (e. g. missing libraries) which can be worked around (e. g. static binaries). – DopeGhoti Jan 30 '18 at 15:38
  • @DopeGhoti OK, that wasn’t clear to me from your answer; I would have expected the solution for missing libraries to be adding those libraries ;-). I still think it would be clearer if you showed the result of moving a binary to another distribution of the same architecture. (I do agree that static linking is a possible solution for people trying to make a distribution-agnostic binary, within reason — see `libnss` and co. Flatpak etc. are perhaps more appropriate, but more cumbersome at first...) – Stephen Kitt Jan 30 '18 at 22:21
51

It depends. Something compiled for IA-32 (Intel 32-bit) may run on amd64 as Linux on Intel retains backwards compatibility with 32-bit applications (with suitable software installed). Here's your code compiled on RedHat 7.3 32-bit system (circa 2002, gcc version 2.96) and then the binary copied over to and run on a Centos 7.4 64-bit system (circa 2017):

-bash-4.2$ file code
code: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.2.5, not stripped
-bash-4.2$ ./code
-bash: ./code: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory
-bash-4.2$ sudo yum -y install glibc.i686
...
-bash-4.2$ ./code ; echo $?
99

Ancient RedHat 7.3 to Centos 7.4 (essentially RedHat Enterprise Linux 7.4) is staying in the same "distribution" family, so will likely have better portability than going from some random "Linux from scratch" install from 2002 to some other random Linux distribution in 2018.

Something compiled for amd64 would not run on 32-bit only releases of Linux (old hardware does not know about new hardware). This is also true for new software compiled on modern systems intended to be run on ancient old things, as libraries and even system calls may not be backwards portable, so may require compilation tricks, or obtaining an old compiler and so forth, or possibly instead compiling on the old system. (This is a good reason to keep virtual machines of ancient old things around.)

Architecture does matter; amd64 (or IA-32) is vastly different from ARM or MIPS so the binary from one of those would not be expected to run on another. At the assembly level the main section of your code on IA-32 compiles via gcc -S code.c to

main:
    pushl %ebp
    movl %esp,%ebp
    movl $99,%eax
    popl %ebp
    ret

which an amd64 system can deal with (on a Linux system--OpenBSD by contrast on amd64 does not support 32-bit binaries; backwards compatibility with old archs does give attackers wiggle room, e.g. CVE-2014-8866 and friends). Meanwhile on a big-endian MIPS system main instead compiles to:

main:
        .frame  $fp,8,$31
        .mask   0x40000000,-4
        .fmask  0x00000000,0
        .set    noreorder
        .set    nomacro
        addiu   $sp,$sp,-8
        sw      $fp,4($sp)
        move    $fp,$sp
        li      $2,99
        move    $sp,$fp
        lw      $fp,4($sp)
        addiu   $sp,$sp,8
        j       $31
        nop

which an Intel processor will have no idea what to do with, and likewise for the Intel assembly on MIPS.

You could possibly use QEMU or some other emulator to run foreign code (perhaps very, very slowly).

However! Your code is very simple code, so will have fewer portability issues than anything else; programs typically make use of libraries that have changed over time (glibc, openssl, ...); for those one may also need to install older versions of various libraries (RedHat for example typically puts "compat" somewhere in the package name for such)

compat-glibc.x86_64                     1:2.12-4.el7.centos

or possibly worry about ABI changes (Application Binary Interface) for way old things that use glibc, or more recently changes due to C++11 or other C++ releases. One could also compile static (greatly increasing the binary size on disk) to try to avoid library issues, though whether some old binary did this depends on whether the old Linux distribution was compiling most everything dynamic (RedHat: yes) or not. On the other hand, things like patchelf can rejigger dynamic (ELF, but probably not a.out format) binaries to use other libraries.

However! Being able to run a program is one thing, and actually doing something useful with it another. Old 32-bit Intel binaries may have security issues if they depend on a version of OpenSSL that has some horrible and not-backported security problem in it, or the program may not be able to negotiate at all with modern web servers (as the modern servers reject the old protocols and ciphers of the old program), or SSH protocol version 1 is no longer supported, or ...

thrig
  • 34,333
  • 3
  • 63
  • 84
  • 14
    re first paragraph: no, Intel calls it "Intel 64" (these days, after going through some other names earlier on). IA-64 refers to Itanium, not anything x86-compatible. – hobbs Jan 30 '18 at 04:50
  • 1
    @hobbs thank you, I have replaced those references with amd64; I'll leave the naming of things to the Intel marketing department. – thrig Jan 30 '18 at 15:02
  • 3
    why not mention static linking? – dcorking Jan 30 '18 at 19:20
  • 2
    It's not just library ABIs that change -- the kernel's syscall interface gets extended over time as well. Note the `for GNU/Linux 2.6.32` (or such) in the output of `file /usr/bin/ls`. – Charles Duffy Jan 30 '18 at 21:01
  • @thrig, they add new syscalls that old kernels don't support *all the time*. Yes, old binaries will still run on your new kernel, but that doesn't mean code you compiled for a new kernel will run on an old one. I recently had the misfortune of trying to build tools to migrate data off a piece of embedded systems hardware from the mid-2000s; getting a build toolchain together was not fun at all. – Charles Duffy Jan 30 '18 at 22:04
  • See ie. https://stackoverflow.com/questions/14363608/compiling-program-for-old-kernel – Charles Duffy Jan 30 '18 at 22:08
  • It depends. On what? On the dependencies it depends on, and on what these dependencies depend. – rackandboneman Feb 01 '18 at 09:10
  • Maybe I misunderstand something, but Red Hat RHEL 7.3 was [released in 2016](https://access.redhat.com/articles/3078).... – Wilbert Feb 01 '18 at 16:57
  • What does that have to do with anything? – DopeGhoti Feb 01 '18 at 17:12
  • Technically, if you were to put the assembled MIPS binary into an executable you could trick the intel-based Linux into running, the processor **would** have an idea of what to do with it. That idea would be plain wrong, but it would still do (seemingly) random stuff and most probably fault within a few cycles. – spectras Feb 02 '18 at 02:11
  • 1
    @Wilbert You're probably missing that thrig was referring to the [Red Hat Linux](https://en.wikipedia.org/wiki/Red_Hat_Linux) that is distinct from Red Hat *Enterprise* Linux. – Bob Feb 02 '18 at 03:31
25

Adding to the excellent @thrig and @DopeGhoti answers: Unix or Unix-like OSes, including Linux, were traditionally always designed and aligned more for the portability of source code than binaries.

If having nothing hardware specific or being a simple source as in your example, you may move it without any problem at all from between pretty much any version of Linux or architecture as source code as long as the destination servers have the C development packages installed, the necessary libraries, and the corresponding development libraries installed.

As far porting more advanced code from older versions of Linux distant in time, or more specific programs like kernel modules for different kernel versions, you might have to adapt and modify source code to account for deprecated libraries/APIs/ABIs.

Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
22

By default, you'll almost certainly run into problems with external libraries. Some of the other answers go into more details about those problems, so I won't duplicate their work.

You can, however, compile many programs - even non-trivial ones - to be portable between Linux systems. The key is toolkit called the Linux Standard Base. The LSB is designed for creating just these types of portable applications. Compile an application for LSB v5.0 and it will run on any other Linux environment (of the same architecture) that implements LSB v5.0. A few Linux distros are LSB compliant, and others include LSB toolkits/libraries as an installable package. If you build your application using the LSB tools (like the lsbcc wrapper for gcc) and link to the LSB version of libraries, you'll create a portable application.

bta
  • 539
  • 2
  • 6
  • and with `qemu` you can even run programs compiled for different archtecture, (not high performace, but you can run them) – Jasen Jan 30 '18 at 09:34
  • 1
    I was not even aware of the Linux Standard Base toolkit, so thank you ! I started working with C/C++ a very long time ago, so much of the info in these answers is new to me. And very helpful. – JCDeen Jan 30 '18 at 15:26
  • 1
    The Wikipedia article says Debian and Ubuntu don't implement LSB (and have no intention to do so.) – BlackJack Jan 30 '18 at 17:12
  • 2
    @BlackJack- The distro itself doesn't implement 100% of it as part of the core OS, but you can install LSB-compatibility libraries and toolkits as optional packages. I've used Ubuntu (for instance) to build LSB-compatible programs that then ran on Suse and Centos, you just need to `apt-get install` a couple of packages. – bta Jan 30 '18 at 17:28
11

Maybe.

Things that tend to break it include.

  1. Different architectures. Obviously totally different architectures won't work (unless you have something like user mode qemu with binfmt_misc but that is hardly a normal configuration). x86 binaries may work on amd64 but only if the required 32-bit libraries are available.
  2. Library versions. If the soversion is wrong then it won't find the library at all. If the soversion is the same but the binary is built against a newer version of the library than it's running against then it may fail to load because of new symbols or new versions of symbols. In particular glibc is a heavy user of symbol versioning, so binaries built against a newer glibc are very likely to fail with an older glibc.

If you avoid using any fast-changing libraries, avoid changes of architecture and build on the oldest distro you want to target you have a good chance of making one binary work on many distros.

plugwash
  • 4,242
  • 1
  • 18
  • 32
5

In addition to some of the things mentioned previously, there have been some changes in executable file format. For the most part, linux uses ELF, but older versions used a.out or COFF.

The start of a wikihole:

https://en.wikipedia.org/wiki/Comparison_of_executable_file_formats

There might be a way to get older versions to run newer formats, but I've personally never looked into it.

Ben
  • 151
  • 2