11

Let's assume process runs in ulimited environment :

(
ulimit  ... -v ... -t ... -x 0 ...
./program
)

Program is terminated.

There might be many reasons : memory/time/file limit exceeded ; just simple segfault ; or even normal termination with return code 0.

How to check what was the reason of program termination, without modifying program?

P.S. I mean "when binary is given". Maybe some wrapper (ptrace-ing etc) might help?

Grzegorz Wierzowiecki
  • 13,865
  • 23
  • 89
  • 137

2 Answers2

7

Generally speaking, I don't think you can unfortunately. (Some operating systems might provide for it, but I'm not aware of the ones I know supporting this.)

Reference doc for resource limits: getrlimit from POSIX 2008.

Take for example the CPU limit RLIMIT_CPU.

  • If the process exceeds the soft limit, it gets sent a SIGXCPU
  • If the process exceeds the hard limit, it gets a plain SIGKILL

If you can wait() on your program, you could tell if it was killed by SIGXCPU. But you could not differentiate a SIGKILL dispatched for breach of the hard limit from a plain old kill from outside. What's more, if the program handles the XCPU, you won't even see that from outside.

Same thing for RLIMIT_FSIZE. You can see the SIGXFSZ from the wait() status if the program doesn't handle it. But once the file size limit is exceeded, the only thing that happens is that further I/O that attempts to test that limit again will simply receive EFBIG - this will be handled (or not, unfortunately) by the program internally. If the program handles SIGXFSZ, same as above - you won't know about it.

RLIMIT_NOFILE? Well, you don't even get a signal. open and friends just return EMFILE to the program. It's not otherwise bothered, so it will fail (or not) in whichever way it was coded to fail in that situation.

RLIMIT_STACK? Good old SIGSEGV, can't be distinguished from the score of other reasons to get delivered one. (You will know that that was what killed the process though, from the wait status.)

RLIMIT_AS and RLIMIT_DATA will just make malloc() and a few others start to fail (or receive SIGSEGV if the AS limit is hit while trying to extend the stack on Linux). Unless the program is very well written, it will probably fail fairly randomly at that point.

So in short, generally, the failures are either not visibly different from other process death reasons, so you can't be sure, or can be handled entirely from the program in which case it decides if/when/how it proceeds, not you from the outside.

The best you can do as far as I know is write a bit of code that forks of your program, waits on it, and:

  • check the exit status to detect SIGXCPU and SIGXFSZ (AFAIK, those signals will only be generated by the OS for resource limit problems). Depending on your exact needs, you could assume that SIGKILL and SIGSEGV were also related to resource limits, but that's a bit of a stretch.
  • look at what you can get out of getrusage(RUSAGE_CHILDREN,...) on your implementation to get a hint about the other ones.

OS-specific facilities might exist to help out here (possibly things like ptrace on Linux, or Solaris dtrace), or possibly debugger-type techniques, but that's going to be even more tied to your specific implementation.

(I'm hoping someone else will answer with some magic thing I'm completely unaware of.)

Mat
  • 51,578
  • 10
  • 158
  • 140
  • Ok. What about just those three: (Mem) exceeding memory limit, (Time) time limit, (Err) other error ? I know about making wrapper around `malloc` but unfortunately it does not solve memory problem in general, cause in general it's about system call `brk` (am I right?). – Grzegorz Wierzowiecki Oct 26 '11 at 19:25
  • 1
    Wrapping malloc won't help if you don't control the program. If you're talking about hacks like `LD_PRELOAD`ing that's borderline for your "not modifying the process" constraint, and it will help a bit, but not really - `malloc`, `brk`, `sbrk` and `mmap` will fail with `ENOMEM`, exactly as if you really were in a low memory situation (but far below the memory limits). Time limit is `RLIMIT_CPU`, I don't know of a wall-clock time limit. – Mat Oct 26 '11 at 19:33
  • Thanks for ensuring me about `brk`. As I see, requirement 'program is not handling signals X,Y,Z...' will solve problems of SIGXCPU, SIGXFSZ, SIGSEGV, thanks to waitpid (If I'm wrong, please correct me). – Grzegorz Wierzowiecki Oct 29 '11 at 10:30
  • 1
    SIGSEGV can be raised in situations that are not resource limit breaches (null pointer dereference being the most common thing that raises it) - you can't be sure its a ulimit hit that causes it. – Mat Oct 29 '11 at 10:32
  • Thanks for ensuring me about `brk`. As I see, requirement 'program is not handling signals X,Y,Z...' will solve problems of SIGXCPU, SIGXFSZ, SIGSEGV, thanks to waitpid. Am I right? – Grzegorz Wierzowiecki Oct 29 '11 at 10:46
  • btw. you might be interested in question related to topic: [Reliable and secure wrapping for system call](http://security.stackexchange.com/questions/8484/wrapping-system-call) – Grzegorz Wierzowiecki Oct 29 '11 at 11:09
  • thanks for all tips. Even I can't be sure of reason, it's always some point of reasoning -> for eaxmple, return set of possibilities and try to narrow it. Btw. what are your feeling about idea of waiting for child process with `wait4(2)`, and based on rusage info, try to guess if some limit has been exceeded ? – Grzegorz Wierzowiecki Oct 29 '11 at 17:51
3

I am currently doing some work on the same issue. I have been able to have a partial solution to it. I have used audit susbsystem. You can track the work at [1].

[1] https://github.com/PaulDaviesC/Logging-limits.conf

PaulDaviesC
  • 537
  • 1
  • 4
  • 12