Kill process if it takes more than xGiB of RAM in a given amount of time

Question

I am a software developer and I come into contact with a lot of unstable software. Recently I made a small game, which for some reason memleaks until the system hangs and is unresponsive. Usually, REISUB helps, but sometimes not even that and you need to do a hard poweroff.

Then it happened to me again with another program, so I thought to myself that this could easily be prevented by writing a script that monitors mem usage and if it crosses a certain value per PID over a certain amount of time, it gets a SIGKILL to take it down immediately.

Any ideas? Thanks

One way via [`prlimit`](https://unix.stackexchange.com/a/422732/123460), i.e. setting a limit for a single process from the OS side. The other option depends on the programming language you use: set a memory limit in the code itself. — FelixJN, Jul 14 '21 at 06:28

score 1 · Answer 1 · answered Jan 28 '21 at 15:40

1

Please install earlyoom and enable or any of its alternatives which are listed here:

https://github.com/hakavlad/nohang

earlyoom is now used by default by Fedora.

The Linux kernel OOM handling is quite horrible and more often than not doesn't work without user space utilities/helpers.

You can also use man limits.conf and cgroups to limit your application RAM use.

answered Jan 28 '21 at 15:40

Artem S. Tashkinov

26,392
4
33
64

Thank you for your response, but I asked about writing it, not just downloading someone else's software. – j3ff Jan 29 '21 at 14:02
1

There are already readily available tested solutions which solve your issue - I'm not sure why you need to code something. – Artem S. Tashkinov Jan 29 '21 at 15:28
1

Possible alternatives: https://github.com/facebookincubator/oomd "oomd leverages PSI and cgroupv2 to monitor a system holistically. oomd then takes corrective action in userspace before an OOM occurs in kernel space." or also launch your program using `systemd-run --user --no-block --scope -p MemoryHigh=1G $CMD` – Patrick Mevzek Feb 01 '21 at 15:07

score 0 · Accepted Answer · answered Jan 28 '21 at 15:47

Simple idea: check if memory exceeds a given value, and check again after some time. Hand out strikes, three strikes in a row will lead to a kill. Need to know: PID of process

#!/bin/bash
pid=$1
strike=0

#as long as process exists
while (kill -0 $pid 2>/dev/null) ; do
  #get RAM usage in kB
  ram=$(pmap -x $pid | tail -1 | awk '{print $3}')
  #compare to threshold, 1,000,000kB
  if [[ 1000000 -lt $ram ]] ; then
    strike=$((strike+1))
    if [[ strike -eq 3 ]] ; then
       kill $pid
       exit
    fi
  else
    strike=0
  fi
  sleep 5
done

Kill process if it takes more than xGiB of RAM in a given amount of time

2 Answers2