How to wake up inactive processes in linux?

Question

I am connected to a linux machine over ssh. There I used screen to run multiple processes that will keep running after I log out of my session. However, after a while the processes stopped producing output (to stdout) even though they are taking up memory. I want to know how I can reactivate them. Here is what the list of processes under my username looks like (top -u MY_USERNAME):

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 1268 abcdef    20   0  100m 2044 1012 S  0.0  0.0   0:00.03 sshd
 1269 abcdef    20   0  105m 1940 1456 S  0.0  0.0   0:00.06 bash
12375 abcdef    20   0  115m 1452  808 S  0.0  0.0   0:00.02 screen   # contains python procs
12615 abcdef    20   0  406m  58m 2140 S  0.0  0.0   0:02.23 python   # parent process
12618 abcdef    20   0 5170m 4.7g 1176 S  0.0  1.9 546:19.26 python   # child process
12619 abcdef    20   0 5164m 4.7g 1172 S  0.0  1.9 539:33.64 python   # child process
12620 abcdef    20   0 5185m 4.7g 1172 S  0.0  1.9 547:46.92 python   # child process
12621 abcdef    20   0 5171m 4.7g 1168 S  0.0  1.9 540:45.53 python   # child process
12622 abcdef    20   0 5179m 4.7g 1172 S  0.0  1.9 546:50.28 python   # child process
12623 abcdef    20   0 5171m 4.7g 1168 S  0.0  1.9 549:20.39 python   # child process
12624 abcdef    20   0 5178m 4.7g 1172 S  0.0  1.9 543:33.55 python   # child process
12625 abcdef    20   0 5177m 4.7g 1172 S  0.0  1.9 545:59.51 python   # child process

If it's relevant: the processes do not require user input. I do not have sudo privileges.

You can see under the S column, all processes are designated S. How do I change them to R? Will that make them resume their output to stdout?

Note: I checked the question here but that refers to manually suspending and waking processes. I am not sure why my processes went to sleep in the first place and I want to wake them up.

Edit: After checking the question here, I ran the command strace -p 12622 for a child process and got: futex(0x984271b0, FUTEX_WAIT_PRIVATE, 0, NULL. Wikipedia says it's some kind of a lock. I am not sure if that is significant.

Answer 1

“Sleep” is not really the right metaphor for this process state if you aren't a kernel designer. “Waiting” would be a better description. “Sleep” in the intuitive sense of waiting to be woken up is called “Stopped”, shown as Z.

The process isn't waiting to be woken up by the user, it's waiting for some specific condition to be met. For example, it's reading data from a file and waiting for the data to be loaded from the disk, or it's serving a network connection and waiting for an incoming packet, or it's paused for a set time and waiting for the timeout to expire, etc. The only way to “wake it up” is to arrange for the condition to be met.

As you've discovered, calling strace -p $PID (on Linux, or the equivalent on other unix variants) may tell you what the process is doing, and in particular what it's waiting for, from a low-level view. However there's no guarantee that this will give you useful data. A futex is a type of lock, and in general the only way to know who's supposed to release the lock is to understand precisely what the program is doing and how the underlying libraries use that core locking feature.

It's possible that the programs have stopped producing output because they've stopped receiving input. It's possible that one of the programs is working and the others are waiting for it. It's possible that there's a bug in the program and the processes are in a deadlock. Whatever it is, it's an application problem, not a system problem. There's no generic way to make the program “wake up”: the program is running normally. It may not be doing what you want, but it's doing what it's been written to do.