-1

I'm running some long running tasks (15 minutes up to a day) in Ubuntu 18.04 under WSL2, itself under Win11.

A couple of times my terminal has stopped responding and never came back, but I could see from resource usage that the task was still running.

How can I best start this task so it's a background task that will persist even if my interactive session ends prematurely?

Important requirements are:

  • The task must be running in the background
  • I should be able to reconnect to the session if something happens to it
  • Output should be logged somewhere in case I don't reconnect in time
  • The task is resource intensive, so the method of starting it should not decrease performance substantially
  • Ideally I should be able to do all of the above locally as well as over SSH

Just in case it's important, the actual task is a tensorflow job running under python 3.9, within an anaconda environment.

Ian Newson
  • 123
  • 1
  • 6
  • 1
    Does this answer your question? [Keep processes running after SSH session disconnects](https://unix.stackexchange.com/questions/479/keep-processes-running-after-ssh-session-disconnects) or [How can I run a command which will survive terminal close?](https://unix.stackexchange.com/questions/4004/how-can-i-run-a-command-which-will-survive-terminal-close) – pLumo Feb 14 '22 at 21:39
  • Does this answer your question? [How can I run a command which will survive terminal close?](https://unix.stackexchange.com/questions/4004/how-can-i-run-a-command-which-will-survive-terminal-close) – NotTheDr01ds Feb 15 '22 at 20:30

1 Answers1

1

You can use screen for that.

You can simply reconnect to a session if the connection gets lost via screen -r.

Sending the task to background as well as logging can additionally be done with & and >> if necessary.

mashuptwice
  • 1,283
  • 5
  • 22