0

Here is a minimal example of what I am trying to do:

# If it does not exist, create a file to persist the unique identifier
if [ ! -f ~/.uid ]; then
   echo 0 > ~/.uid
fi

# Increment the unique identifier in the ~/.uid file
echo 1 + $(<~/.uid) | bc > ~/.uid

# Launch expensive computation that uses this unique identifier
uid=$(<~/.uid)
do_something_long $uid

Except, multiple instances of this script could be running at the same time, and do_something_long should be invoked with an unique identifier (preferably human-readable, so starting from 0 or 1).

I tried to use flock(1) for acquiring a lock, but most safe examples use a subshell to bind to a file descriptor, which prevent me to access the unique identifier in the parent shell. And I don't want do_something_long in the subshell, as it would hog the lock for too long.

r0levrai
  • 11
  • 2
  • This seems like an XY problem - not sure what the ultimate goal is. If it's to run multiple instances of `do_something_long` in parallel with unique value passed to each, then `xargs` with `--process-slot-var` might be useful: https://unix.stackexchange.com/a/449225/70524 – muru Apr 26 '22 at 10:18
  • 1
    Similar (duplicate?): [Bash script, increase/decrease a number in a txt file with arguments](//unix.stackexchange.com/q/618310) – Stéphane Chazelas Apr 26 '22 at 10:32
  • is the PID maybe an identifier that you want? no two running processes would ever have the same PID? But I do agree with muru, it would be helpful to understand what the purpose of this `uid` is – in a lot of use case I can think of (different!) good approaches to avoid doing exactly this :) – Marcus Müller Apr 26 '22 at 12:34
  • Thanks for the comments! To elaborate, in my case, do_something_long is a machine learning experiment, and the uid is mostly to create a directory that store all the logs and results (so, having a logically incrementing number is also useful for bookkeeping). I schedule them on a distributed system, so I can't be sure when they run and they can possibly start at the same time when lots of resources are freed by the end of a subsequent one. In which case I was trying to avoid files from two experiments to clash in one directory. – r0levrai Apr 27 '22 at 00:56
  • I realized in my case, generating the uid in the (sequential) scheduling procedure would do the trick, so it was probably an XY problem :D But I still learned a lot from Nick Matteo's answer so I'll keep it! – r0levrai Apr 27 '22 at 00:56

1 Answers1

1

This appears to do roughly what you described. First define a function which runs in a subshell.

getuid() (
    flock 9
    oldid=$(<~/.uid)
    newid=$((oldid+1))
    echo $newid >&9
    echo $newid
) 9<>~/.uid

Then use myuid=$(getuid) whenever you want a new ID.

Nick Matteo
  • 1,465
  • 11
  • 13
  • Perfect, I was missing the fact that you could read-write with <>, and also use std as a way of communication from the subshell. Thanks! – r0levrai Apr 27 '22 at 00:56
  • Newbie question, does picking an arbitrary file descriptor like 9 ever clash with existing or other descriptors? – r0levrai Apr 27 '22 at 01:02
  • @r0levrai, never say never, but fd numbers < 10 should be relatively safe. E.g. Bash's man page says "Redirections using file descriptors _greater than 9_ should be used with care, as they may conflict with file descriptors the shell uses internally." Scripts that do fd juggling have to use fixed numbers anyway, as there's no standard way to allocate them dynamically. A shell that would make that fail wouldn't be a very good one. (though Bash and others do have `{var} – ilkkachu Apr 27 '22 at 09:27