Pass multiple commands to flock

Question

flock -x -w 5 ~/counter.txt 'COUNTER=$(cat ~/counter.txt); echo $((COUNTER + 1)) > ~/counter.txt'

How would I pass multiple commands to flock as in the example above?

As far as I understand, flock takes different flags (-x for exclusive, -w for timeout), then the file to lock, and then the command to run. I'm not sure how I would pass two commands into this function (set variable with locked file's contents, and then increment this file).

My goal here is to create a somewhat atomic increment for a file by locking it each time a script tries to access the counter.txt file.

score 7 · Accepted Answer · answered Dec 29 '13 at 20:16

Invoke a shell explicitly.

flock -x -w 5 ~/counter.txt sh -c 'COUNTER=$(cat counter.txt); echo $((COUNTER + 1)) > ~/counter.txt'

Note that any variable that you change is local to that shell instance. For example, the COUNTER variable will not be updated in the calling script: you'll have to read it back from the file (but it may have changed in the meantime), or as the output of the command:

new_counter=$(flock -x -w 5 ~/counter.txt sh -c 'COUNTER=$(cat counter.txt); echo $((COUNTER + 1)) | tee ~/counter.txt')

awesome, thanks so much! The explanation you provided definitely saved me from another hour of headaches — d-_-b, Dec 29 '13 at 20:17

score 4 · Answer 2 · edited Oct 05 '19 at 21:31

4

Or you can flock a file descriptor

exec {counterfd}<~/counter.txt
flock -x -w 5 "$counterfd"
COUNTER=$(cat ~/counter.txt)
COUNTER=$(( COUNTER +1 ))
echo "$COUNTER" >~/counter.txt
exec {counterfd}<&-

This also has the benefit of allowing you to use the counter variable directly, unlike subshell based approaches.

edited Oct 05 '19 at 21:31

Phernost

103
2

answered Sep 14 '15 at 18:01

Richard Maw

79
2

Please would you explain the `exec {counterfd}`? On its own, Debian errors `not found`, but it returns 0 with the redirection. Very odd. – Steve Almond Nov 14 '19 at 11:21

score 3 · Answer 3 · answered Jul 31 '17 at 13:25

The flock tool is a little tricky to use and the man page is pretty short. The man page provides three ways to use the tool:

flock [options] <file|directory> <command> [command args]
flock [options] <file|directory> -c <command>
flock [options] <file descriptor number>

The way this question is worded I would definitely use the third form of flock. If you go further down in the man page for flock there are some examples which show the exact syntax for using the third form:

#!/bin/bash
(
 flock -n 9 || exit 1
 echo "commands executed under lock..."
 echo "go here..."
) 9>/tmp/mylockfile

I added the #!/bin/bash.

I have successfully used this form of flock.

score -1 · Answer 4 · answered Aug 31 '21 at 10:48

-1

I had an interesting use case. I had to run my python script as cronjob using flock, every 30 minute. And due to some root path related stuff inside my python script instead of

python /home/myfolder/script.py

I needed to do

cd /home/myfolder/ && python script.py

To pass both commands in crontab -

*/30 * * * * cd /home/myfolder/ /usr/bin/flock -w 0 /home/myfolder/my-file.lock && python my_script.py > /dev/null 2>&1

It worked for me.

answered Aug 31 '21 at 10:48

Pankaj Tanwar

101
1

Ok, but this technically solves the issue of running multiple commands while locking though _not running multiple commands while locking_. – Kusalananda Sep 02 '21 at 20:24
Can you please be more specific about your comment? I might have understood it wrongly. It would help me improve my approach. – Pankaj Tanwar Sep 03 '21 at 13:41
Sorry. It seems that you solved the issue of running two separate commands while using `flock` by rearranging the command in such a way that you run the first command (`cd`) before invoking `flock`. This results in locking only while executing the _single_ command `python`. Hence my comment above. You also have a syntax error in your second crontab schedule (after the argument to `cd` there should be a `&&` or `;`). – Kusalananda Sep 03 '21 at 17:57

Pass multiple commands to flock

4 Answers4