15

I have a bash script I want to run every 5 minutes from cron... but there's a chance the previous run of the script isn't done yet... in this case, i want the new run to just exit. I don't want to rely on just a lock file in /tmp.. I want to make sure sure the process is actually running before i honor the lock file (or whatever)...

Here is what I have stolen from the internet so far... how do i smarten it up a bit? or is there a completely different way that's better?

if [ -f /tmp/mylockFile ] ; then echo 'Script is still running' else echo 1 > /tmp/mylockFile /* Do some stuff */ rm -f /tmp/mylockFile fi 
1

13 Answers 13

24
# Use a lockfile containing the pid of the running process # If script crashes and leaves lockfile around, it will have a different pid so # will not prevent script running again. # lf=/tmp/pidLockFile # create empty lock file if none exists cat /dev/null >> $lf read lastPID < $lf # if lastPID is not null and a process with that pid exists , exit [ ! -z "$lastPID" -a -d /proc/$lastPID ] && exit echo not running # save my pid in the lock file echo $$ > $lf # sleep just to make testing easier sleep 5 

There is at least one race condition in this script. Don't use it for a life support system, lol. But it should work fine for your example, because your environment doesn't start two scripts simultaneously. There are lots of ways to use more atomic locks, but they generally depend on having a particular thing optionally installed, or work differently on NFS, etc...

Sign up to request clarification or add additional context in comments.

3 Comments

Why cat /dev/null >> $lf instead of touch $lf? Why sleep when there's no loop?
touch(1) is good too. The sleep is just for demonstration, to make my script fragment testable on its own.
(That is, the pid will vanish as soon as that script ends)
9

You might want to have a look at the man page for the flock command, if you're lucky enough to get it on your distribution.

 NAME flock - Manage locks from shell scripts SYNOPSIS flock [-sxon] [-w timeout] lockfile [-c] command... 

2 Comments

This solution avoids the race condition between the checking and the obtaining of the lock most cleanly, unlike some other suggested solutions. If you have 'flock(1)' (which comes with util-linux in debian/Ubuntu etc) definitely use it. For the OP use case, you need a short timeout (e.g. -w 1). If the command is already running, flock will give up after 1 sec, and will not run the command, otherwise, it will obtain the lock and start the command.
Instead of using -w 1 to timeout and exit after 1 second, the OP could use -n, --nb, --nonblock Fail (with an exit code of 1) rather than wait if the lock cannot be immediately acquired. See example in my answer which was sadly added way after this topic was active.
8

Never use a lock file always use a lock directory. In your specific case, it's not so important because the start of the script is scheduled in 5min intervals. But if you ever reuse this code for a webserver cgi-script you are toast.

if mkdir /tmp/my_lock_dir 2>/dev/null then echo "running now the script" sleep 10 rmdir /tmp/my_lock_dir fi 

This has a problem if you have a stale lock, means the lock is there but no associated process. Your cron will never run.

Why use a directory? Because mkdir is an atomic operation. Only one process at a time can create a directory, all other processes get an error. This even works across shared filesystems and probably even between different OS types.

1 Comment

Lock files don't work across NFS -- which is common in shared hosting. We often use lock directories for the same reason.
5

If you want to check the process's existence, just look at the output of

ps aux | grep your_script_name

If it's there, it's not dead...

As pointed out in the comments and other answers, using the PID stored in the lockfile is much safer and is the standard approach most apps take. I just do this because it's convenient and I almost never see the corner cases (e.g. editing the file when the cron executes) in practice.

3 Comments

In addition, you can store the PID of the current BASH process in that lock file. BASH provides a variable '$$' (minus the quotes) that gives that number. See here: tldp.org/LDP/abs/html/internalvariables.html for some of those variables
-1 What if "emacs your_script_name" is one of the processes running?
You're both absolutely right. The PID stored in the file would be a much better way to go. I just tend to do it this way because I'm lazy, my scripts aren't mission-critical, and it usually works (pretty rare that I would actually be editing the script when the cron executes).
4

Store your pid in mylockFile. When you need to check, look up ps for the process with the pid you read from file. If it exists, your script is running.

Comments

3

If you use a lockfile, you should make sure that the lockfile is always removed. You can do this with 'trap':

if ( set -o noclobber; echo "locked" > "$lockfile") 2> /dev/null; then trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT echo "Locking succeeded" >&2 rm -f "$lockfile" else echo "Lock failed - exit" >&2 exit 1 fi 

The noclobber option makes the creation of lockfile atomic, like using a directory.

Comments

3

As a one-liner and if you do not want to use a lockfile (e.g. b/c/ of a read only filesystem, etc)

test "$(pidof -x $(basename $0))" != $$ && exit 

It checks that the full list of PID that bear the name of your script is equal to the current PID. The "-x" also checks for the name of shell scripts.

Bash makes it even shorter and faster:

[[ "$(pidof -x $(basename $0))" != $$ ]] && exit 

Comments

1

In some cases, you might want to be able to distinguish between who is running the script and allow some concurrency but not all. In that case, you can use per-user, per-tty or cron-specific locks.

You can use environment variables such as $USER or the output of a program such as tty to create the filename. For cron, you can set a variable in the crontab file and test for it in your script.

Comments

1

you can use this one:

pgrep -f "/bin/\w*sh .*scriptname" | grep -vq $$ && exit 

Comments

1

I was trying to solve this problem today and I came up with the below:

COMMAND_LINE="$0 $*" JOBS=$(SUBSHELL_PID=$BASHPID; ps axo pid,command | grep "${COMMAND_LINE}" | grep -v $$ | g rep -v ${SUBSHELL_PID} | grep -v grep) if [[ -z "${JOBS}" ]] then # not already running else # already running fi 

This relies on $BASHPID which contains the PID inside a subshell ($$ in the subshell is the parent pid). However, this relies on Bash v4 and I needed to run this on OSX which has Bash v3.2.48. I ultimately came up with another solution and it is cleaner:

JOBS=$(sh -c "ps axo pid,command | grep \"${COMMAND_LINE}\" | grep -v grep | grep -v $$") 

Comments

0

You can always just:

 if ps -e -o cmd | grep scriptname > /dev/null; then exit fi 

But I like the lockfile myself, so I wouldn't do this without the lock file as well.

3 Comments

Depending on how portable this needs to be, you might need to tweak your ps options, and you might be able to add -q to grep instead of the /dev/null
Not co-worker proof enough: someone might be running "less scriptname" and that would be enough to prevent the script from running
mobrule: Ya, never thought of that, just because I would never do it this way ;-) The lockfile is better.
0

Since a socket solution has not yet been mentioned it is worth pointing out that sockets can be used as effective mutexes. Socket creation is an atomic operation, like mkdir is as Gunstick pointed out, so a socket is suitable to use as a lock or mutex.

Tim Kay's Perl script 'Solo' is a very small and effective script to make sure only one copy of a script can be run at any one time. It was designed specifically for use with cron jobs, although it works perfectly for other tasks as well and I've used it for non-crob jobs very effectively.

Solo has one advantage over the other techniques mentioned so far in that the check is done outside of the script you only want to run one copy of. If the script is already running then a second instance of that script will never even be started. This is as opposed to isolating a block of code inside the script which is protected by a lock. EDIT: If flock is used in a cron job, rather than from inside a script, then you can also use that to prevent a second instance of the script from starting - see example below.

Here's an example of how you might use it with cron:

*/5 * * * * solo -port=3801 /path/to/script.sh args args args # "/path/to/script.sh args args args" is only called if no other instance of # "/path/to/script.sh" is running, or more accurately if the socket on port 3801 # is not open. Distinct port numbers can be used for different programs so that # if script_1.sh is running it does not prevent script_2.sh from starting, I've # used the port range 3801 to 3810 without conflicts. For Linux non-root users # the valid port range is 1024 to 65535 (0 to 1023 are reserved for root). * * * * * solo -port=3802 /path/to/script_1.sh * * * * * solo -port=3803 /path/to/script_2.sh # Flock can also be used in cron jobs with a distinct lock path for different # programs, in the example below script_3.sh will only be started if the one # started a minute earlier has already finished. * * * * * flock -n /tmp/path.to.lock -c /path/to/script_3.sh 

Links:

Hope this helps.

Comments

0

You can use this.

I'll just shamelessly copy-paste the solution here, as it is an answer for both questions (I would argue that it's actually a better fit for this question).

Usage

  1. include sh_lock_functions.sh
  2. init using sh_lock_init
  3. lock using sh_acquire_lock
  4. check lock using sh_check_lock
  5. unlock using sh_remove_lock

Script File

sh_lock_functions.sh

#!/bin/bash function sh_lock_init { sh_lock_scriptName=$(basename $0) sh_lock_dir="/tmp/${sh_lock_scriptName}.lock" #lock directory sh_lock_file="${sh_lock_dir}/lockPid.txt" #lock file } function sh_acquire_lock { if mkdir $sh_lock_dir 2>/dev/null; then #check for lock echo "$sh_lock_scriptName lock acquired successfully.">&2 touch $sh_lock_file echo $$ > $sh_lock_file # set current pid in lockFile return 0 else touch $sh_lock_file read sh_lock_lastPID < $sh_lock_file if [ ! -z "$sh_lock_lastPID" -a -d /proc/$sh_lock_lastPID ]; then # if lastPID is not null and a process with that pid exists echo "$sh_lock_scriptName is already running.">&2 return 1 else echo "$sh_lock_scriptName stopped during execution, reacquiring lock.">&2 echo $$ > $sh_lock_file # set current pid in lockFile return 2 fi fi return 0 } function sh_check_lock { [[ ! -f $sh_lock_file ]] && echo "$sh_lock_scriptName lock file removed.">&2 && return 1 read sh_lock_lastPID < $sh_lock_file [[ $sh_lock_lastPID -ne $$ ]] && echo "$sh_lock_scriptName lock file pid has changed.">&2 && return 2 echo "$sh_lock_scriptName lock still in place.">&2 return 0 } function sh_remove_lock { rm -r $sh_lock_dir } 

Usage example

sh_lock_usage_example.sh

#!/bin/bash . /path/to/sh_lock_functions.sh # load sh lock functions sh_lock_init || exit $? sh_acquire_lock lockStatus=$? [[ $lockStatus -eq 1 ]] && exit $lockStatus [[ $lockStatus -eq 2 ]] && echo "lock is set, do some resume from crash procedures"; #monitoring example cnt=0 while sh_check_lock # loop while lock is in place do echo "$sh_scriptName running (pid $$)" sleep 1 let cnt++ [[ $cnt -gt 5 ]] && break done #remove lock when process finished sh_remove_lock || exit $? exit 0 

Features

  • Uses a combination of file, directory and process id to lock to make sure that the process is not already running
  • You can detect if the script stopped before lock removal (eg. process kill, shutdown, error etc.)
  • You can check the lock file, and use it to trigger a process shutdown when the lock is missing
  • Verbose, outputs error messages for easier debug

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.