111

I am starting with multi-threads in python (or at least it is possible that my script creates multiple threads). would this algorithm be the right usage of a Mutex? I haven't tested this code yet and it probably won't even work. I just want processData to run in a thread (one at time) and the main while loop to keep running, even if there is a thread in queue. 

from threading import Thread from win32event import CreateMutex mutex = CreateMutex(None, False, "My Crazy Mutex") while(1) t = Thread(target=self.processData, args=(some_data,)) t.start() mutex.lock() def processData(self, data) while(1) if mutex.test() == False: do some stuff break

Edit: re-reading my code I can see that it is grossly wrong. but hey, that's why I am here asking for help.

Improve this question
7
  • It is very difficult to figure out what you are trying to do. You'll need to explain your intent in more detail. Commented Jul 22, 2010 at 14:45
  • @Marcelo Cantos, sorry you are probably right. I want my code in processData to start in a new tread. I only want one thread to be able to process data at a time and in order of which data was sent to process data. I also want the main while loop to keep looping while other threads are in queue. Commented Jul 22, 2010 at 14:51
  • @Richard: Why do you want to use threads if you plan to serialise all processing anyway? What's wrong with a simple loop? Also, why do you want the main thread to keep looping? It will just burn CPU, possibly starving other threads. Commented Jul 22, 2010 at 23:08
  • @Marcelo Cantos, This is not my actual program. My actual script has reached over 500 lines of code, which include database entries and email. I want the main thread to just read a serial port so that there is less of a chance of the buffer being filled before the data received is processed. Correct me if I am wrong but I think this is exactly where one would use either threads or multiprocessing Commented Jul 23, 2010 at 11:39
  • @Richard: That makes sense for the main thread, but if all other processing is going to be sequential, then you could just create one other thread to do all that other work. Commented Jul 23, 2010 at 12:49

4 Answers 4

Reset to default
231

I don't know why you're using the Window's Mutex instead of Python's. Using the Python methods, this is pretty simple:

from threading import Thread, Lock mutex = Lock() def processData(data): with mutex: print('Do some stuff') while True: t = Thread(target = processData, args = (some_data,)) t.start()

But note, because of the architecture of CPython (namely the Global Interpreter Lock) you'll effectively only have one thread running at a time anyway--this is fine if a number of them are I/O bound, although you'll want to release the lock as much as possible so the I/O bound thread doesn't block other threads from running.

An alternative, for Python 2.6 and later, is to use Python's multiprocessing package. It mirrors the threading package, but will create entirely new processes which can run simultaneously. It's trivial to update your example:

from multiprocessing import Process, Lock mutex = Lock() def processData(data): with mutex: print('Do some stuff') if __name__ == '__main__': while True: p = Process(target = processData, args = (some_data,)) p.start()
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

"this is fine if a number of them are I/O bound" Actually it's also fine if they're CPU bound, so long as this is in calls to libraries written in C rather than in pure Python code (e.g. if you're manipulating large matrices in numpy) because the GIL is unlocked during those calls.
@demented hedgehog: the mutex module is deprecated. Creating mutexes using the threading and multiprocessing modules, which is done in the answer, is not deprecated.
excelent for I/O specially alignment of messages
22

I would like to improve answer from chris-b a little bit more.

See below for my code:

from threading import Thread, Lock import threading mutex = Lock() def processData(data, thread_safe): if thread_safe: mutex.acquire() try: thread_id = threading.get_ident() print('\nProcessing data:', data, "ThreadId:", thread_id) finally: if thread_safe: mutex.release() counter = 0 max_run = 100 thread_safe = False while True: some_data = counter t = Thread(target=processData, args=(some_data, thread_safe)) t.start() counter = counter + 1 if counter >= max_run: break

In your first run if you set thread_safe = False in while loop, mutex will not be used, and threads will step over each others in print method as below;

Not Thread safe

but, if you set thread_safe = True and run it, you will see all the output comes perfectly fine;

Thread safe

hope this helps.

Improve this answer

1 Comment

copy thread_safe at the start of processData() and use that copy within the function. thread_safe is an external state variable which could change during the execution, again leading to no releasing the mutex.
16

This is the solution I came up with: 

import time from threading import Thread from threading import Lock def myfunc(i, mutex): mutex.acquire(1) time.sleep(1) print "Thread: %d" %i mutex.release() mutex = Lock() for i in range(0,10): t = Thread(target=myfunc, args=(i,mutex)) t.start() print "main loop %d" %i

Output:

main loop 0 main loop 1 main loop 2 main loop 3 main loop 4 main loop 5 main loop 6 main loop 7 main loop 8 main loop 9 Thread: 0 Thread: 1 Thread: 2 Thread: 3 Thread: 4 Thread: 5 Thread: 6 Thread: 7 Thread: 8 Thread: 9
Improve this answer

2 Comments

There's a potential deadlock. If the print statement throws an Exception, you'll never release the mutex. You need to use try/release or with to ensure the lock is released. See my answer.
Also, there's no need to pass the mutex as an argument to the function. It's available in the global scope.
11

You have to unlock your Mutex at sometime...

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.