Signal or mutex in Python threading

Question

I wrote a Python script which use GCC and dockross to cross build an application on Windows and Linux. Because I want to speed up the time spending on building code, I use threading module to implement all four building processes (Linux and Windows, 32 and 64 bit).

But I found the two GCC building objects (for Linux 32 bit and 64 bit) are in a race condition. If I want to do the two buildings at the same time, some errors will occur, and the same situation happened on the dockcross building process.

The two dockcross building objects are in a race condition.

Are there any function or module I can use in Python for me to implement the two threads as a coupled thread, when one thread is finished, it will signal to its coupled thread to start?

Like the code below, I want the worker[0] and worker[1] have a signal mechanism and also between worker[2] and worker[3].

def main(): linux32_builder = builder( "linux32", "make CFLAGS=-m32 all", "./", "./", "/root/crossCompile/releaseFolder/") linux64_builder = builder( "linux64", "make all", "./", "./", "/root/crossCompile/releaseFolder/") win64_builder = builder( "win64", "dockcross-windows-x64 make all", "./", "./", "/root/crossCompile/releaseFolder/") win32_builder = builder( "win32", "dockcross-windows-x86 make all", "./", "./", "/root/crossCompile/releaseFolder/") # linux32_builder.copySourceCodeFromShare() Worker = [] Worker.append(Make_Thread(1, linux32_builder)) Worker.append(Make_Thread(2, linux64_builder)) Worker.append(Make_Thread(3, win64_builder)) Worker.append(Make_Thread(4, win32_builder)) for i in Worker: i.start() i.join()

Cris P · Accepted Answer · 2016-12-02 22:12:53Z

In Python, thread.join() blocks until the thread terminates. I believe your code sample does not run the threads in parallel, instead waiting for each thread to terminate before starting the next. You may want to verify that each thread does not terminate before the actual build process completes, otherwise you may still have race conditions. (I presume the build uses external processes.)

Assuming that your various _builder objects are functions and that Make_Thread uses them as main thread functions, I would suggest using sequential execution rather than threads to get the effect you desire.

The code could look something like this:

linux = Make_Thread(1, lambda: linux32_builder(), linux64_builder()) windows = Make_Thread(2, lambda: win32_builder(), win64_builder()) linux.start() windows.start() linux.join() windows.join()

(I am guessing at the numeric arguments to Make_Thread.)

The lambda expressions actually return tuples, but it is guaranteed that the builders will be called in order from left to right.

Alternatively, you could make two threads in addition to the four shown in your code, and these two threads could each control two of the other threads. The Linux builder thread main function would look something like:

def linux(): Worker[0].start() Worker[1].start() Worker[0].join() Worker[1].join()

and the the Windows builder thread main function would look like:

def windows(): Worker[2].start() Worker[3].start() Worker[2].join() Worker[3].join()

You would then start each of these two controlling threads and finally join them. This approach is more complex, with more code and more threads.

Hi Cris, I found my problem is on for every thread I change the working directory which will cause problems because the working directory is a global variable for one process. So I change my implement to multiprocess and now it work great. This script is about cross compiling. My work flow is described in here.

Collectives™ on Stack Overflow

Signal or mutex in Python threading

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related