`This code is an attempt to use a queue to feed tasks to a number worker processes.

I wanted to time the difference in speed between different number of process and different methods for handling data.

But the output is not doing what I thought it would.

from multiprocessing import Process, Queue import time result = [] base = 2 data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 23, 45, 76, 4567, 65423, 45, 4, 3, 21] # create queue for new tasks new_tasks = Queue(maxsize=0) # put tasks in queue print('Putting tasks in Queue') for i in data: new_tasks.put(i) # worker function definition def f(q, p_num): print('Starting process: {}'.format(p_num)) while not q.empty(): # mimic some process being done time.sleep(0.05) print(q.get(), p_num) print('Finished', p_num) print('initiating processes') processes = [] for i in range(0, 2): if __name__ == '__main__': print('Creating process {}'.format(i)) p = Process(target=f, args=(new_tasks, i)) processes.append(p) #record start time start = time.time() # start process for p in processes: p.start() # wait for processes to finish processes for p in processes: p.join() #record end time end = time.time() # print time result print('Time taken: {}'.format(end-start)) 

I expect this:

Putting tasks in Queue initiating processes Creating process 0 Creating process 1 Starting process: 1 Starting process: 0 1 1 2 0 3 1 4 0 5 1 6 0 7 1 8 0 9 1 10 0 11 1 23 0 45 1 76 0 4567 1 65423 0 45 1 4 0 3 1 21 0 Finished 1 Finished 0 Time taken: <some-time> 

But instead I actually get this:

Putting tasks in Queue initiating processes Creating process 0 Creating process 1 Time taken: 0.01000523567199707 Putting tasks in Queue Putting tasks in Queue initiating processes Time taken: 0.0 Starting process: 1 initiating processes Time taken: 0.0 Starting process: 0 1 1 2 0 3 1 4 0 5 1 6 0 7 1 8 0 9 1 10 0 11 1 23 0 45 1 76 0 4567 1 65423 0 45 1 4 0 3 1 21 0 Finished 0 

THere seem to be two major problems, I am not sure how related they are:

1. The print statements such as: Putting tasks in Queue initiating processes Time taken: 0.0` are repeated systematically though out the code - I say system systematically the repeat exactly every-time.

2. The second process never finishes, it never recognizes the queue is empty and therefore fails to exit