1

How to run through an array with multiple processes (for example 2) in parallel? For example I have this code:

from multiprocessing import Process, Value import time, os n=Value('i', 0) l=[1, 2, 3, 4, 5] def fun(): try: while True: #time.sleep(0.01) not used, explanation below print os.getpid(), l[n.value] n.value+=1 except: return for i in range(2): p=Process(target=fun) p.start()

Expected output should be like:

111 1 112 2 111 3 112 4 111 5

But I'm getting:

111 1 111 2 111 3 111 4 111 5

Only one process is running through an array.

I'm getting expected result only by putting time.sleep() with any very small value, but I'm looking for solution without it, if possible.

Improve this question
2
  • The output continues on like that? (i.e. always 111)? Commented Oct 31, 2017 at 0:37
  • @information_interchange if there is no sleep - yes, only one process is running Commented Oct 31, 2017 at 0:42

1 Answer 1

Reset to default
1

The problem is because your task executes so quickly the first process executed finishes before the second one even gets started. This is why putting the call to sleep() "fixes" things—it's because it slows the task down enough to give second time enough time to start, so they can both run concurrently for a while

You can see this with your code if you make the l list a lot larger, say l = range(1, 1001).

To further illustrate this, below is a modified version of your code that also shows that eventually both of them will be running at the same time. It also prints out a little more information about what is occurring within each task:

from multiprocessing import Process, Value import time, os n = Value('i', 0) l = [1, 2, 3, 4, 5] def fun(cycles): assert cycles >= 0 while cycles: try: while True: with n.get_lock(): print os.getpid(), l[n.value] n.value += 1 except IndexError: print('list index out of range exception occurred, resetting "n"') with n.get_lock(): # Reset for another cycle n.value = 0 except Exception as exc: print('unexpected exception {} occurred'.format(exc)) break cycles -= 1 print('{} task terminating'.format(os.getpid())) if __name__ == '__main__': cycles = 100 for i in range(2): p = Process(target=fun, args=(cycles,)) p.start() print('done')
Improve this answer
Sign up to request clarification or add additional context in comments.

9 Comments

This would seem to explain it, I was thinking of a lack of context switch/processor scheduling issue
Sorry I just forgot to put brackets here to in question. Btw on pc it's already calling start() and still not working as expected. Edited
Works for me on a Windows PC.
Hmmm... Can it be some PC issue? I'm running linux
I understand what you want. The results are likely because of overhead of task switching verses the fact that the target processes have an extremely tight loop running almost continuously that is causing the them to behave as they do. This would also explain why adding a small amount of sleep() to the tight loop helps.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.