why my Multi-threading code spend More time than Single-threading code

Question

this is my code :

import re,threading class key_value: def __init__(self,filename='a.txt'): self.filename = filename def __getitem__(self,key): file = open(self.filename,'r') data = file.read() value = re.findall(r''+str(key)+' - (\S+)',data) return value def __setitem__(self,key,value): data = open(self.filename).read() b = re.split(r'('+str(key)+' - )\S*', data) # if len(b) == 1: result = str(key)+' - '+str(value) result += '\n' file = open(self.filename,'a') file.write(result) return elif type(value) == type([]): result = "".join(str(x) + value.pop(0) if x == str(key)+' - ' else x for x in b) else : result = "".join(str(x) + str(value) if str(x) == str(key)+' - ' else x for x in b) file = open(self.filename,'w') file.write(result) def run(self): print 'the thread is running!!' class do_thread(threading.Thread): def __init__(self,filename='a.txt',key=None,value=None): threading.Thread.__init__(self) self.filename = filename self.key=key self.value=value def run(self): print 'the thread is running!!' a = key_value(self.filename) if(self.key and self.value): a[self.key] = self.value elif(self.key): self.value = a[self.key] #'''#Multi-threading code for i in range(1000): a = do_thread(key=i,value=i) #print 'the main programme' a.start() #a.join() #print 'game over' '''# Single-threaded code for i in range(1000): a = key_value() a[i] = i '''

my boss told me to add Multi-threading to my code ,

i add it , but i find the Multi-threading spend more time ,

so what is the useful of the Multi-threading that my boss said ,

thanks

You're spawning 1000 threads, and starting a thread has some overhead. Threads give you parallelism, but don't automatically make things go faster. In your example they're all writing to the same file at the same time, which can't be what you intended. — samplebias
– samplebias, Commented Mar 19, 2011 at 14:43
On a side note, have a look at the style guide and if you're using Python 2.x, get rid of that old-style class. — user395760
– user395760, Commented Mar 19, 2011 at 14:43

score 3 · Accepted Answer · 2011-03-19 14:50:55Z

CPython (the main Python implementation) doesn't multi-thread. The GIL gets in the way. So you just have two threads running on one core (i.e. they don't run in parallel) but with all the context switching and other overhead a thread necesarily has. Note though:

Even if you do multi-thread, you're still limited by the number of cores. A dual-core computer can only do two computations at the same time, launching a thousand threads won't change that, no matter whether they're split between two cores or all run on one.
But running more threads than cores can still be useful if you're e.g. doing lots of I/O - the CPU is idle while it waits for the I/O to be completed anyway, so you're not limited by CPU time.

Edit: From the comments it seems your computer is incapable of multi-threading anyway. Well, the above still holds.

Steve-o · Accepted Answer · 2011-03-19 14:52:21Z

The application is IO bound not CPU bound so multi-threading is not going to help.

Also, as noted 1,000 threads is not going to be productive, try smaller numbers, i.e. 2 - 4, it is popular to try up to around 2 × number of cores. Increasing the number of threads too high will result in the overhead of thread management causing the application to be significantly slower.

philnext · Accepted Answer · 2011-03-19 14:42:15Z

The multi-thread is usefull and efficient when you have to access to different ressources (files, network, user interface...) on the same time. In your code it seems to me that you access at only one ressource, a file, so the mutlti-thread is less efficient

BenjaminB · Accepted Answer · 2011-03-19 15:09:22Z

I didn't read your code in details but test it on a multi-core computer, you will probably see an improvement.

Luka Rahne · Accepted Answer · 2011-03-19 15:13:54Z

One reason is that accessing one file at time can be actual much faster than accessing multiple files at same time, due to reading overhead. (you know disk has limited cache, and it is always best to read file as stream from beginning to end).

In anyway bottleneck is disk. And more treads you have asking for resources, the worst it gets.

Y.H Wong · Accepted Answer · 2011-03-19 19:05:41Z

David Beazley has done an excellent investigation of this phenomenon here. This is a video of the talk. In short, your threads are battling each other to send and respond to signals in order to acquire the GIL. And no, this does not only happen to CPU-bound threads, IO-bound threads suffer from the same problem too.

Collectives™ on Stack Overflow

why my Multi-threading code spend More time than Single-threading code

6 Answers 6

Comments

Comments

Comments

Comments

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

Comments

Comments

Comments

Comments

Comments

Comments

Linked

Related