I can see from path= r'C:\Users\final_tweets_de' that your platform is Windows. When you do multiprocessing under Windows your code that creates the sub-processes must absolutely be executed in a block as follows:

import multiprocessing as mp def cube(x): return x**3 if __name__ == '__main__': pool = mp.Pool(processes=2) results = pool.map(cube, range(1,7)) print(results) 

Otherwise you get into a recursive loop where the sub-process will attempt to create a new pool and new sub-processes ad-infinitum. Fix this problem and re-test. The easiest way is to wrap your code in a function (call it main for example) and then add:

if __name__ == '__main_': main() 

I can see from path= r'C:\Users\final_tweets_de' that your platform is Windows. When you do multiprocessing under Windows your code that creates the sub-processes must absolutely be executed in a block as follows:

import multiprocessing as mp def cube(x): return x**3 if __name__ == '__main__': pool = mp.Pool(processes=2) results = pool.map(cube, range(1,7)) print(results) 

Otherwise you get into a recursive loop where the sub-process will attempt to create a new pool and new sub-processes ad-infinitum. Fix this problem and re-test. The easiest way is to wrap your code in a function (call it main for example) and then add:

if __name__ == '__main_': main() 

Also, why are you only using 2 processes or 5 in your actual example. By not specifying an argument to the Pool constructor, you will create a pool size equal to the number of processors actually available on your computer. Not a bad default that.