4

A question. Using multiprocessing is for having a faster code. But after using the following framework, getting the outputs takes the same time or more as it took in normal code. 

import multiprocessing def code() : my code if __name__ == '__main__' : p = multiprocessing.Process(target = code) p.start() p.join()

because of being 2 processors laptop, after running this code the program wants me to import data two times. The problem is time which did not make any sense in this way. I run into a long time as long as the normal code without parallelism. 

import numpy as np from scipy.integrate import odeint from math import * from scipy.integrate import quad import pandas as pd import os import warnings warnings.filterwarnings("ignore") #you need add the following 3 lines from multiprocessing import Pool from multiprocessing import Process import multiprocessing print("Model 4, Equation 11") print("") ###################### STEP NUMBER ####################### N = int(input("PLEASE ENTER NUMBER OF STEP WALKS: ")) # Step walk by user dec=int(input("NUMBER OF DECIMAL PLACES OF OUTPUTS (RECOMENDED 10-15)?")) print("") print("PLEASE WAIT, METROPOLIS HASTINGS IS RUNNING ... ") print("") def FIT(): ########################################################## od0o = np.zeros((N,)) od0o[0]=0.72 od0n = np.zeros((N,)) Mo = np.zeros((N,)) Mo[0]= 0 Mn = np.zeros((N,)) co = np.zeros((N,)) co[0]= 0.84 cn = np.zeros((N,)) bo = np.zeros((N,)) bo[0]= 0.02 bn = np.zeros((N,)) H0o = np.zeros((N,)) H0o[0]= 70 H0n = np.zeros((N,)) Orco = np.zeros((N,)) Orco[0]= 0.0003 Orcn = np.zeros((N,)) temp=1e10 # a big number ########################################################## CovCMB=[[3.182,18.253,-1.429], [18.253,11887.879,-193.808], [-1.429,-193.808,4.556]] # CMB DATA ########################################################## def OD_H(U,z,c,b,Orc): od, H = U Omegai = 3 * b * ((1- od - 2*(Orc)**0.5) + (1- od - 2*(Orc)**0.5)**2/(1-2*(Orc)**0.5)) #equation 10 div1=np.divide((c**2-od),(1+z),where=(1+z)!=0) div2=np.divide(H ,(1+z),where=(1+z)!=0) dMdt = (div1)*(6*od+6-9*(od/c**2)+ Omegai)*(1+c**2+od*(1-3/c**2))**(-1) dHdt = (div2)*(6*od+6-9*(od/c**2)+ Omegai)*(1+c**2+od*(1-3/c**2))**(-1) return [dMdt, dHdt] def solution(H0,z1,z,od0,c,b,Orc): U = odeint(OD_H,[od0,H0],[z1,z], args=(c,b,Orc))[-1] od, H = U return H ########################################################## def DMCMB1(H0,z1,z,od0,c,b,Orc): dm = 1090 * 1/solution(H0,z1,z,od0,c,b,Orc) return dm def R1(H0,z1,z,od0,c,b,Orc): #r=sqrt(Om)*(70/299000)*rszstar(z,Om,Od) r = sqrt(1-od0-2*(Orc)**0.5)*DMCMB1(H0,z1,z,od0,c,b,Orc) return r def lA1(H0,z1,z,od0,c,b,Orc): la=((3.14*299000/H0)*DMCMB1(H0,z1,z,od0,c,b,Orc))/(153) return la def CMBMATRIX1(H0,z1,z,od0,c,b,Orc,M): hmCMB1=[lA1(H0,z1,z,od0,c,b,Orc)-301.57, R1(H0,z1,z,od0,c,b,Orc)-1.7382+M, 0.0222-0.02262] vmCMB1=[[lA1(H0,z1,z,od0,c,b,Orc)-301.57], [R1(H0,z1,z,od0,c,b,Orc)-1.7382], [0.0222-0.02262]] fmCMB1=np.dot(hmCMB1,CovCMB) smCMB1=np.dot(fmCMB1,vmCMB1)[0] return smCMB1 ###################################################### def TOTAL(H0, od0, c, b,Orc, M) : total = CMBMATRIX1(H0,0,1090,od0,c,b,Orc,M) return total ###################################################### ################## MCMC - MH ######################### highest=0 pat='C:/Users/21/Desktop/MHT/Models/outputs' file_path = os.path.join(pat,'Model3.txt') file_path2 = os.path.join(pat,'Model3min.txt') with open(file_path, 'w') as f: # DATA WILL BE SAVED IN THIS FILE, PLEASE BECAREFUL TO CHANGE THE NAME IN EACH RUN TO AVOIDE REWRITING. with open(file_path2, 'w') as d: for i in range (1,N): num = 0 R = np.random.uniform(0,1) while True: num += 1 od0n[i] = od0o[i-1] + 0.001 * np.random.normal() H0n[i] = H0o[i-1] + 0.01 * np.random.normal() bn[i] = bo[i-1] + 0.001 * np.random.normal() cn[i] = co[i-1] + 0.001 * np.random.normal() Mn[i] = Mo[i-1] + 0.01 * np.random.normal() Orcn[i] = Orco[i-1] + 0.00001 * np.random.normal() L = np.exp(-0.5 * (TOTAL(H0n[i], od0n[i], cn[i], bn[i],Orcn[i], Mn[i]) - TOTAL(H0o[i-1], od0o[i-1], co[i-1], bo[i-1],Orco[i-1], Mo[i-1]))) # L = exp(-( x^2 )/2) LL=min(1,max(L,0)) if LL>R: od0o[i]= od0n[i] H0o[i] = H0n[i] bo[i] = bn[i] co[i] = cn[i] Mo[i] = Mn[i] Orco[i] = Orcn[i] chi = TOTAL(H0o[i], od0o[i], co[i], bo[i],Orco[i], Mo[i]) else: od0o[i]= od0o[i-1] H0o[i] = H0o[i-1] bo[i] = bo[i-1] co[i] = co[i-1] Mo[i] = Mo[i-1] Orco[i] = Orco[i-1] chi = TOTAL(H0o[i], od0o[i], co[i], bo[i],Orco[i], Mo[i]) if (Mo[i]>0 and 0<bo[i]<0.09 and Orco[i]>0) or num>100: # constraining the value to stay in positive area highest = max(num, highest) break f.write("{0}\t{1}\t{2}\t{3}\t{4}\t{5}\t{6}\t{7}\t{8}\t{9}\t{10}\t{11}\t{12}\n".format(round(chi,dec),' ',round(H0o[i],dec),' ',round(od0o[i],dec),' ', round(co[i],dec),' ',round(bo[i],dec),' ', round(Orco[i],dec),' ',round(Mo[i],dec))) if chi<temp: temp=chi aa = H0o[i] bb = od0o[i] cc = co[i] dd = bo[i] ee = Mo[i] ff=Orco[i] Om=1-2*sqrt(Orco[i])-od0o[i] # minimum of chi and its parameters d.write("{0}\t{1}\t{2}\t{3}\t{4}\t{5}\t{6}\t{7}\t{8}\t{9}\t{10}\t{11}\t{12},\t{13}\t{14}\n".format(round(temp,dec), "H =", round(aa,dec), "Orc=", round(ff,dec), "OD =",round(bb,dec),"c =", round(cc,dec),"b =", round(dd,dec), "M =",round(ee,dec),"Om =",round(Om,dec))) print(round(temp,dec), "H =", round(aa,dec), "Orc=",round(ff,dec), "OD =",round(bb,dec),"c =", round(cc,dec),"b =", round(dd,dec), "M =",round(ee,dec),"Om =",round(Om,dec)) #print(highest) print("") #test = input("Press the enter to exit...") #print(test) if __name__ == '__main__': p = multiprocessing.Process(target=FIT) p.start() p.join()
Improve this question
7
  • if the data is small but the calculations long then multi may benefit, however, if data is large (so reading is long) and calculations are quick then it won't.... Commented Jun 6, 2018 at 13:59
  • 4
    "Using multiprocessing is for having a faster code" This is a misconception. Sometimes multiprocessing can make code run more slowly if the code running on one core has to wait for that running on another. There are many factors which determine whether or not multiprocessing will decrease the amount of time it takes to run your code. Commented Jun 6, 2018 at 13:59
  • @Code-Apprentice, 2 questions. With the mentioned code do I divide my code calculation on all of processors (CPU) of my lap top? So, how should I improve my calculation speed, apart from writing a optimal codes. Commented Jun 6, 2018 at 14:03
  • "how should I improve my calculation speed?" I cannot answer this without seeing your actual code. Commented Jun 6, 2018 at 14:06
  • @Code-Apprentice I upload the whole code under the code above, just in just 30sec Commented Jun 6, 2018 at 14:07

2 Answers 2

Reset to default
1

I think you missed main concept of multiprocessing. It does not run your code faster, it just let's you run something in another process to bypass GIL (https://wiki.python.org/moin/GlobalInterpreterLock).

It can be used to parallel computing of some function for different input values, like this example from docs

from multiprocessing import Pool def f(x): return x*x if __name__ == '__main__': p = Pool(5) print(p.map(f, [1, 2, 3]))

Which results in computing f in different processes and just each of them returning separate value

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Don't forget to mention that numpy can natively bypass the GIL... And there is no one size fits all when it comes to use multithreading or multiprocessing as an optimization tool. It depends on where is the contention. As a rule of thumb: memory => single process single thread, io => single process multi thread, CPU => multi process in Python because of the GIL
You mean that multiprocessing is not parallelism then. so I do not know how to do that
0

You are creating only one process and all of your other logic is sequential thats why there is no change in performance because all of your code will run sequentially. There are two different scenarios where you can use multiprocessing

  1. Totally Independent functionalities :
    If you have functionalities which are totally Independent and there is no restriction to execute them sequentially you can execute them in parallel this way none of these functionalities will have to wait for each other.
    A good analogy will be reading news paper and having breakfast here no need of doing them sequentially so we can do them at the same time.

  2. Executing same functionality for different inputs :
    If you are executing a functionality repeatedly for different input then you can execute multiple instances of that functionality for several inputs at a time.
    For analogy think of single ticket counter Then think of multiple ticket counters same functionality multiple instances.



Find these scenarios in your code then try to parallelize those functionalities.
Hope it helped.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.