0

Could anybody help me to somehow parallelize this code ? I am about to make some simulations but I STACKED... it takes too long - I even left my computer for 3 days and it did not finished calculating

sapply(1:1000, Take_expected_value, points =10^7, number_of_trajectories = 10^7) Take_expected_value <- function(interval_end = 1, points = 100, number_of_trajectories = 1000){ return( mean( exp( replicate( n = number_of_trajectories, expr = Max_from_Wiener_on_interval(interval_end, points) ) ) ) ) # This function just replicates max_from_... function, then put values # to exp function, and calculates mean of all replications. } Max_from_Wiener_on_interval <- function(interval_end = 1, points = 100){ # time increment Delta <- interval_end/points # time moments time <- seq( 0, interval_end, length = points + 1) # Wiener process W <- cumsum( sqrt(Delta) * rnorm( points + 1 ) ) # return max of "Wiener * sqrt(2) - time moment" return( max(sqrt(2) * W - time) ) }

EDIT

After EDIT I am using this code, but it might be a problem of my weak machine(computer). Still it is very slow for me:

Take_expected_value <- function(interval_end = 1, points = 100, number_of_trajectories = 1000){ return( mean( exp( replicate( n = number_of_trajectories, expr = Max_from_Wiener_on_interval(interval_end, points) ) ) ) ) # This function just replicates max_from_... function, then put values # to exp function, and calculates mean of all replications. } # this function shall not be exported Max_from_Wiener_on_interval <- function(interval_end = 1, points = 100){ # time increment Delta <- interval_end/points # time moments time <- seq( 0, interval_end, length = points + 1) # Wiener process W <- cumsum( sqrt(Delta) * rnorm( points + 1 ) ) # return max of "Wiener * sqrt(2) - time moment" return( max(sqrt(2) * W - time) ) } install.packages("snowfall") require(snowfall) cpucores=as.integer(Sys.getenv('NUMBER_OF_PROCESSORS')) sfInit( parallel=T, cpus=cpucores) # sfExportAll() system.time( sfLapply(as.list(c(1:1000)),fun=Take_expected_value, points =10^6, number_of_trajectories = 10^6) ) sfRemoveAll() sfStop()
Improve this question
3
  • I do not see anything particularly bad in the code, but a sapply is just a giant loop, and loops runs like molasses in R. Pretty trivial to convert this code to Julia, and should be substantially faster. Commented Aug 22, 2014 at 18:16
  • Maybe I'll Have a look at Julia someday, but now I'm trying to do everything in R. I was hoping some1 could divide my code that operations would be split to many cores. Commented Aug 23, 2014 at 12:05
  • The issue is that what you are asking is too computationally intensive. When I ran the code I used the default values for the other 2 arguments in the function. as soon as I tried running it with a the much larger number of points and trajectories, I waited over 5 minutes for the results and no return. Try running the code you have above but with the default arguments: sfLapply(as.list(c(1:1000)),fun=Take_expected_value) It should take just a few seconds at most (took me 10 secs on 8 cores vs 40 seconds running on serial mode (sfInit( parallel=F)) Commented Aug 27, 2014 at 20:09

1 Answer 1

Reset to default
1

I tend to use snowfall, but there are many other ways to parallelize a function. Here is a generic script with a junk function meant to take a while to compute:

Iter_vals=as.list(c(1:16)) # the values to iterate the function with fx_parallel_run=function(Iter_val, multiplier){ #junk function with 2 arguments jnk=round(runif(1)*multiplier) jnk1=runif(jnk) for (i in 1:length(jnk1)){ jnk1[i]=(jnk1[i]*runif(1))+Iter_val[[1]] } return(jnk1) } require(snowfall) cpucores=as.integer(Sys.getenv('NUMBER_OF_PROCESSORS')) #by default snowfall will use the total number of processors, so this is not necessary. #However, if you are using the machine for other purposes, #you can adapt this line to leave at least a core or two free #so the computer is still functional for multi tasking. sfInit( parallel=T, cpus=cpucores) # sfExportAll() results=sfLapply(Iter_vals,fun=fx_parallel_run, multiplier=800) #extra function arguments go after the first two sfLapply arguments sfRemoveAll() sfStop()

in your case, after specifying the functions, I would simply use:

require(snowfall) sfInit( parallel=T, cpus=cpucores) # sfExportAll() results=sfLapply(as.list(c(1:1000)),fun=Take_expected_value, points =10^7, number_of_trajectories = 10^7) sfRemoveAll() sfStop()

This may need a little tweaking but I am not going to do all of the work for you.

Improve this answer
Sign up to request clarification or add additional context in comments.

9 Comments

Thanks I was hoping to see that kind of paralleilzaiton :) Hope it'll make operations faster and finishable
I run this code for smaller sample size and with only 2 iterations but this still does not want finish running and does not want to stop.. I stopped it by myself > system.time( + results <- sfLapply(1:2, Take_expected_value, points = 10^6, + number_of_trajectories = 10^6) + ) Timing stopped at: 0.06 0 3007.29
Hi Marcin, sorry I had a minor oversight- since we are using the sfL(ist)apply, we have to provide the values within a list. I have modified the code above to reflect it.
I tested on a couple of machines and it worked well. Good luck!
I have a monster machine: it took 0.2 secs
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.