5
$\begingroup$

I am experiencing speed issues with ParallelTable. There are a few related problems on SX posted here and here, but no resolution. The various respondents to those problems seem to be focused around PackedArray issues, but in this case I really can't see that it would be an issue (but happy to be shown wrong if it fixes the problems, of course).

Viz., I'm doing a bunch of computational geometry and need to compute RegionDistance over a regular 3D grid to create a volume. Here's a greatly-simplified example ---

f = RegionDistance[Sphere[]]; vol = Table[f[{x, y, z}], {x, -2, 2, .1}, {y, -2, 2, .1}, {z, -2, 2, .1}]; // AbsoluteTiming (* {0.235526, Null} *) vol = ParallelTable[f[{x, y, z}], {x, -2, 2, .1}, {y, -2, 2, .1}, {z, -2, 2, .1}]; // AbsoluteTiming (* {19.9908, Null} *)

So, 100x slower. I'm pretty agile with parallel computing, writ large, and have tried all varieties of Method, DistributeDefinitions and friends, evaluated things in ParallelEvaluate blocks, messed with contexts, &c.

Very frustrating. Has anyone found a solution here?

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ Have you tried ParallelEvaluate[f = RegionDistance[Sphere[]]]; before ParallelTable? $\endgroup$ Commented Jan 13, 2016 at 3:26
  • $\begingroup$ Interesting you mention that... I was just digging through an old notebook and saw that I did something similar back in 9.x. Will try in the AM and report back - thanks for the memory-jog. $\endgroup$ Commented Jan 13, 2016 at 8:34
  • $\begingroup$ Indeed that fixes it! I think I understand why, but then again, maybe not. So now, f 'lives' in the various kernels. DistributeDefinitions doesn't seem to work, so I'm assuming that, when the remote kernel needs f it is getting it from the controlling kernel, perhaps on each iteration? Very messy. Anyway- thank you for the reminder @ilian Next pint is on me. $\endgroup$ Commented Jan 13, 2016 at 15:06
  • 2
    $\begingroup$ After some discussion with the developers, I believe this is due to an autocompilation step, the result of which isn't stored in the InputForm of the distance function and gets recomputed repeatedly on the subkernels. I'd also like to mention that RegionDistanceFunction will do automatic multithreading when given a list of points, e.g. I get a better timing if I don't use parallel kernels and do AbsoluteTiming[vol = f[Table[{x, y, z}, {x, -2, 2, .1}, {y, -2, 2, .1}, {z, -2, 2, .1}]];] instead. $\endgroup$ Commented Jan 13, 2016 at 17:02
  • $\begingroup$ Also good to know. As more internal stuff gets threading the easier this gets, tell them to keep it up. $\endgroup$ Commented Jan 14, 2016 at 0:34

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.