Scala different parallel structures

Question

I tested parallel collections on Scala vs simple collection, here is my code:

def parallelParse() { val adjs = wn.allSynsets(POS.ADJECTIVE).par adjs.foreach(adj => { parse(proc.mkDocument(adj.getGloss)) }) } def serialParse() { val adjs = wn.allSynsets(POS.ADJECTIVE) adjs.foreach(adj => { parse(proc.mkDocument(adj.getGloss)) }) }

The parallel collection speed up about 3 times. What other option do I have in Scala to make it even faster in parallel, I would be happy to test them and put the results here.

axel22 · Accepted Answer · 2014-07-08 07:09:03Z

You can use futures to start asynchronous computations. You could do:

import scala.concurrent._ import scala.concurrent.duration._ import ExecutionContext.Implicits.global val futures = wn.allSynsets(POS.ADJECTIVE).map(adj => Future { parse(proc.mkDocument(adj.getGloss)) }) futures.foreach(f => Await.ready(f, Duration.Inf))

Depending on the amount of work on each element in allSynsets and the number of elements in allSynsets (too many elements -> too many futures -> more overheads), you could get worse results with futures.

To ensure that you are benchmarking correctly, consider using the inline benchmarking feature of ScalaMeter 0.5:

http://scalameter.github.io/home/gettingstarted/0.5/inline/index.html

You could also use actors to achieve this, but it would require a bit more plumbing.

I used future and the benchmark, but it suddenly reports the time which is incorrect for sure, any idea? Here is the code: val time = measure { futureParse() } println(s"Total time future: $time") def futureParse() { wn.allSynsets(POS.ADJECTIVE).foreach(adj => Future { parse(proc.mkDocument(adj.getGloss)) }) }
Sorry, I must have been tired when I was writing the code example. You have to wait for the futures to complete. I've edited my answer to map each element to a Future, and then wait until each of the futures completes its work.

Collectives™ on Stack Overflow

Scala different parallel structures

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related