Future sequence to sequence of futures?

Question

Suppose, I have an abstract "producer" instance:

 trait Producer[T] { def listObjectIds: Future[Seq[String]] def getObject(id: String): Future[T] }

and I need to apply some processing to each (or some) of the objects it yields. So, I do something like:

 producer .listObjectIds .map(maybeFilter) .map(_.map(producer.getObject))

... and end up with Future[Seq[Future[T]]] This is ok, but kinda cumbersome. I would like to get rid of the outer Future, and just have Seq[Future[T]], but can't think of a (non-blocking) transformation, that would let me do that.

Any ideas?

Joe K · Accepted Answer · 2017-08-07 19:40:16Z

4

It's not possible to end up with a Seq[Future[T]]. See Reverse of Future.sequence.

But it is possible to end with a Future[Seq[T]]. Just call .flatMap(Future.sequence) on the Future[Seq[Future[T]].

answered Aug 7, 2017 at 19:40

Joe K

18.5k2 gold badges42 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Dima Over a year ago

Yeah, I know ... I don't what that though. My Seq is lazy (it's actually an Iterator, not even Seq), and the idea is to load those objects one-by-one. They won't even fit into memory more than a couple of so at a time. I understand your point about not knowing how long the result is going to be, but I'd be fine with Iterator[Future[T]], that always ends with a failure (kinda like Future.collect does when the PF is not defined).

Joe K Over a year ago

What should happen if .next was called on the iterator 1000 times immediately, and eventually the Future[Seq] finished with just 1 element? The other 999 would all end in failure?

Dima Over a year ago

Yes. So, we get 1000 futures. The first one satisfies with the result of getObject. All the other ones end up throwing NoSuchElementException when satisfied ... Just like Future(1).collect { case 2 => "two" } would do.

Joe K Over a year ago

Then perhaps something like: Iterator.from(0).map(i => futSeqFut.flatMap(s => s(i))) ?

Joe K Over a year ago

Then listObjectIds would have to return something other than a Future[Seq]. It's sounding more and more like you might want to consider modeling this using some streaming library (akka streams or fs2)

|

Collectives™ on Stack Overflow

Future sequence to sequence of futures?

1 Answer 1

7 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Linked

Related