Scala: Combining a list of Future for all successes and errors-CodePudding

I have a list of Future, with each Future completing with a List of values OR with a failure. I am trying to combine these futures in a way that all available values are captured and errors are logged. This is my attempt:

val individualFutures:  List[Future[Seq[Element]]] = ....
val allElements: Future[List[Element]] = Future.foldLeft(individualFutures)(List[Element]())((acc, 
     elements) => acc    elements)
Try(Await.result(allElements, Duration.Inf)) match {
      case Success(elements) => ....
      case Failure(error)    => ....log
}

But I don't think this is right way to achieve my objective. I guess as soon as a future completes with an error, this would stop there.

How can I achieve same?

CodePudding user response：

Maybe you could traverse the list and recover from throwables by returning an empty list and logging a failure reason? At the end you would flatten the whole sequence.

  val allElements: Future[List[Element]] = Future.traverse(individualFutures) {
    _.recover {
      case throwable: Throwable =>
        println(throwable.getMessage)
        Nil
    }
  }.map(_.flatten)

CodePudding user response：

You can log errors as they become available

    individualFutures.foreach(_.recover {
      case e: Exception => log(e); Nil
    })

And then you can use .sequence and .flatten to collect successful elements:

   val allElements: Future[List[Element]] = Future.sequence(individualFutures).map(_.flatten)

As a general rule, you should avoid Await in most situations. Use allElements.map or allElements.foreach to process collected elements asynchronously.

If you can process elements individually, without assembling them into a single list first, you can do it in one go:

    individualFutures.map(
      _.onComplete { 
        case Failure(e) => log(e)
        case Success(elems) => elems.foreach(process)
     }
    )