66

Still the newbie in Scala and I'm now looking for a way to implement the following code on it:

@Override public void store(InputStream source, String destination, long size) { ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentLength(size); final PutObjectRequest request = new PutObjectRequest( this.configuration.getBucket(), destination, source, metadata); new RetryableService(3) { @Override public void call() throws Exception { getClient().putObject(request); } }; } 

What would be the best way to implement the same funcionality that RetryableService implements but in Scala?

It basically calls the call method N times, if all of them fail the exception is then raised, if they succeed it moves on. This one does not return anything but then I have another version that allows for returning a value (so, i have two classes in Java) and I believe I could do with a single class/function in Scala.

Any ideas?

EDIT

Current implementation in java is as follows:

public abstract class RetryableService { private static final JobsLogger log = JobsLogger .getLogger(RetryableService.class); private int times; public RetryableService() { this(3); } public RetryableService(int times) { this.times = times; this.run(); } private void run() { RuntimeException lastExceptionParent = null; int x = 0; for (; x < this.times; x++) { try { this.call(); lastExceptionParent = null; break; } catch (Exception e) { lastExceptionParent = new RuntimeException(e); log.errorWithoutNotice( e, "Try %d caused exception %s", x, e.getMessage() ); try { Thread.sleep( 5000 ); } catch (InterruptedException e1) { log.errorWithoutNotice( e1, "Sleep inside try %d caused exception %s", x, e1.getMessage() ); } } } try { this.ensure(); } catch (Exception e) { log.error(e, "Failed while ensure inside RetryableService"); } if ( lastExceptionParent != null ) { throw new IllegalStateException( String.format( "Failed on try %d of %s", x, this ), lastExceptionParent); } } public void ensure() throws Exception { // blank implementation } public abstract void call() throws Exception; } 

14 Answers 14

215

Recursion + first class functions by-name parameters == awesome.

def retry[T](n: Int)(fn: => T): T = { try { fn } catch { case e => if (n > 1) retry(n - 1)(fn) else throw e } } 

Usage is like this:

retry(3) { // insert code that may fail here } 

Edit: slight variation inspired by @themel's answer. One fewer line of code :-)

def retry[T](n: Int)(fn: => T): T = { try { fn } catch { case e if n > 1 => retry(n - 1)(fn) } } 

Edit Again: The recursion bothered me in that it added several calls to the stack trace. For some reason, the compiler couldn't optimize tail recursion in the catch handler. Tail recursion not in the catch handler, though, optimizes just fine :-)

@annotation.tailrec def retry[T](n: Int)(fn: => T): T = { val r = try { Some(fn) } catch { case e: Exception if n > 1 => None } r match { case Some(x) => x case None => retry(n - 1)(fn) } } 

Edit yet again: Apparently I'm going to make it a hobby to keep coming back and adding alternatives to this answer. Here's a tail-recursive version that's a bit more straightforward than using Option, but using return to short-circuit a function isn't idiomatic Scala.

@annotation.tailrec def retry[T](n: Int)(fn: => T): T = { try { return fn } catch { case e if n > 1 => // ignore } retry(n - 1)(fn) } 

Scala 2.10 update. As is my hobby, I revisit this answer occasionally. Scala 2.10 as introduced Try, which provides a clean way of implementing retry in a tail-recursive way.

// Returning T, throwing the exception on failure @annotation.tailrec def retry[T](n: Int)(fn: => T): T = { util.Try { fn } match { case util.Success(x) => x case _ if n > 1 => retry(n - 1)(fn) case util.Failure(e) => throw e } } // Returning a Try[T] wrapper @annotation.tailrec def retry[T](n: Int)(fn: => T): util.Try[T] = { util.Try { fn } match { case util.Failure(_) if n > 1 => retry(n - 1)(fn) case fn => fn } } 
Sign up to request clarification or add additional context in comments.

19 Comments

Actually, by-name parameters, though I'm concerned there might be something awkward going on when passing a by-name parameter recursively.
If you still have the habit @dave take a look at the scala.util.Try object! Great answers!
I'll just point that sometimes you will want to retry only on some exceptions while directly throwing others. You can use scala.util.control.Exception.catching(classOf[E1],classOf[E2]).withTry(fn) match...
I read all your solutions and you could really notice how each improves the previous. This is why Scala is love. This is what makes scala great. Beautiful. I am adapting the last one and adding escalating timeouts between each retry. I will try to keep the beauty as much as possible.
one question here. How to modify it if my function returns Future?
|
8

There is a method in scalaz.concurrent.Task[T]: http://docs.typelevel.org/api/scalaz/nightly/#scalaz.concurrent.Task

def retry(delays: Seq[Duration], p: (Throwable) ⇒ Boolean = _.isInstanceOf[Exception]): Task[T] 

Given a Task[T], you can create a new Task[T] which will retry a certain number of times, where the delay between retries is defined by the delays parameter. e.g.:

// Task.delay will lazily execute the supplied function when run val myTask: Task[String] = Task.delay(???) // Retry four times if myTask throws java.lang.Exception when run val retryTask: Task[String] = myTask.retry(Seq(20.millis, 50.millis, 100.millis, 5.seconds)) // Run the Task on the current thread to get the result val result: String = retryTask.run 

Comments

6

Here is one possible implementation:

def retry[T](times: Int)(fn: => T) = (1 to times).view flatMap (n => try Some(fn) catch {case e: Exception => None}) headOption 

You can use it like this:

retry(3) { getClient.putObject(request) } 

retry also returns Some[T] if body was processed successfully and None if body was only throwing exceptions.


Update

If you want to bobble up last exception, then you can take very similar approach but use Either instead of Option:

def retry[T](times: Int)(fn: => T) = { val tries = (1 to times).toStream map (n => try Left(fn) catch {case e: Exception => Right(e)}) tries find (_ isLeft) match { case Some(Left(result)) => result case _ => throw tries.reverse.head.right.get } } 

Also, as you can see, at the end, instead of having only last exception, I have them all. So you can also wrap them in some AggregatingException if you want and then throw it. (for simplicity, I just throw last exception)

6 Comments

If it fails all times the exception must bubble up, I'll add the full implementation in the question.
Note that this is probably not what the OP intends re: side effects - retry(3) { println("foo") } will print three lines.
@themel: With my implementation of retry, retry(3) {println("foo")} will print only once
Did you try? It doesn't for me in Scala 2.8.1.
@themel: I use 2.9.1.final and it prints foo only one time
|
4

I'd suggest this -

def retry[T](n: Int)(code: => T) : T = { var res : Option[T] = None var left = n while(!res.isDefined) { left = left - 1 try { res = Some(code) } catch { case t: Throwable if left > 0 => } } res.get } 

It does:

scala> retry(3) { println("foo"); } foo scala> retry(4) { throw new RuntimeException("nope"); } java.lang.RuntimeException: nope at $anonfun$1.apply(<console>:7) at $anonfun$1.apply(<console>:7) at .retry(<console>:11) at .<init>(<console>:7) at .<clinit>(<console>) at RequestResult$.<init>(<console>:9) at RequestResult$.<clinit>(<console>) at RequestResult$scala_repl_result(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at scala.tools.nsc.Interpreter$Request$$anonfun$loadAndRun$1$$anonfun$apply$17.apply(Interpreter.scala:988) at scala.tools.nsc.Interpreter$Request$$anonfun$loadAndRun$1$$anonfun$apply$17.apply(Interpreter.... scala> var i = 0 ; i: Int = 0 scala> retry(3) { i = i + 1; if(i < 3) throw new RuntimeException("meh");} scala> i res3: Int = 3 

It can probably be improved to be more idiomatic Scala, but I am not a big fan of one-liners that require the reader to know the entire standard library by heart anyways.

Comments

4

You can express the idea in functional style using scala.util.control.Exception:

@annotation.tailrec def retry[T](n: Int)(fn: => T): T = Exception.allCatch.either(fn) match { case Right(v) => v; case Left(e) if (n <= 1) => throw e; case _ => retry(n - 1)(fn); } 

As we can see, tail recursion can be used here.

This approach gives you the additional benefit that you can parametrize the catch container, so you can only retry a certain subset of exceptions, add finalizers etc. So the final version of retry might look like:

/** Retry on any exception, no finalizers. */ def retry[T](n: Int)(fn: => T): T = retry(Exception.allCatch[T], n)(fn); /** Parametrized retry. */ @annotation.tailrec def retry[T](theCatch: Exception.Catch[T], n: Int)(fn: => T): T = theCatch.either(fn) match { case Right(v) => v; case Left(e) if (n <= 1) => throw e; case _ => retry(theCatch, n - 1)(fn); } 

With this, you can do complex stuff like:

retry(Exception.allCatch andFinally { print("Finished.") }, 3) { // your scode } 

Comments

3

There is an existing library that can help with that, called retry, and there is a Java library too, called guava-retrying.

Here are some examples of using retry:

// retry 4 times val future = retry.Directly(4) { () => doSomething } // retry 3 times pausing 30 seconds in between attempts val future = retry.Pause(3, 30.seconds) { () => doSomething } // retry 4 times with a delay of 1 second which will be multipled // by 2 on every attempt val future = retry.Backoff(4, 1.second) { () => doSomething } 

Comments

2

I like the accepted solution, but suggest checking the exception is NonFatal:

// Returning T, throwing the exception on failure @annotation.tailrec def retry[T](n: Int)(fn: => T): T = { Try { fn } match { case Success(x) => x case _ if n > 1 && NonFatal(e) => retry(n - 1)(fn) case Failure(e) => throw e } } 

You don't want to retry a control flow exception, and usually not for thread interrupts...

1 Comment

The NonFatal match line won't compile unless I do: case Failure(e) if n > 1 && NonFatal(e) => retry(n - 1)(fn).
1

If you want control of which exceptions you retry, you can use methods in scala.util.control.Exception:

import java.io._ import scala.util.control.Exception._ def ioretry[T](n: Int)(t: => T) = ( Iterator.fill(n){ failing[T](classOf[IOException]){ Option(t) } } ++ Iterator(Some(t)) ).dropWhile(_.isEmpty).next.get 

(As written, it will also retry on null; that's the Option(t) part. If you want nulls to be returned, use Some(t) inside the iterator fill instead.)

Let's try this out with

class IoEx(var n: Int) { def get = if (n>0) { n -= 1; throw new IOException } else 5 } val ix = new IoEx(3) 

Does it work?

scala> ioretry(4) { ix.get } res0: Int = 5 scala> ix.n = 3 scala> ioretry(2) { ix.get } java.io.IOException at IoEx.get(<console>:20) ... scala> ioretry(4) { throw new Exception } java.lang.Exception at $anonfun$1.apply(<console>:21) ... 

Looks good!

Comments

1

I ended up adapting a previous answer to allow filtering on which exceptions to retry on:

 /** * Attempt 'fn' up to 'attempts' times, retrying only if 'forExceptions' returns true for retry-able exceptions. */ def retry[T](attempts: Int, forExceptions: (Throwable) => Boolean)(fn: => T): T = { // toStream creates a lazily evaluated list, which we map to a try/catch block resulting in an Either val tries = (1 to attempts).toStream map { n => try Left(fn) catch { case e if forExceptions(e) => Right(e) } } // find the first 'Either' where left is defined and return that, or if not found, return last // exception thrown (stored as 'right'). The cool thing is that because of lazy evaluation, 'fn' is only // evaluated until it success (e.g., until Left is found) tries find (_ isLeft) match { case Some(Left(result)) => result case _ => throw tries.reverse.head.right.get } } 

You can call in two ways:

val result = retry(4, _.isInstanceOf[SomeBadException]) { boom.doit() } 

or with partial functions (also showing version where don't care about return value)

 def pf: PartialFunction[Throwable, Boolean] = { case x: SomeOtherException => true case _ => false } retry(4, pf) { boom.doit() } 

Comments

1

This solution is not optimized by compiler to tail recursion for some reason (who knows why?), but in case of rare retries would be an option:

def retry[T](n: Int)(f: => T): T = { Try { f } recover { case _ if n > 1 => retry(n - 1)(f) } get } 

Usage:

val words: String = retry(3) { whatDoesTheFoxSay() } 

End of the answer. Stop reading here


Version with result as a Try:

def reTry[T](n: Int)(f: => T): Try[T] = { Try { f } recoverWith { case _ if n > 1 => reTry(n - 1)(f) } } 

Usage:

// previous usage section will be identical to: val words: String = reTry(3) { whatDoesTheFoxSay() } get // Try as a result: val words: Try[String] = reTry(3) { whatDoesTheFoxSay() } 

Version with a function returning Try

def retry[T](n: Int)(f: => Try[T]): Try[T] = { f recoverWith { case _ if n > 1 => reTry(n - 1)(f) } } 

Usage:

// the first usage section will be identical to: val words: String = retry(3) { Try(whatDoesTheFoxSay()) } get // if your function returns Try: def tryAskingFox(): Try = Failure(new IllegalStateException) val words: Try[String] = retry(3) { tryAskingFox() } 

Comments

1

A reusable object/method with a pause between attempts:

Retry(3, 2 seconds) { /* some code */ } 

Code:

object Retry { def apply[A](times: Int, pause: Duration)(code: ⇒ A): A = { var result: Option[A] = None var remaining = times while (remaining > 0) { remaining -= 1 try { result = Some(code) remaining = 0 } catch { case _ if remaining > 0 ⇒ Thread.sleep(pause.toMillis) } } result.get } } 

Comments

0
//Here is one using Play framework def retry[T](times:Int)(block: => Future[T])(implicit ctx: ExecutionContext):Future[T] = { type V = Either[Throwable,T] val i:Iterator[Future[Option[V]]] = Iterator.continually(block.map(t => Right(t)).recover { case e => Left(e) }.map(t => Some(t))) def _retry:Iteratee[V,V] = { def step(ctr:Int)(i:Input[V]):Iteratee[V,V] = i match { case Input.El(e) if (e.isRight) => Done(e,Input.EOF) case _ if (ctr < times) => Cont[V,V](i => step(ctr + 1)(i)) case Input.El(e) => Done(e,Input.EOF) } Cont[V,V](i => step(0)(i)) } Enumerator.generateM(i.next).run(_retry).flatMap { _ match { case Right(t) => future(t) case Left(e) => Future.failed(e) }} } 

Comments

0

This project seems to provide some nice implementations for different retry mechanisms https://github.com/hipjim/scala-retry

// define the retry strategy implicit val retryStrategy = RetryStrategy.fixedBackOff(retryDuration = 1.seconds, maxAttempts = 2) // pattern match the result val r = Retry(1 / 1) match { case Success(x) => x case Failure(t) => log("I got 99 problems but you won't be one", t) } 

Comments

0

Minor improvement to printout attempt x of N

// Returning T, throwing the exception on failure @annotation.tailrec final def retry[T](n: Int, name: String ="", attemptCount:Int = 1)(fn: => T): T = { logger.info(s"retry count: attempt $attemptCount of $n ....... function: $name") try { val result = fn logger.info(s"Succeeded: attempt $attemptCount of $n ....... function: $name") result } catch { case e: Throwable => if (n < attemptCount) { Thread.sleep(5000 * attemptCount); retry(n, name, attemptCount+1)(fn) } else throw e } } 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.