Retry on a specific condition

[Ranagal Adapla]

Hello forum veterans,

I would like to retry a REST Call with the following conditions. Can anyone here help me on this ?

Scenario: I am making a REST Call that returns a list.

The retry should happen in the below cases:

1. When the list returned from REST Call is empty/null.
2. When the list is not empty/null and it does not contain the value that I am looking for.

Note that in case of any exception after making the REST Call, I don't want to retry.

I am looking for an elegant way to handle this.

Regards,
Ranagal

[Stephan van Hulst]

var list  = fetchList();
var value = findValue(list);

while (!value.isPresent()) {
  list  = fetchList();
  value = findValue(list);
}

return value.get();

This'll just keep going until it's found the value you're looking for, or an exception occurs.

The problem with this approach is that you'll be firing a lot of requests if it takes the server some time to update the list. You'll probably want to use a ScheduledExecutorService to schedule your REST requests after a certain delay, to prevent your application from swamping the server in requests.

[Ranagal Adapla]

Yeah. That is correct. I am aware of @Retry annotation. But I dont know how to customize it and use it for this scenario.
Also, for now the backoff time is 10seconds. I forgot to mention that in the original post.

By the way, thanks for the reply and your time.

[Stephan van Hulst]

Are you using Spring Batch? Are the REST calls made from your web service, or from a client application? How are you making the REST calls in the first place?

[Ranagal Adapla]

Stephan van Hulst wrote:Are you using Spring Batch? Are the REST calls made from your web service, or from a client application? How are you making the REST calls in the first place?

I am making the REST Call like below:

new RestTemplate().exchange(restUrl, httpMethod, httpRequestEntity, responseType, requestParams);

[Stephan van Hulst]

So what's the problem in using @Async and @Retryable?

[Ranagal Adapla]

Stephan van Hulst wrote:So what's the problem in using @Async and @Retryable?

Sure. But I am not sure how I can customize it for my scenario. If you could help me on this, that would be great. Thanks in advance.

[Stephan van Hulst]

You'd need to tell us what customization you need.

[Ranagal Adapla]

Stephan van Hulst wrote:You'd need to tell us what customization you need.

The retry should happen in the below cases:

1. When the list returned from REST Call is empty/null.
2. When the list is not empty/null and it does not contain the value that I am looking for.

Note that in case of any exception after making the REST Call, I don't want to retry.

The above is how I need to customize

[Stephan van Hulst]

Upon further analysis, I don't think @Retryable is the right tool for this. That annotation is good if you want to retry when an exception occurs. You want to retry as part of normal execution flow.

Use @Async and CompletableFuture instead. It might look something like this:

import java.io.*;
import java.util.*;
import java.util.concurrent.*;

import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;

import static java.util.concurrent.CompletableFuture.completedFuture;
import static java.util.concurrent.CompletableFuture.delayedExecutor;
import static java.util.concurrent.CompletableFuture.supplyAsync;
import static java.util.concurrent.TimeUnit.SECONDS;

@Service
public class ExampleService {

  @Async
  public Future<ExampleValue> getValue() {
    return supplyAsync(this::fetchList).thenCompose(this::findValueOrRetry);
  }

  private CompletionStage<ExampleValue> findValueOrRetry(List<? extends ExampleValue> values) {
    var foundValue = findValue(values);
    
    if (foundValue.isPresent()) {
      return completedFuture(foundValue.get());
    }
    
    else {
      return supplyAsync(this::fetchList, delayedExecutor(10, SECONDS)).thenCompose(this::findValueOrRetry);
    }
  }

  private List<ExampleValue> fetchList() throws UncheckedIOException {
    // This is where you make a single REST call to get a list of values.
    throw new UnsupportedOperationException();
  }

  private Optional<ExampleValue> findValue(List<? extends ExampleValue> values) {
    // This is where you look through the list of values to find the one you're interested in.
    throw new UnsupportedOperationException();
  }
}

Please note that I wrote this without testing it. It might contain bugs, or it might not work at all.

[Ranagal Adapla]

Thanks for your time and thoughts on this sir. I will use this as reference and test for my scenario.

[Ranagal Adapla]

I have written something like this to test. And I have two questions on this:

1. It is not delaying for 5sec as mentioned in the code.
2. I would like to limit this retry for 3 times. And I don't see it anywhere.

Can you please help me with that ?

public class TestAsync {
	
	  @Async
	  public Future<String> getValue() {
	    return supplyAsync(this::fetchList).thenCompose(this::findValueOrRetry);
	  }
	 
	  private CompletionStage<String> findValueOrRetry(List<String> values) {
	    var foundValue = findValue(values);
	     
	    if (foundValue.isPresent()) {
	    	System.out.println("Found the value and hence completing the Future");
	      return completedFuture(foundValue.get());
	    } else {
	    	System.out.println("Did not find it and hence retrying again with 5 sec delay");
	      return supplyAsync(this::fetchList, delayedExecutor(5, SECONDS)).thenCompose(this::findValueOrRetry);
	    }
	  }
	 
	  private List<String> fetchList() throws UncheckedIOException {
	    // This is where you make a single REST call to get a list of values.
		return returnList();
	  }
	 
	  private Optional<String> findValue(List<String> values) {
	    // This is where you look through the list of values to find the one you're interested in.
		return values.stream().filter(item -> item.equals("Four")).findFirst();
	    //throw new UnsupportedOperationException();
	  }
	  
	  private List<String> returnList() {
		  List<String> originalList = List.of("One", "Two", "Three", "Four");
		  int lastIndex = new Random().nextInt(0, 4);
		  System.out.println("lastIndex: "+lastIndex);
		  List<String> newList = originalList.subList(0, lastIndex);
		  System.out.println("newList: "+newList);
		  return newList;
	  }

	public static void main(String[] args) {
		TestAsync testAsync = new TestAsync(); 
		Long start = System.currentTimeMillis();
		System.out.println("Calling the getValue method start: "+ start);
		testAsync.getValue();
		System.out.println("End the call of getValue method end: "+ System.currentTimeMillis());
		System.out.println("Time Taken: "+ String.valueOf(System.currentTimeMillis() - start));
		
	}

}

[Stephan van Hulst]

I'll see if I can work out what the problem is tomorrow. In the mean time, please do the following things for me:

Let me know how you are calling that service. Are you calling it from a web controller? Can you show us?

Please remove the quoting of my posts from your last two posts. They're not necessary and they make the topic a bit harder to read.

[Ranagal Adapla]

Let me know how you are calling that service. Are you calling it from a web controller? Can you show us?
-> As a standard in our project, we have created a RESTRepository to make external REST Calls.  And we have written a method in the that repository and then from there, we are making the API call like below:
new RestTemplate().exchange(restUrl, httpMethod, httpRequestEntity, responseType, requestParams);
Please remove the quoting of my posts from your last two posts. They're not necessary and they make the topic a bit harder to read
-> Done

[Ranagal Adapla]

Stephan van Hulst wrote:I'll see if I can work out what the problem is tomorrow.

Just a reminder sir