REST API GET and POST API calls retry logic

[sai rama krishna]

Hi,

I am looking for retry logic for REST API GET and POST API calls when the the fetch and update operations fail in a do-while or for kind of loops. What are best practices around this. What kind of logging I have to implement for this for future debugging, analysis purpose in higher environments like production. Any good links, resources around this. Please advise

[Tim Cooke]

As is often the case, it depends.

I assume you're doing something like this?

for (int itemId : itemIds) {
    Item item = api.getItem(itemId);
    // Modify item
    api.postItem(itemId, item);
}

(Note: Assume 'api' here encapsulates the REST API calling details)

The way I would approach this is to list out the ways in which it could fail, and then decide what you can and want to do in response.

For example:

GET Item returns a 404 Not Found. Now what?

POST Item returns a 404 Not Found. Now what?

POST Item returns a 400 Bad Request. Now what?

POST Item returns a 429 Too Many Requests. Now what?

POST Item returns a 500 Server Error. Now what?

Some of these failure modes might benefit from being retried, some might not. Here are my thoughts on them, although I know nothing about your system so they may be wrong. You'll need to do your own analysis.

A 404 on the GET or POST request is likely to still be a 404 later no matter how many times you retry. Unless you have another process running asynchronously somewhere creating the resources you seek

A 400 Bad Request generally suggests the structure of your request is wrong so retrying the same request again is unlikely to succeed. I'd consider this a programming error and would want as much detail about the request and 400 response as possible for debugging.

A 429 Too Many Requests is probably a good candidate for a retry. A well behaved API will tell you in its header how long you have to wait before it's ready to accept new requests, so you can have your retry loop wait at least that long before trying again. However, if the content of the message you're sending has a short lifespan then maybe there's no value in sending it later. When I say a short lifespan I mean that in the context of your domain and system, not technically. Maybe your message is a "now or never" type thing where you'd rather it fail now than succeed later.

A 500 Server Error might be worth a retry. You never quite know why an API gives a 500 and it might be a transient state, say if the database goes offline for a moment, or a permanent state, say a bug in the server code. To account for transient errors it might be reasonable to try again later

For resources to help you implement retry logic, I'd search for "exponential back off and retry java", or just "back off and retry java". I haven't used this particular one but since most of the Java projects I work on use Spring I'd probably start here Spring Retry.

[Tim Holloway]

This really isn't a "Beginning Java" sort of question, though.

[sai rama krishna]

Thank you very much. This is very informative. I will check further in this direction.