We have 4 nodes in a Weblogic 14c Cluster and all the 4 nodes are fronted by a load balancer. From the UI a service call goes to one of the node in this cluster, based on some conditions another 3rd party service is invoked from the service code. But, the 3rd party service accepts only 3 connections at any time (i.e. only 3 requests can be made), so for ex: if 4 different requests from UI lands to 4 different nodes and if all the 4 requests invoke 3rd party service, then one of them will fail and in that scenario we don't want to throw a error message to user such as try again or something. How do we handle such a situation gracefully i.e. instead of threads are there any other better options to implement retry mechanism? if not please suggest any other better idea.

p.s. 3rd party service cannot increase the total number of open connections on their side.

CodePudding user response：

Weblogic contains a Distributed Singleton service: http://developsimpler.blogspot.com/2012/03/weblogic-clusters-and-singleton-service.html

With this you could have a distributed singleton countdown latch to control the number of calls the full cluster does to the 3rd party service.

CodePudding user response：

if you receive a specific exception from the 3rd party service and one service call goes fast enough you can retry within the same frontend call and decide how many times you retry (maybe with a short delay time)

You mention spring in your tags so I assume you're using it.

I haven't done this myself but apparently spring has a retry mechanism: org.springframework.retry.annotation.Retryable

https://fullstackdeveloper.guru/2020/09/29/how-to-retry-service-calls-in-spring-boot/