I exposed 2 api's
/endpoint/A and /endpoint/B .
@GetMapping("/endpoint/A")
public ResponseEntity<ResponseA> controllerA() throws InterruptedException {
ResponseA responseA = serviceA.responseClient();
return ResponseEntity.ok().body(responseA);
}
@GetMapping("/endpoint/B")
public ResponseEntity<ResponseA> controllerB() throws InterruptedException {
ResponseA responseB = serviceB.responseClient();
return ResponseEntity.ok().body(responseB);
}
Services implemented regarding endpoint A internally call /endpoint/C and endpoint B internally call /endpoint/D.
As external service /endpoint/D taking more time i.e getting response from /endpoint/A takes more time hence whole threads are stucked that is affecting /endpoint/B.
I tried to solve this using executor service having following implementation
@Bean(name = "serviceAExecutor")
public ThreadPoolTaskExecutor serviceAExecutor(){
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(100);
taskExecutor.setMaxPoolSize(120);
taskExecutor.setQueueCapacity(50);
taskExecutor.setKeepAliveSeconds(120);
taskExecutor.setThreadNamePrefix("serviceAExecutor");
return taskExecutor;
}
Even after implementing this if I received more than 200 request on /endpoint/A simultaneously (greater than default max number of threads in Tomcat server) then I am not getting responses from /endpoint/B as all threads are busy for getting response from endpoint A or in queue.
Can someone plz suggest is there any way to apply bucketization on each exposed endpoint level and allow only limited request to process at a time & put remaining into bucket/queue so that request on other endpoints can work properly ?
Edit:- following is solution approach
@GetMapping("/endpoint/A")
public CompletableFuture<ResponseEntity<ResponseA>> controllerA() throws InterruptedException {
return CompletableFuture.supplyAsync(()->controllerHelperA());
}
@GetMapping("/endpoint/B")
public CompletableFuture<ResponseEntity<ResponseB>> controllerB() throws InterruptedException {
return CompletableFuture.supplyAsync(()->controllerHelperB());
}
private ResponseEntity<ResponseA> controllerHelperA(){
ResponseA responseA = serviceA.responseClient();
return ResponseEntity.ok().body(responseA);
}
private ResponseEntity<ResponseB> controllerHelperB(){
ResponseB responseB = serviceB.responseClient();
return ResponseEntity.ok().body(responseB);
}
CodePudding user response:
Spring MVC supports the async servlet API introduced in Servlet API 3.0. To make it easier when your controller returns a Callable
, CompletableFuture
or DeferredResult
it will run in a background thread and free the request handling thread for further processing.
@GetMapping("/endpoint/A")
public CompletableFuture<ResponseEntity<ResponseA>> controllerA() throws InterruptedException {
return () {
return controllerHelperA();
}
}
private ResponseEntity<ResponseA> controllerHelperA(){
ResponseA responseA = serviceA.responseClient();
return ResponseEntity.ok().body(responseA);
}
Now this will be executed in a background thread. Depending on your version of Spring Boot and if you have configured your own TaskExecutor
it will either
- use the
SimpleAsycnTaskExecutor
(which will issue a warning in your logs), - the default provided
ThreadPoolTaskExecutor
which is configurable through thespring.task.execution
namespace - Use your own
TaskExecutor
but requires additional configuration.
If you don't have a custom TaskExecutor
defined and are on a relatively recent version of Spring Boot 2.1 or up (IIRC) you can use the following properties to configure the TaskExecutor
.
spring.task.execution.pool.core-size=20
spring.task.execution.pool.max-size=120
spring.task.execution.pool.queue-capacity=50
spring.task.execution.pool.keep-alive=120s
spring.task.execution.thread-name-prefix=async-web-thread
Generally this will be used to execute Spring MVC tasks in the background as well as regular @Async
tasks.
If you want to explicitly configure which TaskExecutor
to use for your web processing you can create a WebMvcConfigurer
and implement the configureAsyncSupport
method.
@Configuration
public class AsyncWebConfigurer implements WebMvcConfigurer {
private final AsyncTaskExecutor taskExecutor;
public AsyncWebConfigurer(AsyncTaskExecutor taskExecutor) {
this.taskExecutor=taskExecutor;
}
public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
configurer.setTaskExecutor(taskExecutor);
}
}
You could use an @Qualifier
on the constructor argument to specify which TaskExecutor
you want to use.