I am trying to send couple of hundreds http requests from akka actor however I am getting

akka.stream.BufferOverflowException: Exceeded configured max-open-requests value of [16]. This means that the request queue of this pool (HostConnectionPoolSetup(places.api.here.com,443,ConnectionPoolSetup(ConnectionPoolSettings(16,1,5,16,1,Duration.Inf,100 milliseconds,2 minutes,30 seconds,ClientConnectionSettings(Some(User-Agent: akka-http/10.2.0)...

this the application.conf

   http {
          host-connection-pool {
            max-connections = 16
            min-connections = 1
            max-open-requests = 16
          }
        }

This the code

override def receive: Receive = {
      case Foo(_) => 
       val res: Future[HttpResponse] = Http().singleRequest(HttpRequest(uri = "http://..."))
   // do something for the result

I tried to control by state e.g

override def receive: Receive = run(0)
def run(openRequests: Int) : Receive = {
  case Foo(_) if openRequests <= 16 => 
     context.become(run(openRequests   1))
       val responseFuture: Future[HttpResponse] = Http().singleRequest(HttpRequest(uri = "http://..."))
       responseFuture.foreach(context.become(run(openRequests - 1)))
        //...

either way I got the same exception of BufferOverflowException

any advice will be greatly appreciated

CodePudding user response：

Using context asynchronously inside a Future is a bad idea. context it is only valid during calls to the actor.

The bug is that context.become(run(openRequests - 1)) uses the value of openRequests at the time the Future is created, not the value when it is called. So when the first request completes it will call context.become(run(-1)) (which is clearly bogus) even though there may be 15 outstanding requests.

The solution is to send a private message to yourself in the foreach rather than calling context.become directly. When the actor handles that message it decrements the current request count and sends a new request if necessary.