How to send a lot of simultaneous requests to FastAPI endpoint?-CodePudding

With a "normal" couroutine like below, the result is that all requests are printed first and then after circa 5 seconds all responses are printed:

import asyncio

async def request():
    print('request')
    await asyncio.sleep(5)
    print('response')

loop = asyncio.get_event_loop()

tasks = [
    loop.create_task(request())
    for i in range(30)
]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

I want to replicate same behavior in FastApi, so I have and endpoint like this:

import asyncio

from fastapi import FastAPI

app = FastAPI()

@app.post("/")
async def root():
    print('request')
    await asyncio.sleep(5)
    return 'OK'

And I am bombing it with multiple requests from the frontend like this:

 const url = 'http://localhost:8000'
 const data = [1, 2, 3]
 const options = {
     method: 'POST',
     headers: new Headers({'content-type': 'application/json'}),
     body: JSON.stringify({data}),
     mode: 'no-cors',
 }

 for (let i=0; i<30; i  ) {
     fetch(url, options)
 }

However, in the terminal I can clearly see that the FastAPI accepts only 6 requests at a time, returns responses for them, and then accepts another 6:

request
request
request
request
request
request
←[32mINFO←[0m:     127.0.0.1:63491 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:58337 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:50479 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:60499 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:56990 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:56107 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
request
request
request
request
request
request
←[32mINFO←[0m:     127.0.0.1:58337 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:63491 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:60499 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:56990 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:56107 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m
←[32mINFO←[0m:     127.0.0.1:50479 - "←[1mPOST / HTTP/1.1←[0m" ←[32m200 OK←[0m

etc.

Is this because of some FastAPI/uvicorn setting or limitation?

Can the number be increased, and is it even reasonable to do so?

CodePudding user response：

You're doing this in your browser, so you're effectively hitting the parallel request limit in your browser. It's not related to the API itself. If you want to test your API's performance, use a tool that has been designed to do that - such as siege, httperf, ab or similar tools.

From the answer documenting the current parallel request limits in browsers:

Firefox 3 : 6
...
Edge:       6
Chrome:     6