Home > Mobile >  Use fastapi app engine with GCloud Buckets to upload more than 35MB
Use fastapi app engine with GCloud Buckets to upload more than 35MB

Time:09-16

I currently have a Fastapi App Engine app. It handles file uploads from a form, processes the file, adds a record of it to a DB, and saves the file in a Google Cloud storage bucket. However, the app engine limits the file upload size of a POST request to 35mb. I can test the app locally and it works perfectly, but the GCloud GFE causes the endpoint to return a status code of 413 for any large files. Below you can see the code I'm using to handle file uploads.

@router.post('/upload')
def upload(name: str = Form(...), file: UploadFile = File(...), db=Depends(get_session)):
    """
    Uploads a file to the server
    """

    if not file.content_type.startswith('video/'):
        error = "File must be a video"
        return RedirectResponse(url='/files?error_message='   error, status_code=302)

    create_db_record(db, name, file)

    storage_client = storage.Client.from_service_account_json('service_account.json')

    bucket = storage_client.get_bucket('myapp.appspot.com')
    blob = bucket.blob(file.filename)
    blob.upload_from_file(file.file)

    return RedirectResponse(url='/')

If I were to directly upload the file to the bucket, i wouldn't be able to process it first, but sending it through the upload form isn't an option if it is going to be over 35mb. How do I allow this Fastapi post request to accept larger files to be able to upload to the bucket? All of the documentation I have found doesn't show how to handle it from the fastapi/django side.

CodePudding user response:

The limit is 32MB. You cannot change that limit (quota).

That value is determined/configured at the Google Frontend (GFE) which is a global service for much of Google. Services such as App Engine, Cloud Run, and Cloud Functions have the same limitation.

App Engine also has a 32 MB file size limit.

Google Compute Engine does not have the same limitations for file size or upload size.

  • Related