Home > Software design >  Uploading json content on s3 using presigned url - gzipped
Uploading json content on s3 using presigned url - gzipped

Time:09-14

On the backend i'm generating a presigned url to upload a json object to an s3 bucket. I want the body actual json to be compressed.

This is what i'm doing in the backend:

  public String generateUploadLink(String bucketName, String bucketKey, int days) {

    var objectRequestBuilder = PutObjectRequest.builder().bucket(bucketName).key(bucketKey);
    objectRequestBuilder.contentType(MediaType.APPLICATION_JSON_VALUE);
    objectRequestBuilder.contentEncoding(GZIP.name().toLowerCase());

    PutObjectRequest objectRequest = objectRequestBuilder.build();
    PutObjectPresignRequest presignRequest =
        PutObjectPresignRequest.builder()
            .signatureDuration(Duration.ofDays(days))
            .putObjectRequest(objectRequest)
            .build();
    return this.s3Presigner.presignPutObject(presignRequest).url().toString();
  }

Now, in postman, I manually add the content-encoding: gzip header and the file gets uploaded

postman upload gzipped json

Issues:

1. If i go to the aws console and try to download the file i get enter image description here

If i also try to open the file, nothing happens, no errors in the web console.

2. If i generate a download link from the backend

  public String generateDownloadLink(String bucketName, String responseContentDisposition,
      String key, int days) {

    GetObjectRequest objectRequest =
        GetObjectRequest.builder()
            .bucket(bucketName)
            .responseContentDisposition(responseContentDisposition)
            .key(key).build();

    GetObjectPresignRequest preSignRequest =
        GetObjectPresignRequest.builder()
            .signatureDuration(Duration.ofDays(days))
            .getObjectRequest(objectRequest)
            .build();

    PresignedGetObjectRequest presignedRequest = this.s3Presigner.presignGetObject(preSignRequest);

    return presignedRequest.url().toString();
  }

and try to use that in postman, i get

Error: incorrect header check
Request Headers
User-Agent: PostmanRuntime/7.29.2
Accept: */*
Postman-Token: d4ecd208-8474-45aa-aecf-3d41caa863f3
Host: <host>.s3.eu-west-2.amazonaws.com
Accept-Encoding: gzip, deflate, br
Connection: keep-alive

If i remove the gzip encoding tag from file in the aws console, I can download/view the file normally.

CodePudding user response:

The problem you are facing is due to postman being set to raw body type while uploading.

You should gzip the file before hand and uploading using binary body type.

  • Related