How to upload files to AWS S3?, I get an error when doing it (using private buckets)-CodePudding

I am trying to upload files to AWS S3 using getSignedUrlPromise() to obtain the access link since the bucket is completely private, I want it to only be accessible through the links that the server generates with getSignedUrlPromise().

The problem comes when I try to make a Put request to that link obtained since I get the following error, I also leave you the response that I receive.

Here is the code for configuring aws in nodeJS:

import AWS from 'aws-sdk';

const bucketName = 'atlasfitness-progress';
const region =process.env.AWS_REGION;
const accessKeyId = process.env.AWS_ACCESS_KEY
const secretAccessKey = process.env.AWS_SECRET_KEY

const URL_EXPIRATION_TIME = 60; // in seconds

const s3 = new AWS.S3({
    region,
    accessKeyId,
    secretAccessKey,
    signatureVersion: 'v4'
})

export const generatePreSignedPutUrl = async (fileName, fileType) => {

    const params = ({
        Bucket: bucketName,
        Key: fileName,
        Expires: 60
    })

    const url = await s3.getSignedUrlPromise('putObject', params);

   return url;
}

And then I have a express controller to send the link when it's requested:

routerProgress.post('/prepare_s3', verifyJWT, async (req, res) => {
  res.send({url: await generatePreSignedPutUrl(req.body.fileName, req.body.fileType)});
})

export { routerProgress };

But the problem comes in the frontend, here is the function that first asks for the link and then it tryies to upload the file to S3.

const upload = async (e) => {
    e.preventDefault();
    await JWT.checkJWT();

    const requestObject = {
        fileName: frontPhoto.name,
        fileType: frontPhoto.type,
        token: JWT.getToken()
    };

    const url = (await axiosReq.post(`${serverPath}/prepare_s3`, requestObject)).data.url;
    //Following function is the one that doesn't work
    const response = await fetch(url, {
      method: "PUT",
      headers: {
        "Content-Type": "multipart/form-data"
      },
      body: frontPhoto
  });
    console.log(response);
}

And with this all is done, I can say that I am a newbie to AWS so it is quite possible that I have caused a rather serious error without realizing it, but I have been blocked here for some many days and I'm starting to get desperate. So if anyone detects the error or knows how I can make it work I would be very grateful for your help.

CodePudding user response：

The first thing I note about your code is that you await on async operations but do not provide for exceptions. This is very bad practice as it hides possible failures. The rule of thumb is: whenever you need to await for a result, wrap your call in a try/catch block.

In your server-side code above, you have two awaits which can fail, and if they do, any error they generate is lost.

A better strategy would be:

export const generatePreSignedPutUrl = async (fileName, fileType) => {

    const params = ({
        Bucket: bucketName,
        Key: fileName,
        Expires: 60
    })

    let url;
    try {
      url = await s3.getSignedUrlPromise('putObject', params);
    } catch (err) {
      // do something with the error here
      // and abort the operation.
      return;
    }
      

   return url;
}

And in your POST route:

routerProgress.post('/prepare_s3', verifyJWT, async (req, res) => {
  let url; 
  try {
    url = await generatePreSignedPutUrl(req.body.fileName, req.body.fileType);
  } catch (err) {
    res.status(500).send({ ok: false, error: `failed to get url: ${err}` });
    return;
  }
  res.send({ url });
})

And in your client-side code, follow the same strategy. At the very least, this will give you a far better idea of where your code is failing.

Two things to keep in mind:

Functions declared using the async keyword do not return the value of the expected result; they return a Promise of the expected result, and like all Promises, can be chained to both .catch() and .then() clauses.
When calling async functions from within another async function, you must do something with any exceptions you encounter because, due to their nature, Promises do not share any surrounding runtime context which would allow you to capture any exceptions at a higher level.

So you can use either Promise "thenable" chaining or try/catch blocks within async functions to trap errors, but if you choose not to do either, you run the risk of losing any errors generated within your code.

CodePudding user response：

Here's an example of how create a pre-signed URL that can be used to PUT an MP4 file.

const AWS = require('aws-sdk');

const s3 = new AWS.S3({
  apiVersion: '2010-12-01',
  signatureVersion: 'v4',
  region: process.env.AWS_DEFAULT_REGION || 'us-east-1',
});

const params = {
  Bucket: 'mybucket',
  Key: 'videos/sample.mp4',
  Expires: 1000,
  ContentType: 'video/mp4',
};

const url = s3.getSignedUrl('putObject', params);
console.log(url);

The resulting URL will look something like this:

https://mybucket.s3.amazonaws.com/videos/sample.mp4?
    Content-Type=video/mp4&
    X-Amz-Algorithm=AWS4-HMAC-SHA256&
    X-Amz-Credential=AKIASAMPLESAMPLE/20200303/us-east-1/s3/aws4_request&
    X-Amz-Date=20211011T090807Z&
    X-Amz-Expires=1000&
    X-Amz-Signature=long-sig-here&
    X-Amz-SignedHeaders=host

You can test this URL by uploading sample.mp4 with curl as follows:

curl -X PUT -T sample.mp4 -H "Content-Type: video/mp4" "<signed url>"

A few notes:

hopefully you can use this code to work out where your problem lies.
pre-signed URLs are created locally by the SDK, so there's no need to go async.
I'd advise creating the pre-signed URL and then testing PUT with curl before testing your browser client, to ensure that curl works OK. That way you will know whether to focus your attention on the production of the pre-signed URL or on the use of the pre-signed URL within your client.