Can s3.getObject or s3.putObject be used in Promise.then() block?-CodePudding

I need to test if an object exists in an S3 bucket, if yes, I need to download the object, and if not, create the object. My code looks something like below:

let params = {
    Bucket: bucket,
    Key: fileName,        
};

const function async () => {
    await s3.headObject(params).promise()
    .then(() => {
        s3.getObject(params)
        .promise()
        .then(response => {
            console.log(response.Body.toISOString());
        });
    })
    .catch((err) => {
        if (err.code == "NotFound") {
            s3.putObject({
                Body: event,
                Bucket: bucket,
                Key: fileName,                  
            });
        }
    });
}

It doesn't seem to work, problem might be nesting an s3.getObject or s3.putObject call inside an then or catch block. I have tried prepending await before s3.getObject/s3.putObject, still no good.

CodePudding user response：

You must return nested promises from within their .then() handlers so that the promises are properly chained. But, if you're going to use async/await, it's better to not even use .then() and just use await with no nesting. In general, you do not want to mix await and .then() as it complicates proper flow and error handling.

Here's a version where you just use await and all the promises are properly sequenced:

const params = {
    Bucket: bucket,
    Key: fileName,
};

async function someFunction() {
    try {
        await s3.headObject(params).promise();
        let response = await s3.getObject(params).promise();
        console.log(response.Body.toISOString());
        return response.Body;
    } catch (err) {
        if (err.code == "NotFound") {
            await s3.putObject({
                Body: event,
                Bucket: bucket,
                Key: fileName,
            }).promise();
        } else {
            throw err;
        }
    }
}

FYI, it looks like you could/should get rid of the headObject() part of your function. It appears that all you're doing is checking to see if the object exists and getObject() should already do the same thing. So, just handle errors properly on getObject() and you can remove the headObject() like this:

const params = {
    Bucket: bucket,
    Key: fileName,
};

async function someFunction() {
    try {
        let response = await s3.getObject(params).promise();
        console.log(response.Body.toISOString());
        return response.Body;
    } catch (err) {
        if (err.code == "NotFound") {
            await s3.putObject({
                Body: event,
                Bucket: bucket,
                Key: fileName,
            }).promise();
        } else {
            throw err;
        }
    }
}

Note: This code is subject to race conditions if two pieces of code are both attempting to add an object with the same key at the same time. That's because there's a race condition window between the time you check for the object's existence and the time you add it where another thread of code could add the object which you would then overwrite or end up with duplicates (depending upon how the data-store is configured). Your code to check if it exists and then add it if it isn't is not atomic.

This is a classic data-store mistake/error. I don't know the s3 API in detail, but in general, you should either configure the data store so only one object with a given key is ever allowed and you just attempt to add the new object and if it already exists with the same key, you just get an error back that it already exists or you use a specific atomic function in the data-store to add it if it doesn't already exist.