I have lots of .tar.gz files (millions) stored in a bucket on Amazon S3.

I'd like to untar them and create the corresponding folders on Amazon S3 (in the same bucket or another).

Is it possible to do this without me having to download/process them locally?

CodePudding user response：

It's not possible with only S3. You'll have to have something like EC2, ECS or Lambda, preferably running in the same region as the S3 bucket, to download the .tar files, extract them, and upload every file that was extracted back to S3.

CodePudding user response：

With AWS lambda you can do this! You won't have the extra costs with downloading and uploading to AWS network.

You should follow this blog but then unzip instead of zipping! https://www.antstack.io/blog/create-zip-using-lambda-with-files-streamed-from-s3/