How to delete thousands of objects from s3 bucket with in specific object folder?-CodePudding

Im having thousands of objects in all the folders gocc1, gocc2,etc

s3://awss3runner/gocc1/gocc2/goccf/

i just want to delete the objects(50,000 ) from goccf and its versions

import boto3
session = boto3.Session()
s3 = session.resource(service_name='s3')
#bucket = s3.Bucket('awss3runner','goccf')if we use this getting error
bucket = s3.Bucket('awss3runner') # (working but if we use this everything in the bucket getting deleted)
bucket.object_versions.delete()

is there anyway to delete goccf objects and its versions

CodePudding user response：

You can use the DeleteObjects API in S3 (https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html)

I would first perform a list operation to enumerate all the objects you wish to delete, then pass that into DeleteObjects. Be very careful as you could accidentally delete other objects in your bucket.

Another option, is to use an S3 lifecycle policy, if this is going to be a one-off operation. With a lifecycle policy you can specify a path in your S3 bucket and set the objects to Expire. They will be asynchronously removed from your S3 bucket https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-expire-general-considerations.html