Copy file from s3 subfolder in another subfolder in same bucket-CodePudding

I'd like to copy file from subfolder into another subfolder in same s3 bucket. I've read lots of questions in SO and I came finally with this code. It has an issue, when I run it it works, but it doesn't copy the file only, it copy the folder that contain the file into the destination wanted I've have the file but inside a folder(root). How do I only copy the files inside that subfolder?

XXXBUCKETNAME:
    -- XXXX-input/  # I want to copy from here
    -- XXXX-archive/ # to here


import boto3
from botocore.config import Config
s3 = boto3.resource('s3', config=Config(proxies={'https': getProperty('Proxy', 'Proxy.Host')}))
bucket_obj = s3.Bucket('XXX')
destbucket = 'XXX'

jsonfiles = []
for obj in bucket_obj.objects.filter(Delimiter='/', Prefix='XXXX-input/', ):
    if obj.key.endswith('json'):
        jsonfiles.append(obj.key)
for k in jsonfiles:
    if k.split("_")[-1:][0] == "xxx.txt":
        dest = s3.Bucket(destbucket)
        source= { 'Bucket' : destbucket, 'Key': k}
        dest.copy(source, "XXXX-archive/" k)

it give:

XXXBUCKETNAME:
    -- XXXX-input/
    -- XXXX-archive/
        -- XXXX-input/file.txt

I want:

XXXBUCKETNAME:
    -- XXXX-input/
    -- XXXX-archive/
        -- file.txt

CodePudding user response：

In S3 there really aren't any "folders." There are buckets and objects, as explained in documentation. The UI may make it seem like there are folders, but the key for an object is the entire path. So if you want to copy one item, you will need to parse its key and build the destination key differently so that it has the same prefix (path) but end with a different value.

In Amazon S3, buckets and objects are the primary resources, and objects are stored in buckets. Amazon S3 has a flat structure instead of a hierarchy like you would see in a file system. However, for the sake of organizational simplicity, the Amazon S3 console supports the folder concept as a means of grouping objects. It does this by using a shared name prefix for objects (that is, objects have names that begin with a common string). Object names are also referred to as key names.

In your code you are pulling out each object's key, so that means the key already contains the full "path" even though there isn't really a path. So you will want to split the key on the / character instead and then take the last element in the resulting list and append that as the file name:

dest.copy(source, "XXXX-archive/"   k.split("/")[-1])