I'm new to python. I have an event triggered AWS Lambda function that copies files from an S3 bucket to another S3 bucket. The destination S3 path where I want to copy the file is: "dest_bucket/folder1/test". It gives me this error when I try to run it:

Invalid bucket name "dest_bucket/folder1/test": Bucket name must match the regex "^[a-zA-Z0-9.-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).:(s3|s3-object-lambda):[a-z-0-9]:[0-9]{12}:accesspoint[/:][a-zA-Z0-9-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z-0-9] :[0-9]{12}:outpost[/:][a-zA-Z0-9-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9-]{1,63}$"

The source bucket does not have any folder structure. The destination bucket has a folder structure and the files need to be copied under "dest_bucket/folder1/test". The error is occurring here in the lambda function: "destination_bucket_name = 'dest_bucket/folder1/test". Because, if I simply write the destination bucket name without the slashes, it works! Any idea how i should write this?

import json
import boto3
import os
import uuid

def lambda_handler(event, context):
    try:
        client = boto3.client('sts')
        response = client.assume_role(RoleArn='arn:aws:iam::xxx:role/xxx_lambda_role',RoleSessionName="cross_acct_lambda")
        session = boto3.Session(aws_access_key_id=response['Credentials']['AccessKeyId'],aws_secret_access_key=response['Credentials']['SecretAccessKey'],aws_session_token=response['Credentials']['SessionToken'])
        print(session)
        print("successfully assumed role")
        
        s3_client = boto3.client("s3", aws_access_key_id=response['Credentials']['AccessKeyId'],aws_secret_access_key=response['Credentials']['SecretAccessKey'],aws_session_token=response['Credentials']['SessionToken'])
        #s3_client = boto3.client("s3")
        
        #base = read from parameter store
        #table_partion = read from file
        destination_bucket_name = 'dest_bucket/folder1/test'

        # event contains all information about uploaded object
        print("Event :", event)

        # Bucket Name where file was uploaded
        source_bucket_name = event['Records'][0]['s3']['bucket']['name']
        print(source_bucket_name)

        # Filename of object (with path)
        file_key_name = event['Records'][0]['s3']['object']['key']
        #file_key_name = 'empty_test.txt'
        print(file_key_name)

        # Copy Source Object
        copy_source_object = {'Bucket': source_bucket_name, 'Key': file_key_name}
        print(copy_source_object)

        # S3 copy object operation
        s3_client.copy_object(CopySource=copy_source_object, Bucket=destination_bucket_name, Key=file_key_name)


        return {
            'statusCode': 200,
            'body': json.dumps('S3 events Lambda!')
        }

    except Exception as e:
        print(e)
        raise e

CodePudding user response：

From the docs:

The bucket name can be between 3 and 63 characters long, and can contain only lower-case characters, numbers, periods, and dashes.

Each label in the bucket name must start with a lowercase letter or number.

The bucket name cannot contain underscores, end with a dash, have consecutive periods, or use dashes adjacent to periods.

The bucket name cannot be formatted as an IP address (198.51.100.24).

https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-s3-bucket-naming-requirements.html

CodePudding user response：

Make sure you just use the bucket name for bucket name :) as for the "path", it's really a fake thing in S3 - the only real thing is object key. Slashes are just characters in that name, they have no special meaning.

CodePudding user response：

You can use:

destination_bucket = 'dest_bucket'
destination_path = 'folder1/test/'

for record in event['Records']:
    source_bucket = record['s3']['bucket']['name']
    source_key = record['s3']['object']['key']

    copy_source_object = {'Bucket': source_bucket, 'Key': source_key}

    destination_key = destination_path   source_key
    s3_client.copy_object(CopySource=copy_source_object, Bucket=destination_bucket, Key=destination_key)

This will loop through all incoming Records within the event.

It will then create an object with the name folder1/test/ the name (Key) of the source object.