is there a way to push newly uploaded file from S3 to an FTP or SFTP server within AWS Services?

my s3 looks something like this: s3-bucket/some_path/yyyymm/yyyymmdd/file_yyymmdd.csv.gz

and everytime we generate a new file based on date, we need to upload or transfer to FTP server

CodePudding user response：

You can have S3 send event notifications to other AWS services when a new object is uploaded to a bucket.

You could have that trigger a Lambda function each time a new object is uploaded. The Lambda function would receive an event object with information about the S3 bucket and the path of the object in the bucket. It can use that info to download the file from S3, and upload it to an FTP server.

I would recommend having S3 send the events to an SQS queue, and having your Lambda function pull events from the queue, that way you have both built-in error handling, and concurrency throttling of your Lambda function invocations.

If you don't want to use a Lambda function for this, you could have S3 send the events to SQS, and then run some code that polls SQS anywhere, such as on an EC2 server, or in an ECS task.