I have some data to be ingested from redshift to snowflake.currently I use unload command to get it to s3 bucket and then load it to snowflake for redshift and aws cp for other files. I am looking for someway to automate this unloading .Looking for suggestions .I also have a use case to load the delta of these data so will have to run it on a schedule. Any suggestions would be appreciated

CodePudding user response：

You could use a lambda if you are confident that the execution time will be less than 15 minutes Otherwise you could use a

Configure the pipeline and press Edit in architect

You will see a graph similar to the one below. You can easily adjust it to use Redshift instead of RDS (As it's displayed in the screenshot)