AWS Step Functions Consuming messages from SQS-CodePudding

I am consuming messages from SQS to trigger queries. When I normally consume a message from SQS in Python, I need to delete the message from SQS. Do I have to manually delete the message from SQS in a Step Function? What is the best/simplest way to do so?

I believe SQS has done the integration:

{
  "Comment": "Run Redshift Queries",
  "StartAt": "ReceiveMessage from SQS",
  "States": {
    "ReceiveMessage from SQS": {
      "Type": "Task",
      "Parameters": {
        "QueueUrl": "******"
      },
      "Resource": "arn:aws:states:::aws-sdk:sqs:receiveMessage",
      "Next": "Run Analysis Queries",
      "ResultSelector": {
        "body.$": "States.StringToJson($.Messages[0].Body)"
      }
    },
    "Run Analysis Queries": {
      "Type": "Task",
      "Parameters": {
        "ClusterIdentifier": "******",
        "Database": "prod",
        "Sql": "select * from ******"
      },
      "Resource": "arn:aws:states:::aws-sdk:redshiftdata:executeStatement",
      "End": true
    }
  },
  "TimeoutSeconds": 3600
}

I just did a test and it seems that the messages goes down temporarily but then goes up again.

Is the best way to insert a Lambda in between the "ReceiveMessage from SQS" stage & Redshift stage?

This raised another question. I have only run this manually. How do I activate this Step Function eventually to run on any message?

CodePudding user response：

If you must use SQS, then you will need to have a lambda function to act as a proxy. You will need to set up the queue as a lambda trigger, and you will need to write a lambda that can parse the SQS message and make the appropriate call to the Step Functions StartExecution API.