Within a GAE application, we want to process Pub Sub messages by batches, for example: every 10 seconds read all pending messages on a subscription and process them in a batch (because that will be more efficient). A synchronous subscriber.pull() would nicely allow us to read a batch of pending messages. The question is what would I do next ? Sleep for 10 seconds then read again ? But that would require a permanent background task, which is sort of difficult to set up in App Engine. An endpoint called by a cron every minute (or every hour), that runs a number of cycles of [ read and process messages, sleep for 10 seconds ] cycles for an hour, then exits ? Any better idea ?

CodePudding user response：

You can use Cloud Scheduler to call every minutes your App Engine endpoint. This endpoint reads the pubsub subscription for a while (let say 45 seconds) processes the messages and then return a 200 HTTP code.

If you want to read by time window of 10s, you need to build the process on your side. Continue to call the endpoint every minutes (it's the Serverless pattern, the process is perform only inside a request handling), but the endpoint listen the subscription for 10s, processes the messages, sleep 10s et repeat that 5 time, and then return a 200 HTTP code.