Lets say we have following apps ,

• API app : Responsible for serving the user requests.
• Backend app: Responsible for handling the user requests which are long running tasks. It updates the progress to database (postgres) and distributed cache (Redis).

Both apps are scalable service. Single Backend app handles multiple tenants e.g. Customer here but one customer is assigned to single backend app only.

I have a usecase where I need API layer to connect to specific replica which is handling that customer. Do we have a common Pattern for this ?

Few strategies in mind

1. Pub/Sub: Problem is we want sync guranteed response , probably using Redis
2. gRPC : Using POD IP to connect to specific pod is not a standard way
3. Creating a Service at runtime by adding labels to the replicas and use those. -- Looks promising

Do let me know if there is common pattern or example architecture of this or standard way of doing this?

Note :[Above is a simulation of production usecase, names and actual use case is changed]

CodePudding user response：

You should aim to keep your services stateless, in a Kubernetes environment there is no telling when one pod might be replaced by another due to worker node maintenance.

If you have long running task that cannot be completed during the configured grace period for pods to shutdown during a worked node drain/evacuation you need to implement some kind of persistent work queue as your are think about in option 1. I suggest you look into the saga pattern.

Another pattern we usually employ is to let the worker service write the current state of the job into the database and let the client pull the status every few seconds. This does however require some way of handling half finished jobs that might be abandoned by pods that are forced to shutdown.