I'm working on a web app and plan to deploy using Kubernetes. The web, app and database tiers will all be containerized. I plan to use a cluster with 3 master and 12 worker nodes. I know that each node is limited to ~100 pods. I'm expecting around 10k concurrent requests and want to decide on the number of replicas in my Kubernetes deployments. I can't find a connection limit number online; one source says that a maximum of 8 concurrent connections are possible per pod. So would I need at least 1250 pods?

Searched for hard limits online but could not find any numbers. Found a pod limit of 110 per node but no connections per pod limit

CodePudding user response：

I can't find a connection limit number online

There is no connection limit.

Your app will be constrained by CPU or memory resources and it may affect your response latency. Exactly where your bottleneck will be, depends totally on your application.

Invest in good observability using e.g. Prometheus and Grafana and you will see what and when you need to improve.

CodePudding user response：

This is a number you have to figure out based on your code. I generally go by the following steps:

1. Install Prometheus-Grafana setup to measure the health metrics of your pod
2. Install your service with one pod and run a load test to find out when your pod reaches the limit of CPU/Memory. Whichever of these (CPU/Memory) reaches 100% first, is called the saturation metric for your application. Your pod is bound to fail beyond 100% usage of your saturation metric.
3. Find the number of requests at which your pod will reach x% if your saturation metrics. You may start with x being 70. This becomes your operative range. This is kept lower than 100% to manage spikes. You may fine-tune it later.
4. Set an HPA and make it follow your operating range. This will scale the number of pods based on traffic and utilization of saturation metric. Keep the min pod limit to >=2 to handle any unexpected pod restarts.

I have given you a general guideline. There is a lot of fine text that you will figure out along the way.