I have sharded services, where each one have to have its ordinal. Also, each service consume different amount RAM, for example one - 4Gib, another - 20Gib. I would like to roll out these services on kubernates cluster.
I thought of using Stateful Set, where each service gets its ordinal from pod name. But there is the problem relating to resourse limits. As I wrote above, RAM consumption varies between shards, and as far as I know, high resourse limits may lead to less effective service distribution on nodes by scheduler.
Which solution suits to my problem or is the suggested one acceptable?
Requirements:
- each pod have to get unique number (that may be acquired inside service) starting from 0
- easy to increase shard number; new shard must get a new unique number
- easy to deploy new service changes
- there are dns entries for each pod, where the differences between them are only pod numbers
- possibility to set limits for each pod (this requirement stems from the fact that each shard consumes different amount of RAM)
CodePudding user response:
The standard Kubernetes Deployment and StatefulSet objects require the replicas to be more or less the same. A StatefulSet has more specific rules on how the Pod name is constructed and the order Pods are created and creates a PersistentVolumeClaim for each replica, but in particular every Pod has the same resource constraints.
That means you don't have a choice but to create a separate Deployment for each of your shards. This lines up will with your requirement that the Pods be individually addressable, though, since that requirement also means you probably need a separate Service for each shard.
If you're deploying in a Helm chart then you could use a range
loop to construct the Deployments and Services, and a well-constructed chart should be able to meet your other requirements as well. For example, you could write a Helm YAML values file containing something like
# values-prod.yaml
registry: registry.example.com
image: myimage
tag: a1b2c3
shards:
- memory: 4GiB
- memory: 20GiB
You could update this file to change the tag, or add shards, and run helm update -f values-prod.yaml ...
to redeploy it.
Inside the chart, you'd have to create Deployments and Services in a loop, which would look like
{{- $image := printf "%s:%s/%s" .Values.registry .Values.image .Values.tag }}
{{- range $index, $shard := range .Values.shards -}}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: shard-{{ $index }}
...
spec:
template:
spec:
containers:
- name: shard
image: {{ $image }}
resources:
limits:
memory: {{ $shard.memory }}
---
apiVersion: v1
kind: Service
...
{{ end }}