One pod/job per kubernetes node-CodePudding

I would like to schedule periodically kubernetes jobs (with different images)

These jobs are required to run on a node with GPU support (1 GPU device)

Currently If I create two jobs at the same time - the pods will be scheduled both on the same node - while only one pod will have access to GPU device

Is there a way to configure nodes/pods so that scheduler only places one pod per node and once it is completed places next job ?

CodePudding user response：

You could set an inter-pod anti-affinity as described in the docs here.

Inter-pod affinity and anti-affinity rules take the form "this Pod should (or, in the case of anti-affinity, should not) run in an X if that X is already running one or more Pods that meet rule Y", where X is a topology domain like node, rack, cloud provider zone or region, or similar and Y is the rule Kubernetes tries to satisfy.

Similar to node affinity are two types of Pod affinity and anti-affinity as follows:

requiredDuringSchedulingIgnoredDuringExecution

preferredDuringSchedulingIgnoredDuringExecution

Consider the following Pod spec:

apiVersion: v1
kind: Pod
metadata:
  name: with-pod-affinity
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: security
            operator: In
            values:
            - S1
        topologyKey: topology.kubernetes.io/zone
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: security
              operator: In
              values:
              - S2
          topologyKey: topology.kubernetes.io/zone
  containers:
  - name: with-pod-affinity
    image: registry.k8s.io/pause:2.0

CodePudding user response：

Since you are preferring to use a simple way you can use nodeSelector for selecting your desired node using the node labels and use queueSort for scheduling pods one after another. In short you are defining pods with certain labels to run on a certain node that too on a priority basis. This document gives you a better understanding for achieving your desired functionality.