Home > Enterprise >  How can I let parallel Jenkins jobs share a limited resource pool?
How can I let parallel Jenkins jobs share a limited resource pool?

Time:10-27

Our project uses Jenkins pipeline for the automation tests and packaging. The pipeline is defined in a Jenkinsfile script having multiple stages. Multiple jobs (triggered by the push events from different dev branches) might be running in parallel.

Now that we need to test a function against an external system which has limited resources.

Let's say, that system has 10 resource "slots" for tests: #1, #2, ..., #10. When each pipeline job wants to test that function, it needs to "reserve" a slot number (just an integer will suffice) and then the test program talks to the external system using the slot number (like a token or something). When the job finishes, it releases the number.

Is it possible in Jenkins? In other words, the Jenkins needs to maintain a small integer array for all parallel jobs. Whenever a job asks for a "slot" number, it finds a free number in the array and lock it up.

I Googled it and found a Jenkins plugin called "lockable resource plugin" (Jenkins Configuration - Lockable Resources

  • Use the lock step in your pipeline code to acquire a slot. The key here is to use the quantity: 1 argument to acquire only a single slot (by default it would try to acquire all resources that match the given label). It will be released automatically when the code block of the lock step ends. When all slots are in use, the lock step waits until the next slot becomes available.

    pipeline {
        agent any
        stages {
            stage('Stage 1') {
                steps {
                    script {
                        // Acquire a single slot
                        lock(label: 'slots', quantity: 1, variable: 'slotName') {
    
                            // Extract the slot number from the resource name                           
                            def slotNumber = slotName.split('-')[1] as int
    
                            // Code that uses the slot
    
                            echo "Acquired slot $slotNumber"                        
    
                            // Slot gets released automatically
                        }
                    }
                }
            }
        }
    }
    
    • Related