Looking for a way to pause gitlab job and resume based on AWS lambda input.

Due to restrictive permissions in my organization, below is my current CI workflow:

In diagram above, on push event lambda triggers gitlab job through webhook. the gitlab job only gets latest code, zip files and copy to certain s3 bucket. AFTER gitlab job is finished, same lambda then triggers codebuild build which does gets latest zip file from s3 bucket, creates UI chunk files and artifacts are pushed to a different s3 bucket.

### gitlab-ci.yml ###
variables:
  environment: <aws-account-number>

stages:
  - get-latest-code

get-latest-code:
  stage: get-latest-code
  script:
    - zip -r project.zip $(pwd)/*
    - export PATH=$PATH:/tmp/project/.local/bin
    - pip install awscli
    - aws s3 cp $(pwd)/project.zip s3://project-input-bucket-dev
  rules:
  - if: ('$CI_PIPELINE_SOURCE == "merge_request_event"' || '$CI_PIPELINE_SOURCE == "push"')

### lambda code ###
def runner_lambda_handler(event, context):
    
    cb = boto3.client( 'codebuild' )

    builds_dir = os.environ.get('BUILDS_DIR', '/tmp/project/builds')
    logger.debug("STARTING GIT LAB RUNNER")
    gitlab_runner_cmd = f"gitlab-runner --debug run-single -u https://git.company.com/ -t {token} " \
                        f"--builds-dir {builds_dir} --max-builds 1 " \
                        f"--wait-timeout 900 --executor shell"
    s3_libraries_loader.subprocess_cmd(gitlab_runner_cmd)

    cb.start_build(projectName='PROJECT-Deploy-dev')

    return {
        "statusCode": 200,
        "body": "Gitlab build success."
    }

#### Codebuild Stack ####
codebuild.Project(self, f"Project-{env_name}",
                        project_name=f"Project-{env_name}",
                        role=codebuild_role,
                        environment_variables={
                            "INPUT_S3_ARTIFACTS_BUCKET": {
                                "value": input_bucket.bucket_name
                            },
                            "INPUT_S3_ARTIFACTS_OBJECT": {
                                "value": "project.zip"
                            },
                            "OUTPUT_S3_ARTIFACTS_BUCKET": {
                                "value": output_bucket.bucket_name
                            },
                            "PROJECT_NAME": {
                                "value": f"Project-{env_name}"
                            }
                        },
                        cache=codebuild.Cache.bucket(input_bucket),
                        environment=codebuild.BuildEnvironment(
                            build_image=codebuild.LinuxBuildImage.STANDARD_5_0
                        ),
                        vpc=vpc,
                        security_groups=[codebuild_sg],
                        artifacts=codebuild.Artifacts.s3(
                            bucket=output_bucket,
                            include_build_id=False,
                            package_zip=False,
                            encryption=False
                        ),
                        build_spec=codebuild.BuildSpec.from_object({
                            "version": "0.2",
                            "cache": {
                                "paths": ['/root/.m2/**/*', '/root/.npm/**/*', 'build/**/*', '*/project/node_modules/**/*']
                            },
                            "phases": {
                                "install": {
                                    "runtime-versions": {
                                        "nodejs": "14.x"
                                    },
                                    "commands": [
                                        "aws s3 cp --quiet s3://$INPUT_S3_ARTIFACTS_BUCKET/$INPUT_S3_ARTIFACTS_OBJECT .",
                                        "unzip $INPUT_S3_ARTIFACTS_OBJECT",
                                        "cd project",
                                        "export SASS_BINARY_DIR=$(pwd)",
                                        "npm cache verify",
                                        "npm install",
                                    ]
                                },
                                "build": {
                                    "commands": [
                                        "npm run build"
                                    ]
                                },
                                "post_build": {
                                    "commands": [
                                        "echo Clearing s3 bucket folder",
                                        "aws s3 rm --recursive s3://$OUTPUT_S3_ARTIFACTS_BUCKET/$PROJECT_NAME"
                                    ]
                                }
                            },
                            "artifacts": {
                                "files": [
                                    "**/*.html",
                                    "**/*.js",
                                    "**/*.css",
                                    "**/*.ico",
                                    "**/*.woff",
                                    "**/*.woff2",
                                    "**/*.svg",
                                    "**/*.png"
                                ],
                                "discard-paths": "yes",
                                "base-directory": "$(pwd)/dist/proj"
                            }
                        })
        )

What's needed:
Currently there is a disconnect between gitlab job and codebuild job. I'm looking to find a way to PAUSE gitlab job after all steps are executed. later on codebuild job successful completion I can resume the same gitlab job and mark as done

thanks in advance

CodePudding user response：

you need to change gitlab's webhook update part and there by curl you can check the status of other parts. if their status are not finished pause git push to pushing event

CodePudding user response：

You'll need to implement your own logic to wait for the codebuild job you finish, you may use batch-get-builds you check the status.

Check out this, you can have something similar in you gitlab job waiting for the codebuild job to finish