Login to aws through Gitlab CI-CD pipeline-CodePudding

My .gitlab-ci.yml pipeline worked like a charm in the last year and today, from nothing, I am unable to login to my aws account with this error:

$ echo `aws ecr get-login --no-include-email --region eu-central-1` | sh
Traceback (most recent call last):
  File "/usr/local/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/local/lib/python3.5/dist-packages/awscli/clidriver.py", line 17, in <module>
    import botocore.session
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 30, in <module>
    import botocore.client
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 16, in <module>
    from botocore.args import ClientArgsCreator
  File "/usr/local/lib/python3.5/dist-packages/botocore/args.py", line 26, in <module>
    from botocore.signers import RequestSigner
  File "/usr/local/lib/python3.5/dist-packages/botocore/signers.py", line 19, in <module>
    import botocore.auth
  File "/usr/local/lib/python3.5/dist-packages/botocore/auth.py", line 121
    pairs.append(f'{quoted_key}={quoted_value}')
                                              ^
SyntaxError: invalid syntax

Environment

I'm using docker to build images, push them to ECR and then force the deployment inside my ECS cluster. I'm also using gitlab in my self-hosted server and have 3 defined variables set in the Gitlab CI/CD section. The variables are: AWS_ACCESS_KEY_ID,AWS_DEFAULT_REGION,AWS_SECRET_ACCESS_KEY.

This is my .gitlab-ci.yml file:


services:
  - docker:dind

stages:
  - test_build
  - deploy_staging
  - deploy_production

test_build:
  stage: test_build
  only:
    - merge_requests
  tags:
    - genuino.webapp.runner
  image: ubuntu:16.04
  script:
    # Add some dependencies for docker and the AWS CLI
    - apt-get update -y  # Get the most up-to-date repos.
    - apt-get install -y apt-transport-https ca-certificates software-properties-common python-software-properties curl python3-pip
    # Install Docker
    - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
    - apt-key fingerprint 0EBFCD88
    - add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    - apt-get update -y
    - apt-get install -y docker-ce
    # Build our image
    - docker build -t $APP_NAME -f ./deploy/Dockerfile .

deploy_staging:
  stage: deploy_staging
  image: ubuntu:16.04
  only:
    - tags
  except:
    - branches
  tags:
    - genuino.webapp.runner
  script:
    # Add some dependencies for docker and the AWS CLI
    - apt-get update -y  # Get the most up-to-date repos.
    - apt-get install -y apt-transport-https ca-certificates software-properties-common python-software-properties curl python3-pip
    # Install Docker
    - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
    - apt-key fingerprint 0EBFCD88
    - add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    - apt-get update -y
    - apt-get install -y docker-ce
    # Install the AWS CLI and login to our registry
    - pip3 install awscli
    - pip3 install rsa
   - echo `aws ecr get-login --no-include-email --region eu-central-1` | sh
    # Build and push our image
    - docker build -t $APP_NAME -f ./deploy/Dockerfile .
    - docker tag $APP_NAME:$VERSION $REPOSITORY_URL/$APP_NAME:$VERSION
    - docker push $REPOSITORY_URL/$APP_NAME:$VERSION
    # Force deploy
    - aws ecs update-service --cluster genuino-staging --service webapp --force-new-deployment --region eu-central-1

deploy_production:
  stage: deploy_production
  image: ubuntu:16.04
  when: manual
  only:
    refs:
      - develop
      - tags
  except:
    - branches
  tags:
    - genuino.webapp.runner
  script:
    # Add some dependencies for docker and the AWS CLI
    - apt-get update -y  # Get the most up-to-date repos.
    - apt-get install -y apt-transport-https ca-certificates software-properties-common python-software-properties curl python3-pip
    # Install Docker
    - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
    - apt-key fingerprint 0EBFCD88
    - add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
    - apt-get update -y
    - apt-get install -y docker-ce
    # Install the AWS CLI and login to our registry
    - pip3 install awscli
    - pip3 install rsa
    - echo `aws ecr get-login --no-include-email --region eu-central-1` | sh
    # Build and push our image
    - docker build -t $PROD_APP_NAME -f ./deploy/Dockerfile.production .
    - docker tag $PROD_APP_NAME:$VERSION $REPOSITORY_URL/$PROD_APP_NAME:$VERSION
    - docker push $REPOSITORY_URL/$PROD_APP_NAME:$VERSION
    # Force deploy
    - aws ecs update-service --cluster genuino-production --service webapp --force-new-deployment --region eu-central-1

What I already done

I tried to change the authentication line as this: aws ecr get-login-password | docker login -u AWS --password-stdin $REPOSITORY_URL, it works in localhost, but during the deploy I get this error:

$ aws ecr get-login-password | docker login -u AWS --password-stdin $REPOSITORY_URL
Traceback (most recent call last):
  File "/usr/local/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/local/lib/python3.5/dist-packages/awscli/clidriver.py", line 17, in <module>
    import botocore.session
  File "/usr/local/lib/python3.5/dist-packages/botocore/session.py", line 30, in <module>
    import botocore.client
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 16, in <module>
    from botocore.args import ClientArgsCreator
  File "/usr/local/lib/python3.5/dist-packages/botocore/args.py", line 26, in <module>
    from botocore.signers import RequestSigner
  File "/usr/local/lib/python3.5/dist-packages/botocore/signers.py", line 19, in <module>
    import botocore.auth
  File "/usr/local/lib/python3.5/dist-packages/botocore/auth.py", line 121
    pairs.append(f'{quoted_key}={quoted_value}')
                                              ^
SyntaxError: invalid syntax
Error: Cannot perform an interactive login from a non TTY device

CodePudding user response：

AWS cli v1 require Python 3.6 while you are using Python 3.5 in GitLab CI. Upgrading Python should solve your problem

https://docs.aws.amazon.com/cli/latest/userguide/welcome-versions.html#welcome-versions-v1