Running python unittests in azure pipeline doesn't fail even though it should-CodePudding

I want to run unittests in my docker-compose application as part of an Azure DevOps pipeline. The problem is that the tests fail locally but they don't fail when I run them in the pipeline. These are the relevant bits of my code:

The docker-compose configuration sets up a python app and a MySQL image. The unittests are running against the db service in the app service:

services: 
  app: 
    build: . 
    ports: 
      - '8080:8080'
    command: ./make.sh unittests_entrypoint
    stdin_open: false # docker run -i
    tty: false        # docker run -t
    depends_on:
      - db
    environment:
      - DB_USER_TEST
      - DB_PASSWORD_TEST
      - DB_HOST_TEST
      - DB_NAME_TEST

  db:
    image: mysql:8.0.29
    tty: false        # docker run -t
    stdin_open: false
    ports: 
      - '8000:8000'
    environment:
      - MYSQL_ROOT_PASSWORD
      - MYSQL_DATABASE
      - DB_USER_TEST
      - DB_PASSWORD_TEST
      - DB_HOST_TEST
      - DB_NAME_TEST

When running docker-compose up -d --build the app service starts the unittests_entrypoint which waits for the database to boot and then runs the unittests against the dbservice:

# Inside make.sh
unittests_entrypoint () {
    echo "Waiting for database boot process to finish..."
    sleep 30 
    python -m unittest  # Run unittests
}

When running this locally, the tests fail, which is what I expect at this point:

app_1  | .F...
app_1  | ======================================================================
app_1  | FAIL: test_status_get (test.test_app.MyTest)
app_1  | Tests the status code of successful GET on route '/api/v1/book'.
app_1  | ----------------------------------------------------------------------
app_1  | Traceback (most recent call last):
app_1  |   File "/app/test/test_app.py", line 32, in test_status_get
app_1  |     self.assertEqual(response.status_code, 404)
app_1  | AssertionError: 200 != 404
app_1  | 
app_1  | ----------------------------------------------------------------------
app_1  | Ran 5 tests in 2.410s
app_1  | 
app_1  | FAILED (failures=1)

However, when I run this in my azure pipeline, it doesn't fail. The relevant part of my pipeline looks like this:

    - task: Bash@3
      inputs:
        targetType: 'inline'
        script: 'docker-compose up -d --build'
      env:
        DB_HOST_TEST: $(DB_HOST_TEST_ENV)
        DB_USER_TEST: $(DB_USER_TEST_ENV)
        DB_PASSWORD_TEST: $(DB_PASSWORD_TEST_ENV)
        DB_NAME_TEST: $(DB_NAME_TEST_ENV)
        MYSQL_ROOT_PASSWORD: $(MYSQL_ROOT_PASSWORD_ENV)
        MYSQL_DATABASE: $(MYSQL_DATABASE_ENV)

End of the output of my docker-compose task in azure pipelines (it just continues with the next pipeline stage after this, but it should actually stop executing the pipeline because the tests failed):

Removing intermediate container 4cd86b544f4f
 ---> 800f62d5bf2d
Step 5/8 : COPY . .
 ---> 8a951e53d1a3
Step 6/8 : RUN chmod  x src/app.py src/***.py test/test_app.py
 ---> Running in 6fb14e7391b6
Removing intermediate container 6fb14e7391b6
 ---> 9d4ca1466dd2
Step 7/8 : EXPOSE 8080
 ---> Running in b994322b776e
Removing intermediate container b994322b776e
 ---> 840f747945ae
Step 8/8 : CMD ["./src/app.py"]
 ---> Running in 36552f8c4a73
Removing intermediate container 36552f8c4a73
 ---> 281f6467578c
Successfully built 281f6467578c
Successfully tagged s_app:latest
Creating s_***_1 ... 
Creating s_***_1 ... done
Creating s_app_1 ... 
Creating s_app_1 ... done
Finishing: Bash

As one can see, the tests don't fail as they do locally, even though they should. What am I doing wrong?

EDIT:

Running only docker-compose up --build gives a strange output, where 0 tests run and the database complains that "localhost is created with an empty password", which it doesn't when I run it locally. Also the pipeline can't continue because the database container keeps on running. All the environment variables required for authentication are set correctly.

***_1   | 2022-05-10 16:56:44 00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.29-1debian10 started.
***_1   | 2022-05-10 16:56:44 00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
***_1   | 2022-05-10 16:56:44 00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.29-1debian10 started.
***_1   | 2022-05-10T16:56:44.256776Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.29) initializing of server in progress as process 43
***_1   | 2022-05-10 16:56:44 00:00 [Note] [Entrypoint]: Initializing database files
***_1   | 2022-05-10T16:56:44.261370Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
***_1   | 2022-05-10T16:56:44.618226Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
app_1  | Waiting for database boot process to finish...
***_1   | 2022-05-10T16:56:45.346210Z 6 [Warning] [MY-010453] [Server] ***@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
***_1   | 2022-05-10 16:56:47 00:00 [Note] [Entrypoint]: Database files initialized
***_1   | 2022-05-10 16:56:47 00:00 [Note] [Entrypoint]: Starting temporary server
***_1   | 2022-05-10T16:56:47.675567Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29) starting as process 90
***_1   | 2022-05-10T16:56:47.688327Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
***_1   | 2022-05-10T16:56:47.838394Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
***_1   | 2022-05-10T16:56:48.007316Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
***_1   | 2022-05-10T16:56:48.007356Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
***_1   | 2022-05-10T16:56:48.010147Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
***_1   | 2022-05-10T16:56:48.026597Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
***_1   | 2022-05-10T16:56:48.027127Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.29'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
app_1  | 
app_1  | ----------------------------------------------------------------------
app_1  | Ran 0 tests in 0.000s
app_1  | 
app_1  | OK
s_app_1 exited with code 0
##[error]The operation was canceled.
Finishing: Bash

CodePudding user response：

You need to start the database detached and then run your app as one shot command, and make sure it returns with non-zero exit code on failed tests:

steps:
  - script: |
      docker compose run --use-aliases --detach mysql
      docker compose run app
  - script: docker compose down --volumes
    condition: always()

I am running compose down with the always condition to clean up, even if the test failed.

Alternatively, you could use a container resource as a sidecar service. This could look something like this:

resources:
  containers:
    - container: mysql
      image: mysql:8.0.29
      ports:
        - 127.0.0.1:3306:3306
      environment:
        - MYSQL_ROOT_PASSWORD=test
        - MYSQL_DATABASE=test
        - DB_USER_TEST=test
        - DB_PASSWORD_TEST=test
        - DB_HOST_TEST=test
        - DB_NAME_TEST=test

services:
  mysql: mysql

steps:
  - script: docker build -t app .
  - script: |
      docker run --network host \
        -e DB_HOST_TEST=127.0.0.1 \
        -e DB_USER_TEST=test \
        -e DB_PASSWORD_TEST=test \
        -e DB_NAME_TEST=test \
        app ./make.sh unittests_entrypoint

Doing it this way would clean up for you.