How to not exit a for loop inside a pytest although few items fail-CodePudding

I would like to run the pytest for all the items in the for loop. The pytest should fail in the end but it should run all the elements in the for loop.

The code looks like this

@pytest.fixture
def library():
    return Library( spec_dir = service_spec_dir)
 
@pytest.fixture
def services(library):
    return list(library.service_map.keys())

def test_properties(service, services):
    for service_name in services:
        model = library.models[service_name]
        proxy = library.get_service(service_name)
        if len(model.properties ) != 0 :
            for prop in model.properties:
                try:
                    method = getattr(proxy, f'get_{prop.name}')
                    method()
                except exception as ex:
                    pytest.fail(ex)

The above code fails if one property of one service fails. I am wondering if there is a way to to run the test for all the service and get a list of failed cases for all the services.

I tried parametrize But based on this stackoverflow discussion. The parameter list should be resolved during the collection phase and in our case the library is loaded during the execution phase. Hence I am also not sure if it can be parametrized.

The goal is run all the services and its properties and get the list of failed items in the end.

CodePudding user response：

I moved the variables to the global scope. I can parametrize the test now\

library = Library( spec_dir = service_spec_dir)
service_names = list(library.service_map.keys())

@pytest .mark.paramertize("serivce_name", service_names)
def test_properties(service):
    pass

CodePudding user response：

Don't use pytest.fail, but pytest_check.check instead. fail point, is that you stop test execution on condition, while check made for collect how much cases were failed.

import logging
import pytest
import pytest_check as check

def test_000():
    li = [1, 2, 3, 4, 5, 6]
    for i in li:
        logging.info(f"Test still running. i = {i}")
        if i % 2 > 0:
            check.is_true(False, msg=f"value of i is odd: {i}")

Output:

tests/main_test.py::test_000 
-------------------------------- live log call --------------------------------
11:00:05 INFO Test still running. i = 1
11:00:05 INFO Test still running. i = 2
11:00:05 INFO Test still running. i = 3
11:00:05 INFO Test still running. i = 4
11:00:05 INFO Test still running. i = 5
11:00:05 INFO Test still running. i = 6
FAILED                                                                   [100%]

================================== FAILURES ===================================
__________________________________ test_000 ___________________________________
FAILURE: value of i is odd: 1
assert False
FAILURE: value of i is odd: 3
assert False
FAILURE: value of i is odd: 5
assert False