Capture a custom exception with 'pytest.raises()' and assert the error message-CodePudding

I am using pytest to run some unit tests. Some of my tests involve initialising an object with invalid arguments, and testing that the exception that is raised contains the expected error message.

To do this, I am using raises, however pytest is failing the test as opposed to capturing the exception. The test output shows that the expected exception was raised.

>>> def test_400_invalid_org_id(self):
... 
...     # Setting host, org_id and body here.
... 
...     with pytest.raises(InvalidOrgIdException) as e_info:
...         OrgRequest(host, org_id, body)
... 
...     assert str(e_info.value) == 'Invalid organisation ID.'
E           InvalidOrgIdException: Invalid organisation ID.

Obviously InvalidOrgIdException is a custom type (properly imported), a subclass of Exception. And for some reason pytest.raises(Exception) works as expected. Looking at the documentation, it suggests that I should be able to assert the type of the exception that has been caught, but this too fails.

>>> def test_400_invalid_org_id(self):
... 
...     # Setting host, org_id and body here.
... 
...     with pytest.raises(Exception) as e_info:
...         OrgRequest(host, org_id, body)
...
...     assert str(e_info.value) == 'Invalid organisation ID.'
...     assert e_info.type is InvalidOrgIdException
E       AssertionError: assert <class 'InvalidOrgIdException'> is InvalidOrgIdException
E           where <class 'InvalidOrgIdException'> = <ExceptionInfo InvalidOrgIdException('xxx', '^[a-z0-9]{16}$') tblen=3>.type

When comparing e_info.type and InvalidOrgIdException, there is a difference between the __init__ and __str__ methods. Note that imported objects are both from the same module - I am not mocking either.

>>> pprint(vars(e_info.type))
mappingproxy({'__doc__': 'Organisation ID in path is invalid.',
              '__init__': <function InvalidOrgIdException.__init__ at 0x7f58b867e8b0>,
              '__module__': 'builtins',
              '__str__': <function InvalidOrgIdException.__str__ at 0x7f58b867e9d0>,
              '__weakref__': <attribute '__weakref__' of 'InvalidOrgIdException' objects>})
...
>>> pprint(vars(InvalidOrgIdException))
mappingproxy({'__doc__': 'Organisation ID in path is invalid.',
              '__init__': <function InvalidOrgIdException.__init__ at 0x7f58b867e670>,
              '__module__': 'builtins',
              '__str__': <function InvalidOrgIdException.__str__ at 0x7f58b867e700>,
              '__weakref__': <attribute '__weakref__' of 'InvalidOrgIdException' objects>})

So why is pytest behaving in this way, and is it possible to change the behaviour?

Class that raises `InvalidOrgIdException`

class OrgRequest():
    def __init__(self) -> None:
        raise InvalidOrgIdException() from None

Full output from `pytest` run

Tests are being run from the myproject directory (see file structure at the bottom of the question).

python -m pytest tests/unit/test_requests.py --verbose

The tests were run with the --verbose argument. I'm not a pytest expert, but I don't think there is more detailed output available.

========================================================================================= test session starts ==========================================================================================
platform linux -- Python 3.8.10, pytest-7.0.1, pluggy-1.0.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/me/myproject/tests, configfile: pytest.ini
plugins: env-0.6.2, mock-3.7.0
collected 1 item                                                                                                                                                                                       

tests/unit/test_requests.py::TestRequests::test__400_org_id_invalid FAILED

=============================================================================================== FAILURES ===============================================================================================
_________________________________________________________________________________ TestRequests.test_400_org_id_invalid _________________________________________________________________________________

self = <tests.unit.test_requests.TestRequests object at 0x7f7627e07e20>

    def test_400_org_id_invalid(self):
        with pytest.raises(InvalidOrgIdException) as e_info:
>           OrgRequest()

tests/unit/test_requests.py:9: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <myfunction.core.request.OrgRequest object at 0x7f7627e20130>

    def __init__(self) -> None:
>       raise InvalidOrgIdException() from None
E       InvalidOrgIdException: Invalid organisation ID.

myfunction/core/request.py:19: InvalidOrgIdException
======================================================================================= short test summary info ========================================================================================
FAILED tests/unit/test_requests.py::TestRequests::test_400_org_id_invalid - InvalidOrgIdException: Invalid organisation ID.
========================================================================================== 1 failed in 0.06s ===========================================================================================

File Structure

myproject
├── myfunction
|   ├── app.py
|   └── core
|       ├── exception.py
|       └── request.py
└── tests
    └── unit
        └── test_requests.py

Hacky "Fix"

If I move InvalidOrgIdException into the same module as OrgRequest then import the exception from there, the test passes. Obviously though I'd prefer not to do this, as it makes sense for all of the exceptions to live together. And although the "fix" exists, I'd still like to usnderstand what's happening and exactly why it works.

from app import InvalidOrgIdException

CodePudding user response：

As mentioned in my question, a comparison of the exception captured by pytest.raises() seemed to differ from that which was imported from my code. I must confess, I'm not a Python expect and I don't understand why exactly this worked, but the solution was to add init.py to the core module, and in there import all of the modules in that package.

from core.exception import *
from core.logging import *
# etc.

And then in tests/unit/test_requests.py I was able to import all objects from the core package.

from myfunction.core import *

Now pytest sees the exception it captured and the imported exception as the same and my tests are passing.

Class that raises InvalidOrgIdException

Full output from pytest run

File Structure

Hacky "Fix"

Class that raises `InvalidOrgIdException`

Full output from `pytest` run