I am using pytest to run some unit tests. Some of my tests involve initialising an object with invalid arguments, and testing that the exception that is raised contains the expected error message.
To do this, I am using raises, however pytest is failing the test as opposed to capturing the exception. The test output shows that the expected exception was raised.
>>> def test_400_invalid_org_id(self): ... ... # Setting host, org_id and body here. ... ... with pytest.raises(InvalidOrgIdException) as e_info: ... OrgRequest(host, org_id, body) ... ... assert str(e_info.value) == 'Invalid organisation ID.' E InvalidOrgIdException: Invalid organisation ID. Obviously InvalidOrgIdException is a custom type (properly imported), a subclass of Exception. And for some reason pytest.raises(Exception) works as expected. Looking at the documentation, it suggests that I should be able to assert the type of the exception that has been caught, but this too fails.
>>> def test_400_invalid_org_id(self): ... ... # Setting host, org_id and body here. ... ... with pytest.raises(Exception) as e_info: ... OrgRequest(host, org_id, body) ... ... assert str(e_info.value) == 'Invalid organisation ID.' ... assert e_info.type is InvalidOrgIdException E AssertionError: assert <class 'InvalidOrgIdException'> is InvalidOrgIdException E + where <class 'InvalidOrgIdException'> = <ExceptionInfo InvalidOrgIdException('xxx', '^[a-z0-9]{16}$') tblen=3>.type When comparing e_info.type and InvalidOrgIdException, there is a difference between the __init__ and __str__ methods. Note that imported objects are both from the same module - I am not mocking either.
>>> pprint(vars(e_info.type)) mappingproxy({'__doc__': 'Organisation ID in path is invalid.', '__init__': <function InvalidOrgIdException.__init__ at 0x7f58b867e8b0>, '__module__': 'builtins', '__str__': <function InvalidOrgIdException.__str__ at 0x7f58b867e9d0>, '__weakref__': <attribute '__weakref__' of 'InvalidOrgIdException' objects>}) ... >>> pprint(vars(InvalidOrgIdException)) mappingproxy({'__doc__': 'Organisation ID in path is invalid.', '__init__': <function InvalidOrgIdException.__init__ at 0x7f58b867e670>, '__module__': 'builtins', '__str__': <function InvalidOrgIdException.__str__ at 0x7f58b867e700>, '__weakref__': <attribute '__weakref__' of 'InvalidOrgIdException' objects>}) So why is pytest behaving in this way, and is it possible to change the behaviour?
Class that raises InvalidOrgIdException
class OrgRequest(): def __init__(self) -> None: raise InvalidOrgIdException() from None Full output from pytest run
Tests are being run from the myproject directory (see file structure at the bottom of the question).
python -m pytest tests/unit/test_requests.py --verbose The tests were run with the --verbose argument. I'm not a pytest expert, but I don't think there is more detailed output available.
========================================================================================= test session starts ========================================================================================== platform linux -- Python 3.8.10, pytest-7.0.1, pluggy-1.0.0 -- /usr/bin/python cachedir: .pytest_cache rootdir: /home/me/myproject/tests, configfile: pytest.ini plugins: env-0.6.2, mock-3.7.0 collected 1 item tests/unit/test_requests.py::TestRequests::test__400_org_id_invalid FAILED =============================================================================================== FAILURES =============================================================================================== _________________________________________________________________________________ TestRequests.test_400_org_id_invalid _________________________________________________________________________________ self = <tests.unit.test_requests.TestRequests object at 0x7f7627e07e20> def test_400_org_id_invalid(self): with pytest.raises(InvalidOrgIdException) as e_info: > OrgRequest() tests/unit/test_requests.py:9: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <myfunction.core.request.OrgRequest object at 0x7f7627e20130> def __init__(self) -> None: > raise InvalidOrgIdException() from None E InvalidOrgIdException: Invalid organisation ID. myfunction/core/request.py:19: InvalidOrgIdException ======================================================================================= short test summary info ======================================================================================== FAILED tests/unit/test_requests.py::TestRequests::test_400_org_id_invalid - InvalidOrgIdException: Invalid organisation ID. ========================================================================================== 1 failed in 0.06s =========================================================================================== File Structure
myproject ├── myfunction | ├── app.py | └── core | ├── exception.py | └── request.py └── tests └── unit └── test_requests.py Hacky "Fix"
If I move InvalidOrgIdException into the same module as OrgRequest then import the exception from there, the test passes. Obviously though I'd prefer not to do this, as it makes sense for all of the exceptions to live together. And although the "fix" exists, I'd still like to usnderstand what's happening and exactly why it works.
from app import InvalidOrgIdException 
pytestrun.InvalidOrgIdExceptionis somehow imported from elsewhere and is actually not of the same class as the raised exception? (Unlikely, but nothing else comes to mind at the moment)