How to trap this nested httpx exception?-CodePudding

I'm trapping thus:

with httpx.Client(**sessions[scraperIndex]) as client:
    try:
        response = client.get(...)
    except TimeoutError as e:
        print('does not hit')
    except Exception as e:
        print(f'⛔️ Unexpected exception: {e}')
        print_exc()  # hits!

However I'm getting the below crashdump.

Pulling out key lines:

TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:
    httpcore.ReadTimeout: The read operation timed out

The above exception was the direct cause of the following exception:
    httpx.ReadTimeout: The read operation timed out

Why isn't my TimeoutError catching this?

And what's the correct catch? Can someone give a logic for deducing it?

CrashDump:

⛔️ Unexpected exception: The read operation timed out
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_exceptions.py", line 8, in map_exceptions
    yield
  File "/usr/local/lib/python3.10/dist-packages/httpcore/backends/sync.py", line 26, in read
    return self._sock.recv(max_bytes)
  File "/usr/lib/python3.10/ssl.py", line 1258, in recv
    return self.read(buflen)
  File "/usr/lib/python3.10/ssl.py", line 1131, in read
    return self._sslobj.read(len)
TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 218, in handle_request
    resp = self._pool.handle_request(req)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection_pool.py", line 253, in handle_request
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection_pool.py", line 237, in handle_request
    response = connection.handle_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/connection.py", line 90, in handle_request
    return self._connection.handle_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/http11.py", line 105, in handle_request
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/http11.py", line 84, in handle_request
    ) = self._receive_response_headers(**kwargs)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/http11.py", line 148, in _receive_response_headers
    event = self._receive_event(timeout=timeout)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_sync/http11.py", line 177, in _receive_event
    data = self._network_stream.read(
  File "/usr/local/lib/python3.10/dist-packages/httpcore/backends/sync.py", line 24, in read
    with map_exceptions(exc_map):
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/httpcore/_exceptions.py", line 12, in map_exceptions
    raise to_exc(exc)
httpcore.ReadTimeout: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/scraper-pi/Scrape.py", line 148, in main
    cursor, _nScraped = scrape(client, cursor)
  File "/root/scraper-pi/Scrape.py", line 79, in scrape
    response = client.get(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1039, in get
    return self.request(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 815, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 902, in send
    response = self._send_handling_auth(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 930, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 967, in _send_handling_redirects
    response = self._send_single_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1003, in _send_single_request
    response = transport.handle_request(request)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 217, in handle_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout: The read operation timed out

CodePudding user response：

The base class for all httpx timeout errors is not the built-in TimeoutError (presumably because that would also make timeouts OSErrors, which doesn't sound correct), but httpx.TimeoutException.

import httpx

with httpx.Client() as client:
    try:
        response = client.get("http://httpbin.org/get", timeout=0.001)
    except httpx.TimeoutException as e:
        print('gottem')

prints gottem just fine.