How to get more details on a requests.exceptions.SSLError?-CodePudding

When I request a URL with an expired HTTPS certificate I do not get a meaningful error from requests. Instead it gives me a cascade of "ssl.SSLError: A failure in the SSL library occurred".

See this example with https://expired.badssl.com/ :

>python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get("https://expired.badssl.com/")
Traceback (most recent call last):
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\connection.py", line 362, in connect
    self.sock = ssl_wrap_socket(
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\util\ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\me\apps\Python39\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
  File "C:\Users\me\apps\Python39\lib\ssl.py", line 1040, in _create
    self.do_handshake()
  File "C:\Users\me\apps\Python39\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: A failure in the SSL library occurred (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\me\apps\Python39\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "C:\Users\me\apps\Python39\lib\site-packages\urllib3\util\retry.py", line 446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='expired.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, 'A failure in the SSL library occurred (_ssl.c:1129)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\me\apps\Python39\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\me\apps\Python39\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\me\apps\Python39\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\me\apps\Python39\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\me\apps\Python39\lib\site-packages\requests\adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='expired.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, 'A failure in the SSL library occurred (_ssl.c:1129)')))
>>>

requests is 2.24.0 and ssl.OPENSSL_VERSION says "OpenSSL 1.1.1h 22 Sep 2020". I can not update the packages.

How can I get a meaningful error or error message that tells me that the certificate is expired?

CodePudding user response：

This is normally how you would handle such a thing:

import requests
try:
    requests.get("https://expired.badssl.com/")
except requests.exceptions.SSLError as e:
    print(f'oops! got the following SSL error: {e}')

And on my test machine the output contains [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired

Since "certificate has expired" is not in your exception text, I suspect you have a bug in your version of something. requests 2.24.0 is not the latest version. Also, requests uses both urllib3 and certifi. Perhaps you should try:

pip install --upgrade urllib3 requests certifi

CodePudding user response：

Requirements:

Environment: Python3.9.5 / requests 2.24.0 / OpenSSL 1.1.1h
if a contact to a server fails, it should be determined whether the problem is an expired server certificate

Simplest solution would be to update the installed OpenSSL to, say, the current version 1.1.1n, but even 1.1.1i would be sufficient to get a message which contains certificate has expired. But updating the OpenSSL library cannot be done, as you mention in your post.

Alternatively, you can catch exceptions during a request and then explicitly check if the server certificate has expired.

To do this, you could, for example, take the get_cert_for_hostname function from a previous stackoverflow answer of mine and compare the not_valid_after field with the current date accordingly.

A simple, self-contained example might then look something like this:

import ssl
import requests
import platform
from datetime import datetime
from cryptography import x509


def get_cert_for_hostname(hostname, port):
    conn = ssl.create_connection((hostname, port))
    context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
    sock = context.wrap_socket(conn, server_hostname=hostname)
    certDER = sock.getpeercert(True)
    certPEM = ssl.DER_cert_to_PEM_cert(certDER)
    conn.close()
    return x509.load_pem_x509_certificate(certPEM.encode('ascii'))


def is_cert_expired(hostname, port):
    cert = get_cert_for_hostname(hostname, port)
    return datetime.now() > cert.not_valid_after


if __name__ == '__main__':
    print(f"Python version: {platform.python_version()}")
    print(f"OpenSSL version: {ssl.OPENSSL_VERSION}")
    print(f"requests version: {requests.__version__}")  #
    hosts = ['software7.com', 'expired.badssl.com']
    for host in hosts:
        try:
            requests.get(f"https://{host}")
            print(f"request for host {host} was successful")
        except BaseException as err:
            if is_cert_expired(host, 443):
                print(f"certificate for {host} expired")
            else:
                print(f"error {err} with {host}")

This would then log the following to the debug console:

Python version: 3.9.5
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020
requests version: 2.24.0
request for host software7.com was successful
certificate for expired.badssl.com expired