I am currently using python 3.8.8 with version 12.9.0 of azure.storage.blob and 1.14.0 of azure.core.

I am downloading multiple files using the azure.storage.blob package. My code looks something like the following

from azure.storage.blob import ContainerClient
from azure.core.exceptions import ResourceNotFoundError, AzureError
from time import sleep

max_attempts = 5
container_client = ContainerClient(DETAILS)

for file in multiple_files:

  attempts = 0

  while attempts < max_attempts:
  
    try:
    
      data = container.download_blob(file).readall()
      break
      
    except ResourceNotFoundError:
      # log missing data
      break
      
    except AzureError:
      # This is mainly here as connections seem to drop randomly.
      attempts  = 1
      sleep(1)
      
  if attempts >= max_attempts:
    #log connection error
      
  #do something with the data.

It seems to be running fine, and I don't see any loss of data. However, within my terminal I keep getting the message

Unable to stream download: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

This appears to be a TCP 104 return message but isn't being handled by the azure module. My questions are as follows.

1. Where is this message coming from? I can't see it in any of the packages I am using.
2. How do I handle this error better? It doesn't appear to be caught as an exception as it isn't crashing my code.
3. Can I get this to print to a log?

CodePudding user response：

1. Where is this message coming from? I can't see it in any of the packages I am using.

Looks like The clients seemed to be connected to the server, but when they attempted to transfer data, they received a Errno 104 Connection reset by peer error. This also means, that the other side has reset the connection else the client would encounter with [Errno 32] Broken pipe exception.

2. How do I handle this error better? It doesn't appear to be caught as an exception as it isn't crashing my code.

One of the workarounds you can try is to have try and catch block to handle that exception:

from socket import error as SocketError
import errno

try:
    response = urllib2.urlopen(request).read()
except SocketError as e:
    if e.errno != errno.ECONNRESET:
        raise # Not error we are looking for
    pass # Handle error here.

Also try referring to this similar issue where sudo pip3 install urllib3 solved the issue.

3. Can I get this to print to a log?

One workaround is that you can pass exception instance in exc_info argument:

import logging
try:
    1/0
except Exception as e:
   logging.error('Error at %s', 'division', exc_info=e)

For more information you can refer How to log python exception?

Here is a related issue that you can follow up azure storage blob download: ConnectionResetError(104, 'Connection reset by peer')

REFERENCE: Connection broken: ConnectionResetError(104, 'Connection reset by peer') error while streaming