catch errors if s3 file is not downloaded-CodePudding

I download a file from S3 like this:

s3 = boto3.client('s3')
s3.download_file('testunzipping','DataPump_10000838.zip','/tmp/DataPump_10000838.zip')

For now it always works. However, I wanted to add some sort of error handling. How can I check or get an error message if the download fails. How would I know that there's something wrong?

Does boto3 offer any error handling functions?

I read this: Check if S3 downloading finish successfully but I am looking for alternatives as well.

CodePudding user response：

You can have something like the below. Download and make sure it was created.

import boto3 
import os


def download_and_verify(Bucket, Key, Filename):
  try:
    os.remove(Filename)
    s3 = boto3.client('s3')
    s3.download_file(Bucket,Key,Filename)
    return os.path.exists(Filename)
  except Exception: # should narrow the scope of the exception
    return False

CodePudding user response：

This is just to improve the answer of @balderman, to actually check in exception what caused your BOTO request to fail.

def download_and_verify(Bucket, Key, Filename):
  try:
    os.remove(Filename)
    s3 = boto3.client('s3')
    s3.download_file(Bucket,Key,Filename)
    return os.path.exists(Filename)
  except botocore.exceptions.ClientError as error:
    print(error.response['Error']['Code']) #a summary of what went wrong
    print(error.response['Error']['Message']) #explanation of what went wrong
    return False