how to use Amazon S3 Uri links to download image?-CodePudding

The scenario is, I got s3:// type links to work with and i am supposed to write a script to download images from that links, I don't have much idea how to do it, tried reading some posts, docs did not help much, ended up with script which was throwing some exception which I did not understand, I used boto3 here.

So basically I got these kind links s3://some-name-with-hyphens/other/and_one_more/some.jpg I need write python script to download that object.

These images are hosted on public AWS S3 bucket.

Here is Script I used, I am showing here fake s3 uri:

import boto3
def find_bucket_key(s3_path):
    """
    This is a helper function that given an s3 path such that the path is of
    the form: bucket/key
    It will return the bucket and the key represented by the s3 path
    """
    s3_components = s3_path.split('/')
    bucket = s3_components[0]
    s3_key = ""
    if len(s3_components) > 1:
        s3_key = '/'.join(s3_components[1:])
    return bucket, s3_key


def split_s3_bucket_key(s3_path):
    """Split s3 path into bucket and key prefix.
    This will also handle the s3:// prefix.
    :return: Tuple of ('bucketname', 'keyname')
    """
    if s3_path.startswith('s3://'):
        s3_path = s3_path[5:]
    return find_bucket_key(s3_path)


client = boto3.client('s3')
bucket_name, key_name = split_s3_bucket_key(
    's3://some-name-with-hyphens/other/and_one_more/some.jpg')
response = client.get_object(Bucket=bucket_name, Key=key_name)

And the exception I got:

File "C:\Users\BASAVARAJ\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\botocore\auth.py", line 373, in add_auth
raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials

CodePudding user response：

If you don't have AWS credentials to perform the request, then you need to perform an unsigned request. Replace the client creation with this to create a client that won't sign any requests:

import boto3
from botocore import UNSIGNED
from botocore.config import Config
client = boto3.client('s3', config=Config(signature_version=UNSIGNED))