Unable to access data from S3 bucket to jupyter notebook of aws sagemaker-CodePudding

I need to train a model on aws sagemaker. I'm unable to access data in Jupiter notebook of sagemaker from S3 bucket. My bucket name is "riceleaf" there are four folders in the bucket named as s1,s2,s3,s4 and each folder contains 330 images named as 1.jpg and so on. It is created in Us-east zone. Bucket is private.

One way i did was to access the object and when i displayed the key it shows me 1.jpg and so on. But when i try to open that image it didn't work. So i think I'm unable to get exact data path.

In my code I need exact data path since I'm doing some random data generation in the code so need to access different folders. Therefore, I need a path till bucket so i can change next folder name and image name randomly in my code.

Please help me to so that I can access the images in the Jupiter notebook of sagemaker.

CodePudding user response：

If you can list the keys but not open a file (or download), make sure your notebook's execution role has s3:GetObject permissions on your riceleaf bucket. The default execution role will only have permissions to access a bucket that has sagemaker in its name.

Once your permissions are set, you can use the S3 Paginator from boto3 to list all your objects for your training.

CodePudding user response：

what is the error message you are getting?

using boto3 sdk with s3 you can do somehting like this:

import boto3
s3 = boto3.resource('s3')
for key in bucket.objects.all():
  print 's3://{}/{}.format(bucket,key.key)