How to upload a file from Vertex AI Workbench to AWS S3?-CodePudding

I have an access in Google Cloud and AWS. I wanted to upload a file from Vertex AI Workbench to AWS S3, is that possible? Or there is an alternative way?

I have read some tread that might help me, and have try some code, but still can't solve my problem, and raise an error

Could not connect to the endpoint URL: "https://xyz.s3.auto.amazonaws.com/uploaded.csv?uploads"

Here is my code

import boto3
import os
import io

s3 = boto3.resource('s3')

key_id="my_key"
access_key="my_access_key"

client = boto3.client("s3", region_name="auto", aws_access_key_id=key_id, aws_secret_access_key=access_key)

client.upload_file(
    Filename="path_file.csv",
    Bucket="bucket_name",
    Key="uploaded.csv",
)

CodePudding user response：

I think the issue here is you're using region=auto for AWS which is not supported. The region needs to be real region because (you can see in the error) it's being used to pick the endpoint. Try it without that.

import os
import io

s3 = boto3.resource('s3')

key_id="my_key"
access_key="my_access_key"

client = boto3.client("s3", aws_access_key_id=key_id, aws_secret_access_key=access_key)

client.upload_file(
    Filename="path_file.csv",
    Bucket="bucket_name",
    Key="uploaded.csv",
)