When we send data from an encrypted AWS S3 bucket to an encrypted Google Cloud Storage bucket, is th-CodePudding

I am copying files from my AWS S3 bucket to GCS bucket using gsutil and boto3 in google Cloud function. See code below. In my cloud function I pass my AWS ACCESS_KEY and SECRET_KEY.

The data is encrypted in the AWS S3 bucket using the default Amazon S3 master-key (SSE-S3) and files in GCS bucket are also encrypted using the default 'Google-managed key' option.

So, I am not worried about the data encryption in storage since it is already taken care of , but I am not sure if the data is encrypted in transit. Does anyone know, that in this particular scenario if the data is already encrypted in transit? OR do I need to encrypt the files before writing to S3 using a thrid party tool such as "cryptography" to manually encrypt my data and then decrypt the files using this custom generated key before writing to GCS bucket?

import boto3
from boto3.session import Session 
from botocore.client import Config 
from botocore.handlers import set_list_objects_encoding_type_url 
import os
from io import BytesIO
from google.cloud import storage

'''
    copy all new files from AWS S3 to GCS
    Also delete files after copying
'''
def main(request):

    ACCESS_KEY = os.environ['ACCESS_KEY']
    SECRET_KEY = os.environ['SECRET_KEY']
    S3_BUCKET_NAME = os.environ['S3_BUCKET']
    GCS_BUCKET_NAME = os.environ['GCS_BUCKET']
    session = Session(aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY)

.....
.....
for s3_file in original_s3_file_list:
        file_name = s3_file.key
        s3_obj = s3.Object(S3_BUCKET_NAME,file_name)

        #copy S3 file to GCS
        data=s3_obj.get()['Body'].read()
        gcs_blob = gcs_bucket.blob(file_name)
        gcs_blob.upload_from_string(data)

CodePudding user response：

Your data is transferred encrypted because the transports use HTTPS (AWS <-> client <-> GCP).

The boto3 client's default transport is https. This can be configured with the use_ssl parameters link. The default is True.

The default transport for Google Cloud Storage is also https. To change this to use http requires modifying the API_BASE_URL link.