Loading pandas data frame from pickle file in S3 bucket to AWS Lambda

I created a machine-learning model with a KNN classifier. Then, I made a pickle file of the test dataset and uploaded it to the AWS S3 bucket using AWS SDK.

For testing purposes, I have downloaded it and tested the type with the following:

with open("C:\\...path...\\test_features.pkl", 'rb') as f:
    test_data= pickle.load(f)
print(type(test_data))

The result is <class 'pandas.core.frame.DataFrame'>, which is ok.

However, when reading through AWS Lambda, the following part

s3 = boto3.client('s3')
test_features = s3.get_object(Bucket=bucket, Key= key)
print(type(test_features))

gives <class 'dict'>

How to get DataFrame type in AWS Lambda too?

CodePudding user response：

You will need to read content first then use pickle to load the content and create data frame

test_features = s3.get_object(Bucket=bucket, Key= key)
body = test_features['Body'].read()
test_data = pickle.loads(body)
print(type(test_data))