Error when deploying pre trained Tensorflow models to one endpoint (multimodel for one endpoint) in-CodePudding

I am following this example from aws https://github.com/aws-samples/sagemaker-multi-model-endpoint-tensorflow-computer-vision/blob/main/multi-model-endpoint-tensorflow-cv.ipynb to apply same workflow with two pre trained models (trained outside of sagemaker).

But when I do the following, logs say that models can't be found:

import boto3
import datetime
from datetime import datetime
import time
import sagemaker
from sagemaker import get_execution_role
from sagemaker.tensorflow.serving import TensorFlowModel
from sagemaker.multidatamodel import MultiDataModel

model_data_prefix = f's3://{BUCKET}/{PREFIX}/mme/'
output = f's3://{BUCKET}/{PREFIX}/mme/test.tar.gz'

modele = TensorFlowModel(model_data=output, 
                          role=role, 
                          image_uri=IMAGE_URI)

mme = MultiDataModel(name=f'mme-tensorflow-{current_time}',
                     model_data_prefix=model_data_prefix,
                     model=modele,
                     sagemaker_session=sagemaker_session)

predictor = mme.deploy(initial_instance_count=1,
                       instance_type='ml.m5.2xlarge',
                       endpoint_name=f'mme-tensorflow-{current_time}')

When I give an image as input to predict, I have this message:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from model with message "<html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>
".

Logs give:

Could not find base path /opt/ml/models/.../model for servable ...

What did I missed ?

CodePudding user response：

In the sample notebook, the model is trained within SageMaker. So it is created with certain environment variables like the "SAGEMAKER_PROGRAM"(I think, need to check the documentation) with value set to entry point script.

But while you are creating the model with models trained outside the SageMaker you need to add those environment variables.

Without an entry point script SageMaker is not in a position to know what to do with the request.