Home > other >  How can I use Sagemaker built-in algorithms locally without estimators?
How can I use Sagemaker built-in algorithms locally without estimators?

Time:11-11

I am interested in using the AWS Sagemaker built-in algorithms (or pre-trained models) without calling an Estimator class, just like I would do with tensorflow or scikit-learn locally.

I have found this but it uses an Estimator class to do the training: https://sagemaker.readthedocs.io/en/stable/overview.html?highlight=local mode#local-mode

Is there a way or do I need to call an Estimator class every time?

CodePudding user response:

Pretrained models

If you want to use a pretrained model, you don't have to go through the estimator because you already have the artifact of the model.

You can see a guide here: Host a Pretrained Model on SageMaker

In a nutshell, the turn it takes is to load the pre-trained model and package it in the usual model.tar.gz archive (which would be created following normal training with the estimator). If a custom inference script is required, there is the step of repacking the model. At this point, it loads the uri of the model artifact and deploys the endpoint.

Extra: Using JumpStart (on SageMaker Studio)

A more interactive way is to use SageMaker JumpStart on SageMaker Studio:Use Amazon SageMaker Built-in Algorithms or Pre-trained Models


Built-in algorithms (not pretrained)

If the model is not pre-trained (so, will be trained), the Estimator (a high level interface for SageMaker training) must be used by definition.

See Launch a Training Job Using the SageMaker Python SDK:

The SageMaker Python SDK supports managed training of models with ML frameworks such as TensorFlow and PyTorch. To launch a training job using one of these frameworks, you define a SageMaker TensorFlow estimator, a SageMaker PyTorch estimator, or a SageMaker generic Estimator to use the modified training script and model parallelism configuration

  • Related