I need to change the flavors "sklearn_version" in mlflow from "0.22.1" to "1.0.0" on azure machine learning when I log my trained model, since this model will be incompatible with the sklearn version that I am using for deployment during inference. I could change the version of sklearn in conda.yml file by setting "conda_env" in

mlflow.sklearn.log_model(conda_env= 'my_env')

here is the screen shot of requirements.txt

however, sklearn version under flavors in MLmodel file remains unchanged and that is the file that causes problem:

and here is script that I use to create this mlflow experiment in azure machine learning notebooks.

import mlflow
from sklearn.tree import DecisionTreeRegressor

from azureml.core import Workspace
from azureml.core.model import Model
from azureml.mlflow import register_model


def run_model(ws, experiment_name, run_name, x_train, y_train):
    
    # set up MLflow to track the metrics
    mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
    mlflow.set_experiment(experiment_name)  
    
    with mlflow.start_run(run_name=run_name) as run:
        
        # fit model
        regression_model = DecisionTreeRegressor()
        regression_model.fit(x_train, y_train)
    
        # log training score 
        training_score = regression_model.score(x_train, y_train)
        mlflow.log_metric("Training score", training_score)

        my_conda_env = {
                    "name": "mlflow-env",
                    "channels": ["conda-forge"],
                    "dependencies": [
                        "python=3.8.5",
                        {
                            "pip": [
                                "pip",
                                "scikit-learn~=1.0.0",
                                "uuid==1.30",
                                "lz4==4.0.0",
                                "psutil==5.9.0",
                                "cloudpickle==1.6.0",
                                "mlflow",
                            ],
                        },
                    ],
                }

        
        # register the model
        mlflow.sklearn.log_model(regression_model, "model", conda_env=my_conda_env)

    model_uri = f"runs:/{run.info.run_id}/model"
    model = mlflow.register_model(model_uri, "sklearn_regression_model")

if __name__ == '__main__':

    # connect to your workspace
    ws = Workspace.from_config()

    # create experiment and start logging to a new run in the experiment
    experiment_name = "exp_name"

    # mlflow run name
    run_name= '1234'

  
    # get train data
    x_train, y_train  = get_train_data()
    
    run_model(ws, experiment_name, run_name, x_train, y_train)

Any idea how can change the flavor sklearn version in MLmodel file from "0.22.1" to "1.0.0" in my script?

With many thanks in advance!

CodePudding user response：

The modifications of the versions must be from "requirements.txt". We need to manually override the versions we need and move the build to the pipeline.

Manually edit the following code

conda_env = {
    'channels': ['conda-forge'],
    'dependencies': [
        'python=3.8.8',
        'pip'],
    'pip': [
        'mlflow',
        'scikit-learn==0.23.2',
        'cloudpickle==1.6.0'
    ],
    'name': 'mlflow-env'
}
mlflow.sklearn.log_model(model, "my_model", conda_env=conda_env)

The below code should be in conda.yml

name: mlflow-env
channels:
  - conda-forge
dependencies:
- python=3.8.8
- pip
- pip:
  - mlflow
  - scikit-learn==0.23.2
  - cloudpickle==1.6.0

The following code block must be there in python_env.yaml

python: 3.8.8
build_dependencies:
  - pip==21.1.3
  - setuptools==57.4.0
  - wheel==0.37.0
dependencies:
  - -r requirements.txt

The following thing must be there in requirements.txt

mlflow
scikit-learn==0.23.2
cloudpickle==1.6.0

CodePudding user response：

I was finally able to solve this issue. Apparently flavors within mlflow MLfile use the version of installed scikit-learn within the workspace. all I needed to to was to upgrade the scikit-learn from cli within workspace.

1. open cli in azure machine learning notebooks
2. upgarde your intended flavor package. My case was scikit-learn, in my case I used 1.0.2:

 pip install --upgrade scikit-learn==1.0.2

3. log your model in mlflow by running your training script one more time.