I have written a simple neural network (MLP Regressor), to fit simple data frame columns. To have an optimum architecture, I also defined it as a function to see whether it is converging to a pattern. But every time that I run the model, it gives me a different result than the last time that I tried, and I do not know why? Due to the fact that it is fairly difficult to make the question reproducible, I can not post the data but I can post the architecture of the network here:

def MLP(): #After 50
    nn=30
    nl=25
    a=2
    s=0
    learn=2
    learn_in=4.22220046e-05
    max_i=1000
    return nn,nl,a,s,learn,learn_in,max_i#,



def process(df):
    y = df.iloc[:,-1]
    X = df.drop(columns=['col3'])
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=27)
    return X_train, X_test, y_train, y_test

def MLPreg(x_train, y_train):#
    nn,nl,a,s,learn,learn_in,max_i=MLP()#nl,
    act=['identity', 'logistic', 'relu','tanh'] #'identity'=Linear
    activ=act[a]
    sol=['lbfgs', 'sgd', 'adam']
    solv=sol[s]
    l_r=['constant','invscaling','adaptive']
    lr=l_r[learn]
    model = MLPRegressor(hidden_layer_sizes=(nl,nn), activation=activ, solver=solv, alpha=0.00001, batch_size='auto', 
    learning_rate=lr, learning_rate_init=learn_in, power_t=0.5, max_iter=max_i, shuffle=True, random_state=None,
    tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True, early_stopping=False, 
    validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08, n_iter_no_change=10, max_fun=15000)
    model.fit(x_train, y_train)
    return model

Even I have tried to keep all variables that make the model produce randomness Off but I receive different mse in every run. The new model which is below is the simplest version.

def MLPreg(x_train, y_train):
    model = MLPRegressor(hidden_layer_sizes=(100,),
    activation='relu',
    solver='adam',
    alpha=0.0001,
    batch_size='auto',
    learning_rate='constant',
    learning_rate_init=0.001,
    power_t=0.4,
    max_iter=100,)
    model.fit(x_train, y_train)
    
    return model

The first time mse:

2.6935335013259937e-05
2.7836293087120013e-05
7.218691932722961e-05
4.950603795598673e-05
4.743424330664441e-06

The second time mse:

3.6520542498579784e-06
1.151821946860996e-05
3.0840569586230768e-06
1.4008729128558944e-05
9.326142225670172e-06

And so on.

CodePudding user response：

Looking at the documentation of MLPRegressor, these two arguments are important for reproducible results:

shuffle:bool, default=True

Whether to shuffle samples in each iteration. Only used when solver=’sgd’ or ‘adam’.

Set shuffle=False to have the same behavior between runs.

random_state: int, RandomState instance, default=None

Determines random number generation for weights and bias initialization, train-test split if early stopping is used, and batch sampling when solver=’sgd’ or ‘adam’. Pass an int for reproducible results across multiple function calls.

Follow these instructions and set an int value for this argument. Neither of your code examples currently does that (one sets it to None, which is the non-reproducible default, the other omits it).