I'm using GridsearchCV for hyperparameter tuning. Im having problems fitting a variation of "hidden_layer_sizes", as they have to be tuples.
Is there a way to check multiple hidden layer sizes, without manually typing them in?
My code looks like this:
param_list = {"hidden_layer_sizes": [(200,70), (150,100,50), (120,80,40), (100,50,30)], "activation": ["identity", "logistic", "tanh", "relu"], "solver": ["lbfgs", "sgd", "adam"], "alpha": [0.0001,0.0005]}
MLP_gridCV = GridSearchCV(
estimator=MLPRegressor(max_iter=10000, n_iter_no_change=30),
param_grid=param_list,
n_jobs=-1,
cv=3,
verbose=5,
)
MLP_gridCV.fit(X_train, y_train.ravel())
# Prediction
y_pred8 = MLP_gridCV.predict(X_test)
For my SVR I used:
gamma = np.arange(0.001, 0.1, 0.001).tolist()
Is there any possibility to use the same logic for the MLPRegressor so that I dont have to type in (150,100,50) for example?
I tried something like this, but it doesnt work:
learning_rate_init= [0.0001,0.0002]
first_layer_neurons= np.arange(10, 200, 10).tolist()
second_layer_neurons= np.arange(10, 200, 10).tolist()
hidden_layer_sizes = [first_layer_neurons,second_layer_neurons]
activation= ['identity', 'tanh', 'relu']
params_grid = {
'hidden_layer_sizes':hidden_layer_sizes,
'learning_rate_init':learning_rate_init,
'activation':activation}
print(params_grid)
{'hidden_layer_sizes': [[10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190],
[10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190]],
'learning_rate_init': [0.0001, 0.0002],
'activation': ['identity', 'tanh', 'relu']}
CodePudding user response:
You can use product
from itertools
:
from itertools import product
first_layer_neurons = np.arange(10, 200, 10)
second_layer_neurons = np.arange(10, 200, 10)
hidden_layer_sizes = list(product(first_layer_neurons, second_layer_neurons))
Output:
>>> hidden_layer_sizes
[(10, 10),
(10, 20),
(10, 30),
(10, 40),
(10, 50),
...
(190, 150),
(190, 160),
(190, 170),
(190, 180),
(190, 190)]