optimizing neural network with genetic algoritms - evaluating multiple candidates at once-CodePudding

I built a neural network as a surrogate model for some optimization problem I am trying to solve. This neural network approximates the real-world problem quite decently, with R_squared around 0.92. I used a differential_evolution by scipy, and the part of the code that does it is this:

def func_to_opt(x, TRANS_MIN_BV=TRANS_MIN_BV, SUBS_VAL=100, model=model):
    """Returns Rsp if BV is above TRANS_MIN_BV, SUBS_VAL if not. Rsp and BV is calculated by the neural network model"""
    y = model.predict(np.array([x]))
    y = np.array(y)[:, : ,0]
    y = np.swapaxes(y, 0, 1)
    if y[0, 0] > TRANS_MIN_BV:
        Rsp = y[0, 1]
    else:
        Rsp = SUBS_VAL
    return Rsp

results = scipy.optimize.differential_evolution(func_to_opt, bounds=trans_bounds.T.tolist(), maxiter=int(5e2), seed=1)

Basically, the parameters we want to optimize look like x=[0.1, 1.2, -0.4, 0.8]. The model outputs two values, Rsp and BV, I want Rsp to be as low as possible with the constraint that BV needs to be higher than some constant.

The reason why I am writing here is that the differential_evolution ran for much longer than I thought it would, and while examining what is the part that slows it down the most, I realized that it would be much faster if the func_to_opt could give evaluations of multiple x's at once. It is because Tensorflow in which the neural network is implemented is vectorized and it takes similar time to give predictions for a single x as for a huge batch of hundreds of x's.

Is there anywhere such an implementation of a genetic algorithm, that would let me write a func_to_opt that gives evaluations of multiple x's (for the whole population simultaneously), or generally that would let me take advantage of the fact that the neural network that is the most crucial part of the func_to_opt takes similar time to spit out predictions for a single x as for a batch of them? I have no trouble writing func_to_opt in a vectorized form, but afaik scipy.optimize.differential_evolution only let's me use a function that gives evaluation of a single x.

Thank you very much - if there is some part of the question that is not comprehensible or if I messed up something with the tags, let me know, I am quite new to stackoverflow.

CodePudding user response：

I did not read the documentation for https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.differential_evolution.html properly, since version 1.9, it supports vectorized function, using vectorized=True. So yeah, one can calculate fitness for the whole population simultaneously.