Keras Graph Disconnected Error - Appending an Encoder & Decoder-CodePudding

Hi everyone,

Hope you're all doing great.

I am currently struggling with a problem that seems trivial at a first glance - however, I was not able to find a solution to this no matter how many research I've tried to make.

I am currently working on implementing a new method (layerwise relevance propagation) on top of an existing pipeline concerning an Autoencoder architecture, which is coded by a class as follows :

class Autoencoder(tf.keras.Model):

def __init__(self):

    self._encoder = tf.keras.Sequential(
        [
            tf.keras.layers.Dense(500, activation='relu', input_shape=(input_size, )),
            tf.keras.layers.Dense(200, activation='relu'),
            tf.keras.layers.Dense(50, activation='relu'),
        ]
    )
    
    self._decoder = tf.keras.Sequential(
        [
            tf.keras.layers.Dense(200, activation='relu', input_shape=(50, )),
            tf.keras.layers.Dense(500, activation='relu'),
            tf.keras.layers.Dense(input_size)
        ]
    )

The autoencoder is implemented as a sequence of two sequential models - an encoder followed by a decoder.

For this reason, running autoencoder.layers yields :

[<keras.engine.sequential.Sequential at 0x7f190014f080>,
 <keras.engine.sequential.Sequential at 0x7f190012a400>]

I am currently working on a neural network method that requires that the input model to be a sequence of layers - in other words, the output of autoencoder.layers should be the following :

[<keras.layers.core.Dense at 0x7f191a2aa438>,
 <keras.layers.core.Dense at 0x7f191a2aa748>,
 <keras.layers.core.Dense at 0x7f191a2aad30>,
 <keras.layers.core.Dense at 0x7f1900123cf8>,
 <keras.layers.core.Dense at 0x7f190012ab00>,
 <keras.layers.core.Dense at 0x7f190012ae10>]

To that end, I have tried to define a new model as follows :

model = Sequential([autoencoder.encoder.layers[0],
                    autoencoder.encoder.layers[1],
                    autoencoder.encoder.layers[2],
                    autoencoder.decoder.layers[0],
                    autoencoder.decoder.layers[1],
                    autoencoder.decoder.layers[2]])

Unfortunately, using this method, I get the following error when building a relevance propagation graph (which is basically a backward propagation graph):

ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 1114), dtype=tf.float32, name='dense_input'), name='dense_input', description="created by layer 'dense_input'") at layer "dense". The following previous layers were accessed without issue: []

In case you might want to look deeper into the code responsible for the error :

class LayerwiseRelevancePropagation:

def __init__(self, model, alpha=2, epsilon=1e-7):
    
    # Initialization
    self.model = model
    self.alpha = alpha
    self.beta = 1 - alpha
    self.epsilon = epsilon
    
    # Retrieve network parameters
    self.names, self.activations, self.weights = get_model_params(self.model)
    self.num_layers = len(self.names)
    
    # Build relevance propagation graph
    self.relevance = self.relevance_propagation()
    
    print(self.model.input)
    
    self.lrp_runner = K.function(inputs=self.model.input,
            outputs=self.relevance)

def relevance_propagation(self):
    """Builds graph for relevance propagation."""
    
    # Forward pass
    r = self.model.output
    
    # Relevance propagation
    for i in range(self.num_layers-2, -2, -1):

        if i==-1:
            r = self.backprop_fc(self.weights[i   1][0], self.weights[i   1][1], tf.ones_like(self.model.input), r)
        elif 'dense' in self.names[i   1]:
            r = self.backprop_fc(self.weights[i   1][0], self.weights[i   1][1], self.activations[i], r)
        else:
            raise Exception("Error: layer type not recognized.")
    return r

def backprop_fc(self,w,b,a,r):
    
    # Positive relevance
    w_p = K.maximum(w, 0.)
    b_p = K.maximum(b, 0.)
    z_p = K.dot(a, w_p)   b_p   self.epsilon
    s_p = r / z_p
    c_p = K.dot(s_p, K.transpose(w_p))
    
    # Negative relevance
    w_n = K.minimum(w, 0.)
    b_n = K.minimum(b, 0.)
    z_n = K.dot(a, w_n)   b_n - self.epsilon
    s_n = r / z_n
    c_n = K.dot(s_n, K.transpose(w_n))
    
    return a * (self.alpha * c_p   self.beta * c_n)

Any help would be greatly appreciated!

Thank you in advance for your time :)

CodePudding user response：

If I understood your question correctly, then this short code snippet should give you an idea of how you can achieve the desired behavior:

def model_test():
    encoder = tf.keras.Sequential(
        [
            tf.keras.layers.Dense(500, activation='relu', input_shape=(10,)),
            tf.keras.layers.Dense(200, activation='relu'),
            tf.keras.layers.Dense(50, activation='relu'),
        ]
    )

    decoder = tf.keras.Sequential(
        [
            tf.keras.layers.Dense(200, activation='relu', input_shape=(50,)),
            tf.keras.layers.Dense(500, activation='relu'),
            tf.keras.layers.Dense(10)
        ]
    )

    inputs = layers.Input(shape=(10,))
    x = inputs

    for layer in encoder.layers:
        x = layer(x)

    for layer in decoder.layers:
        x = layer(x)

    model = Model(inputs=inputs, outputs=x)
    model.summary()
    print(model.layers)

CodePudding user response：

it seems like I managed to come up with a solution - it is not as elegant as I would have hoped, but it does solve the problem.

# Create the model with the same architecture and a "clean" network graph
model = tf.keras.Sequential(
    [tf.keras.layers.Dense(500, activation='relu', input_shape=(input_size, )),
     tf.keras.layers.Dense(200, activation='relu'),
     tf.keras.layers.Dense(50, activation='relu'),
     tf.keras.layers.Dense(200, activation='relu', input_shape=(50, )),
     tf.keras.layers.Dense(500, activation='relu'),
     tf.keras.layers.Dense(input_size)])

# Copy the weights
model.set_weights(autoencoder.get_weights())

Thanks for your help again !