compiling tensoflow model using TPU-CodePudding

During compilation of my model on tpu I run into error but using it on cpu it worsk just fine.This i how my data looks like

age           int64
sex           int64
cp            int64
trestbps      int64
chol          int64
fbs           int64
restecg       int64
thalach       int64
exang         int64
oldpeak     float64
slope         int64
ca            int64
thal         object
target        int64
dtype: object

target = df.pop('target')

numeric_feature_names = ['age', 'thalach', 'trestbps',  'chol', 'oldpeak']
numeric_features = df[numeric_feature_names]
tf.convert_to_tensor(numeric_features)
normalizer = tf.keras.layers.Normalization(axis=-1)
normalizer.adapt(numeric_features)

def get_basic_model():
  
    return  tf.keras.Sequential([
    normalizer,
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(1)
  ])

with strategy.scope():
    model = get_basic_model()
    
    model.compile(optimizer='adam',
                loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
                metrics=['accuracy'])

This is the part where it crashes with the following error

ValueError: Variable (<tf.Variable 'mean:0' shape=(5,) dtype=float32, numpy=
array([ 54.594067 , 149.19473  , 131.78549  , 246.54785  ,   1.0577558],
      dtype=float32)>) was not created in the distribution strategy scope of (<tensorflow.python.distribute.tpu_strategy.TPUStrategyV2 object at 0x7f1e64e3f9d0>). It is most likely because some layers, model, or optimizer was being created outside the distribution strategy scope. Try to make sure your code looks similar to the following.
with strategy.scope():
  model=_create_model()
  model.compile(...)

CodePudding user response：

As stated in the error,

"It is most likely because some layers, model, or optimizer was being created outside the distribution strategy scope.Try to make sure your code looks similar to the following. with strategy.scope(): model=_create_model() model.compile(...) "

The Normalization layer was created outside of the distribution strategy scope, thus the error. Please include that layer in the scope of the distribution strategy, as shown below.

def get_basic_model():

    normalizer = tf.keras.layers.Normalization(axis=-1)
    normalizer.adapt(numeric_features)
    
    model=tf.keras.Sequential([
    normalizer,
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(10, activation='relu'),
    tf.keras.layers.Dense(1)])
    return  model

with strategy.scope():
    model = get_basic_model()
    
    model.compile(optimizer='adam',
                loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
                metrics=['accuracy'])