I'm trying to load a pretrained BERT model in a sagemaker training job using the transformers library and I'm getting "No modul named keras error". You can find the relevant code, imports and requirements.txt below
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.wrappers.scikit_learn import KerasClassifier
from tensorflow.keras import applications
from tensorflow.keras import layers
from tensorflow.keras import losses
from tensorflow.keras import optimizers
from tensorflow.keras import metrics
from tensorflow.keras import Model
from transformers import TFAutoModel
from transformers import BertTokenizer
from tensorflow.keras.layers import LeakyReLU
bert1 = TFAutoModel.from_pretrained('path/to/BERT', from_pt=True)
requirements.txt (I haven't set any versions for these):
transformers
torch
SentencePiece
Other env settings:
Python= 3.7
tensorflow= 2.3
I had used these exact settings a few months back and faced no issues, so not sure why I'm getting this error now
Edit: based on the answers I received, I added Keras to my requirements and also added from TensorFlow import keras statement, and now I'm getting the following error:
ModuleNotFoundError: No module named 'keras.saving'
CodePudding user response:
Please Try to run the following command, may be keras is not preinstalled in the workspace !pip install keras
CodePudding user response:
Your TensorFlow module may be installed in a different env. Please Try to add conda environment as the python interpreter and install all your libraries to it. it will work properly.