I am trying to do Anomaly detection with LSTM. I am able to plot all features with local and global anomaly but I am not able to print all anomaly values, datetime, loss, threshold and date together (like a table).
After calculating test and train MAE in the following way:
Y_train_pred = self.model.predict(self.X_train)
train_mae_loss = np.mean(np.abs(self.Y_train_pred - self.Y_train), axis=1)
Y_test_pred = self.model.predict(self.X_test)
test_mae_loss = np.mean(np.abs(self.Y_test_pred - self.Y_test), axis=1)
test = self.test[:len(Y_test_pred)]
I tried to make a table by matching date, loss, threshold, and anomaly in this way:
test_score_df = pd.DataFrame(index=self.test.index)
print(test_score_df)
test_score_df['loss'] = loss_mean_vec
test_score_df['threshold'] = threshold_mean_exp
test_score_df['anomaly'] = test_score_df.loss > test_score_df.threshold
test_score_df['dckw'] = self.test[:].dckw
anomalies = test_score_df[test_score_df.anomaly == True]
print(anomalies.head())
But it throws and error :
AttributeError: 'DataFrame' object has no attribute 'dckw'
When I print self.test it has all the features with header datetimeAt, dckw ......
When I remove this line test_score_df['dckw'] = self.test[:].dckw
It gives me this result:
loss threshold anomaly
0 0.106414 0.037134 True
1 0.107169 0.037134 True
2 0.107001 0.037134 True
3 0.105836 0.037134 True
4 0.103779 0.037134 True
So how can I get the last table with datetime and other features which are in csv file so that I can plot datetime and see when was the anomaly appeared?
My code and files are quite heavy so I uploaded them in git hub: