Home > Blockchain >  Need help in LSTM Autoencoder - Anomaly detection
Need help in LSTM Autoencoder - Anomaly detection

Time:12-23

I am trying to do Anomaly detection with LSTM. I am able to plot all features with local and global anomaly but I am not able to print all anomaly values, datetime, loss, threshold and date together (like a table).

After calculating test and train MAE in the following way:

Y_train_pred = self.model.predict(self.X_train)
train_mae_loss = np.mean(np.abs(self.Y_train_pred - self.Y_train), axis=1)
Y_test_pred = self.model.predict(self.X_test)
test_mae_loss = np.mean(np.abs(self.Y_test_pred - self.Y_test), axis=1)
test = self.test[:len(Y_test_pred)]

I tried to make a table by matching date, loss, threshold, and anomaly in this way:

test_score_df = pd.DataFrame(index=self.test.index)
print(test_score_df)
test_score_df['loss'] = loss_mean_vec
test_score_df['threshold'] = threshold_mean_exp
test_score_df['anomaly'] = test_score_df.loss > test_score_df.threshold
test_score_df['dckw'] = self.test[:].dckw
anomalies = test_score_df[test_score_df.anomaly == True]
print(anomalies.head())

But it throws and error :

AttributeError: 'DataFrame' object has no attribute 'dckw' 

When I print self.test it has all the features with header datetimeAt, dckw ......

When I remove this line test_score_df['dckw'] = self.test[:].dckw

It gives me this result:

       loss  threshold  anomaly
0  0.106414   0.037134     True
1  0.107169   0.037134     True
2  0.107001   0.037134     True
3  0.105836   0.037134     True
4  0.103779   0.037134     True

So how can I get the last table with datetime and other features which are in csv file so that I can plot datetime and see when was the anomaly appeared?

My code and files are quite heavy so I uploaded them in git hub: enter image description here

  • Related