Comparing two datetime columns returns FALSE when NaT

I'm extracting data from the database using sqlalchemy. I need to compare the col 'fields_resolved_at' with 'resolved_at' and mark False/True if there is a mismatch/match.

Running:

comparison_column = np.where(merged_df["fields_resolved_at"] == merged_df["resolved_at"], True, False)
merged_df["equal"] = comparison_column

Returns:

However, I would expect rows with index 1, 2, 3 to be marked as True because both are NaT. I am thinking that maybe the easiest way will be to replace NaT with '0' and then, run the comparison but I wonder if there is a more elegant way to do it.

CodePudding user response：

You can't actually do this, because NaN != NaN (and therefore NaT != NaT). Instead, you can check if the two columns are both equal, OR if they're both NaN/NaT:

comparison_column = (merged_df["fields_resolved_at"] == merged_df["resolved_at"]) | (merged_df["fields_resolved_at"].isna() & merged_df["resolved_at"].isna())
merged_df["equal"] = comparison_column