I wrote the below test code to check it is working properly and I get a "True" Value in the DF, but when i sure the same calculation on the just one value, it gives me "False". The correct value should be "False" as the date_time column is not in epoch/UNIX time
import pandas as pd
from datetime import datetime, timedelta
temp_list = []
i = 1
while i < 10:
d = {
'ticker': 'TEST',
'date_time': datetime.now() timedelta(days=i),
'price': 100 i,
'volume': i
}
temp_list.append(d)
i = 1
test_df = pd.DataFrame(data=temp_list)
test_df['isepoch'] = pd.notnull(pd.to_numeric(
test_df['date_time'], errors='coerce'))
print(test_df)
print(pd.notnull(pd.to_numeric(test_df['date_time'][0], errors='coerce')))
Output of the two print statements:
ticker date_time price volume isepoch
0 TEST 2021-09-16 10:33:43.285935 101 1 True
1 TEST 2021-09-17 10:33:43.285935 102 2 True
2 TEST 2021-09-18 10:33:43.285935 103 3 True
3 TEST 2021-09-19 10:33:43.285935 104 4 True
4 TEST 2021-09-20 10:33:43.285935 105 5 True
5 TEST 2021-09-21 10:33:43.285935 106 6 True
6 TEST 2021-09-22 10:33:43.285935 107 7 True
7 TEST 2021-09-23 10:33:43.285935 108 8 True
8 TEST 2021-09-24 10:33:43.285935 109 9 True
False
I adapted the solution from Pandas: Check if value is epoch time using python, to create the 'isepoch' column in the dataframe.
CodePudding user response:
If you heck pandas.to_numeric, scalar
, list
, tuple
, 1-d array
, or Series
can be converted as an argument for pandas.to_numeric
.
Above your code, the part as following :
test_df['isepoch'] = pd.notnull(pd.to_numeric(test_df['date_time'], errors='coerce'))
returns true as pandas.to_numeric
accepts series
value.
However, the test_df['date_time'][0]
is neither, scalar
, list
, tuple
, 1-d array
, or Series
as following :
In [1]: print(type(test_df['date_time'][0]) == list)
...: print(type(test_df['date_time'][0]) == tuple)
...: print(type(test_df['date_time'][0]) == np.array)
...: print(type(test_df['date_time'][0]) == pd.Series)
...: print(np.isscalar(test_df['date_time'][0]))
False
False
False
False
False
Therefore, if you change test_df['date_time'][0]
as argument type which can be acceptable in pandas.to_numeric
, it will return True value. Just like as following :
In [2]: pd.notnull(pd.to_numeric(np.array(test_df['date_time'][0]), errors='coerce'))
Out[2]: True
If you want to know more about pandas.to_numeric
source code, you can visit here.