Home > Enterprise >  formatting unique dates in a database with Pandas Python
formatting unique dates in a database with Pandas Python

Time:11-10

How would I be able to write a code where it gives out only 1 output for each given dates in the data.csv. I want to select the first date value as a unique row and remove all the other ones just like how it is shown in the Expected Output.3 rows have the date value as 2021-10-31 13:17:00 of which it selects the very first one and disregards the rest. How would i be able to achieve this?

Code:

data = pd.read_csv('data.csv')
dates= pd.to_datetime(data['Date'].to_list())

data.csv file contents:

Date,Symbol,Open,High,Low,Close,Volume
2021-10-31 13:16:00,BTCUSD,60568.0,60640.0,60568.0,60638.0,3.9771881707839967
2021-10-31 13:15:00,BTCUSD,60620.0,60633.0,60565.0,60568.0,1.3977284440628714
2021-10-31 13:17:00,BTCUSD,60638.0,60640.0,60636.0,60638.0,0.4357009185659157
2021-10-31 13:17:00,BTCUSD,60648.0,60650.0,60638.0,60638.0,0.42475009155665155
2021-10-31 13:17:00,BTCUSD,60638.0,60640.0,60636.0,60638.0,0.4564009185659157
2021-10-31 13:16:00,BTCUSD,60588.0,60620.0,60510.0,60618.0,3.9771881707839967

Expected Output:

Date,Symbol,Open,High,Low,Close,Volume
2021-10-31 13:17:00,BTCUSD,60638.0,60640.0,60636.0,60638.0,0.4357009185659157
2021-10-31 13:16:00,BTCUSD,60568.0,60640.0,60568.0,60638.0,3.9771881707839967
2021-10-31 13:15:00,BTCUSD,60620.0,60633.0,60565.0,60568.0,1.3977284440628714

CodePudding user response:

We do sort_values first then drop_duplicates

out = df.sort_values('Date',ascending=False).drop_duplicates('Date')
Out[44]: 
                  Date  Symbol     Open     High      Low    Close    Volume
2  2021-10-31 13:17:00  BTCUSD  60638.0  60640.0  60636.0  60638.0  0.435701
0  2021-10-31 13:16:00  BTCUSD  60568.0  60640.0  60568.0  60638.0  3.977188
1  2021-10-31 13:15:00  BTCUSD  60620.0  60633.0  60565.0  60568.0  1.397728
  • Related