Python : Replace a column in a dataframe by datetime values-CodePudding

I'm trying to replace a column an array of 4 columns by datetime values that I treated. The problem is that it's difficult to keep the same form between the different formats of dataframe, array,....


dataw = ds.variables["pr"][:]
dataw = np.array(dataw[:,0,0])
lat = ds.variables["lat"][:]
long = ds.variables["lon"][:]
time = ds.variables["time"][:]

time = pd.to_datetime(ds.variables["time"][:],origin=pd.Timestamp('1850-01-01'),unit='D')
#np.datetime64(ds.variables["time"][:],'D')
x2 = pd.DataFrame(np.zeros((len(dataw),4), float))
x = np.zeros((len(dataw),4), float)


x[:,0] = time
x[:,1] = long
x[:,2] = lat[:]
x[:,3] = dataw[:]*86400


x=pd.DataFrame(x)
x[:,0] = pd.to_datetime(time,origin=pd.Timestamp('1850-01-01'),unit='D')

If I put directly the dates transformed in the array, the result is like: 1.32542e 18

I tried

time = ds.variables["time"][:]

and include it in the array, and then use

x[:,0]=pd.to_datetime(x[:,0],origin=pd.Timestamp('1850-01-01'),unit='D')

I get the error:

TypeError: unhashable type: 'slice'

I tried also directly put:

time=pd.to_datetime(time,origin=pd.Timestamp('1850-01-01'),unit='D')
x[:,0] = time[:]

TypeError: unhashable type: 'slice'

CodePudding user response：

try this instead

        import numpy as np
    import pandas as pd
    
    dataw = ds.variables["pr"][:]
    dataw = np.array(dataw[:, 0, 0])
    lat = ds.variables["lat"][:]
    long = ds.variables["lon"][:]
    time = ds.variables["time"][:]
    
    time = np.datetime64(time, 'D')
x = np.zeros((len(dataw), 4), dtype='datetime64[D]')
    x[:, 0] = time
    x[:, 1] = long
    x[:, 2] = lat
    x[:, 3] = dataw * 86400
    
    df = pd.DataFrame(x, columns=["Time", "Longitude", "Latitude", "Data"])

CodePudding user response：

Xarray makes the netcdf->pandas workflow quite straightforward:

import xarray as xr

ds = xr.open_dataset('file.nc', engine='netcdf4')
df = ds.to_pandas()

Presuming your time variable is using cf-conventions, Xarray will automatically decode it into datetime objects.