numpy.array.tolist() converts numpy.datetime64 to int-CodePudding

I have an array of datetimes that I need to convert to a list of datetimes. My array looks like this:

import numpy as np

my_array = np.array(['2017-06-28T22:47:51.213500000', '2017-06-28T22:48:37.570900000',
                     '2017-06-28T22:49:46.736800000', '2017-06-28T22:50:41.866800000',
                     '2017-06-28T22:51:17.024100000', '2017-06-28T22:51:24.038300000'], dtype='datetime64[ns]')

my_list = my_array.tolist()

I need a list of datetime values, but when I do my_array.tolist(), I get a list of numerical time stamps:

[1498690071213500000,
 1498690117570900000,
 1498690186736800000,
 1498690241866800000,
 1498690277024100000,
 1498690284038300000]

My question is how do I preserve the datetime format when going from an array to a list, or how do I convert the list of time stamps to a list datetime values?

CodePudding user response：

NumPy can't convert instances of 'datetime64[ns]' to Python datetime.datetime instances, because datetime instances do not support nanosecond resolution.

If you cast the array to 'datetime64[us]', so the timestamps have only microsecond resolution, then the .tolist() method will give you datetime.datetime instances:

In [25]: my_array
Out[25]: 
array(['2017-06-28T22:47:51.213500000', '2017-06-28T22:48:37.570900000',
       '2017-06-28T22:49:46.736800000', '2017-06-28T22:50:41.866800000',
       '2017-06-28T22:51:17.024100000', '2017-06-28T22:51:24.038300000'],
      dtype='datetime64[ns]')

In [26]: my_array.astype('datetime64[us]').tolist()
Out[26]: 
[datetime.datetime(2017, 6, 28, 22, 47, 51, 213500),
 datetime.datetime(2017, 6, 28, 22, 48, 37, 570900),
 datetime.datetime(2017, 6, 28, 22, 49, 46, 736800),
 datetime.datetime(2017, 6, 28, 22, 50, 41, 866800),
 datetime.datetime(2017, 6, 28, 22, 51, 17, 24100),
 datetime.datetime(2017, 6, 28, 22, 51, 24, 38300)]

CodePudding user response：

Explicitly casting the numpy.ndarray as a native Python list will preserve the contents as numpy.datetime64 objects:

>>> list(my_array)
[numpy.datetime64('2017-06-28T22:47:51.213500000'),
 numpy.datetime64('2017-06-28T22:48:37.570900000'),
 numpy.datetime64('2017-06-28T22:49:46.736800000'),
 numpy.datetime64('2017-06-28T22:50:41.866800000'),
 numpy.datetime64('2017-06-28T22:51:17.024100000'),
 numpy.datetime64('2017-06-28T22:51:24.038300000')]

However, if you wanted to go back from an integer timestamp to a numpy.datetime64 object, the number given here by numpy.ndarray.tolist is given in nanosecond format, so you could also use a list comprehension like the following:

>>> [np.datetime64(x, "ns") for x in my_list]
[numpy.datetime64('2017-06-28T22:47:51.213500000'),
 numpy.datetime64('2017-06-28T22:48:37.570900000'),
 numpy.datetime64('2017-06-28T22:49:46.736800000'),
 numpy.datetime64('2017-06-28T22:50:41.866800000'),
 numpy.datetime64('2017-06-28T22:51:17.024100000'),
 numpy.datetime64('2017-06-28T22:51:24.038300000')]

And if you want the final result as a Python datetime.datetime object instead of a numpy.datetime64 object, you can use a method like this (adjusted as needed for locality):

>>> from datetime import datetime
>>> list(map(datetime.utcfromtimestamp, my_array.astype(np.uint64) / 1e9))
[datetime.datetime(2017, 6, 28, 22, 47, 51, 213500),
 datetime.datetime(2017, 6, 28, 22, 48, 37, 570900),
 datetime.datetime(2017, 6, 28, 22, 49, 46, 736800),
 datetime.datetime(2017, 6, 28, 22, 50, 41, 866800),
 datetime.datetime(2017, 6, 28, 22, 51, 17, 24100),
 datetime.datetime(2017, 6, 28, 22, 51, 24, 38300)]

Edit: Warren Weckesser's answer provides a more straightforward approach to go from a numpy.datetime64[ns] array to a list of Python datetime.datetime objects than is described here.

CodePudding user response：

Try

# convert to string type first
my_list = my_array.astype(str).tolist()
my_list
# ['2017-06-28T22:47:51.213500000', '2017-06-28T22:48:37.570900000', '2017-06-28T22:49:46.736800000', '2017-06-28T22:50:41.866800000', '2017-06-28T22:51:17.024100000', '2017-06-28T22:51:24.038300000']

The other answers provide a more straightforward ways but for completeness, you can call datetime.datetime.fromtimestamp in a loop

from datetime import datetime
[datetime.fromtimestamp(x) for x in my_array.astype(object)/1e9]

#[datetime.datetime(2017, 6, 28, 15, 47, 51, 213500),
# datetime.datetime(2017, 6, 28, 15, 48, 37, 570900),
# datetime.datetime(2017, 6, 28, 15, 49, 46, 736800),
# datetime.datetime(2017, 6, 28, 15, 50, 41, 866800),
# datetime.datetime(2017, 6, 28, 15, 51, 17, 24100),
# datetime.datetime(2017, 6, 28, 15, 51, 24, 38300)]