I was able to use Numba to solve a slow itertuples iteration issue with code provided in the answer of this question however when I try it on a new function with a similar loop format i run into this error
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(float64, 1d, C), int64, datetime64[ns])
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(float64, 1d, C), int64, datetime64[ns])':
No match.
During: typing of setitem at C:\....
File "dtb.py", line 774:
def RI_finder(idx, start_dtm, end_dtm):
<source elided>
if #check conditions:
start_dtm[i] = start_idx
^
There are some existing questions like this one, and this one, that have a similar issue however the solutions provided don't fix my error.
Here is a sample of my code
@njit
def RI_finder(idx, start_dtm, end_dtm):
for i in prange(len(start_dtm)):
end_idx = idx[i]
if #check conditions:
for j in prange(len(start_dtm)):
start_idx = idx[j]
if #check conditions:
for k in prange(len(start_dtm)):
if #check conditions
if #check conditions
start_dtm[i] = start_idx
end_dtm[i] = end_idx
and here is a sample dataframe from which I am extracting the data used in the code provided
import numpy as np
import pandas as pd
start_dtm = ['2022-10-29 06:59:59.999', '2022-10-29 07:14:59.999', '2022-10-29 07:29:59.999', '2022-10-29 07:44:59.999',
'2022-10-29 07:59:59.999', '2022-10-29 08:14:59.999', '2022-10-29 08:29:59.999']
end_dtm = ['2022-10-29 06:59:59.999', '2022-10-29 07:05:59.999', '2022-10-29 07:10:59.999', '2022-10-29 07:15:59.999',
'2022-10-29 07:20:59.999', '2022-10-29 07:25:59.999', '2022-10-29 07:30:59.999']
idx = ['2022-10-29 06:59:59.999', '2022-10-29 07:01:59.999', '2022-10-29 07:02:59.999', '2022-10-29 07:03:59.999',
'2022-10-29 07:04:59.999', '2022-10-29 07:05:59.999', '2022-10-29 07:06:59.999']
df = pd.DataFrame({'start_dtm': start_dtm, 'end_dtm': end_dtm}, index=idx)
What I am attempting is to loop through the arrays and find when certain conditions are met.
Once those conditions are met I update start_dtm, end_dtm.
Seeing as my code is so similar in structure to the code in this questions answer, (multiple loops to check conditions and then make changes). I can't see why mine is not working the same
Edit: As suggested, making sure the variables are the same type worked to stop the error I was getting however now I am getting a new error which I have been unable to solve also.
non-precise type array(pyobject, 1d, C)
During: typing of argument at C: ....
File "dtb.py", line 734:
def RI_finder(idx, start_dtm, end_dtm):
<source elided>
for i in prange(len(open)):
^
CodePudding user response:
It would help if you simplified the question and added a reproducible example.
But based on the Exception, it looks like you're trying to set the item with a different incompatible type:
setitem(array(float64, 1d, C), int64, datetime64[ns])
start_dtm = float64
i = int64
start_idx = datetime64[ns]
Making sure both start_dtm
and start_idx
share the same datatype would probably solve the issue.
edit:
The error related to the "pyobject 1D" seems to be cause by the len()
function not supporting object type arrays, which is caused by Pandas in this case. It can be reproduced by something like:
@njit
def func(x):
return len(x)
func(np.asarray(["x"], dtype=object))
I'm not sure what the idea behind it all is, but casting all three arrays to datetime (pd.to_datetime(start_dtm)
) solves that, or making it a string type should also work.