np.float64 and dtype=object can not be serialized-CodePudding

i have the below posted list and it contais lists, so it is a list of lists, and i have a webservice that returns return jsonify(resultsDict). the problem i am facing is when i run the App i recieve either of the error messages posted below. as shown in the code below, i tried to set dtype=np.float64 and dtype=object, but each of them generates an error associated to it as shown below in the code. please let me know how to fix it

attempt_1

resultsDict={
    "extras": {
        "pvTreatment":np.array(pvTreatment,dtype=np.float64).tolist(),
        ...
        ...
        ...
    }
}
    
**error associated**:

    ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (7,)   inhomogeneous part.

attempt_2

resultsDict = {
    "extras": {
        "pvTreatment":np.array(pvTreatment,dtype=object).tolist(),
        ...
        ...
        ...
    }
}   
    
**error associated**:

    raise TypeError(f'Object of type {o.__class__.__name__} '
    TypeError: Object of type float32 is not JSON serializable

CodePudding user response：

If I try to make an array with two arrays that differ in length I get your kind of error:

In [186]: np.array([np.ones((3)), np.zeros((4))],float)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [186], in <cell line: 1>()
----> 1 np.array([np.ones((3)), np.zeros((4))],float)

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,)   inhomogeneous part.

I can make an object dtype array:

In [187]: np.array([np.ones((3)), np.zeros((4))],object)
Out[187]: array([array([1., 1., 1.]), array([0., 0., 0., 0.])], dtype=object)

But when I use tolist I get a list with two arrays:

In [188]: np.array([np.ones((3)), np.zeros((4))],object).tolist()
Out[188]: [array([1., 1., 1.]), array([0., 0., 0., 0.])]

Arrays are not JSON serializable.

If the inner arrays were changed to lists, we'd get a list of lists:

In [191]: np.array([np.ones((3)).tolist(), np.zeros((4)).tolist()],object).tolist()
Out[191]: [[1.0, 1.0, 1.0], [0.0, 0.0, 0.0, 0.0]]

For this kind of thing, avoid arrays - both for the outer list and the inner ones.

CodePudding user response：

My suggestion is to use make lists equal in size first (if they don't differ by orders of magnitude) as in this example - List of lists into numpy array

In this way you will have an easier time deserializing it into list of lists. Also You can use external libraries like pickle if you want.