ValueError: could not convert string to float, When loading a list of arrays with different shape-CodePudding

So I have a list of 2D arrays and they have different shapes. For example,(60,40) and (42,40).

A sample code for starting can be

y = []
y.extend([np.ones((60,40)),np.ones((42,40)),np.ones((60,40))])

I saved them using np.savetxt(os.path.join("D:\A\code_testing\dataset",'mfccs.csv'), y,fmt='%s',delimiter=",") for later use.

However, a ValueError occurred when I tried to load them back.

ValueError                                Traceback (most recent call last)
d:\A\Pycodes\Dataset_prep(Watkins) 2.0.ipynb Cell 9 in <cell line: 1>()
----> 1 X_mfcc = np.loadtxt(os.path.join("D:\A\code_testing\dataset",'mfccs.csv'),  delimiter=",")

File e:\Anaconda\lib\site-packages\numpy\lib\npyio.py:1148, in loadtxt(fname, dtype, comments, delimiter, converters, skiprows, usecols, unpack, ndmin, encoding, max_rows, like)
   1143 # read data in chunks and fill it into an array via resize
   1144 # over-allocating and shrinking the array later may be faster but is
   1145 # probably not relevant compared to the cost of actually reading and
   1146 # converting the data
   1147 X = None
-> 1148 for x in read_data(_loadtxt_chunksize):
   1149     if X is None:
   1150         X = np.array(x, dtype)

File e:\Anaconda\lib\site-packages\numpy\lib\npyio.py:999, in loadtxt.<locals>.read_data(chunk_size)
    995     raise ValueError("Wrong number of columns at line %d"
    996                      % line_num)
    998 # Convert each value according to its column and store
--> 999 items = [conv(val) for (conv, val) in zip(converters, vals)]
   1001 # Then pack it according to the dtype's nesting
   1002 items = pack_items(items, packing)

File e:\Anaconda\lib\site-packages\numpy\lib\npyio.py:999, in <listcomp>(.0)
    995     raise ValueError("Wrong number of columns at line %d"
...
    734 if '0x' in x:
    735     return float.fromhex(x)
--> 736 return float(x)

ValueError: could not convert string to float: '[[-4.4259256e 02  4.0493172e 01  2.5947336e 01 ...  3.9495504e 00'

So i later tried using method(Python: How to save lists of 2D numpy arrays of different lengths)

np.savez("D:\A\code_testing\dataset\mfcc.npz",y)
data = np.load("D:\A\code_testing\dataset\mfcc.npz")
list2 = list2 = [v for k, v in data.items()]
print(len(list2))

I got error ValueError: Object arrays cannot be loaded when allow_pickle=False

CodePudding user response：

I think you need to do these separately. Again, ASSUMING y really is a Python list of numpy arrays:

for i,a in enumerate(y):
    name = 'mfccsd.csv' % i
    np.savetxt( name, a, delimiter=',', fmt='%s')

...

y = [np.loadtxt(name, delimiter=',') for name in glob.glob('mfcc*.csv')]

CodePudding user response：

The solution is to put a * before the list when doing the saving

np.savez("D:\A\code_testing\dataset\mfcc.npz",*y)