Convert np.ndarray of tuples (dtype=object) into array with dtype=int-CodePudding

I need to convert np arrays (short) of tuples to np arrays of ints.

The most obvious method doesn't work:

# array_of_tuples is given, this is just an example:
array_of_tuples = np.zeros(2, dtype=object)
array_of_tuples[0] = 1,2
array_of_tuples[1] = 2,3

np.array(array_of_tuples, dtype=int)

ValueError: setting an array element with a sequence.

CodePudding user response：

It looks like placing the tuples into a pre-allocated buffer of fixed size and dtype is the way to go. It seems to avoid a lot of the overhead associated with computing sizes, raggedness and dtype.

Here are some slower alternatives and a benchmark:

You can cheat and create a dtype with the requisite number of fields, since numpy supports conversion of tuples to custom dtypes:

 dt = np.dtype([('', int) for _ in range(len(array_of_tuples[0]))])
 res = np.empty((len(array_of_tuples), len(array_of_tuples[0])), int)
 res.view(dt).ravel()[:] = array_of_tuples

You can stack the array:
```
 np.stack(array_of_tuples, axis=0)
```
Unfortunately, this is even slower than the other proposed methods.

Pre-allocation does not help much:

 res = np.empty((len(array_of_tuples), len(array_of_tuples[0])), int)
 np.stack(array_of_tuples, out=res, axis=0)

Trying to cheat using np.concatenate, which allows you to specify the output dtype does not help much either:
```
 np.concatenate(array_of_tuples, dtype=int).reshape(len(array_of_tuples), len(array_of_tuples[0]))
```

And neither does pre-allocating the array:

 res = np.empty((len(array_of_tuples), len(array_of_tuples[0])), int)
 np.concatenate(array_of_tuples, out=res.ravel())

You can also try to do the concatenation in python space, which is slow too:

 np.array(sum(array_of_tuples, start=()), dtype=int).reshape(len(array_of_tuples), len(array_of_tuples[0]))

 np.reshape(np.sum(array_of_tuples), (len(array_of_tuples), len(array_of_tuples[0])))

array_of_tuples = np.empty(100, dtype=object)
for i in range(len(array_of_tuples)):
    array_of_tuples[i] = tuple(range(i, i   100))

%%timeit
res = np.empty((len(array_of_tuples), len(array_of_tuples[0])), int)
for i, res[i] in enumerate(array_of_tuples):
    pass
305 µs ± 8.55 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

dt = np.dtype([('', 'int',) for _ in range(100)])
%%timeit
res = np.empty((100, 100), int)
res.view(dt).ravel()[:] = array_of_tuples
334 µs ± 5.59 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.array(array_of_tuples.tolist())
478 µs ± 12.9 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%%timeit
res = np.empty((100, 100), int)
np.concatenate(array_of_tuples, out=res.ravel())
500 µs ± 2.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.concatenate(array_of_tuples, dtype=int).reshape(100, 100)
504 µs ± 7.72 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%%timeit
res = np.empty((100, 100), int)
np.stack(array_of_tuples, out=res, axis=0)
557 µs ± 25.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.stack(array_of_tuples, axis=0)
577 µs ± 6.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.array(sum(array_of_tuples, start=()), dtype=int).reshape(100, 100)
1.06 ms ± 11.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit np.reshape(np.sum(array_of_tuples), (100, 100))
1.26 ms ± 24.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)