I have an array a as follow:
import numpy as np
a= np.array([[1, 3, 5, 7, 8, 7, 1],
[11, 13, 51, 17, 18, 17, 10]])
I want to build a list of that array with a sliding window. Here is the output that I want:
I have using the following code, however it does not provide the output which I want:
lag = 3
out = []
for i in range(2):
eachrow =[]
for col in range(a.shape[1]-lag):
X_row = []
xtmp = a[i, col:col lag]
X_row.append(xtmp)
ytmp = a[i, col lag]
X_row.append(ytmp)
eachrow.append(X_row)
out.append(eachrow)
Any help appreciated. Thanks.
CodePudding user response：
There are two issues I see:
The last element in the window isn't enclosed in an
np.array.Then, a quick fix would be to change this line:
X_row.append(ytmp)to
X_row.append(np.array([ytmp]))which produces the desired output.
The result is two dimensions, since you create a separate sublist for each row in the array, and then append that sublist to the result. To resolve, change:
out.append(eachrow)to
out.extend(eachrow)
CodePudding user response：
You can use numpy.lib.stride_tricks.sliding_window_view for a fast, vectorized solution:
x = np.lib.stride_tricks.sliding_window_view(a, (1,3))[:, :-1]
x.shape = (*x.shape[:2], *x.shape[3:])
y = a[:, -x.shape[1]:, None]
Output:
>>> x
array([[[ 1, 3, 5],
[ 3, 5, 7],
[ 5, 7, 8],
[ 7, 8, 7]],
[[11, 13, 51],
[13, 51, 17],
[51, 17, 18],
[17, 18, 17]]])
>>> y
array([[[ 7],
[ 8],
[ 7],
[ 1]],
[[17],
[18],
[17],
[10]]])
Now, just use zip list:
out = [list(zip(x[i], y[i])) for i in range(len(y))]
Output:
>>> out
[[(array([1, 3, 5]), array([7])),
(array([3, 5, 7]), array([8])),
(array([5, 7, 8]), array([7])),
(array([7, 8, 7]), array([1]))],
[(array([11, 13, 51]), array([17])),
(array([13, 51, 17]), array([18])),
(array([51, 17, 18]), array([17])),
(array([17, 18, 17]), array([10]))]]
CodePudding user response：
- Flatten the array. So it's not 2d anymore.
- Start a loop from
0tolength - 3of the array. - Now just choose from
nth(loop's) element to3rd. and3rdelement
Here some code:
fa = a.flatten()
res = np.array(
[
[fa[i:i 3], fa[i 3]]
for i in range(fa.size - 3)
]
)
CodePudding user response：
Your code produces:
In [3]: out
Out[3]:
[[[array([1, 3, 5]), 7],
[array([3, 5, 7]), 8],
[array([5, 7, 8]), 7],
[array([7, 8, 7]), 1]],
[[array([11, 13, 51]), 17],
[array([13, 51, 17]), 18],
[array([51, 17, 18]), 17],
[array([17, 18, 17]), 10]]]
That's a list of length 2. Within that lists.
If we make an array from that - with object dtype, we get a 3d array, where some elements are arrays, and some are integers:
In [6]: arr = np.array(out, object)
In [7]: arr
Out[7]:
array([[[array([1, 3, 5]), 7],
[array([3, 5, 7]), 8],
[array([5, 7, 8]), 7],
[array([7, 8, 7]), 1]],
[[array([11, 13, 51]), 17],
[array([13, 51, 17]), 18],
[array([51, 17, 18]), 17],
[array([17, 18, 17]), 10]]], dtype=object)
In [8]: arr.shape
Out[8]: (2, 4, 2)
Change one line of your code to
X_row.append(np.array([ytmp]))
In [11]: np.array(out,object)
Out[11]:
array([[[array([1, 3, 5]), array([7])],
[array([3, 5, 7]), array([8])],
[array([5, 7, 8]), array([7])],
[array([7, 8, 7]), array([1])]],
[[array([11, 13, 51]), array([17])],
[array([13, 51, 17]), array([18])],
[array([51, 17, 18]), array([17])],
[array([17, 18, 17]), array([10])]]], dtype=object)
or displayed with the str/print array formatting:
In [12]: print(_)
[[[array([1, 3, 5]) array([7])]
[array([3, 5, 7]) array([8])]
[array([5, 7, 8]) array([7])]
[array([7, 8, 7]) array([1])]]
[[array([11, 13, 51]) array([17])]
[array([13, 51, 17]) array([18])]
[array([51, 17, 18]) array([17])]
[array([17, 18, 17]) array([10])]]]
We could reshape that to a (8,2) array (still object dtype):
In [14]: print(Out[11].reshape(-1,2))
[[array([1, 3, 5]) array([7])]
[array([3, 5, 7]) array([8])]
[array([5, 7, 8]) array([7])]
[array([7, 8, 7]) array([1])]
[array([11, 13, 51]) array([17])]
[array([13, 51, 17]) array([18])]
[array([51, 17, 18]) array([17])]
[array([17, 18, 17]) array([10])]]
Since the inner most arrays have a mixed size - some 3 some 1, the result can only be object dtype - or list of lists. That's isn't optimal for array calculations.
Commas are part of the display, along with [] and works like array. Together they give us clues as to the underlying objects, whether they are lists or arrays. Equally important are the shape and dtype (if the object is an array) or length if a list.
===
Another answer uses a striding_tricks function. Here's that method in more detail. While x is a view, slicing and reshaping will make copies, so it's hard to say whether this is any faster. For this small example I bet your code is faster, but for larger case it might not be.
In [16]: np.lib.stride_tricks.sliding_window_view(a,(1,4))
Out[16]:
array([[[[ 1, 3, 5, 7]],
[[ 3, 5, 7, 8]],
[[ 5, 7, 8, 7]],
[[ 7, 8, 7, 1]]],
[[[11, 13, 51, 17]],
[[13, 51, 17, 18]],
[[51, 17, 18, 17]],
[[17, 18, 17, 10]]]])
In [17]: x = np.lib.stride_tricks.sliding_window_view(a,(1,4))
In [18]: x.shape
Out[18]: (2, 4, 1, 4)
That's a 4d view of the original 1d array.
Your size 3 'arrays' can be sliced from that:
In [19]: x[:,:,0,:3]
Out[19]:
array([[[ 1, 3, 5],
[ 3, 5, 7],
[ 5, 7, 8],
[ 7, 8, 7]],
[[11, 13, 51],
[13, 51, 17],
[51, 17, 18],
[17, 18, 17]]])
In [20]: x[:,:,0,:3].reshape(-1,3)
Out[20]:
array([[ 1, 3, 5],
[ 3, 5, 7],
[ 5, 7, 8],
[ 7, 8, 7],
[11, 13, 51],
[13, 51, 17],
[51, 17, 18],
[17, 18, 17]])
and the 1 element column:
In [21]: x[:,:,0,-1].reshape(-1,1)
Out[21]:
array([[ 7],
[ 8],
[ 7],
[ 1],
[17],
[18],
[17],
[10]])
These 2 arrays may be more useful than your object out.
The arrays shown in [14] could be split into 2 similar arrays:
In [27]: np.stack(arr.reshape(-1,2)[:,0])
Out[27]:
array([[ 1, 3, 5],
[ 3, 5, 7],
[ 5, 7, 8],
[ 7, 8, 7],
[11, 13, 51],
[13, 51, 17],
[51, 17, 18],
[17, 18, 17]])
In [28]: arr.reshape(-1,2)[:,1].astype(int)
Out[28]: array([ 7, 8, 7, 1, 17, 18, 17, 10])