splitting two lists in chunks in unison in python-CodePudding

I have to two lists X and Y of same length. I want to split the two lists in unison of chunks of length 2000. List X and Y each are of length 3671460. So for example:

#input
X = [1,2,3,4,5]
Y = [0,1,0,1,1]

#Expected output
X = [[1,2],[3,4],[5]]
Y = [[0,1],[0,1],[1]]

The example ofcourse shows a much smaller list. But my real lists are 3671460 long. How can I divide X and Y in chunks where each chunk has 2000 examples? These chunks will be input to my deep learning model later. Insights will be appreciated.

CodePudding user response：

what about this?

a = [1, 2, 3, 4, 5]
b = [a[x:x 2] for x in range(0, len(a), 2)]

2 might be 2000 in your case.

CodePudding user response：

If your lists are of different lengths, or aren't evenly divisible, you might want to use zip_longest to 'pad' out the empty values. The code here pairs up the chunks from each list:

from itertools import zip_longest

#input
X = [1,2,3,4,5,6,7,8]
Y = [0,1,0,1,0]

# Number of items in each 'chunk'
count = 3

print(
    list(
        zip_longest(
            zip_longest(*[iter(X)] * count),
            zip_longest(*[iter(Y)] * count)
        )
    )
)
# ((1, 2, 3), (0, 1, 0)), ((4, 5, 6), (1, 0, None)), ((7, 8, None), None)]

You can replace None in the output by passing the argument fillvalue='x'.