Python: Memory-efficient random sampling of list of permutations-CodePudding

I am seeking to sample n random permutations of a list in Python.

This is my code:

obj = [    5     8     9 ... 45718 45719 45720]
#type(obj) = numpy.ndarray

pairs = random.sample(list(permutations(obj,2)),k= 150)

Although the code does what I want it to, it causes memory issues. I sometimes receive the error Memory error when running on CPU, and when running on GPU, my virtual machine crashes.

How can I make the code work in a more memory-efficient manner?

CodePudding user response：

You can avoid listing the permutation iterator that could be massive in memory. You can generate random permutations by sampling the list with replace=False.

import numpy as np
obj = np.array([5,8,123,13541,42])
k = 15
permutations = [tuple(np.random.choice(obj, 2, replace=False)) for _ in range(k)]
print(permutations)

This problem becomes much harder, if you for example impose no repetition in your random permutations.

CodePudding user response：

This avoids using permutations at all:

count = len(obj)
pairs = [(obj[i%count],obj[i//count]) for i in random.sample(range(count*count),k=3)]