I have a nested for loop. I am trying to implement a function on a numpy array. Sadly, some data point are bad and start taking ram till system freezes. But, i can figure out those faulty indexes manually by interrupting the loop and removing that data.

The the best thing i can think of is executing a time bound 'for' loop where the loop exists when execution time exceeds, say 2 seconds. here is a sample code for my implementation.

channel=6
for index in range(len(X_train)):
      for i in range(channel):
               X_train[index][:,channel] = function_that_creates_issue_for_some_index_values(X_train[index][]......)

CodePudding user response：

If a function_that_creates_issue_for_some_index_values() is stuck, it's two-way to break this

1. Add timeout mechanism in the function_that_creates_issue_for_some_index_values() and break it from the inside

2. Start for index in range(len(X_train)): loop as multiprocessing. Then you can start another process of monitoring that loop and break it when it gets stuck.

If only this loop get stuck for i in range(channel): not function_that_creates_issue_for_some_index_values(), just use @Jacob solution

CodePudding user response：

Use a break with a timer of your choice. For example you could do:

channel=6
startTime = time.time()
for index in range(len(X_train)):
      for i in range(channel):
               X_train[index][:,channel] = function_that_creates_issue_for_some_index_values(X_train[index][]......)
               if time.time() - startTime > 2:
                   break

CodePudding user response：

Try this:

import time
channel=6
for index in range(len(X_train)):
      start_time = time.clock()
      for i in range(channel):
          if (time.clock() - start_time > 2 ) :
              break;