Immediately raise exceptions in concurrent.futures-CodePudding

I run several threads concurrently using concurrent.futures. All of them are necessary to run successfully for the next steps in the code to succeed.

While at the end of all processes I can raise any exceptions by running .result(), ideally any exception raised in a single thread would immediately stop all threads. This would be helpful to identify bugs in any task sooner, rather than waiting until all long-running processes complete.

Is this possible?

CodePudding user response：

It's possible to exit after the first exception and not submit any new jobs to the Executor. However, once a job has been submitted, it can't be cancelled, you have to wait for all submitted jobs to finish (or timeout). See this question for details. Here's a short example that cancels any unsubmitted jobs once the first exception occurs. However, it still waits for the already submitted jobs to finish. This uses the "FIRST EXCEPTION" keyword listed in the concurrent.futures docs.

import time
import concurrent.futures

def example(i):
    print(i)
    assert i != 1
    time.sleep(i)
    return i

if __name__ == "__main__":
    futures = []
    with concurrent.futures.ThreadPoolExecutor() as executor:
        for number in range(5):
            futures.append(executor.submit(example, number))

        exception = False
        for completed, running_or_error in concurrent.futures.wait(futures, return_when="FIRST_EXCEPTION"):
            try:
                 running_or_error.result()
            except Exception as e:
                for future in futures:
                    print(future.cancel()) # cancel all unstarted futures
                raise e