operating system: windows 10

python 3.7.6

1. I have a task function

def mytest(task_queue):
    while True:
        print(task_queue.get())

2. I want run sub thread and waiting for others put something into task_queue.
3. If I use concurrent.futures.ThreadPoolExecutor().submit() start thread, then put something into queue, it will block, task_queue.put(1) never run.

if __name__ == '__main__':
    import queue
    task_queue = queue.Queue()
    task_queue.put(0)
    with concurrent.futures.ThreadPoolExecutor() as executor:
        executor.submit(mytest, task_queue)
    task_queue.put(1)
    task_queue.put(2)
# only print 0, then block 

4. If I start thread by Thread().start(), it works as I expect.

if __name__ == '__main__':
    import queue
    task_queue = queue.Queue()
    task_queue.put(0)
    t1 = threading.Thread(target=mytest, args=(task_queue,))
    t1.start()
    task_queue.put(1)
    task_queue.put(2)
# print 0, 1, 2. but the main thread does not exit

5. But I don't think either of these methods will block the code because they just start the thread.
6. So I have 2 question:
   • Why does submit() block the code?
   • Why main thread does not exit when use start() to start sub thread without join()?

THX

CodePudding user response：

Q-1) Why does submit() block the code?

A-1) No submit() method not blocking, it is schedules the callable mytest to be executed as mytest(task_queue) and returns a Future object. Look at below code, you will see that submit() method will not block the main thread

if __name__ == '__main__':
    import queue
    task_queue = queue.Queue()
    task_queue.put(0)
    executor = ThreadPoolExecutor()
    executor.submit(mytest, task_queue)
    task_queue.put(1)
    task_queue.put(2)


print("hello")

>> 0
   hello
   1
   2

Or you can do like :

if __name__ == '__main__':
    import queue
    task_queue = queue.Queue()
    task_queue.put(0)
    with concurrent.futures.ThreadPoolExecutor() as executor:
        executor.submit(mytest, task_queue)
        task_queue.put(1)
        task_queue.put(2)

You will see that task_queue.put(1) and other will be called immediately

As you see above examples, submit() method not blocking, but when you use with statement with concurrent.futures.Executor(), __exit__() method will be called end of the with statement. This __exit__() method will call shutdown(wait=True) method of Executor() class. When we look at the doc about shutdown(wait=True) method :

If wait is True then this method will not return until all the pending futures are done executing and the resources associated with the executor have been freed. If wait is False then this method will return immediately and the resources associated with the executor will be freed when all pending futures are done executing. Regardless of the value of wait, the entire Python program will not exit until all pending futures are done executing.

That's why your main thread blocked end of the with statement.

I want to give an answer to your second question, but i am confused with something about main thread exit or not. I will edit this answer later (for second question)

CodePudding user response：

Thread(...).start() creates a new thread. End of story. You can always create a new thread if there's still some memory left in which to create it.

executor.submit(mytest, task_queue) creates a new task and adds it to the task_queue. But adding the task to the queue will force the caller to wait until there is room for it in the queue.

Some time later, when the task eventually reaches the head of the queue, a worker thread will take the task and execute it.