Python: What's the best way to block a thread so that it can be interrupted?-CodePudding

I'm writing the server part of a simple chat program that is composed of different threads: One called accept_thread listening for new connections from clients, one thread for every client that receives the next message and forwards it to all the other clients, and lastly the main thread that should simply wait for me giving a KeyboardInterrupt or pressing ENTER in the console when I want to stop the server, subsequently closing all the sockets and thereby interrupting all the other threads which are probably blocked in socket.accept() or socket.recv(). Also accept_thread should be able to somehow notify the main thread in case of an error so that the main thread can gracefully stop everything.

Of course I want the main thread to not waste any resources by running in an infinite loop, so I tried different ways of blocking it like input() or accept_thread.join(). But with input() it can't be interrupted by accept_thread and with accept_thread.join() it doesn't respond to KeyboardInterrupts. The only working solutions I found are using join(timeout) or sleep(timeout) where accept_thread would signal an error to the main thread by simply terminating:

try:
  while accept_thread.is_alive():
    accept_thread.join(1)
except KeyboardInterrupt:
  # ...

try:
  while accept_thread.is_alive():
    time.sleep(1)
except KeyboardInterrupt:
  # ...

But these don't seem like good solutions to me because the main thread is still taking up resources. Even if I used a very large number for the timeout like sleep(1000000) there still has to be something going on in the background that repeatedly checks if the timeout is due or not which would take up resources.

So is there a better way to do this?

CodePudding user response：

You can't interrupt blocked python thread directly as i know. But we can shutdown the socket object, this will cause throw an OSError exception in a thread which has doing socket operation, we can catch this exception and we can exit the subthread. If some other exceptions occurs in subthread/subthreads we need to notify this exception to the main thread, we do this with event.set() that will wakeup the main thread. Below solution will not consume any CPU resource in the main thread until event.wait() return

You can modify below example according to your case

import socket,threading,signal,os,sys

def accept_thread():
    try:

        port = 12345               

        s.bind(('', port))        

        s.listen(1)    

        c, addr = s.accept() # Blocking I/O operation
    
    except OSError:
        print("error occured when do blocking I/O")
        # do your thread cleanup operation here, and after that 
        if STOP:
            sys.exit() # will kill only calling thread

    except Exception: # Any error occured here, so we need to notify the main thread using event.set()
         # cleanup operation and then notify the main thread
         event.set() #notify the main thread, main thread will wakeup and event.wait() will return 

event=threading.Event()
STOP=False
s=socket.socket() # This must be declared in the main thread because we will do s.shutdown() in main thread

t=threading.Thread(target=accept_thread)        
t.start()

try:
    event.wait() # wait for accept_thread that will call the event.set()
    print("Error occured in subthread")
    s.shutdown(how) # to be kill the other threads
    s.close()
    # cleanup operation

except KeyboardInterrupt:
    print("Interrupted, shutting down the socket object")
    STOP=True

    """
    s.shutdown(how) --> This will shutdown the connection but will not close the socket object.
    When we shutdown the socket object, the OSError will be raised 
    in any thread which has any operation on this socket object
    """

    s.shutdown(how) # This will cause an OSError exception in subthread
    s.close()
    # cleanup operation

You need to give how parameter to shutdown(how) method like socket.SHUT_RD, this is described in the doc:

socket.shutdown(how) Shut down one or both halves of the connection. If how is SHUT_RD, further receives are disallowed. If how is SHUT_WR, further sends are disallowed. If how is SHUT_RDWR, further sends and receives are disallowed.

If i am wrong somewhere, please correct me

CodePudding user response：

It's hard to answer without knowing how the threads interact. However, here is a stab at it.

threading.excepthook() is a module level functions that gets called whenever an exception is raised in a thread. By default, it ignores SystemExit and prints a message for other Exceptions. But, you can provide your own excepthook function to handle things differently.

threading.Event() provides an easy way to send a simple on/off signal to a thread.

Combine these two and you can cause the accept and client threads to stop when a KeyboardInterrupt is raised.

import threading

# set this event to signal the threads to stop
done = threading.Event()

# insert our custom exception hook
oldhook = threading.excepthook

def new_excepthook(args, /):
    # signal the threads to shutdown on Ctrl-C
    # otherwise call the old excepthook
    if args.exc_type == KeyboardInterrupt:
        done.set()

    else:
        oldhook(args)
    
threading.excepthook = new_excepthook



def accept_target():
    # thread start up here ...

    while not done.is_set():
        time.sleep(0.3)
        # thread working code here ...
        # perhaps starting client threads?

    # thread shutdown code here ...
    # perhaps joining on the client threads?        


def client_target():
    # thread start up here ...

    while not done.is_set():
        time.sleep(0.1)
        # thread working code here ...

    # thread shutdown code here ...        

    
accept_thread = threading.Thread(name='accept', target=accept_target)
accept_thread.start()

accept_thread.join()
# perhaps join client threads if not done in accept_target

On receiving a Cntl-C, new_eventhook() should be called. Because it is a KeyboardInterrupt, the done event is set. This causes all the threads to exit their while not done.is_set(): ... loops. When all the threads end, accept_thread.join() is unblocked and the program ends.