Thread blocks the main thread-CodePudding

I'm not python programmer. But I need to write middleware for Openstack Swift. I don't know what environment the code running in. I have no control over the main thread.

I'm tyring to start sub-thread that will service the queue, but this thread blocks main thread. The main thread prints 1 and hangs. Pressing Ctrl C leads to contine execution, but sub-thread stops.

Where to dig?

class ProxyLoggingMiddleware(object):
    """
    Middleware that logs Swift proxy requests in the swift log format.
    """

    def __init__(self, app, conf, logger=None):
        self.queue = Queue(0)
        print "1\n"
        self.processor = self.init_queue_processor()
        print "2\n"


    def init_queue_processor(self):
        processor = threading.Thread(target=self.process_queue, args=(self.fifo_pipe_pathname, self.queue, self.logger))
        processor.setDaemon(True)
        processor.start()
        return processor

    @staticmethod
    def process_queue(fifo_pipe_pathname, queue, logger):
        json_encoder = json.JSONEncoder()
        while True:
            stat = queue.get(True) # <----------------------- Blocks here

UPD: I add next code in the middleware and it prints 1, 2, 3

def m():
    print "--1\n"
    time.sleep(3)
    print "--2\n"


t = threading.Thread(target=m)
t.daemon = True
t.start()

print "--3\n"

But should 1 3 2

THE PROBLEM OCCURS ONLY WHEN THIS CODE RUNS IN Openstack Swift environment!

UPD2:

/opt/swift # python --version
Python 2.7.18
/opt/swift # uname -a
Linux 45cefc56fd0a 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 Linux

UPD3:

I updated my code

    def __init__(self, app, conf, logger=None):

        print threading.currentThread()
        self.processor = self.init_queue_processor()

    def init_queue_processor(self):
        processor = threading.Thread(target=self.process_queue, args=(self.fifo_pipe_pathname, self.queue, self.logger))
        processor.setDaemon(True)
        processor.start()
        return processor

    @staticmethod
    def process_queue(fifo_pipe_pathname, queue, logger):
        print threading.currentThread()
        json_encoder = json.JSONEncoder()
        while True: ....

The output is

/opt/swift # /usr/bin/python /usr/local/bin/swift-proxy-server /etc/swift/proxy-server.conf
<_MainThread(MainThread, started 140613464898888)>


<_MainThread(MainThread, started 140613464898888)>

I see process_queue executing in main thread.

CodePudding user response：

A call to queue.get(False) will raise an Exception (Empty) if there are no data waiting on the queue.

There is no code shown that tries to handle that exception.

Therefore, as the exception is unhandled, the thread will terminate

CodePudding user response：

OMG, I understod. The Openstack Swift uses green threads from eventlet.

I need to use

from eventlet import sleep
from eventlet.green import threading

insetead of python standart threading, because it patched, how I understod.

so it's explains why the Main thread using everywhere.