Home > database >  Merge using threads not working in python
Merge using threads not working in python

Time:10-16

I have to merge two lists and every time a full the lists in order to merge them , but what is happening that I did it like this :

   def repeated_fill_buffer(self):
        """
      repeat the operation until reaching the end of file
        """
        # clear buffers from last data
        self.block = [[] for file in self.files]

        filling_buffer_thread = threading.Thread(self.fill_buffer())
        filling_buffer_thread.start()

        # create inverted index thread
        create_inverted_index_thread = threading.Thread(self.create_inverted_index())
        create_inverted_index_thread.start()

        # check if buffers are not empty to merge and start the thread
        if any(self.block):
            self.block = [[] for file in self.files]
            filling_buffer_thread.join()
            create_inverted_index_thread.join()

but what is happening that filling_buffer_thread and create_inverted_index_thread just called one time, and not working again, when I debugged the code I see that

filling_buffer_thread stopped

I don't know if I explain my question good, but what I want that I can called same thread multi time and run them..

CodePudding user response:

If there is any operation which is CPU Bound then, using thread is of no use. Because of Python GIL, which prevents multiple byte-code instruction to be executed at a time. use multiprocessing module since, every process has its own GIL.

All number crunching or any operations which depends on CPU for its completion are CPU-Bound. Threads are useful for I/O Bound Operations (like Database Calls, Network Calls)

To summarize your error, your filling_buffer_thread got blocked due to create_inverted_index_thread

  • Related