I'd always assumed a threading.Lock object would act as a mutex to prevent race conditions from occuring in a multithreading Python script, but I'm finding that either

1. my assumption is false (contradicting years of experience), or
2. Python itself has a bug (in, at the very least, versions 2.7-3.9) regarding this.

Theoretically, incrementing a value shared between two threads should be fine as long as you protect the critical section (ie the code incrementing that value) with a Lock, ie. mutex.

Running this code, I find mutexes in Python not to work as expected. Can anyone enlighten me on this?

#!/usr/bin/env python

from __future__ import print_function
import sys
import threading
import time

Stop = False

class T(threading.Thread):

  def __init__(self,list_with_int):
    self.mycount = 0
    self.to_increment = list_with_int
    super(T,self).__init__()

  def run(self,):
    while not Stop:
      with threading.Lock():
        self.to_increment[0]  = 1
      self.mycount  = 1

intList = [0]
t1 = T(intList)
t2 = T(intList)
t1.start()
t2.start()

Delay = float(sys.argv[1]) if sys.argv[1:] else 3.0

time.sleep(Delay)

Stop = True
t1.join()
t2.join()
total_internal_counts = t1.mycount   t2.mycount

print("Compare:\n\t{total_internal_counts}\n\t{intList[0]}\n".format(**locals()))

assert total_internal_counts == intList[0]

CodePudding user response：

It's possible that this is the answer, the lock object must be persistent and shared amongst the threads.

This works :

#!/usr/bin/env python

from __future__ import print_function
import sys
import threading
import time

Stop = False


class T(threading.Thread):

  lock = threading.Lock()
  def __init__(self,list_with_int):
    self.mycount = 0
    self.to_increment = list_with_int
    super(T,self).__init__()

  def run(self,):
    while not Stop:
      with self.lock:
        self.to_increment[0]  = 1
      self.mycount  = 1

intList = [0]
t1 = T(intList)
t2 = T(intList)
t1.start()
t2.start()

Delay = float(sys.argv[1]) if sys.argv[1:] else 3.0

time.sleep(Delay)

Stop = True
t1.join()
t2.join()
total_internal_counts = t1.mycount   t2.mycount

print("Compare:\n\t{total_internal_counts}\n\t{intList[0]}\n".format(**locals()))

CodePudding user response：

It's possible that...the lock object must be persistent and shared amongst the threads

That's exactly right. threading.Lock() is a constructor call. The run loop in your original example created a new Lock object for each iteration:

while not Stop:
    with threading.Lock():         //creates a new Lock object each time 'round
        self.to_increment[0]  = 1

You fixed it by creating a single Lock object that is used by all the threads and, by every iteration of the loop in each thread.

class T(threading.Thread):

    // create one Lock object that will be shared by all instances of
    // class T
    //
    lock = threading.Lock()

    def run(self,):    
        while not Stop:
            with self.lock:    // lock and release the one Lock object.
                self.to_increment[0]  = 1

When you lock a Lock object, the only thing that it prevents is, it prevents other threads from locking the same Lock at the same time.