I have a very large project of a Web API using Flask and Python. It is used for testing some electronic hardware automatically. The program uses some threading in order to run a web UI while a server runs some services (SSH, serial, VISA) among others.
The program was originally coded in python 2.7 and works just fine with this version. Right now, I am trying to update it to python 3.8 for obvious reasons.
As I am updating the project, I'm having trouble with the copy library. It is supposed to serialize a _thread.RLock object and to send it to another thread, but it keeps giving me an error. Here is the traceback that I get:
Traceback (most recent call last):
File "c:\git_files\[...]\nute\route_config\flask_api_testbench.py", line 208, in _hook_run
super(FlaskAPITestbench, self).hook_run()
File "c:\git_files\[...]\nute\core\testbench\base.py", line 291, in hook_run
while self.state_machine():
File "c:\git_files\[...]\nute\core\testbench\base.py", line 304, in state_machine
on_input=self.state_testrun
File "c:\git_files\[...]\nute\core\testbench\base.py", line 380, in wait_for_input_or_testrun
self.hook_load_testrun(config_with_input)
File "c:\git_files\[...]\nute\core\testbench\base.py", line 428, in hook_load_testrun
self.interface.load_testrun(self.load_testrun(config))
File "c:\git_files\[...]\nute\core\testbench\base.py", line 461, in load_testrun
testrun = self.test_loader.load_testrun(config, context_type=self.TestRunContext)
File "c:\git_files\[...]\nute\core\testrun\loader.py", line 89, in load_testrun
testrun_template = process_all_loaders(self.batchers, _process_batcher)
File "c:\git_files\[...]\nute\core\config\loader.py", line 127, in process_all_loaders
return fn(loader)
File "c:\git_files\[...]\nute\core\testrun\loader.py", line 85, in _process_batcher
batcher.batch_testrun(testrun_template, config, context)
File "c:\git_files\[...]\nute\batcher\python_module_batcher.py", line 21, in batch_testrun
batch_module.main(testrun, context)
File "C:\GIT_Files\[...]\pyscripts\script\patest\_batch.py", line 168, in main
test.suite(ImpedanceTest)
File "c:\git_files\[...]\nute\core\testrun\base.py", line 213, in suite
testsuite = testsuite_instance_or_class()
File "c:\git_files\[...]\nute\core\functions\helpers.py", line 233, in __new__
cls._attach_nodes_to(template)
File "c:\git_files\[...]\nute\core\functions\helpers.py", line 271, in _attach_nodes_to
node = root.import_testcase(testcase)
File "c:\git_files\[...]\nute\core\functions\specific.py", line 307, in import_testcase
test_node = testcase.copy(cls=self.__class__)
File "c:\git_files\[...]\nute\core\functions\base.py", line 645, in copy
value = copy(value)
File "c:\users\[...]\.conda\envs\py37\lib\copy.py", line 96, in copy
rv = reductor(4)
TypeError: can't pickle _thread.RLock objects
It works fine in Python 2.7, but not with Python 3.x. I've tried it on 3.7.10, 3.8.9 and 3.9.6 with the same result.
Here's the implementation of my wrap method of copy:
from copy import copy
...
def copy(self, cls=None): # class method
if cls is None:
cls = self.__class__
new_self = cls()
for key, value in self.__dict__.items():
# if key == "multithread_lock":
# continue
if self.should_copy_attribute(key, value):
# Handle recursion by pointing to the new object instead of copying.
if value is self:
value = new_self
else:
value = copy(value) # This is where it fails
new_self.__dict__[key] = value
return new_self
As you can see with the commented part, skipping the pickling of any _thread.RLock object makes the program work, but I need to refresh the web UI manually to see it running since the thread doesn't work.
Any idea why it's working on python 2.7 but not on newer versions? Thanks in advance.
CodePudding user response:
So I found out that a _thread.RLock() object cannot be copied. I just added a condition to skip an object like this to be copied, and it works fine. For the web UI not refreshing, I changed to a lower version of Flask-CoketIO and it worked just fine.