My goal is to run an SQL engine in one process and the query, data entry and other actions in other at least two separate multiple processes.

I get an error with the shared variable

ENGINE

as follows:

import pandas as pd
import time 
from sqlalchemy import create_engine
from multiprocessing import Process
from multiprocessing.sharedctypes import Value, Array, RawArray


def sql(ENGINE):
    print("Starting SQL ENGINE ...")
    engine = create_engine('sqlite:///TestDB.db') 
    print(f" {engine} Type: {type(engine)}")
    ENGINE.value = f"{engine}" # because of "TypeError: unicode string expected instead of instance ENGINE"
    print(ENGINE.value)

def test(ENGINE):
    time.sleep(1)
    try:
        print(f" Type: {ENGINE.value}")
        print(pd.read_sql('TEST', ENGINE.value))
    except Exception as e:
        # import sys
        print(f"ERROR: {e} ")

def main():
    
    ENGINE = RawArray('u', 99) # Number is the length of the array
    p1 = Process( target=sql, args=(ENGINE, ) )
    p3 = Process( target=test, args=(ENGINE,) ) 
    p1.start()
    p3.start()

if __name__ == "__main__":
    main()

Output (and error):

Starting SQL ENGINE ...
 Engine(sqlite:///TestDB.db) Type: <class 'sqlalchemy.engine.base.Engine'>
Engine(sqlite:///TestDB.db)
 Type: Engine(sqlite:///TestDB.db)
ERROR: Could not parse rfc1738 URL from string 'Engine(sqlite:///TestDB.db)'

As it it evident, the shared variable ENGINE is a string, converted from a class. I had to do this (ENGINE.value = f"{engine}"), because I get an error that it can't use classes as shared variable.

TypeError: unicode string expected instead of Engine instance

Is there way to either

• 1. make it to the class again or
• 2. to share a class variable

among other processes?

CodePudding user response：

Turns out simply re-mounting the database for every instance is the solution!

Also this was very helpful: Python share objects between independent processes