How to invoke concurrent.futures with a function that uses two arguments (one List argument and one-CodePudding

I've built a function where I want to leverage some multi-threading in Python. This function takes two arguments: A List type of argument (where I leverage multi-threading) and a non iterable variable. The way that I've coded is not working and I don't know how to invoke it

("ssns" variable would be a List of ssn and cursor is simply the cursor used for writing in SQL Server DB). Thank you!

def main(ssns):

t1 = time.perf_counter()
create_folders()

conn = pymssql.connect(db.server, db.user, db.password, db.db)
cursor = conn.cursor(as_dict=True)

with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(f.get_pdf_multi_thread, ssns, cursor)

conn.commit()
cursor.close()
conn.close()

CodePudding user response：

If I'm understanding the question correctly:

You have the function f.get_pdf_multi_thread which takes 2 arguments: a List of SSNs and a cursor. You want to call f.get_pdf_multi_thread(ssns, cursor), using the ThreadPoolExecutor class for asynchronicity.

Answer: I think you want to either:

(1) use executor.submit instead of executor.map in order to accept multiple arguments, or

(2) change your function to accept two non-iterables, e.g. f.get_pdf_single(ssn, cursor), and use an anonymous (lambda) function within the call to executor.map:

results = executor.map(lambda s, c: f.get_pdf_single(s, c), ssns)

The method executor.map actually does the work of making multiple calls to the given function AND distributing them across threads for you. The function passed in should be a function that takes one item of your iterable as an argument.