Threads reading the same line from one file-CodePudding

I'm trying to use threads with python, I'm pretty new to threads. I wanted the threads to read random lines from the same file but all the threads read the same line. So the file I'm trying to read has all the lines in email:pass:another_line format. I expected to read different lines from the same file with multiple threads but its reading the same line with multiple threads. So for example 1 thread will return line1, second thread will return line2 but its random lines.

import random
import threading

def email_pass_token():
    global email, pass2, token

    file = open("testing/pokens.csv").read().splitlines()
    acc_str = random.choice(file)

    num_lines = sum(1 for _ in file)
    print(num_lines)

    email = ":".join(acc_str.split(":", 1)[:1])

    pass2 = ":".join(acc_str.split(":", 2)[:2][1:])

    token = ":".join(acc_str.split(":", 3)[:3][2:])
email_pass_token()

def gen_acc():
    print(email, pass2, token)

threads = []
num_thread = input("Threads: ")
num_thread = int(num_thread)
for i in range(num_thread):
    t = threading.Thread(target=gen_acc)
    threads.append(t)
    t.start()

File Sample:

[email protected]:#354946345e696$e30*417:another_line1
[email protected]:2e5548c543709!8@305-8(:another_line2
[email protected]:41c!954=7543cc^1#48fd_$*b5:another_line3
[email protected]:1f@e54d78^feb54355&6$50:another_line4
[email protected]:#3946345e696$e30*417:another_line5
[email protected]:2e58c5437709!8@305-8(:another_line6
[email protected]:41c!9=7543cc^1#48fd_$*b5:another_line7
[email protected]:1f@ed78^feb53455&6$50:another_line8

CodePudding user response：

You are not running the function for selecting line in the threads. What you are doing is you are running gen_acc in the threads which only prints the three variables email, pass2 and token. Your code for selecting lines resides email_pass_token which is called only once.

CodePudding user response：

Here's one way you could do it. Note that this will not work if the password contains a colon. Also, I've hard-coded the number of threads but you get the idea.

import random
from concurrent.futures import ThreadPoolExecutor

FILENAME = '/Users/andy/pokens.csv'
NTHREADS = 5

def myfunc():
    with open(FILENAME) as infile:
        lines = infile.readlines()
        line = random.choice(lines).strip()
        tokens = line.split(':')
        return ' '.join(tokens)

def main():
    with ThreadPoolExecutor() as executor:
        futures = []
        printable = set()
        for _ in range(NTHREADS):
            future = executor.submit(myfunc)
            futures.append(future)
        for future in futures:
            printable.add(future.result())
        for p in printable:
            print(p)

if __name__ == '__main__':
    main()

CodePudding user response：

Short answer

You have to call email_pass_token inside the thread worker, i.e. in your case the function gen_acc:

import random
import threading

def email_pass_token():
    global email, pass2, token

    file = open("testing/pokens.csv").read().splitlines()
    acc_str = random.choice(file)

    num_lines = sum(1 for _ in file)
    print(num_lines)

    email = ":".join(acc_str.split(":", 1)[:1])

    pass2 = ":".join(acc_str.split(":", 2)[:2][1:])

    token = ":".join(acc_str.split(":", 3)[:3][2:])

def gen_acc():
    email_pass_token()
    print(email, pass2, token)

threads = []
num_thread = input("Threads: ")
num_thread = int(num_thread)
for i in range(num_thread):
    t = threading.Thread(target=gen_acc)
    threads.append(t)
    t.start()

Long answer

In your code, you only execute email_pass_token() once, which reads the file one time only. The threads you create execute the gen_acc() function which only prints the result; it does not invoke file reading again.

In addition, there are a lot of improvements to make in your code. For instance, you have a file descriptor leak because you open the file and never close it. Also, it is almost always wrong to parse csv files manually, you can use the csv module in the standard library to do that. Here is an improved version of your code:

Edit: fix the race condition due to global variables access by multiple threads

import csv
import random
import threading

def email_pass_token():
    with open("testing/pokens.csv") as file:  # `with open ...` automatically closes the file
        reader = csv.reader(file, delimiter=':')
        return random.choice(list(reader))

def gen_acc():
    email, pass2, token = email_pass_token()
    print(email, pass2, token)

threads = []
num_thread = int(input("Threads: "))
for i in range(num_thread):
    t = threading.Thread(target=gen_acc)
    threads.append(t)
    t.start()
for thread in threads:
    thread.join()