I know there are tons of questions like this one, I tried to read them all. What I'm doing is to use the multiprocessing library to parse web pages via Python Selenium. So, I have 3 lists to give to a function that processes them. First I write the function, then initiate the browser istance and lastly start the 3 processes.

import ...

def parsing_pages(list_with_pages_to_parse):
    global browser
    #do stuff

if __name__ == '__main__':
    browser = webdriver.Chrome(..., options = ...)
    browser.get(...)

    lists_with_pages_to_parse = [ [...], [...], [...] ]
    
    pool.mp.Pool(3)
    pool.map(parsing_pages, lists_with_pages_to_parse)
    pool.close
    pool.join

The error:

NameError: name 'browser' is not defined

Traceback (most recent call last):
  File "c:\Users\39338\Desktop\program.py", line 323, in <module>
    pool.map(parsing_pages, lists_with_pages_to_parse)
  File "C:\Users\39338\AppData\Local\Programs\Python\Python310\lib\multiprocessing\pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "C:\Users\39338\AppData\Local\Programs\Python\Python310\lib\multiprocessing\pool.py", line 771, in get
    raise self._value
NameError: name 'browser' is not defined

I used global to allow "browser" to be used inside the function. I thought the problem was that the function is written before I create "browser", but when I try to put it after the main part, I get the error that the function cannot be found when called.

CodePudding user response：

First thing first: always try to avoid using the global keyword. It causes instability i code as it gets longer and complex.

Anyways, your code says global is not defined because you don't have the global variable named browser defined outside of the function scopes.

Remove the global keyword. You don't need it since you are returning browser to the function itself.

don't forget to check out these resources:

NameError: global name 'browser' is not defined

https://python-forum.io/thread-12073.html

https://githubhot.com/repo/Matrix07ksa/Brute_Force/issues/24

https://github.com/MasonStooksbury/Free-Games/issues/41

CodePudding user response：

Calling this function when __name__ != '__main__' (from another: file, thread or process) will never initialize browser. Example:

def f():
    global browser
    browser

if __name__ == '__main__':
    browser = None

# Calling f will not raise an error
f()

def f():
    global browser
    browser

if __name__ != '__main__':
    browser = None

# Calling f will will now raise an error
f()

I think what's happening is you are making a pool and the pool runs parsing_pages() from another process where __name__ != '__main__'.

You need to do one of the following:

• Pass browser into your function as an argument
• Initialize browser outside of the if statement

You should add print(__name__) to check what it equals. It will probably return the name of your file, rather than __main__.