How to kill multiprocess with selenium chrome driver-CodePudding

I'm terminating multiprocess thread after certain period and when I terminate it won't kill the chromedriver. This is a sample of the code (kept it simple):

def start_thread(url):
    driver = webdriver.Chrome()
    driver.get(url)

urls=[]
threads=[]
for counter in range(urls):
    thread = multiprocessing.Process( target=start_thread,name=f'start_thread_{counter}', args=[url])
    threads.append(thread)
    thread.start()

sleep(10)
for t in threads:
    t.terminate()

The thread is indeed terminated but the chromedriver isn't. I wanted to know if there's any option for me to kill the driver when I terminate the process.

CodePudding user response：

because ways to hijack the session from children seem to no-longer work, you are stuck with having to manually kill chrome and the webdriver and all its children using psutils, until they put sessions hijacking into the webdrivers.

import time
import multiprocessing
from selenium import webdriver
import psutil


def start_thread(url):
    driver = webdriver.Chrome()
    driver.get(url)
    time.sleep(30)


def get_all_children(proc: psutil.Process):
    try:
        if len(proc.children()) == 0:
            return []
        else:
            returned_list = []
            for item in proc.children():
                returned_list.append(item)
                returned_list.extend(get_all_children(item))
            return returned_list
    except psutil.NoSuchProcess:
        return []


if __name__ == "__main__":
    urls = ['http://www.python.org','http://www.python.org']
    threads = []
    for counter in range(len(urls)):
        thread = multiprocessing.Process(target=start_thread, name=f'start_thread_{counter}', args=(urls[counter],))
        threads.append(thread)
        thread.start()

    time.sleep(5)
    for t in threads:
        children = get_all_children(psutil.Process(pid=t.pid))
        t.terminate()
        for child in children:
            try:
                child.terminate()
            except psutil.NoSuchProcess:
                pass  # in case child was already dead

        time.sleep(5)  # to see what's happening

searching for children this way is somewhat necessary because chrome tends to open a lot of children, althought it might be limited to only 2 children on windows, this is may not be the case for other systems.