Selenium WebDriverWait exception with Twitter-CodePudding

I am trying to find any new Tweets from a certain account as they're posted. I have the following code:

while True:
    browser.get("https://twitter.com/"   twitter_username)
    try:
       WebDriverWait(browser, 10).until(lambda x: x.find_element(by=By.XPATH, value='//article[@data-testid="tweet"]'))
    except Exception as e:
        print(e)
        return
     
    # More code to get the newest Tweet

This all works for the first ~10-20 minutes (web page loads properly and I get the newest Tweet), but then it starts throwing the following exception:

Message:
Stacktrace:
#0 0x5583bfb217d3 <unknown>
#1 0x5583bf87d688 <unknown>
#2 0x5583bf8b3c21 <unknown>
#3 0x5583bf8b3de1 <unknown>
#4 0x5583bf8e6d74 <unknown>
#5 0x5583bf8d16dd <unknown>
#6 0x5583bf8e4a0c <unknown>
#7 0x5583bf8d15a3 <unknown>
#8 0x5583bf8a6ddc <unknown>
#9 0x5583bf8a7de5 <unknown>
#10 0x5583bfb5249d <unknown>
#11 0x5583bfb6b60c <unknown>
#12 0x5583bfb54205 <unknown>
#13 0x5583bfb6bee5 <unknown>
#14 0x5583bfb48070 <unknown>
#15 0x5583bfb87488 <unknown>
#16 0x5583bfb8760c <unknown>
#17 0x5583bfba0c6d <unknown>
#18 0x7fbd0c6b76db <unknown>

Any help would be much appreciated. Thanks.

CodePudding user response：

at the end of each loop add:

browser.refresh()

CodePudding user response：

Looks like the memory consumption of your driver groves up with each loop iteration.
You say after 10-20 minutes your script stops working correctly?
Let's say your code works correct for about 2000 iterations? In this case add a counter to your look so that after 500 iterations You will dispose the currently used driver object instance and create a new one, something like this:

counter = 0
while True:
    counter = counter   1
    if(counter>500):
        browser = webdriver.Chrome(executable_path='chromedriver.exe')
        counter = 0
    browser.get("https://twitter.com/"   twitter_username)
    try:
       WebDriverWait(browser, 10).until(lambda x: x.find_element(by=By.XPATH, value='//article[@data-testid="tweet"]'))
    except Exception as e:
        print(e)
        return