Variable don't get update on a for loop web scrapping selenium-CodePudding

I'm traying to filter web data but my variables don't update!

for i in range(len(links)):
    cupons = recupTahmin(links[i])
    try:
        for j in range(len(cupons)):
            eventName = cupons[j].find_element(by=By.XPATH, value="//a[@class='eventName ']").text
            eventDate = cupons[j].find_element(by=By.XPATH, value="//div[@class='eventDate']").text
            bahis = cupons[j].find_element(by=By.XPATH, value="//span[@class='type']").text
            tahmin = cupons[j].find_element(by=By.XPATH, value="//span[@class='choice  ']").text
            oran = cupons[j].find_element(by=By.XPATH, value="//div[@class='eventOutcome']").text
            predictions.append(cupon(eventName,eventDate, tahmin, oran, bahis))
    except:
        pass
for j in range(len(predictions)):
    print(vars(predictions[j]))

I already check cupons if all is okay. Cupons is a list of web element. The result of the code is -> my terminal print len(cupons) * predictions's first elemnent for i different event. Can someone help me ? I hope it's understable :)

CodePudding user response：

Did you define predictions somewhere above?

predictions = []

Did you check the except part if it actually lands there?

If you want, you could easier iterate through the list with

for c in cupons:
    bahis = c.find_element(...

CodePudding user response：

Looks like you have to make your XPath relative. This done by adding a dot . in front of the XPath expression, as following.
Without that dot driver will each time search for a match from the top of the page while when you use this dot making the XPath relative it will start searching from the current element, cupons[j] in your case.

for i in range(len(links)):
    cupons = recupTahmin(links[i])
    try:
        for j in range(len(cupons)):
            eventName = cupons[j].find_element(By.XPATH, ".//a[@class='eventName ']").text
            eventDate = cupons[j].find_element(By.XPATH, ".//div[@class='eventDate']").text
            bahis = cupons[j].find_element(By.XPATH, ".//span[@class='type']").text
            tahmin = cupons[j].find_element(By.XPATH, ".//span[@class='choice  ']").text
            oran = cupons[j].find_element(By.XPATH, ".//div[@class='eventOutcome']").text
            predictions.append(cupon(eventName,eventDate, tahmin, oran, bahis))
    except:
        pass
for j in range(len(predictions)):
    print(vars(predictions[j]))