Selenium no data output-CodePudding

I've been struggeling for a while I am an beginner python learner and I want to webscrape a bookie (bwin) but i just cant scrape the data I want I only get a "Process finished with exit code 0" Nothing else

Here is the Code I wrote

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
import pandas as pd
import time
import pickle
import re
import unittest

options = Options()
options.headless = False
web = 'https://sports.bwin.be/nl/sports/live/voetbal-4'
path = r'C:\Users\Senne\Desktop\Chromedriver\chromedriver.exe'

options.add_argument('window-size=1920x1080')
driver = webdriver.Chrome(path, options=options)
driver.get(web)

Cookies = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="onetrust-accept-btn-handler"]')))
Cookies.click()

box = driver.find_element_by_xpath('//*[@id="main-view"]/ms-live/ms-live-event-list/div') #livebox
rows = WebDriverWait(box, 10).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'grid-event-wrapper')))

dropdowns = WebDriverWait(driver, 5).until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'grid-dropdown.ms-dropdown')))

Teams = driver.find_elements_by_xpath('//*[@id="main-view"]/ms-live/ms-live-event-list/div/ms-grid/ms-event-group[1]/ms-event/div/a/ms-event-detail/ms-event-name/ms-inline-tooltip/div/div[1]')
Teams_list = []
for p in range(len(Teams)):
    Teams_list.append(Teams[p].text)

driver.quit()

if __name__ == '__main__':
    unittest.main()

I don't know what I'm doing wrong

CodePudding user response：

Lots of fixes here but your code runs fine for me.

options = Options()
options.headless = False # Not needed. This is the default argument.

Next, get rid of options.add_argument('window-size=1920x1080'). Also i use webdriver-manager since it keeps updated automatically and you dont have to bother with getting the path every time.

from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
driver.set_window_size(1920,1080) # You dont need this

Next, i just made your Teams_list variable a generator instead of using a for loop. They are perfect in situations like this and a great tool for beginners to learn. Also, You don't need to use for p in range(len(Teams)) you can just use for team in Teams since you are not changing anything.

Teams = driver.find_elements_by_xpath('//*[@id="main-view"]/ms-live/ms-live-event-list/div/ms-grid/ms-event-group[1]/ms-event/div/a/ms-event-detail/ms-event-name/ms-inline-tooltip/div/div[1]')
Teams_list = [team.text for team in Teams]

Finally, You need to either print:

print(Teams_list)
driver.quit()

Or save to file.

with open("scrape.txt",'a') as file:
    file.write(Teams_list.__str__())