not getting table data printed-CodePudding

code

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
#from selenium.webdriver.support.ui import Select
import time


#driver path
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH) 


driver.get("https://www.adamchoi.co.uk/overs/detailed")
print(driver.title)


all_matches=driver.find_element(By.XPATH,"//label[@analytics-event='All matches']")
all_matches.click()

driver.find_element(By.XPATH,"//select[@id='season']/option[@label='20/21']").click()


matches=driver.find_elements(By.XPATH,'//tr')

for match in matches:
    print(match.text)


#driver.quit()

terminal

DevTools listening on ws://127.0.0.1:61934/devtools/browser/30c3f68e-40f3-4a78-b193-d358ba7bbee2
Overs - Total Goals
PS C:\Users\mrmad\Desktop\selenium test> [9208:2332:0731/115957.877:ERROR:device_event_log_impl.cc(214)] [11:59:57.877] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[9208:2332:0731/115957.879:ERROR:device_event_log_impl.cc(214)] [11:59:57.878] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)

using

chrome - 103.0.5060.134
chromewebdriver - 103.0.5060.134
python- 3.10.5
selenium - 4.3.0

code till select option 20/21 is working after that i have selected tag (also cross checked in inspect that have around 760 result so i guess find elements statement is not problem ) please help me print all data in terminal

CodePudding user response：

You can select date dropdown option using Select(). After that, the selected date contains 20 tables. I pull the first table data as an example using selenium with pandas.

import pandas as pd
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
options = Options()
options.add_argument("--window-size=1920,1200")


s = Service("./chromedriver") ## path to where you saved chromedriver binary
driver = webdriver.Chrome(service=s, options=options)

url = 'https://www.adamchoi.co.uk/overs/detailed'
driver.get(url)

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[@analytics-event='All matches']"))).click()

date=WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '(//*[@])[3]/select')))
dropdown= Select(date)
dropdown.select_by_value('object:53')

table_1 = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, '(//table)[1]'))).get_attribute('outerHTML')
df=pd.read_html(table_1)[0]
print(df)
driver.quit()

Output:

0          1              2                     3
0   13-08-2021       Brentford  2 - 0         Arsenal
1   22-08-2021         Arsenal  0 - 2         Chelsea
2   28-08-2021        Man City  5 - 0         Arsenal
3   11-09-2021         Arsenal  1 - 0         Norwich
4   18-09-2021         Burnley  0 - 1         Arsenal
5   26-09-2021         Arsenal  3 - 1       Tottenham
6   02-10-2021        Brighton  0 - 0         Arsenal
7   18-10-2021         Arsenal  2 - 2  Crystal Palace
8   22-10-2021         Arsenal  3 - 1     Aston Villa
9   30-10-2021       Leicester  0 - 2         Arsenal
10  07-11-2021         Arsenal  1 - 0         Watford
11  20-11-2021       Liverpool  4 - 0         Arsenal
12  27-11-2021         Arsenal  2 - 0       Newcastle
13  02-12-2021      Man United  3 - 2         Arsenal
14  06-12-2021         Everton  2 - 1         Arsenal
15  11-12-2021         Arsenal  3 - 0     Southampton
16  15-12-2021         Arsenal  2 - 0        West Ham
17  18-12-2021           Leeds  1 - 4         Arsenal
18  26-12-2021         Norwich  0 - 5         Arsenal
19  01-01-2022         Arsenal  1 - 2        Man City
20  23-01-2022         Arsenal  0 - 0         Burnley
21  10-02-2022          Wolves  0 - 1         Arsenal
22  19-02-2022         Arsenal  2 - 1       Brentford
23  24-02-2022         Arsenal  2 - 1          Wolves
24  06-03-2022         Watford  2 - 3         Arsenal
25  13-03-2022         Arsenal  2 - 0       Leicester
26  16-03-2022         Arsenal  0 - 2       Liverpool
27  19-03-2022     Aston Villa  0 - 1         Arsenal
28  04-04-2022  Crystal Palace  3 - 0         Arsenal
29  09-04-2022         Arsenal  1 - 2        Brighton
30  16-04-2022     Southampton  1 - 0         Arsenal
31  20-04-2022         Chelsea  2 - 4         Arsenal
32  23-04-2022         Arsenal  3 - 1      Man United
33  01-05-2022        West Ham  1 - 2         Arsenal
34  08-05-2022         Arsenal  2 - 1           Leeds
35  12-05-2022       Tottenham  3 - 0         Arsenal
36  16-05-2022       Newcastle  2 - 0         Arsenal
37  22-05-2022         Arsenal  5 - 1         Everton