If you visit this site, https://www.premierleague.com/players, you will be able to see a list of players with their positions and Nationality. I am only required to scrape the position but I am unable to
playerss = driver.find_elements(By.XPATH, '//*[@id="mainContent"]/div[2]/div[1]/div/div/table/tbody/tr')
for player in playerss:
position = player.find_element(By.XPATH,'//[@id="mainContent"]/div[2]/div[1]/div/div/table/tbody/tr/td[2]')
print(position)
This is what I have but it is not working. If anyone knows how to help, I will appreciate it.
Thanks!
CodePudding user response:
This is how I was able to get the list of positions:
...
driver.get('https://www.premierleague.com/players')
positions = driver.find_elements(By.CSS_SELECTOR,
'table tbody tr td:nth-child(2)')
for pos in positions:
print(pos.text)
if you have to use xpath:
positions = driver.find_elements(By.XPATH, '//table/tbody/tr/td[2]')
CodePudding user response:
The position of the players are within:
<td >Midfielder</td>
where the <td>
is having class as hide-s
Solution
To print the positions you need to induce WebDriverWait for the presence_of_element_located() and you can use either of the following locator strategies:
Using text attribute:
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "table tbody tr td:nth-of-type(2)")))])
Using
innerHTML
:print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "table tbody tr td:nth-of-type(2)")))])
Using
innerText
:print([my_elem.get_attribute("innerText") for my_elem in WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "table tbody tr td:nth-of-type(2)")))])
Console Output:
['Midfielder', 'Defender', 'Defender', 'Forward', 'Forward', 'Forward', 'Defender', 'Goalkeeper', 'Forward', 'Defender', 'Defender', 'Defender', 'Defender', 'Defender', 'Midfielder', 'Defender', 'Defender', 'Midfielder', 'Defender', 'Defender', 'Defender', 'Midfielder', 'Goalkeeper', 'Midfielder', 'Midfielder', 'Midfielder', 'Defender', 'Midfielder', 'Forward', 'Forward']