Retrieving td value from tr that has certain other td value-CodePudding

Need to get the links from a td in rows that has a certain td value.

this is a tr in the table and I want to get the link from the div "Match" if the div "Home team" is of a certain value. There are many rows and I want to find every link that is matching. I have tried this and every time I only get the first row of the table. Here is the link https://wp.nif.no/PageTournamentDetailWithMatches.aspx?tournamentId=403373&seasonId=200937&number=all . Note that I translated some of the values to English in the examples below

homegames = browser.find_elements_by_xpath('//div[@data-title = "Home team"]/a[text()="Cleveland"]//parent::div//parent::td//parent::tr')

for link in homegames:
    print(link.find_element_by_xpath('//td[3]/div/a').get_attribute('href'))

<td><div data-title="Date">23.10.2021</div></td>
<td><div data-title="Tid">16:15</div></td>
<td>div data-title="Matchnr">
        <a href="/MatchDetails?id=743062">2121503051</a>
</div>
</td><td><div data-title="Home team"><a href="/PageOrgDetail.aspx?LinkId=86613" title="Cleveland Basket - MEN 4">Cleveland</a></div></td>
<td><div data-title="Away team">
        <a href="/PageOrgDetail.aspx?LinkId=890549" title="SSK Ohio Basketball - MEN 3">Ohio Travellers</a></div></td>
<td><div data-title="Court"><a href="">F21</a></div></td><td><div data-title="Result">71 - 64</div></td>
<td><div data-title="Referee">John Doe<br>Will Smith<br></div></td></tr>```

CodePudding user response：

The data is within the html source (so no need to use Selenium). But regardless of using Selenium or not, what you can do here is let BeautifulSoup find the specific tags you are after.

Without Selenium, it requires a little manipulation as decode the html.

import requests
from bs4 import BeautifulSoup
import json
import html

keyword = 'Askim'

url = 'https://wp.nif.no/PageTournamentDetailWithMatches.aspx?tournamentId=403373&seasonId=200937&number=all'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

jsonStr = soup.find('div', {'class':'xwp_table_bg'}).find_next('input')['value']
jsonData = json.loads(jsonStr)

links_list = []
for each in jsonData['data']:
    #each = jsonData['data'][6]
    htmlStr = ''.join(each)
    htmlStr = html.unescape(htmlStr)
    soup = BeautifulSoup(htmlStr, 'html.parser')

    if soup.find('div', {'data-title':'Hjemmelag'}, text=keyword):
        link = soup.find('div', {'data-title':'Kampnr'}).find('a')['href']
        links_list.append(link)