I want to scrape the opening odds of a certain bookmaker from this website:

I want to find by the name of a bookmaker and scrape his data-opening-odd

CodePudding user response：

took a bit of debugging, but the response is a dictionary with the only key being 'odds'. the value is the html. You'll see below where I put the value into the bs4 parser.

The odds are in a table, so I find all and loop through. The first and last elements don't seem to matter, so you'll see [1:-1] in the for loop.

def get_odds(ids):
  for id in ids:
    url = f'https://www.betexplorer.com{id}'
    matchup = url.split('/')[6]
    match_id = url.split('/')[7]  # <-- this is the last part of URL
    api_url = "https://www.betexplorer.com/match-odds/{}/1/1x2/".format(match_id)
    headers = {"Referer": "https://www.betexplorer.com",
                    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36'}
    s = requests.Session()
    s.headers.update(headers)
    response = s.get(api_url, headers=headers)
    soup = BeautifulSoup(response.json()['odds'],'html.parser')
    # print(soup)
    return soup

ids=['/soccer/argentina/primera-division-2016/gimnasia-l-p-colon-santa-fe/4YWZmAJt/']

for id in ids:
    soup = get_odds(ids)
    trs = soup.find_all('tr')
    for tr in trs[1:-1]:
        print(tr.find(class_="in-bookmaker-logo-link").text)
        print(tr.find( class_="table-main__detail-odds").get('data-opening-odd'))

10Bet
2.00
188BET
2.00
1xBet
1.92
888sport
1.83
bet-at-home
1.89
bet365
2.00
Betfair
2.00
Betsafe
2.00
Betsson
2.00
BetVictor
1.95
Betway
2.00
bwin
1.90
ComeOn
2.00
Interwetten
1.95
Pinnacle
2.06
SBOBET
2.05
Unibet
1.95
William Hill
2.00
youwin
1.85
Betfair Exchange
1.02