How can I extract the table from the given URL-CodePudding

I am trying to scrape the data from the table ETH ZERO SEK of the given URL, however I can't make it work. Does anyone have some advise how I can get it to work?

from selenium import webdriver
from selenium.webdriver.common.by import By

url = 'https://www.ngm.se/marknaden/vardepapper?symbol=ETH ZERO SEK'
driver = webdriver.Chrome()
driver.get(url)

element = driver.find_element(By.Xpath, './/*[@id="detailviewDiv"]/table/tbody/tr[1]/td/div')

CodePudding user response：

What happens?

Content you are looking for is provided via iframe, so you xpath won't work.

How to fix?

Option#1

Change your url to https://mdweb.ngm.se/detailview.html?locale=sv_SE&symbol=ETH ZERO SEK and call content directly

Option#2

Grab the source of iframe from your original url:

driver.get('https://www.ngm.se/marknaden/vardepapper?symbol=ETH ZERO SEK')

get the src of iframe that holds your table

iframe = driver.find_element(By.XPATH, '//iframe').get_attribute("src")

get the iframe

driver.get(iframe)

wait until your tbody of table is located and store it in element

element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//div[@id="detailviewDiv"]//thead[.//span[contains(text(),"Volym")]]/following-sibling::tbody')))

Assign values from cells to variables, by split of elements text:

volym = element.text.split('\n')[-3]
vwap = element.text.split('\n')[-2]

Note waits requires - from selenium.webdriver.support.ui import WebDriverWait