Access the table hidden inside an iframe using python and selenium-CodePudding

I'm trying to access announcements table from this web page. The table is in an iframe, the contents of which are not visible in the source when the page loads. The table only shows up in source if I do inspect element "twice". Once the table is visible I can execute the below javascript code via chrome console to access the table.

document.getElementsByTagName('html')[0].getElementsByTagName('body')[0].getElementsByTagName('section')[4].getElementsByTagName('article')[0].getElementsByTagName('div')[2].getElementsByTagName('announcement_data')[0].getElementsByTagName('table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr')

However, I'm struggling to find a way to make the elements inside the iframe visible programmatically using python and selenium. I've tried to switch to the iframe but that has not helped.

seq = driver.find_elements_by_tag_name('iframe')
print("No of frames present in the web page are: ", len(seq))

iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to.frame(iframe)
table = driver.execute_script("return document.getElementsByTagName('html')[0].getElementsByTagName('body')[0].getElementsByTagName('section')[4].getElementsByTagName('article')[0].getElementsByTagName('div')[2].getElementsByTagName('announcement_data')[0].getElementsByTagName('table')[0].getElementsByTagName('tbody')[0].getElementsByTagName('tr');")

I get the following error if I try to run the above code in my jupyter notebook -

No of frames present in the web page are:  4

Error getting the length of the table: list index out of range

Any pointers to access the length and content of the table would be appreciated.

Thank you.

CodePudding user response：

You should be able to scrape the iframe url programmatically and then load that up as a new page in selenium.

https://www.asx.com.au/asx/v2/statistics/todayAnns.do is the url for the iframe.

To do it programmatically try something like this:

url = driver.find_element_by_class_name('external-form__iframe default').get_attribute("src")
driver.get(url)