Right now I'm trying to scrape the dividend yield from a chart using the following code.

df = pd.read_html('https://www.macrotrends.net/stocks/charts/BMO/Bank-of-Montreal/dividend-yield-history')

df = df[0].dropna()

But the code wont pick up the chart's data.

Any suggestions on pulling it from the website? Here is the specific link I'm trying to use: https://www.macrotrends.net/stocks/charts/BMO/Bank-of-Montreal/dividend-yield-history

I've used the code for picking up the book values but the objects they're using for the dividends and book values must be different.

Maybe I could use Beautiful Soup?

Thanks, R

CodePudding user response：

Sadly that website is rendered dynamically, so there's nothing in the html pandas is getting to scrape from. (The chart is loaded after the page). Scraping manually isn't going to help you here, because the data isn't there. (It's fetched after the page is loaded.)

You can either find an api which provides the data (best, quite possible given the content), work out where the page is fetching its data from and see if you can get it directly (better if possible), or use something like selenium to control a real browser, render the page, get the html, and then use that.

CodePudding user response：

Data is generating dynamically by javascript and it's a bit bad new that data is also not populating from backdoor meaning api calls json response. Actually data is in script tag with in a iframe.