web scrap stock data from Reuters-CodePudding

I am a programming beginner and trying to extract key metric data (e.g. Beta) for a stock from Reuters. However, it always come back as blank.

my codes are like this:

from bs4 import BeautifulSoup as bs
import requests
import re

url = 'https://www.reuters.com/markets/companies/TSLA.OQ/key-metrics/price-and-volume'
page = requests.get(url)
bs1 = bs(page.text, 'html.parser')

beta=bs1.find_all('th', class_ ='text__text__1FZLe text__dark-grey__3Ml43 text__regular__2N1Xr text__body__yKS5U body__base__22dCE body__body__VgU9Q',text=re.compile('Beta'))
print(beta)

I know it is not correct but I cannot figure out what to do. please help. Ultimate I want to be extract the Beta info for a stock from Reuters. thank you for your help!!!

CodePudding user response：

Here's one way of collecting the data you need:

from bs4 import BeautifulSoup as bs 
import requests
import re

url = 'https://www.reuters.com/markets/companies/TSLA.OQ/key-metrics/price-and-volume'
page = requests.get(url)
soup = bs(page.text, 'html.parser')

# Locate the Table you wish to scrape
table = soup.select_one('table.table__table__2px_A')

# Locate the Keys and Value for each of the rows
keys = [i.text for i in table.select('tr th') if i]
values = [i.text for i in table.select('tr td') if i]

# Convert the two lists into a dictionary for a neater output
data = dict(zip(keys,values))

This will return:

{'% Change': '671.00',
 'Brent Crude Oil': '-1.40%Negative',
 'CBOT Soybeans': '1,626.00',
 'Copper': '111.91',
 'Future': '1,805.20',
 'Gold': '-0.57%Negative',
 'Last': ' 0.35%Positive'}

CodePudding user response：

You can scrape the site (without inspecting the javascript/json) using Selenium, using bs4 from my previous answer but you can use seleniums functions instead.

from selenium import webdriver
from bs4 import BeautifulSoup as bs


# Initiate webdriver
driver = webdriver.Firefox()

# Fetch the web page
driver.get('https://www.reuters.com/markets/companies/TSLA.OQ/key-metrics/price-and-volume')

# Convert the driver page source to a soup object
soup = bs(driver.page_source, 'html.parser')

# Find the table you want to scrape
table = soup.find('table', attrs={'aria-label':'KeyMetrics'})

# Locate the Keys and Value for each of the rows
keys = [i.text for i in table.select('tr th') if i]
values = [i.text for i in table.select('tr td') if i]

# Convert the two lists into a dictionary for a neater output
data = dict(zip(keys,values))

driver.quit()
print(data)

This will return:

{'Title': '681.79', 'Value': 'Jul 01', 'Price Closing Or Last Bid': '1,243.25', 'Pricing Date': 'Nov 04', '52 Week High': '620.50', '52 Week High Date': 'Jul 08', '52 Week Low': '31.62', '52 Week Low Date': '611.33', '10 Day Average Trading Volume': '706,600.80', '3 Month Average Trading Volume': '2.13', 'Market Capitalization': '1.24', 'Beta': '-7.51', '1 Day Price Change': '-37.14', '5 Day Price Return (Daily)': '-35.48', '13 Week Price Return (Daily)': '0.57', '26 Week Price Return (Daily)': '1.24', '52 Week Price Return (Daily)': '-35.48', 'Month To Date Price Return (Daily)': '4.08', 'Year To Date Price Return (Daily)': '-25.30', 'Price Relative To S&P500 (4 Week)': '-19.62', 'Price Relative To S&P500 (13 Week)': '13.57', 'Price Relative To S&P500 (26 Week)': '-19.62'}

To access the data within that dictionary you can search by key: data['Beta'] to show "-7.51".

CodePudding user response：

i want details more from Thailand