I need to web scrap a particular value from a page which is contained in a table-CodePudding

I want to scrap net sales value for Dec 2021 that is contained in a table from a webpage. I am using simple beautifulsoup module.I have included the python code I am using for extracting some of the other values.I want to extract the value 9644.8. The code for the webpage is give below

<table >
                            <thead>
                                <tr>
                                    
                                            <th scope="col">PARTICULARS</th>
                                        
                                            <th scope="col">Dec 2020</th>
                                        
                                            <th scope="col">Mar 2021</th>
                                        
                                            <th scope="col">Jun 2021</th>
                                        
                                            <th scope="col">Sep 2021</th>
                                        
                                            <th scope="col">Dec 2021</th>
                                        
                                </tr>
                            </thead>
                            <tbody>
                                
                                        <tr >
                                            <th scope="row">Net Sales <span  data-tooltip="tooltip" title="" data-original-title="It is companys core revenue net of discounts and returns."><svg  aria-hidden="true" focusable="false" data-prefix="fas" data-icon="info-circle" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" data-fa-i2svg=""><path fill="currentColor" d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 110c23.196 0 42 18.804 42 42s-18.804 42-42 42-42-18.804-42-42 18.804-42 42-42zm56 254c0 6.627-5.373 12-12 12h-88c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h12v-64h-12c-6.627 0-12-5.373-12-12v-24c0-6.627 5.373-12 12-12h64c6.627 0 12 5.373 12 12v100h12c6.627 0 12 5.373 12 12v24z"></path></svg><!-- <i ></i> --></span></th>
                                            <td>
                                                <span  value="10824.4">10,824.40</span>
                                                

                                            </td>
                                            <td>
                                                <span  value="9530.9">9,530.90</span>
                                                
                                            </td>
                                            <td>
                                                <span  value="9088.2">9,088.20</span>
                                                
                                            </td>
                                            <td>
                                                <span  value="9321.5">9,321.50</span>
                                                
                                            </td>
                                            <td>
                                                <span  value="9644.8">9,644.80</span>
                                                
                                            </td>
                                            
                                        </tr>

'''

import pandas as pd
from bs4 import BeautifulSoup
import requests
a=input("Enter symbol of the company\n")
url="https://ticker.finology.in/company/" a
print(url)
response=requests.get(url)
soup=BeautifulSoup(response.text,'html.parser')
CP = soup.find("div", {"id":"mainContent_clsprice"}).find("span", {"class": "Number"}).getText()

''' The ticker I am using is IDEA

CodePudding user response：

You can pass the URL directly to pandas.read_html() which will return a list of dataframes of tables found.

>>> dfs = pd.read_html('https://ticker.finology.in/company/idea')
>>> dfs[1]
          PARTICULARS  Dec 2020  Mar 2021  Jun 2021  Sep 2021  Dec 2021
0           Net Sales  10824.40   9530.90   9088.20    9321.5    9644.8
1   Total Expenditure   6717.40   5296.20   5529.30    5672.9    5995.2
2    Operating Profit   4107.00   4234.70   3558.90    3648.6    3649.6
3        Other Income     36.30     31.10     29.20      22.8      25.2
4            Interest   4782.60   4711.00   5223.20    5112.8    5324.7
5        Depreciation   5638.90   5629.50   5831.90    5743.8    5550.5
6   Exceptional Items   -439.50   -972.60     51.30      13.5      11.6
7   Profit Before Tax  -6717.70  -7047.30  -7415.70   -7171.7   -7188.8
8                 Tax      0.00    -20.80      0.00       0.0       0.0
9    Profit After Tax  -6717.70  -7026.50  -7415.70   -7171.7   -7188.8
10  Adjusted EPS (Rs)     -2.34     -2.45     -2.58      -2.5      -2.5

CodePudding user response：

Provided you have located the correct table you can use select_one and the css selector tbody td:last-child > .Number. tbody will restrict to the table body rows, select_one with td:last-child will grab the first last column, then the > .Number will grab the direct child span. You could of course, combine and shorten the selector list rather than separately grabbing table, however, I was unsure if you would do other things with the table.

from bs4 import BeautifulSoup as bs
import requests

r = requests.get('https://ticker.finology.in/company/idea', headers = {'User-Agent':'Mozilla/5.0'})
soup = bs(r.content, 'lxml')
table = soup.select_one('.card:has(> #mainContent_hfSizeForQuarter) table')
table.select_one('tbody td:last-child > .Number').text.strip()