Home > Enterprise >  Correctly refer to a button to scrape aspx webpage using requests
Correctly refer to a button to scrape aspx webpage using requests

Time:12-30

I'm trying to extract data from an aspx site. Trying to find the name of the submit button to include in the post request, but it doesn't appear when I inspect the page.

import csv
import requests

form_data = {
    'CalendarioDesde': '01/11/2022',
    'CalendarioHasta': '28/12/2022',
    'IdEstacion': 'Aeropuerto San Luis (REM)',
    '': 'Consultar'   # trying to get the button name
}

response = requests.post('http://clima.edu.ar/InformePorPeriodo.aspx', data=form_data)

reader = csv.reader(response.text.splitlines())

Based on the posts I've seen, it's standard to use beautiful soup for scraping aspx pages, but I wasn't able to find a tutorial on the complex parameters (i.e. VIEWSTATE). So I'm starting with this basic requests version.

If this isn't the correct method, please suggest a post / resource that I can use as a template to extract data.

CodePudding user response:

Yesterday, I answered a question on how to get data loaded dynamically. See: enter image description here

  • Related