Home > Enterprise >  Web Scraping XML file that's Downloaded After Submitting Form (Python)
Web Scraping XML file that's Downloaded After Submitting Form (Python)

Time:11-28

How do you scrape an xml file that automatically downloads to your computer after submitting a form? I haven't seen any examples that contain a file that is from submitted data. I haven't been able to find it in the python documentation. https://docs.python.org/3/library/xml.etree.elementtree.html . Any suggestions, or links would be very much appreciated.

   from selenium import webdriver
   from webdriver_manager.chrome import ChromeDriverManager
   from selenium.webdriver.support.select import Select
   from selenium.webdriver.common.action_chains import ActionChains
   
   url = 'https://oui.doleta.gov/unemploy/claims.asp'
   driver = webdriver.Chrome(executable_path=r"C:\Program Files (x86)\chromedriver.exe")
   
   driver.implicitly_wait(10)
   driver.get(url)
   driver.find_element_by_css_selector('input[name="level"][value="state"]').click()
   Select(driver.find_element_by_name('strtdate')).select_by_value('2020')
   Select(driver.find_element_by_name('enddate')).select_by_value('2022')
   driver.find_element_by_css_selector('input[name="filetype"][value="xml"]').click()
   select = Select(driver.find_element_by_id('states'))

   # Iterate through and select all states
   for opt in select.options:
       opt.click()
   input('Press ENTER to submit the form')
   driver.find_element_by_css_selector('input[name="submit"][value="Submit"]').click()

CodePudding user response:

This post on stackoverflow seems to be in the direction of what I'm looking for; How to download XML files avoiding the popup This type of file may harm your computer through ChromeDriver and Chrome using Selenium in Python

This link helped the most for my needs, so I'll add it here, however, the first link is very helpful as well so I want to leave it. How to control the download of files with Selenium Python bindings in Chrome

  • Related