I am trying to scrape data off a website using Selenium in Python, where the data only appears once I click a 'Details' button, the problem is that this button isn't initially visible and requires scrolling on an internal scroll bar (circled in red)
For reference, here's the website: https://www.abt-sportsline.com/tuning/configurator#80706385!43C6B47
The goal is to scroll this internal scroll bar so I can click the button and view/scrape the data.
Here's my initial code:
scrollBar = driver.find_element(By.XPATH,'//*[@id="mCSB_1_dragger_vertical"]/div')
driver.execute_script("arguments.scrollBy(0,arguments[0].scrollHeight)", scrollBar)
temp_element = in_tab_object.find_element(By.CSS_SELECTOR,"div > div > div > div.row > div.col-xs-12.product-details > div > div.text-right.details-link > span.icon-ArrowRight")
temp_element.click()
This does not scroll the internal bar at all and I get the following error:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
I have another idea of solving this problem, where I find the scrollbar container and click the bottom of it, hence, scrolling all the way down, but I am unsure how to specify clicking local coordinates of a container.
Some help would be appreciated!
CodePudding user response:
As it is not inside frame, You can easily use scrolled_once_into_view
which is the same as scrollIntoView() in js.
for instance:
element = driver.find_element('xpath', "DETAILS_BUTTON_XPATH")
element.scrolled_once_into_view # Ignore IDE errors for this command. It is working well.
Alternatively you can use following (using JS command):
element = driver.find_element('xpath', "DETAILS_BUTTON_XPATH")
driver.execute_script("arguments[0].scrollIntoView();", element)
CodePudding user response:
You could try ActionChains.
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains as AC
dr = webdriver.Chrome("[path_to_your_chromedriver]")
dr.get("https://www.abt-sportsline.com/tuning/configurator#80713308!5972E27")
aa = dr.find_element_by_id("mCSB_1_scrollbar_vertical")
action = AC(dr)
action.click_and_hold(aa).move_by_offset(0,1).perform()
# Don't forget to release the click_and_hold, for example with:
action.reset_actions()
This does not work very well, however. I think it's something to do with that it is a custom scroll bar.
Read more about action chains here:
https://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.common.action_chains
.