Selenium :Cant find and click element by X-path-CodePudding

I am trying to click a button on website using selenium. But selenium cant find it. It is not iframe or problem with X-path. What I think is there is some event to make the element available by javascript. I tried to get it using class although it had very bad class value. If any can help. It will be a relief for me. I tried my best. I got the X-path from firefox developer tools.The url of site is :https://www.bedbathandbeyond.com/store/category/kitchen/trash-recycling/14367 I want to click the "next" button (available almost in the bottom)

from selenium import webdriver
from selenium.webdriver.chrome import service
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

option = webdriver.ChromeOptions()
option.add_argument("start-maximized")
option.add_experimental_option("excludeSwitches", ["enable-automation"])
option.add_experimental_option('useAutomationExtension', False)
option.add_argument("--disable-blink-features")
option.add_argument("--disable-gpu")
#option.add_argument("--headless")
option.add_argument('--disable-blink-features=AutomationControlled')
wd = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=option)
wait = WebDriverWait(wd, 40)

wd.get('https://www.bedbathandbeyond.com/store/category/kitchen/trash-recycling/14367')
wait.until(EC.element_to_be_clickable((By.XPATH, "/body/div[3]/div[8]/div[1]/div[3]/div[2]/amp-list/div/div/button[2]")))
wd.find_element(By.XPATH,'/body/div[3]/div[8]/div[1]/div[3]/div[2]/amp-list/div/div/button[2]').click()
# 
time.sleep(15)
wd.quit()

EDIT: I think it is some shadow root. I dont know much but tried this code but it did not worked

def expand_shadow_element(element):
  shadow_root = wd.execute_script('return arguments[0].shadowRoot', element)
  return shadow_root
wd.get('https://www.bedbathandbeyond.com/store/category/kitchen/trash-recycling/14367')
wait.until(EC.element_to_be_clickable((By.XPATH, '/body/div[3]/div[8]/div[1]/div[3]/div[2]/amp-list/div/div')))
ele = wd.find_element(By.XPATH, '/body/div[3]/div[8]/div[1]/div[3]/div[2]/amp-list/div/div')
root = expand_shadow_element(ele)

It gives Timeout error on the wait

CodePudding user response：

you could use javascript instead..this will click the next button

driver.execute_script(
    'document.querySelector("#wmHostPrimary").shadowRoot.querySelector("button.plpPage.plpNext.flex.mid.pwaOnly").click()')