Error finding element with selenium python of transfer market website?(tried with xpath,class name e-CodePudding

Hi Dear Programmers and webscrapers.I have tried so much times to locate the element and then press it but it always gives me error in selenium.I have treid using XPATH,class name and id of the element is not present so I tried with multiple things but nothing works and returns me an error. This is the website link that I want to scrap.

This element had class selector-title so I tried with class name and also with xpath of this element and with its sub elements also but unfortunately was not able to locate it. Here is the code that i used.This code is working for other functions like I have located search box and have searched pressed button it works.Right.Where it dosent work is the element I have been trying it to work.Please let me know what mistake is from mine side so this can be workable or I can locate it and then click on it. Actually this I have to do for web scraping the data of all the players in a club or in a league so for that I have to select the league of country first then I will be able to scrap the data so its very much important for me to actually be able to locate and click the marked element.Please help me if any one of you can.Thanks.

XPATH of the marked element

//*[@id="main"]/header/div[3]/tm-quick-select-bar//div/tm-quick-select[1]/div/div

Here is my code

import os
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait

os.environ['PATH']  = r"C:/ChromeDriver"
browser = webdriver.Chrome()
browser.get("https://www.transfermarkt.us/")
element = browser.find_element(By.XPATH, '//*[@id="schnellsuche"]/input[1]')
element.send_keys("J1 League")
time.sleep(5)

# element2 = browser.find_element(By.XPATH, '//*[@id="main"]/header/div[3]/tm-quick-select-bar//div/tm-quick-select[1]/div/div[1]/strong/text()')
# element2.click()

button = wait(browser, 10).until(EC.presence_of_element_located(
    (By.CLASS_NAME, 'selector-title')))
button.click()

time.sleep(10)

The error I get is the browser instantly closes as soon as it reaches the last element target element to click

DevTools listening on ws://127.0.0.1:61330/devtools/browser/d6f23f8f-6bea-41dd-8500-08f85c9cb502
[18424:9224:0523/200949.930:ERROR:device_event_log_impl.cc(214)] [20:09:49.929] USB: usb_device_handle_win.cc:1049 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[18424:9224:0523/200949.936:ERROR:device_event_log_impl.cc(214)] [20:09:49.935] USB: usb_device_handle_win.cc:1049 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[18424:9224:0523/200949.951:ERROR:device_event_log_impl.cc(214)] [20:09:49.951] Bluetooth: bluetooth_adapter_winrt.cc:1075 Getting Default Adapter failed.
Traceback (most recent call last):
  File "d:\WeBScraping\FreecodecampWebScrapingTutorial\selenium_tutorial_freecodecamp\scrapingtarget.py", line 18, in <module>
    button = wait(browser, 10).until(EC.presence_of_element_located(
  File "C:\Users\bilal\AppData\Roaming\Python\Python310\site-packages\selenium\webdriver\support\wait.py", line 89, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Stacktrace:
Backtrace:
        Ordinal0 [0x00E1B8F3 2406643]
        Ordinal0 [0x00DAAF31 1945393]
        Ordinal0 [0x00C9C748 837448]
        Ordinal0 [0x00CC92E0 1020640]
        Ordinal0 [0x00CC957B 1021307]
        Ordinal0 [0x00CF6372 1205106]
        Ordinal0 [0x00CE42C4 1131204]
        Ordinal0 [0x00CF4682 1197698]
        Ordinal0 [0x00CE4096 1130646]
        Ordinal0 [0x00CBE636 976438]
        Ordinal0 [0x00CBF546 980294]
        GetHandleVerifier [0x01089612 2498066]
        GetHandleVerifier [0x0107C920 2445600]
        GetHandleVerifier [0x00EB4F2A 579370]
        GetHandleVerifier [0x00EB3D36 574774]
        Ordinal0 [0x00DB1C0B 1973259]
        Ordinal0 [0x00DB6688 1992328]
        Ordinal0 [0x00DB6775 1992565]
        Ordinal0 [0x00DBF8D1 2029777]
        BaseThreadInitThunk [0x76CDFA29 25]
        RtlGetAppContainerNamedObjectPath [0x76F57A7E 286]
        RtlGetAppContainerNamedObjectPath [0x76F57A4E 238]

CodePudding user response：

Not sure if I understood you correctly, but the code below return the following text "United States".

import time
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait

driver = webdriver.Chrome(options=options, desired_capabilities=capabilities)

# open url:
driver.get('https://www.transfermarkt.us/')

# switch to cookies frame
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.ID, "sp_message_iframe_575847")))
time.sleep(3)

# click "Accept" cookies button
driver.find_element(By.XPATH, '//button[@title="ACCEPT ALL"]').click()
time.sleep(3)

# go back to main frame
driver.switch_to.default_content()

# here's the trick, what you are looking for is inside a "shadow-root" DOM so to access it you need to execute the script and then use CSS selector, I don't think XPATH works here:
country = driver.execute_script("return document.querySelector('tm-quick-select-bar').shadowRoot.querySelector('div > tm-quick-select:nth-child(2) > div > div > strong')")

# print
print(country.text)

# close the driver
driver.close()