Selenium popup data extraction using Python-CodePudding

I am a Selenium beginner looking to extract data from popup windows (HTML) via Selenium and Python.

This is the website: https://www.drayage.com/directory/results.cfm?city=SAV&port=y&OceanCntrs=y&drvrs=y&showClicks=y

I am trying to click on the "detail" popup for every single Company and extract all of the data. This is what I have so far (sorry, not much!) which simply clicks the first "detail" popup.

#import packages
from selenium import webdriver

#create filepath for correct chrome webdriver
path = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(path)

driver.get("https://www.drayage.com/directory/results.cfm?city=SAV&port=y&OceanCntrs=y&drvrs=y&showClicks=y")

driver.find_element_by_link_text("detail").click()

If I can get a text block for each of the popups that would even be enough for me to then separate via delimiters and clean in SQL.

My trouble is how to extract all of this as well as how to continually click each "detail" popup.

Thank you so much in advance for the help!

Update 1

Thanks for the comments so far!

Once I get into the popup window by clicking "detail" I am trying to extract everything from the first table which has the XPATH "/html/body/table[1]".

Unfortunately the below code isn't working. Let me know if you are able to help and thank you again!

table = driver.find_element_by_xpath("/html/body/table[1]")
for i in driver.find_elements_by_xpath("/html/body/table[1]/tbody"):
    data = [table.text for table in i.find_elements_by_xpath(".//*[self::td]")]
    print(data)

CodePudding user response：

wait=WebDriverWait(driver,10)
driver.get("https://www.drayage.com/directory/results.cfm?city=SAV&port=y&OceanCntrs=y&drvrs=y&showClicks=y")
trs=wait.until(EC.visibility_of_all_elements_located((By.XPATH,"//html/body/table/tbody/tr/td/table[1]//tr[position()>2]")))
window_before = driver.window_handles[0]
for tr in trs:
    try:
        detail=tr.find_element(By.XPATH,".//a[contains(.,'detail')]")
        detail.click()
        #Handle the tab switch
        window_after = driver.window_handles[1]
        driver.switch_to.window(window_after)
        #Do what you want here.
        driver.close()
        driver.switch_to.window(window_before)
    except:
        print('No detail')

Here's a sample to go through links handling going to the tab, close the tab and then go back to the default tab. I'm not sure how you want the values out you can try searching for the table.

Import:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC