I am using selector gadget to get the xpath from the "read more" button from the first review on this website

This is the xpath it gave:

//*[contains(concat( " ", @class, " " ), concat( " ", "Z", " " ))]

Here is the first part of the code I am using:

import selenium
import csv #This package lets us save data to a csv file
from selenium import webdriver #The Selenium package we'll need
import time #This package lets us pause execution for a bit
from selenium.webdriver.common.by import By

path_to_file = "/Users/user/Desktop/HotelReviews.csv"

pages_to_scrape = 3

url = "https://www.tripadvisor.com/Hotel_Review-g60982-d209422-Reviews-Hilton_Waikiki_Beach-Honolulu_Oahu_Hawaii.html"

# open the file to save the review
csvFile = open(path_to_file, 'a', encoding="utf-8")
csvWriter = csv.writer(csvFile)

for i in range(0, pages_to_scrape):
    
    driver = webdriver.Chrome()
    driver.get("url")
    # give the DOM time to load
    time.sleep(2) 
    driver.find_element_by_xpath("//*[contains(concat( " ", @class, " " ), 
    concat( " ", "Z", " " ))], 'Read more')]").click()

This is the error I get:

File "/var/folders/6c/jpl964752rv_72zjclrp_8ym0000gn/T/ipykernel_24978/2812702568.py", line 8
    driver.find_element_by_xpath("//*[contains(concat( " ", @class, " " ), concat( " ", "Z", " " ))], 'Read more')]").click()
                                                                                         ^
SyntaxError: invalid syntax

Looks like it's the quotation marks that seems to be the issue.

So I followed this advice. I tried making the code a variable, but it spit out the same error. I tried removing the extra quotes, same error. I tried removing the space between the quotes, same error.

I tried a different xpath, one for the whole review //*[contains(concat( " ", @class, " " ), concat( " ", "F1", " " ))] Same error.

Then I tried adjusting the quotation marks on the first xpath

driver.find_element_by_xpath("//*[contains(concat( " ", @class, " " ), 
    concat( " ", "Z", " " ))]", "Read more")]).click()

same error

CodePudding user response：

Selenium supports XPath 1.0 only and XPath 1.0 AFAIK does not support concat() method.
As about that specific element locator you can use this XPath expression:

"//div[@id='ABOUT_TAB']//span[text()='Read more']"

So instead of

driver.find_element_by_xpath("//*[contains(concat( " ", @class, " " ), 
    concat( " ", "Z", " " ))], 'Read more')]").click()

You can use

driver.find_element_by_xpath("//div[@id='ABOUT_TAB']//span[text()='Read more']").click()

Also instead of hardcoded sleeps

time.sleep(2)

You should use explicit waits

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
...
wait = WebDriverWait(driver, 20)
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[@id='ABOUT_TAB']//span[text()='Read more']"))).click()

CodePudding user response：

You XPath expression contains several errors:

1. You are breaking the surrounding python command by using the quotes wrong. Essentially, your command looks like

   driver.find_element_by_xpath("..." "..." "..." ", "...", " "...", "...")]).click()
   

   And this is not a correct parameter (list) for driver.find_element_by_xpath("xpath").

2. You're using the contains function wrong. It should be contains(node, "string").

So try this expression instead:

driver.find_element_by_xpath("//*[contains(@class, 'Z')] and contains(text(),'Read more')]").click()

This should select all elements that have a class attribute containing the letter Z and a text node containing the string Read more.