So I have a webscraper which first of all needs to get past the cookie banner of a given website. Normally I'd just locate the element by id or classname and be done with it, but on this site none of the elements can be located. I've tried/checked the following:

• The element of interest is <div id="cookiescript_accept" tabindex="0" role="button" data-cs-i18n-text="[]">Alles accepteren</div>
• The element is not part of an iframe
• The element is not part of a shadow DOM
• Using wait.until(ExpectedConditions.visibilityOfElementLocated hits the 15 seconds timeout
• Using driver.executeScript("return document.getElementById('cookiescript_accept');"); doesn't work either
• The parent element and parent of parent can also not be found

I'm still fairly new to Selenium and HTML so I must be missing something, please tell me if you know what that is

Code:

public void loadUrl(String url) {
    System.out.println("\t\t- loadUrl "   url);
    idle5000();

    driver.get(url);
    idle5000();

    setWindowSize();
    idle5000();
    printFirefoxCPU();

    scrollViewport();
    idle5000();
    printFirefoxCPU();
}
loadUrl("https://www.schoolplaten.com/");
prepRunnable.getDriver().findElement(By.id("cc-cookiescript_accept")).click();
//  -> NoSuchElementException
WebDriverWait wait = new WebDriverWait(prepRunnable.getDriver(), 15);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.id("cookiescript_accept")));
//  -> Timeout
WebElement elem = (WebElement) prepRunnable.getDriver().executeScript("return document.getElementById('cookiescript_accept');");
elem.click();
//  -> elem is null

CodePudding user response：

I could locate the web element with the below XPath

//*[name()='div' and @id='cookiescript_accept']

and could perform click on it with the help of below code :

WebDriverWait wait = new WebDriverWait(driver, 30);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[name()='div' and @id='cookiescript_accept']"))).click();

CodePudding user response：

To click() on the element ALLES ACCEPTEREN you can use either of the following Locator Strategies:

• cssSelector:

  driver.findElement(By.cssSelector("div#cookiescript_accept")).click();
  

• xpath:

  driver.findElement(By.xpath("//div[@id='cookiescript_accept']")).click();
  

However, the element is a dynamic element so to click() on the element you need to induce WebDriverWait for the elementToBeClickable() and you can use either of the following Locator Strategies:

• cssSelector:

  new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.cssSelector("div#cookiescript_accept"))).click();
  

• xpath:

  new WebDriverWait(driver, 20).until(ExpectedConditions.elementToBeClickable(By.xpath("//div[@id='cookiescript_accept']"))).click();
  

Reference

You can find a detailed discussion on NoSuchElementException in:

• NoSuchElementException, Selenium unable to locate element