Puppeteer: Click within page.evaluate() not working

In the following code snippet, I try to click a button (after some Timeout) within the page.evaluate function. It does not work. Yet, when I open the console in the launched browser and manually type const btn = document.querySelectorAll("form button")[1]; btn.click() it does.

Can anyone explain to me the cause of this difference in behavior and how to fix it?

Here's a minimal reproducible example:

import { resolve } from 'path';
import puppeteer from 'puppeteer'

//go to page and handle cookie requests
const browser = await puppeteer.launch({defaultViewport: {width: 1920, height: 1080},
        headless:false, args: ['--start-maximized']});
const page = await browser.newPage();
const url = "https://de.finance.yahoo.com/";
await page.goto(url);
await page.waitForSelector("div.actions");
await page.evaluate( () => {
let z= document.querySelector("div.actions"); z.children[4].click()
})

await page.waitForSelector("input[id=yfin-usr-qry]");
await page.evaluate( () => {let z= document.querySelector("input[id=yfin-usr-qry]");
    z.value = "AAPL"; const btn = document.querySelectorAll("form button")[1]; 
    return new Promise((resolve) => setTimeout(() => {btn.click();resolve()},1000))})
})

CodePudding user response：

The form button selector appears to be incorrect, selecting a non-visible element with class .modules_clearBtn__uUU5h.modules_noDisplay__Qnbur. I'd suggest selecting by .finsrch-btn or #UH-0-UH-0-Header .finsrch-btn if you have to select this, but it's not really necessary, so I won't use it in my suggested solution below.

Beyond that, I'd tighten up some of the selectors, skip the timeout and prefer using trusted Puppeteer events when possible.

I'm not sure what data you want on the final page but this should give you a screenshot of it, showing all of the content:

const puppeteer = require("puppeteer"); // ^18.0.4

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const $ = (...args) => page.waitForSelector(...args);
  const url = "https://de.finance.yahoo.com/";
  await page.goto(url, {waitUntil: "domcontentloaded"});
  await (await $('button[name="agree"]')).click();
  const input = await $("#yfin-usr-qry");
  await input.type("AAPL");
  await page.keyboard.press("Enter");
  await $("#AAPL-interactive-2col-qsp-m");
  await page.evaluate("scrollTo(0, document.body.scrollHeight)");
  await $("#recommendations-by-symbol");
  await page.screenshot({path: "aapl.png", fullPage: true});
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close())
;

That said, rather than navigating to the homepage, typing in a search, then pressing a button, you could consider building the URL directly, e.g. https://de.finance.yahoo.com/quote/${symbol} and navigating right to it. This is generally faster, more reliable, and easier to code.