Puppeteer: Uncaught TimeoutError TimeoutError: Timeout exceeded while waiting for event-CodePudding

I'm trying to get the response headers and cookies for

url= 'https://www.asr.pima.gov/Parcel/GetParcel'

I have the following code using node and puppeteer:

const browser = await puppeteer.launch({
    headless: false,
    executablePath: executablePath(),

});
let req;
const page = await browser.newPage();
await page.goto(url);
const finalResponse = await page.waitForResponse(response =>
    response.url() === 'https://www.asr.pima.gov/Parcel/GetParcel' && response.status() === 200
);

.......

As I step through the code there is a delay at the final line ( starting at: const finalResponse = ..), The error is in the title.

What am I doing wrong?

CodePudding user response：

You may be overthinking it. page.goto returns the response you seem to be looking for:

const puppeteer = require("puppeteer"); // ^19.1.0

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const url = "https://www.asr.pima.gov/Parcel/GetParcel";
  const finalResponse = await page.goto(url);
  console.log(finalResponse.headers());
  console.log(await page.cookies());
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

Now, let's say you're looking for some other response that's kicked off by the initial page load. A way to do it is by planting the promise without await before goto, then beginning navigation.

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const url = "https://www.asr.pima.gov/Parcel/GetParcel";
  const finalResponseP = page.waitForResponse(res => res.url() === url);
  await page.goto(url, {waitUntil: "domcontentloaded"});
  const finalResponse = await finalResponseP;
  console.log(finalResponse.headers());
  console.log(await page.cookies());
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

This works for the base url as shown above, but not necessarily, so it's a more general solution than the first code block. If you have multiple responses, you can try Promise.all or add a listener with page.on("response", res => {}).

Based on the follow-up, it sounds like you were expecting cookies to be set on this request, but I don't see any on a visit to the page in my browser.

If you're looking for the cookie that's returned after you enter a search, you can access it as follows:

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  const ua =
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36";
  await page.setUserAgent(ua);
  const url = "https://www.asr.pima.gov/Parcel/GetParcel";
  const finalResponse = await page.goto(url, {waitUntil: "domcontentloaded"});
  console.log(finalResponse.headers());
  await page.type("#parcel", "123-45-678A");
  await Promise.all([
    page.waitForResponse(res => res.url().includes("GetSearchResults")),
    page.click('[type="submit"]'),
  ]);
  console.log(await page.cookies());
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());