Why does Puppeeteer cause my test suite to hang for 30 seconds when I use "waitForSelector&quot-CodePudding

I have a Node.js Mocha test suite (I've created a minimal reproduction based on the real world application I was trying to create an automated test for).

package.json:

{
  "name": "puppeteer-mocha-hang-repro",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "chai": "4.3.7",
    "express": "4.18.2",
    "mocha": "10.2.0",
    "puppeteer": "19.6.2"
  }
}

index.spec.js:

const expect = require('chai').expect;
const express = require('express');
const puppeteer = require('puppeteer');

const webServerPort = 3001;

describe('test suite', function () {
    this.timeout(10000);

    let webServer;
    let browser;

    beforeEach(async () => {
        // Start web server using Express
        const app = express();
        app.get('/', (_, res) => {
            res.send('<html>Hello, World from the <span id="source">Express web server</span>!</html>');
        });
        webServer = app.listen(webServerPort, () => {
            console.log(`Web server listening on port ${webServerPort}.`);
        });

        // Start browser using Puppeteer
        browser = await puppeteer.launch();
        console.log('Browser launched.');
    });

    afterEach(async () => {
        // Stop browser
        await browser.close();
        console.log('Browser closed.');

        // Stop web server
        await webServer.close();
        console.log('Web server closed.');
    });

    it('should work', async () => {
        const page = await browser.newPage();

        await page.goto(`http://localhost:${webServerPort}/`);
        console.log('Went to root page of web server via Puppeteer.');

        if (process.env['PARSE_PAGE'] === 'true') {
            const sel = await page.waitForSelector('#source');
            const text = await sel.evaluate(el => el.textContent);
            console.log('According to Puppeteer, the text content of the #source element on the page is:', text);
            expect(text).eql('Express web server');
        }

        await page.close();
        console.log('Page closed.');
    });
});

If I run the test suite with the command npx mocha index.spec.js, which causes lines 45-48 to be skipped, the test suite passes and the Mocha process ends quickly:

$ time npx mocha index.spec.js


  test suite
Web server listening on port 3001.
Browser launched.
Went to root page of web server via Puppeteer.
Page closed.
    ✔ should work (70ms)
Browser closed.
Web server closed.


  1 passing (231ms)


real    0m0.679s
user    0m0.476s
sys     0m0.159s

Note that it finished in 0.679s.

If I instead run it with the command PARSE_PAGE=true npx mocha index.spec.js, which causes none of my code to be skipped, the tests pass quickly but the process hangs for about 30 seconds:

$ time PARSE_PAGE=true npx mocha index.spec.js


  test suite
Web server listening on port 3001.
Browser launched.
Went to root page of web server via Puppeteer.
According to Puppeteer, the text content of the #source element on the page is: Express web server
Page closed.
    ✔ should work (79ms)
Browser closed.
Web server closed.


  1 passing (236ms)


real    0m30.631s
user    0m0.582s
sys     0m0.164s

Note that it finished in 30.631s.

I suspected that this meant I was leaving things open, forgetting to call functions like close. But, I am calling close on the Express web server, Puppeteer browser, and Puppeteer page. I tried calling close on the objects I use when I don't skip any of that code, which are sel and text. But if I try that, I get error messages telling me that those objects have no such functions.

System details:

$ node --version
v18.13.0
$ npm --version
9.4.0
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy
$ uname -r
5.10.16.3-microsoft-standard-WSL

CodePudding user response：

I'm able to reproduce the hang. The culprit is page.waitForSelector, which seems to run its full 30 second default timeout even after resolving, somehow preventing the process from exiting.

Possible workarounds:

Avoid using waitForSelector, since the data you want is in the static HTML (may not apply to your actual page though).
Call with page.waitForSelector("#source", {timeout: 0}) which seems to fix the problem, with the risk of stalling forever (practically not a serious issue since mocha will time the test out). This seems to be the best solution for now.
Call with page.waitForSelector("#source", {timeout: 1000}) which reduces the impact of the delay, with the risk of a false positive if the element takes longer than a second to load. This doesn't seem to stack, so if you use a 1-3 second delay across many tests, mocha should exit within a few seconds of all tests completing rather than the sum of all delays across all waitForSelector calls.
Run npx mocha --exit index.spec.js. Not recommended--this suppresses the issue.

I'm not sure if the behavior is specific to waitForTimeout or if it may apply to other waitFor-family methods.

As an aside, your server listen and close calls are technically race conditions, so:

await new Promise(resolve =>
  webServer = app.listen(webServerPort, () => {
    console.log(`Web server listening on port ${webServerPort}.`);
    resolve();
  })
);

and

await new Promise(resolve => webServer.close(() => resolve()));

I'll try to investigate further when I get the time. This might be a good issue to open on Puppeteer's GitHub repo if it hasn't been mentioned already.

System details:

$ node --version
v18.7.0
$ npm --version
9.3.0
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:    22.04
Codename:   jammy
$ uname -r
5.15.0-56-generic

CodePudding user response：

I am not sure how much it will be helpful but you can try this:

if (process.env['PARSE_PAGE'] === 'true') {
   const sel = await page.waitForSelector('#source');
   const text = await page.evaluate(el => el.textContent, sel);
   console.log('According to Puppeteer, the text content of the #source element on the page is:', text);
   expect(text).eql('Express web server');
}

Also, check for global hooks!