Puppeteer needs to restart server for each scrape-CodePudding

I'm scraping a website with Puppeteer using a Nest.js module.

The scraping works the first time I call the GET method, but it won't scrape again after that. Which means I have to restart the server each for each scraping, which isn't ideal.

Scraping code (Simplified):

@Injectable()
export class AppService {
  constructor(@InjectBrowser() private readonly browser: Browser) {}

  async getFoodoraRestaurantMenus(url: string): Promise<string> {
    console.log("Calling method");
    const page = await this.browser.newPage();
    console.log('Never gets here the second time');

    await page.goto(url);

    const content = await page.evaluate(() => {});

    await this.browser.close();

    return JSON.stringify(content, null, 4);
  }
}

The second time I call getFoodoraRestaurantMenus I never get to the second console log, so a new page is never opened. I'm unsure if I need to do something more than browser.close(), that was all I could find.

I have no idea why I need to restart the server each time I want to perform a scrape. All help I can get is greatly appreciated.

Thanks.

CodePudding user response：

If you only want to free the memory and reuse the same browser, you could try only closing the page instead of the browser:

await page.close();