I'm using Puppeteer for webscraping, with a small NodeJs webapp that I made. This webapp is hosted on Heroku and use jontewks/puppeteer-heroku-buildpack
to works.
The problem I'm facing is that my app do not build anymore because of the Heroku size limit:
Compiled slug size: 537.4M is too large (max is 500M).
I've tried severals things:
- Using Firefox instead of Chromium
- It's a "no go" for me because of a current issue with puppeteer/firefox:
- Reducing the size of Chromium by removing the file
interactive_ui_tests.exe
- I can't do this because Heroku use Linux instead of Windows, and this file does not exist in the Linux Chromium distribution
- Using
headless_shell
instead of Chromium - Using Playwright instead of Puppeteer
- It might be a solution, but I'm using stuffs like
puppeteer-extra
andpuppeteer-extra-plugin-stealth
, so it bother me to change
- It might be a solution, but I'm using stuffs like
- Reducing the size of Chromium by removing the folder
locales
- It helps a bit, but not much
- Using an older version of Puppeteer (
2.1.1
), which is using an older version Chromium who was slighlty lighter- At the moment, it's the only working solution that I have
- Use the command
heroku repo:gc -a myapp
andheroku builds:cache:purge -a myapp
My last three points reduced the size of my slug to 490M
. So my app is working, but it's not great for the (close) future, like having an up to date Puppeteer version.
So here I am, asking for help, as I do not have any more ideas at the moment.
Thank you very much for your help