Firebase functions: RangeError [ERR_FS_FILE_TOO_LARGE]: File ... is greater than 2 GB-CodePudding

Been deploying a ton w/ no problems. Made a small change and now it bonks on deploy.

I rolled back a few commits and still get same error. Tried deploying only 1 function, same issue. Deleted a bunch of log files, etc. Same. Quotas all look fine.

There's about 20 functions in the project.

i  functions: Loaded environment variables from .env. 
i  functions: preparing functions directory for uploading... 
[2023-01-17T07:51:09.242Z] RangeError [ERR_FS_FILE_TOO_LARGE]: File size (5067767843) is greater than 2 GB
    at new NodeError (node:internal/errors:372:5)
    at readFileHandle (node:internal/fs/promises:377:11)
    at async getSourceHash (/Users/adrian/.nvm/versions/node/v16.16.0/lib/node_modules/firebase-tools/lib/deploy/functions/cache/hash.js:12:18)
    at async packageSource (/Users/adrian/.nvm/versions/node/v16.16.0/lib/node_modules/firebase-tools/lib/deploy/functions/prepareFunctionsUpload.js:61:30)
    at async prepare (/Users/adrian/.nvm/versions/node/v16.16.0/lib/node_modules/firebase-tools/lib/deploy/functions/prepare.js:131:36)
    at async chain (/Users/adrian/.nvm/versions/node/v16.16.0/lib/node_modules/firebase-tools/lib/deploy/index.js:35:9)
    at async deploy (/Users/adrian/.nvm/versions/node/v16.16.0/lib/node_modules/firebase-tools/lib/deploy/index.js:79:5)

Error: Could not read source directory. Remove links and shortcuts and try again.

What file is it referring to?

Appreciate any ideas.

CodePudding user response：

RangeError [ERR_FS_FILE_TOO_LARGE]: File ... is greater than 2 GB

Error indicates that is the maximum size a Buffer can have. Note that this operation will load all the contents of the file into memory at once.

According to this answer on GitHub, 2GB is the limit:

That is the max buffer size in node. To import large files, the code will need to change the imports to streams instead of putting the whole file in a buffer (...)

Have a look at this Link1 which explains how to load heavy files.

You can use the command firebase deploy --debug which will help you to find out what causes the problem in detail.

For more information, you can refer to these Link2 and Link3 which may help you.

CodePudding user response：

A new type of log file had been generated which didn't match the current ignore pattern in firebase.json. So the log file was being processed during deploy.

Solution: update ignore pattern to cover any .log files

firebase.json

...

"functions": [
    {
      "source": "functions",
      "codebase": "default",
      "ignore": ["node_modules", ".git", "*.log"]
    }
  ]

...