CloudRun cold start issue and 429 Rate exceeded error-CodePudding

We have a nodejs monorepo project with >179 packages where each package may have >30 files. It also contains proxy with routing and several forked processes (as usual). So, when we packed all of this stuff into Docker image and moved it into CloudRun (with min instance = 1, max instance >10, concurrency=1000), users randomly start see the error '429 Rate exceeded'. (As far as we understood from the documentation, it happens when 'max instances' limit is reached by CloudRun and it can not scale our application anymore. Indirect reason of that maybe too long cold start (which is also limited in CloudRun = 10s). We measured our cold start - it was ~20sec.

To identify the issue of cold start we used this package - https://www.npmjs.com/package/require-so-slow
It showed us that each our small module requires ~5ms for 'import'. So, average calculation may show why do we have so long cold start: 170 packages * 30 files * 5ms > 25s

For monorepo we use pnmpm and each package builds with tsc.

So, the question is how to improve cold start in CloudRun?

Note: Locally, on dev environment on laptops, we do not have issue with cold start, only in CloudRun. So, looks like, this issue is a platform specific issue.

CodePudding user response：

...this is continuation for comments above...

3.2) Starting point for nodejs bundling can be this webpack.config.js:

const path = require('path');
const nodeExternals = require('webpack-node-externals');

module.exports =  (env) => {
  return {
  target: 'node',
  externals: [nodeExternals()],
    node: {
    global: false,
    __filename: false,
    __dirname: false,
  },
  entry: './src/index.ts',
  optimization: {
        minimize: false
    },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        loader: 'ts-loader',
        exclude: /node_modules/,
        options: {
          compilerOptions: {
              outDir: env[1].output
          }
        }
      },
    ],
  },
  resolve: {
    extensions: ['.tsx', '.ts', '.js'],
  },
  output: {
    filename: 'index.js',
    pathinfo: false,
    libraryTarget: 'commonjs2'
  },
}};

package.json

    "devDependencies": {
        "typescript": "4.5.4",
        "ts-loader": "8.0.3",
        "webpack": "4.33.0",
        "webpack-node-externals": "3.0.0"
    }

command to start bundling:

node_modules/.bin/webpack \
            --config=./webpack.config.js \
            --output-path="./dist" \
            --env=development \
            --env.output="./dist"

env.output - uses for emitting *.d.ts files

CodePudding user response：

To improve cold starts in CloudRun. Please try to implement the points below .
when it comes to applications written in dynamic languages like node.js,etc. You should be aware of some modules that run initialization code upon importing.
Minimize the number and size of dependencies if you’re using a dynamic language.
Instead of computing things upon startup, compute them lazily. The initialization of global variables always occurs during startup, which increases cold start time. Use lazy initialization for infrequently used objects to defer the time cost and decrease cold start times. For the details you can check the documentation
For detailed description you can check Article and Document