I get internal server error when I have a long running query. Actually, I have to fetch historic data through an API, which sometime can take longer than 30 seconds. It depends on the query how complex it is. It can take 1 min also.

Not sure but guessing, API gateway timeout is set to 30 seconds (and I cann't increase it) and my query execution time is more then 30 seconds. So I get internal server error I believe.

HOW can I say above statement ?

because If I run the same query locally, I mean in node/express locally by running npm run start, it works fine even if takes1 mins, response will always come back.

But when I deploy node/express code to lambda function, it throws error if any query takes longer period to execute.

I have following setup of node/express

const express = require("express");
const serverless = require("serverless-http");
const app = express();

app.use(cors());
app.use((req, res, next) => {
    res.setHeader('Connection', 'keep-alive');                   // I added this line as suggested in some post but not helping
    res.setHeader('Keep-Alive', 'timeout=30');                   // I added this line as suggested in some post but not helping
    res.setHeader("Access-Control-Allow-Headers", "X-Requested-With,content-type");
    res.setHeader("Access-Control-Allow-Origin", "*");
    res.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS, PUT, PATCH, DELETE");
    res.setHeader("Access-Control-Allow-Credentials", true);
    next();
});

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.use(`api-end-point/user`, userRoute);
....

if (process.env.NODE_ENV !== "lambda") {

    PORT = process.env.PORT || 7000;
    const server = app.listen(PORT, () => {
        console.log(`node-express server running in ${process.env.NODE_ENV} mode on ${PORT}`);
    });
    server.timeout = 0;

}else {

    module.exports.handler = serverless(app);     // this is for lambda function

}

I deploy this code to AWS lambda function.

HTTP API gateway is configured with two routes /ANY, /{proxy }

TIMEOUT

API gateway is set to default 30 seconds. [I can not increase this time as not allowed by AWS]

Lambda is set to 10 **mins**

CORS

I really have no idea how can I fix this problem ?

How can I increase API gateway timeout or How can I keep connection alive ?

CodePudding user response：

https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html

Since timeout cannot be incresed, you might change using single HTTP request to

• https://en.wikipedia.org/wiki/Post/Redirect/Get Pattern

1. Client POST query
2. Server response an url for the result
3. Client GET the url multiple times -- it will be 200 OK when the result ready

• or WebSocket

The document says Idle Connection Timeout for WebSocket is up to 10 minutes

CodePudding user response：

You cannot increase the API Gateway timeout to greater than 30 seconds, as has already been mentioned.

The only solution I know of at this time is to run your Lambda asynchronously, but this cannot be done in an Http API. But if you're willing to change it to a REST API, then this can be done with a combination of turning on Lambda Proxy Integration in the REST API and invoking the Lambda asynchronously utilizing an invoke header X-Amz-Invocation-Type. This will allow your Lambda to run asynchronously (up to 15 minutes) with an API call.