0

I have an API Gateway with some AWS Lambda functions that make a database call (using sequelize.js) and return the result as a JSON object.

When I run the code locally I get the results, but when I deploy the code, any method that has a database call in it results in an internal server error.

The following code returns the message:

module.exports.testFunction = (event, context, callback) => { return callback(null, { statusCode: 200, body: JSON.stringify({ message: 'Test function is working.' }) }); };

The following code logs the database results in CloudWatch, but the API call times out after 6 seconds and then an internal server error (502) is returned:

module.exports.getAll = (event, context, callback) => { Entity.findAll().then(result => { console.log(JSON.stringify(result)); return callback(null, { statusCode: 200, body: JSON.stringify(result) }); }); };

Any solutions to make the return work?

Improve this question
7
  • 1
    Is the DB inside a VPC? Commented May 17, 2019 at 11:59
  • Yes, the DB is in a VPC Commented May 17, 2019 at 12:02
  • 1
    And is your Lambda inside the same VPC? Commented May 17, 2019 at 12:05
  • Yes, in the serverless.yml the security group and subnet from the vpc are defined Commented May 17, 2019 at 12:07
  • 1
    Are your security groups correct? Does the lambda's allow outbound to the db and does the db's allow inbound from the lambda? Commented May 17, 2019 at 12:15

3 Answers 3

Reset to default
4

First of all congratulations, that you were able to successfully identify the issue. However, there are many concepts that can be learnt from the occurred issue. Let’s discuss few of them.

Lambda cold start issue

To understand the concept of cold start in lambda it is inevitable to go any further without understanding how AWS Lambda works. Please refer to the undermentioned link which explains the overall working of lambda as shared in AWS re:Invent 2018

AWS Lambda under the hood (Video Link) : https://www.youtube.com/watch?v=QdzV04T_kec

Coming back to the issue of cold start, as our lambda functions are executed in a container, post execution the container gets killed if there are no further function invocations within  ~15 minutes of inactivity. Once the container is killed, any future call to execute the same function will require setting up a new container which can take >5 seconds (most probable reason behind your api call getting terminated post 6 seconds). However, there are many available options available that can keep your lambda warm. Please refer the undermentioned link from Serverless community.

Keeping Lambda Functions Warm (Blog Link) : https://serverless.com/blog/keep-your-lambdas-warm/

Increasing Lambda timeout

“I had to go into the AWS console and increase the timeout to 20 seconds”.

Although, this approach is totally acceptable but since you are using server less technology (explicit serverless.yml file), you can directly change the default lambda timeout period (which is 6 seconds). Please refer to the undermentioned code snippet and link for further understanding.

provider: name: was runtime: nodejs6.10 memorySize: 512 # optional, in MB, default is 1024 timeout: 10 # optional, in seconds, default is 6

Serverless AWS Lambda function Guide (Blog Link) : https://serverless.com/framework/docs/providers/aws/guide/functions/

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

Adding another answer, as I found what was the actual problem.

Setting the timeout to 20 made it work, but every call took around 10s, not only cold start. However, the query results were already received after just a few milliseconds.

Apparently AWS Lambda passes a context argument in handler functions and one of its properties is callbackWaitsForEmptyEventLoop, which defaults to true:

callbackWaitsForEmptyEventLoop – Set to false to send the response right away when the callback executes, instead of waiting for the Node.js event loop to be empty. If false, any outstanding events will continue to run during the next invocation.

I set the property to false in the beginning of the function and now every call only takes ~70ms.

module.exports.getAll = (event, context, callback) => { context.callbackWaitsForEmptyEventLoop = false; Entity.findAll().then(result => { console.log(JSON.stringify(result)); return callback(null, { statusCode: 200, body: JSON.stringify(result) }); }); };
Improve this answer

Comments

1

I fixed it. I had to go into the AWS console and increase the timeout to 20 seconds. So the lambda function works but takes a very long time to run. Amazon Support now recommended using X-Ray traces to see which calls take long.

Improve this answer

1 Comment

Cold starts inside VPCs really take quite a while. AWS announced they are working on it and the improvement should be released in the course of this year. They are making it as fast as non-VPC Lambda functions

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.