104

I've followed the tutorial here to create a VPC with public and private subnets.

Then I set up an AWS lambda function inside the public subnet to test if it could connect to the outside internet.

Here's my lambda function written in python3

import requests def lambda_handler(event, context): r = requests.get('http://www.google.com') print(r)

The function above failed to fetch the content of http://www.google.com when I set it inside the public subnet in a VPC.

Here's the error message:

"errorMessage": "HTTPConnectionPool(host='www.google.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 110] Connection timed out',))", "errorType": "ConnectionError",

I don't understand why.

The route table of the public subnet looks like this:

enter image description here

The GET request to http://www.google.com should match igw-XXXXXXXXX target. Why can't the internet-gateway(igw) deliver the request to http://www.google.com and get back the website content?

This article says that I must set the lambda function inside the private subnet in order to have internet access.

If your Lambda function needs to access private VPC resources (for example, an Amazon RDS DB instance or Amazon EC2 instance), you must associate the function with a VPC. If your function also requires internet access (for example, to reach a public AWS service endpoint), your function must use a NAT gateway or instance.

But it doesn't explain why I can't set the lambda function inside the public subnet.

Improve this question
3
  • What does the Lambda logs in Cloudwatch say? Assume you have included requests modules with your deployment package? Could it be NACL preventing outbound traffic? Commented Oct 25, 2018 at 16:52
  • Do you actually need to deploy the Lambda function into a VPC? Commented Oct 25, 2018 at 17:10
  • 2
    You need to setup a NAT gateway docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html Commented Oct 25, 2018 at 18:04

4 Answers 4

Reset to default
238

Lambda functions attached to a public subnet in your VPC cannot access the internet.

To access the internet from a public subnet you would need the requests to originate from a network interface that has an associated public IP or you would need to route the requests via a Network Address Translation (NAT) device that itself has a public IP. In both cases you would also need an Internet Gateway (IGW).

However:

  1. Lambda functions do not and cannot** have public IP addresses, and
  2. the default route target in a VPC public subnet is the IGW, not a NAT

So, because the Lambda function only has a private IP and its traffic is routed to the IGW rather than to a NAT, all packets to the internet from the Lambda function will be dropped at the IGW.

** there is an alleged workaround whereby an Elastic IP can be associated with the Lambda function's ENI. See this post for more details and, importantly, some potentially significant limitations.

Should I Configure my Lambda Function for VPC Access?

If your Lambda function does not need to reach private resources inside your VPC (e.g. an RDS database or Elasticsearch cluster) then do not attach the Lambda function to your VPC.

If your Lambda function does need to reach private resources inside your VPC, then attach the Lambda function to one or more private subnets (and only private subnets) of your VPC.

NAT or Not?

If the Lambda function only needs access to resources in the VPC (e.g. an RDS database in a private subnet) then you don't need to route through NAT.

If the Lambda function only needs access to resources in the VPC and access to AWS services that are all available via private VPC Endpoint then you don't need to route through NAT. Use VPC Endpoints.

If your Lambda function needs to reach endpoints on the internet then ensure a default route from the Lambda function's private subnet(s) to a NAT instance or NAT Gateway in a public subnet. And configure an IGW, of course, to enable internet access.

Be aware that NAT gateway charges per hour and per GB processed so it's worth understanding how to reduce data transfer costs for NAT gateway.

Best Practices

When configuring Lambda functions for VPC access, it is a High Availability (HA) best practice to configure multiple (private) subnets across different Availability Zones (AZs).

Intermittent Connectivity

Be sure that all the subnets you configure for your Lambda function are private subnets. It is a common mistake to configure, for example, 1 private subnet and 1 public subnet. This will result in your Lambda function working OK sometimes and failing at other times without any obvious cause.

For example, the Lambda function may succeed 5 times in a row, and then fail with a timeout (being unable to access some internet resource or AWS service). This happens because the first launch was in a private subnet, launches 2-5 reused the same Lambda function execution environment in the same private subnet (a so-called "warm start"), and then launch 6 was a "cold start" where the AWS Lambda service deployed the Lambda function in a public subnet where the Lambda function has no route to the internet.

Improve this answer
Sign up to request clarification or add additional context in comments.

13 Comments

Is there a reason why you advise to run the lambda in a private subnet? Any drawbacks vs running it on a public one?
@LLL stepping back, you run Lambda functions in a VPC if you need them to run there, e.g. because they need to access private resources such as a MySQL DB inside the VPC or an S3 bucket restricts access to a specific VPC via private endpoint. The drawback of running in VPC is cold start latency is higher than when not run in VPC (because an ENI must be attached). The difference between public and private subnets is routing (what the 0.0.0.0/0 default route points to, IGW or NAT). If your Lambda needs outbound access then it won't work in a public subnet, because the default route is the IGW.
Thanks! I didn't realize that only private subnets can have a NAT GW, so it makes perfect sense. BTW while going this rabbit hole I also found out that you actually CAN have a lambda access the internet from a public subnet, but an elastic IP needs to be attached to its ENI. I know it's silly but it's nice to know.
And it all changes again. AWS have listened regarding the cold start and ENI consumption issues and now support Hyperplain for Lambda, you still have to add the appropriate IAM permissions extra, but it looks like significant improvements : aws.amazon.com/blogs/compute/…
Technically the public/private subnet + NAT solution is great. But the NAT Gateway is quiet expensive. That's why I would like to avoid it!
|
29

You can make a lambda function access the public internet from within your VPC. Solution A is the actual answer, Solution B is a more elegant alternative solution.

Solution A - Lambda in VPC + Public IP associated with ENI

For accessing resources external to AWS such as Google API (like OP's example) you do need a Public IP. For other cases like RDS or S3 you don't need a Public IP, you can use a VPC Endpoint, so communication between your Lambda and the desired AWS Service doesn't leave AWS network.

By default some AWS Services are indeed reached via public internet, but it doesn't have to be.

Now if you want an actual external resource (e.g. google), you need to assign Elastic Public IPs to the Network Interfaces for each subnet linked to your lambda. First let's figure which subnets and security groups are linked to your lambda:

Lambda screen

Next, go to EC2 Service, find the Public IPs menu under Network & Security. Allocate one IP for each subnet (in the example above there are two subnets).

Go to Network Interfaces menu, find the network interfaces attached to your lambda (same subnet and security group).

Network Interfaces

Associate the Public IPs in the actions menu for each one:

Actions menu

Associate IP

That's it, now your Lambda can reach out to public internet.

[EDIT] Someone was concerned about Solution's A scalability issues saying each lambda instance has a new network interface but they missed this from AWS Docs:

"Multiple Lambda functions can share a network interface, if the functions share the same subnet and security group"

So whatever scalability issues you may face has nothing to do with this solution and how Lambda uses ENI, you'd face the same issues using EC2, ECS, EKS, not just Lambda.

Solution B - Decompose into multiple Lambdas

Requiring access both to external resources and VPC resources would seem like too much responsibility for a single function. You may want to rethink your design and decompose your single lambda function into at least two lambda functions:

  • Lambda A goes to external resources (e.g. Google API), fetches whatever data you need, add to SQS. No need to attach to VPC, no need to manually associate Elastic Public IP to ENI.
  • Lambda B processes the message from SQS, stores results to a storage (db, s3, efs, another queue, etc). This one lives within your VPC, and don't need external access.

This way seems more scalable, more secure, each individual lambda is less complex and more maintainable, the architecture looks better overall.

Of course life is not always rainbows and butterflies, so Solution A is good and scalable enough, but improving the architecture is even better.

Improve this answer

8 Comments

Does someone know when AWS allowed this? I'm finding lots of "doesn't work" material (which is old) but I also saw that this works now (at least for some time >=2021)
I think this method does not work reliably with lambdas as when it scales, each lambda instance has a new network interface. So while you can do this, it's not going to work when you scale out.
@Capaj from AWS docs: "Multiple Lambda functions can share a network interface, if the functions share the same subnet and security group". Also you must have a Public IP for reaching Public Internet, regardless of it being a Lambda, EC2, ECS, even a NAT Gateway needs an Elastic Public IP if you want to reach the public internet through it.
This indeed works for me. And instead of having NAT gateway which is expensive, this sounds good to me. But i hope there won't be any issues if we scale.
@Veer you'd only explicitly attach a VPC to your lambda if you need to access resources inside it. You'd also only need a Public IP (via NAT or ENI) if you need external resources (outside AWS). One way to improve things even more is trying to decompose your lambda into two smaller lambdas, one that gets access to external resources, another for VPC resources. e.g. lambda A gets data from external API, adds output to SQS, lambda B processes the message, store to database. I'll update my answer with this suggestion.
"Solution B - Decompose into multiple Lambdas" this is want I was looking for. Thanks a lot for this suggestion.
|
4

A bit late, but nonetheless. @jarmod answer is correct but I'd like to develop further on it. The explanation for the inability of a Lambda function linked to a public subnet to access the Internet is as follows:

Lambda functions operate within an AWS-managed Virtual Private Cloud (VPC). When you establish a "connection" between the Lambda function and your designated VPC, an Elastic Network Interface (ENI), specifically a Hyperplane ENI, is generated within the chosen subnets for Lambda's execution. This ENI assumes a "private" IP, directing all of the Lambda function's network traffic through it. Yet, why is a public IP not assigned to it???

This is because of how public IP assignment within AWS works. The configuration of a public IP for an EC2 instance doesn't reside within the instance itself; rather, it's orchestrated through Network Address Translation (NAT) within the Internet Gateway (IGW). Therefore, deploying an EC2 instance within a public subnet doesn't guarantee automatic possession of a public IP. This also applies to Lambda functions as well, but unlike EC2 instances, Lambdas lack the capacity for manual public IP assignment.

Hope it helps.

Improve this answer

Comments

0

This is a common wrong assumptions or misleading blog posts, among cloud developers, about how Lambda functions work within VPCs. Even though your Lambda function is in a public subnet, it cannot directly access the internet through the Internet Gateway (IGW). being Public doesnt mean it can access directly, And Implementation is like this because of:

  1. Lambda functions in a VPC do not receive public IP addresses, even when placed in a public subnet

  2. Internet Gateways only route traffic for resources that have public IP addresses

To enable internet access for your Lambda function in a VPC, you have two options:
Option 1: Use a NAT Gateway
Option 2: Use VPC Endpoints

The key reason is that Lambda functions in a VPC do not receive public IP addresses, even if they have placed in a public subnet. Here's why it happens like that, You need to understand:

  1. No Public IP Assignment

    • When a Lambda function is placed in a VPC, AWS assigns it a private IP address only

    • Without a public IP address, the Internet Gateway (IGW) cannot route return traffic back to the Lambda function

    • This is true even if the subnet is "public" (has a route table with IGW)

  2. How Internet Gateway Works

    • An Internet Gateway only works with resources that have public IP addresses

    • It needs the public IP to know where to send return traffic

    • Since Lambda functions don't get public IPs, the IGW can't route traffic back to them

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.