0

I'm trying to create a EKS cluster in a private subnet. I'm having issues getting it working. I get the error unhealthy nodes in the kubernetes cluster. Wonder if its due to security group or some other issues like VPC endpoints?

When I use NAT gateway setup then it works fine. But I don't want to use nat gateway anymore.

One think I'm not sure is should the EKS cluster subnet_ids be only private subnets?

In the below config I'm using both public and private subnets.

resource "aws_eks_cluster" "main" { name = var.eks_cluster_name role_arn = aws_iam_role.eks_cluster.arn vpc_config { subnet_ids = concat(var.public_subnet_ids, var.private_subnet_ids) security_group_ids = [aws_security_group.eks_cluster.id, aws_security_group.eks_nodes.id, aws_security_group.external_access.id] endpoint_private_access = true endpoint_public_access = false } # Ensure that IAM Role permissions are created before and deleted after EKS Cluster handling. # Otherwise, EKS will not be able to properly delete EKS managed EC2 infrastructure such as Security Groups. depends_on = [ "aws_iam_role_policy_attachment.aws_eks_cluster_policy", "aws_iam_role_policy_attachment.aws_eks_service_policy" ] }
Improve this question
4
  • 2
    Do you have any NAT gateway to provide internet access for nodes in private subnets? Commented Jan 6, 2021 at 6:50
  • No nat gateway. I don't want to use Nat gateway since its price is higher. I had a different setup which had nat gateway and that worked fine. But I want to implement without a nat gateway and use vpc endpoints. Commented Jan 6, 2021 at 8:47
  • What should be the method for using VPC endpoints instead of nat gateway? Commented Jan 6, 2021 at 8:48
  • Did you ever figure this out without a NAT gateway? Commented Jul 30, 2021 at 1:30

1 Answer 1

Reset to default
1

Since you don't have NAT gateway/instance, your nodes can't connect to the internet and fail as they can't "communicate with the control plane and other AWS services" (from here).

Thus, you can use VPC endpoints to enable communication with the plain and the services. To view the properly setup VPC with private subnets for EKS, you can check AWS provided VPC template for EKS (from here).

From the template, the VPC endpoints in us-east-1:

  • com.amazonaws.us-east-1.ec2
  • com.amazonaws.us-east-1.ecr.api
  • com.amazonaws.us-east-1.s3
  • com.amazonaws.us-east-1.logs
  • com.amazonaws.us-east-1.ecr.dkr
  • com.amazonaws.us-east-1.sts

Please note that all these endpoints, escept S3, are not free. So you have to consider if running cheap NAT instances or gateway would be cheaper or more expensive then maintaining these endpoints.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.