Elastic Beanstalk setup with public ALB and EC2 on private subnet falling health check-CodePudding

I am trying to setup a sample Elastic beanstalk app with ALB being in public subnets(internet facing) and ec2 instances in private subnets in terraform. If I put ec2 instances in public subnets then the elastic beanstalk app get created successfully but in private subnets I get the following error.

The EC2 instances failed to communicate with AWS Elastic Beanstalk, either because of configuration problems with the VPC or a failed EC2 instance. Check your VPC configuration and try launching the environment again.

aws_elastic_beanstalk_environment

setting {
    namespace = "aws:ec2:vpc"
    name      = "Subnets"
    value     = join(",", module.vpc.private_subnets) 
  }

  setting {
    namespace = "aws:ec2:vpc"
    name      = "DBSubnets"
    value     = join(",", module.vpc.private_subnets)
  }

  setting {
    namespace = "aws:ec2:vpc"
    name      = "ELBSubnets"
    value     = join(",", module.vpc.public_subnets)
  }


  setting {
    namespace = "aws:ec2:vpc"
    name      = "AssociatePublicIpAddress"
    value     =  "false"
  }

I have also setup vpc endpoints as describe in https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-instance-failure/

module "endpoints" {
  source  = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"

  vpc_id = module.vpc.vpc_id
  security_group_ids = [data.aws_security_group.default.id]

  endpoints = {
    dynamodb = {
      service      = "dynamodb",
      service_type = "Gateway"
      route_table_ids = module.vpc.private_route_table_ids
      tags            = { Name = "dynamodb-vpc-endpoint" }
    },
    s3 = {
      service      = "s3",
      service_type = "Gateway"
      route_table_ids = module.vpc.private_route_table_ids
      tags            = { Name = "s3-vpc-endpoint" }
    },
    elasticbeanstalk-app = {
      # interface endpoint
      service_name             = aws_vpc_endpoint_service.elasticbeanstalk.service_name
      subnet_ids = module.vpc.private_subnets
      tags                = { Name = "elasticbeanstalk-app-vpc-endpoint" }
    },
    elasticbeanstalk = {
      # interface endpoint
      service_name             = "com.amazonaws.${var.aws_region}.elasticbeanstalk"
      subnet_ids = module.vpc.private_subnets
      private_dns_enabled = true
      tags                = { Name = "elasticbeanstalk-${var.aws_region}-elasticbeanstalk-vpc-endpoint" }
    }
    elasticbeanstalk-hc = {
      # interface endpoint
      service_name             = "com.amazonaws.${var.aws_region}.elasticbeanstalk-health"
      subnet_ids = module.vpc.private_subnets
      private_dns_enabled = true
      tags                = { Name = "elasticbeanstalk-${var.aws_region}-elasticbeanstalk-health-vpc-endpoint" }
    },
    sqs = {
      # interface endpoint
      service_name             = "com.amazonaws.${var.aws_region}.sqs"
      subnet_ids = module.vpc.private_subnets
      private_dns_enabled = true
      tags                = { Name = "elasticbeanstalk-${var.aws_region}-sqs-vpc-endpoint" }
    },
    cloudformation = {
      # interface endpoint
      service_name             = "com.amazonaws.${var.aws_region}.cloudformation"
      subnet_ids = module.vpc.private_subnets
      private_dns_enabled = true
      tags                = { Name = "elasticbeanstalk-${var.aws_region}-cloudformation-vpc-endpoint" }
    },
    ec2 = {
      # interface endpoint
      service_name             = "com.amazonaws.${var.aws_region}.ec2"
      subnet_ids = module.vpc.private_subnets
      private_dns_enabled = true
      tags                = { Name = "elasticbeanstalk-${var.aws_region}-ec2-vpc-endpoint" }
    },
    ec2messages = {
      # interface endpoint
      service_name             = "com.amazonaws.${var.aws_region}.ec2messages"
      subnet_ids = module.vpc.private_subnets
      private_dns_enabled = true
      tags                = { Name = "elasticbeanstalk-${var.aws_region}-ec2messages-vpc-endpoint" }
    },
  }
}

I have a vpc endpoint even for the elasticbeanstalk-app .The setup based on AWS beanstalk PrivateLink not connecting .

Security group

data "aws_security_group" "default" {
  name   = "default"
  vpc_id = module.vpc.vpc_id
}

data "aws_vpc_endpoint_service" "dynamodb" {
  service = "dynamodb"

  filter {
    name   = "service-type"
    values = ["Gateway"]
  }
}

data "aws_vpc_endpoint_service" "s3" {
  service = "s3"

  filter {
    name   = "service-type"
    values = ["Gateway"]
  }
}

CodePudding user response：

In order to be able to connect to service endpoints such as com.amazonaws.[aws_region].elasticbeanstal or com.amazonaws.[aws_region].elasticbeanstalk-health you need to have a security group which allows HTTP/HTTPS inbound connection.

My assumption is that aws_security_group.default security group, which is referenced from a data block, is a default security group and it does not allow HTTP/HTTPS inbound connectivity.