I manage our ECS tasks with terraform and it's been all well and good but recently, I've run into this error when the task tries to fetch the image from ECR:

CannotPullContainerError: failed to register layer: devmapper: Thin Pool has 4143 free data blocks which is less than minimum required 4449 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior

I'm a bit confused here on what to update. Our ECS service uses an EC2 launch type, but I think I have a fundamental misunderstanding of what's happening here. Reading https://docs.aws.amazon.com/AmazonECS/latest/developerguide/CannotCreateContainerError.html it seems like it's because the volume is too small but the AWS help page suggests docker prune. This confuses me since it was my understanding that the ECS task would have access to the full volume as a fresh start every time -- why would there be stale images/containers on the volume? Or is the page just suggesting that in cases where your ECS task might be fetching too multiple images, you should remove stale ones.

Other than that, I'm not sure how to proceed. Would increasing the ephemeral_storagein the aws_ecs_task_definition help here, or do I need to manage my own volume that's attached to the aws_launch_template instance that's tied to the aws_autoscaling_group that's tied to relevant aws_ecs_service

CodePudding user response：

Just had to add this block to my aws_launch_template

resource "aws_launch_template" "instance" {
  ...
  block_device_mappings {
    device_name = "/dev/xvdcz"

    ebs {
      volume_size           = 50
      delete_on_termination = true
    }
  }

}