I have set up EFS to mount on a lambda, I seem to have it working correctly, the Lambda is able to mount the filesystem and has read/write access.

The issue I'm having is that CloudFormation freezes when I try to destroy the stack completely and also when I tried to change the ID of the lambda.

import * as cdk from 'aws-cdk-lib';
import {RemovalPolicy} from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as efs from 'aws-cdk-lib/aws-efs';
import {ThroughputMode} from 'aws-cdk-lib/aws-efs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import {Architecture} from 'aws-cdk-lib/aws-lambda';
import {Construct} from 'constructs';
import * as path from "path";

export class SimilarityEmbeddingsCdkStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props?: cdk.StackProps) {
        super(scope, id, props);
        const vpc = new ec2.Vpc(this, 'VPC', {
            maxAzs: 2
        });

        const efsSecurityGroup = new ec2.SecurityGroup(this, 'EfsSimilarityEmbeddingSecurityGroup', {
            vpc
        });

        const fs = new efs.FileSystem(this, 'EfsSimilarityEmbeddingFileSystem', {
            vpc: vpc,
            performanceMode: efs.PerformanceMode.GENERAL_PURPOSE,
            throughputMode: ThroughputMode.BURSTING,
            removalPolicy: RemovalPolicy.DESTROY,
            securityGroup: efsSecurityGroup
        });

        const accessPoint = fs.addAccessPoint('LambdaAccessPoint', {
            path: '/export/lambda',
            createAcl: {
                ownerGid: '1001',
                ownerUid: '1001',
                permissions: '755',
            },
            posixUser: {
                uid: '1001',
                gid: '1001',
            }
        });

        const lambdaSecurityGroup = new ec2.SecurityGroup(this, 'SimilarityEmbeddingLambdaSecurityGroup', {
            vpc
        });

        const createEmbeddingHandler = new lambda.DockerImageFunction(this, 'CreateEmbeddingLambdaFunction', {
            code: lambda.DockerImageCode.fromImageAsset(
                path.join(__dirname, '../lambdas'),
                {
                    cmd: ["create_embedding.index.handler"]
                }
            ),
            architecture: Architecture.ARM_64,
            securityGroups: [lambdaSecurityGroup],
            filesystem: lambda.FileSystem.fromEfsAccessPoint(accessPoint, '/mnt/filesystem'),
            vpc,
        });
    }
}

Update

After further attempts, I left it running for ~20 mins, and it eventually succeeds in destroying the stack. I guess that as long as the file system is mounted on the Lambda, it can't be removed.

Is there any way around this, or do I just have to wait it out?

CodePudding user response：

Your Lambda function is using a network interface in your VPC. This is (most likely) the reason why the function takes some time to delete. Network interfaces can't be deleted while the network interface is in use. The network interface will be in use if the function is running or even in a "warm" state.

Even if your function is not running, AWS may keep the function 'warm' for a while (keeping the interface in use, preventing deletion).

A trick you can try to use to speed up this process is to first modify the network security group to disallow all traffic first, which will stop all connections allowing the interface (and subsequently the function) to be deleted faster.

Another trick to help dispose of 'warm' lambda instances is to update the function environment (like increasing or decreasing memory) and setting its concurrency to 0 (to prevent new invocations/warm instances). That may also help your delete run faster.

It's also important to make sure you don't have cyclical dependencies, like if you're using that lambda for a custom resource provider that is also involved in the stack delete.