Using AWS CDK v2, I want to deploy two lambda functions. As both depend on pandas, I want to deploy a layer first, which has pandas and its dependencies in requirements.txt located at entry.

One of the lambda functions also depends on pyarrow, which I put into the requirements.txt located at entry of that lambda function. As both pyarrow and pandas depend numpy, I currently end up with numpy and its dependencies being installed both in the layer and that lambda function, which obviously isn't what I want.

In the end, I'm looking for a way to pass --no-deps to the pip installation of requirements.txt of the lambda function that depends on pyarrow. Any ideas how to achieve this?

CodePudding user response：

I don't think this is possible to specify no-deps when using a requirements.txt. CDK also can use pipenv's Pipfile and Poetry's poetry.lock. That being said, in my research none of those tools seem to support installing with --no-deps either unless you manually mess with the lock files (not advised or maintainable really).

Looking deeper at the specific problem, I don't see pyarrow listing pandas as a dependency but I do see it installing numpy which is a dependency of pandas

I think your best option presently is to just deal with having numpy in both places or put both pyarrow and pandas in the layer.