I've built a web scraper in python using seleniumwire that runs fine on my local machine. I've adjusted it for AWS Lambda, but when I went to deploy I received lots of various errors with components that seleniumwire needs to run. After a lot of troubleshooting and creating lambda packages I've gotten stuck at this error:

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named '_cffi_backend' Traceback (most recent call last):

I know that Lambda cannot support the arm architecture and I've been using docker to generate the required packages for this, however am stuck with this, as creating and using the package that contains the cffi module, still doesn't get me through.

So we come to my ask: I'd like to create an environment on either my machine, or on an EC2 machine, or any other suggestions are welcome. Where I can have the same OS/environment that Lambda uses, that I can ssh/connect into, and 1. run my code to ensure it's working. 2. create a deployment package from that machine.

I really appreciate you reading this, and am grateful for your feedback.

CodePudding user response：

Instead of uploading your code as an image to lambda, experiment adding your dependencies as layers and code your function directly on the console.

Here is a video of how you can create layers to add libraries and other dependencies on your lambda function.

Create layer lambda function

CodePudding user response：

I ended up creating an EC2 instance with AWS-Linux.

I then followed the instructions found here to build the layers needed for my code within the Linux environment. These instructions were amazing. Ultimately the issue for the layers was

1. Folder structure
2. Compiling on ARM architecture.