How to install python modules in a Azure function app-CodePudding

I need help debugging this or working out how to run this code in Azure functions. it's currently failing and I'm unsure on pythons modules. Assumed it trying to install but is failing at the first step

my error

Result: Failure Exception: ModuleNotFoundError: No module named 'azure.storage' Stack: File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py", line 458, in _handle__invocation_request call_result = await self._loop.run_in_executor( File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/dispatcher.py", line 701, in _run_sync_func return ExtensionManager.get_sync_invocation_wrapper(context, File "/azure-functions-host/workers/python/3.8/LINUX/X64/azure_functions_worker/extension.py", line 215, in _raw_invocation_wrapper result = function(**args) File "/home/site/wwwroot/httptrigger-pmaas-data-transformation/init.py", line 30, in main from .run_transformation import run File "/home/site/wwwroot/httptrigger-pmaas-data-

My __Init.py

import azure.functions as func 
import logging
import os
from subprocess import check_call
try:
    check_call(['python3', '-m', 'pip', 'install', 'numpy==1.21.4'])
    check_call(['python3', '-m', 'pip', 'install', 'pandas==1.2.4'])
    check_call(['python3', '-m', 'pip', 'install', 'azure-storage-blob==12.8.0'])
    check_call(["python3", '-m', 'pip', 'install', 'openpyxl==3.0.10'])
    from .run_transformation import run
    from azure.storage.blob import BlobServiceClient
except:
    pass  def main(req: func.HttpRequest) -> func.HttpResponse:
    try:
        check_call(['python3', '-m', 'pip', 'install', 'numpy==1.21.4'])
        check_call(['python3', '-m', 'pip', 'install', 'pandas==1.2.4'])
        check_call(['python3', '-m', 'pip', 'install', 'azure-storage-blob==12.8.0'])
        check_call(["python3", '-m', 'pip', 'install', 'openpyxl==3.0.10'])
        from .run_transformation import run
        from azure.storage.blob import BlobServiceClient
    except:
        check_call(['python3', '-m', 'pip', 'install', 'numpy==1.21.4'])
        check_call(['python3', '-m', 'pip', 'install', 'pandas==1.2.4'])
        check_call(['python3', '-m', 'pip', 'install', 'azure-storage-blob==12.8.0'])
        check_call(["python3", '-m', 'pip', 'install', 'openpyxl==3.0.10'])
        from .run_transformation import run
        from azure.storage.blob import BlobServiceClient
    logging.info('Starting Data Transformation')
    connectionString = os.environ['SA_CONNECTION_STRING']
    exceptions = run(connectionString)     if len(exceptions) > 0:
        return func.HttpResponse(
            "Failed! :: One more files were not processed. List of exceptions. {exceptions}",
            status_code=200
        )
    return func.HttpResponse(
            "Success! :: Data Transfomration is completed. Check log for more details and troubleshooting",
            status_code=200
    )

Thanks

CodePudding user response：

The modules you would like to install should be included in the requirements.txt file. The requirements.txt contains the list of Python packages the system installs when publishing to Azure. Assuming that you are using the v1 programming model, you can find more details here.

In your case the requirements.txt file would contain the corresponding modules...

numpy==1.21.4
pandas==1.2.4
azure-storage-blob==12.8.0
openpyxl==3.0.10

... which will then be installed during publishing so that you import them into your code as usual.

Hope this helps.

CodePudding user response：

Thank you @holger for pointing in the right direction. After reproducing from my end, I faced the same issue but after adding the modules in requirements.txt file I could able to solve this. I have achieved your requirement using

Popen(['pip', 'freeze'], stdout=open('requirements.txt', 'w'))

Below is the complete code that worked for me.

import azure.functions as func 
import logging
import os
from subprocess import check_call, Popen

try:
    check_call(['pip', 'install', 'numpy==1.21.4'])
    check_call(['pip', 'install', 'pandas==1.2.4'])
    check_call(['pip', 'install', 'azure-storage-blob==12.8.0'])
    check_call(['pip', 'install', 'openpyxl==3.0.10'])
    from .run_transformation import run
    from azure.storage.blob import BlobServiceClient
    
except:
    def main(req: func.HttpRequest) -> func.HttpResponse:
        try:
            check_call(['pip', 'install', 'numpy==1.21.4'])
            check_call(['pip', 'install', 'pandas==1.2.4'])
            check_call(['pip', 'install', 'azure-storage-blob==12.8.0'])
            check_call(['pip', 'install', 'openpyxl==3.0.10'])
            from .run_transformation import run
            from azure.storage.blob import BlobServiceClient
        except:
            check_call(['pip', 'install', 'numpy==1.21.4'])
            check_call(['pip', 'install', 'pandas==1.2.4'])
            check_call(['pip', 'install', 'azure-storage-blob==12.8.0'])
            check_call(['pip', 'install', 'openpyxl==3.0.10'])
            Popen(['pip', 'freeze'], stdout=open('requirements.txt', 'w'))
            
            from .run_transformation import run
            from azure.storage.blob import BlobServiceClient
            
            logging.info('Starting Data Transformation')
            connectionString = os.environ['SA_CONNECTION_STRING']
            exceptions = run(connectionString) 
                
            if len(exceptions) > 0:
                return func.HttpResponse(
            "Failed! :: One more files were not processed. List of exceptions. {exceptions}",
            status_code=200
            )
                
        return func.HttpResponse(
            "Success! :: Data Transfomration is completed. Check log for more details and troubleshooting",
            status_code=200
        )

RESULTS:

enter image description here