I am trying to deploy a simple function as shown below:

import pandas as pd
import glob

def hello_gcs(event, context):
    """Triggered by a change to a Cloud Storage bucket.
    Args:
         event (dict): Event payload.
         context (google.cloud.functions.Context): Metadata for the event.
    """
    file = event
    print(f"Processing file: {file['name']}.")


    paths = []
    all_dfs = []
    for files in glob.glob("gs://BUCKET/*/*.csv"):
      paths.append(files)
    
    for i in range(len(paths)):

      temp_list = (paths[i].split("_"))
      date_temp_list = (temp_list[2])
      read_date = (date_temp_list.split("T")[0])

      globals()['table%s' % i] = pd.read_csv('{}' .format(paths[i]), index_col=None, header=0) #create new dfs based on subfolder structure
      globals()['table%s' % i]["Read Date"] = read_date
      all_dfs.append(globals()['table%s' % i])

    output_df = pd.concat(all_dfs, axis=0, ignore_index=True)
    output_df.to_csv("NAME.csv")  

With my requirements txt:

pandas==1.0.3
google-cloud-storage==1.26.0
numpy==1.23.0

I am unable to deploy the function and returned with an error:

Deployment failure: Build failed: ... In file included from /opt/python3.9/include/python3.9/unicodeobject.h:1026:0, from /opt/python3.9/include/python3.9/Python.h:93, from numpy/random/mtrand/mtrand.c:4: /opt/python3.9/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( ^~~~~~~~~~~~~~~~~~~ numpy/random/mtrand/mtrand.c:45406:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations] (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 : ^ In file included from /opt/python3.9/include/python3.9/unicodeobject.h:1026:0, from /opt/python3.9/include/python3.9/Python.h:93, from numpy/random/mtrand/mtrand.c:4: /opt/python3.9/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) { ^~~~~~~~~~~~~~~~~~~~~~~~~~ error: Command "x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fstack-protector-strong -g -Wformat -Werror=format-security -fPIC -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.9/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/opt/python3.9/include/python3.9 -Ibuild/src.linux-x86_64-3.9/numpy/core/src/private -Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.9/numpy/core/src/private -Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.9/numpy/core/src/private -Ibuild/src.linux-x86_64-3.9/numpy/core/src/npymath -c numpy/random/mtrand/mtrand.c -o build/temp.linux-x86_64-3.9/numpy/random/mtrand/mtrand.o -MMD -MF build/temp.linux-x86_64-3.9/numpy/random/mtrand/mtrand.o.d" failed with exit status 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> numpy

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure. [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.; Error ID: c84b3231

Now:

1. In my code I am not using Numpy; I've added it to Requirements txt just to check if that would solve the error - it didn't
2. I checked if numpy was installed at all by going to Shell and running python -m pip list; It's there

Is there anything that I am missing here in the invokation or code part? I have no explanation for this error to happen

I would appreciate any help!

CodePudding user response：

Pandas depends on Numpy so you need to have it.

According to their docs for Pandas 1.0.3 it has the following dependencies:

setuptools=24.2.0

NumPy=1.13.3

python-dateutil=2.6.1

pytz=2017.2

CodePudding user response：

so after contacting GCP Support turns out that this is an error that shouldn't be there and internal produc teams will work on resolving it.

Quick solution they proposed me, which worked was to switch the runtime from Python 3.9 to Python 3.8