Writing address of a numpy array to a file and then opening it in C via ctypes-CodePudding

I was wondering if it's possible to actually write in a file an address of a numpy array, via e.g. ctypeslib.ndpointer or something similar and then open this file in a C function, also called through ctypes in the same python process and read this address, convert it to e.g. C double array.

This will all be happening in the same python process.

I am aware that it's possible to pass it as a function argument and that works, but that isn't something I'd need.

This is how the code would look like, don't mind the syntax errors:

test.py
with open(path) as f:
  f.write(matrix.ctypes.data_as(np.ctypeslib.ndpointer(dtype=np.float64, ndim=2, flags='C_CONTIGUOUS')))

and cpp:

void function()
{
... read file, get address stored into double* array;
e.g. then print out the values
}

Where could I be wrong?

I work on a project where we are writing np array to a file and then reading that file in cpp, which is wasteful. I want to try adjusting it to write and later on read just this address. Sending a ndpointer or something else as a function argument wont work, as that would require editing big partion of the project.

CodePudding user response：

I think that the data of your np.array will be lost once the python program terminates therefore you will not be able to access its memory location once the program ends.

Unfortunately, I don't know how to do it using ctypes but only using the C-API Extention. With it, you access directly the python variable from c. It is represented by a pointer therefore you could access the address of any python object( therefore also ndarrays).

in python you would write:

import c_module
import NumPy as np
...
a = np.array([...])
#generate the numpy array
...
c_module.c_fun(a)

and then in your c code, you will receive the memory address

static PyObject* py_f_roots(PyObject* self, PyObject* args) {
  PyObject *np_array_py;
  if (!PyArg_ParseTuple(args, "OO", &np_array_py))
    return NULL;
  //now np_array_py points to the memory cell of the python numpy array a
  //if you want to access it you need to cast it to a PyArrayObject *
  PyArrayObject *np_array = (PyArrayObject *) np_array_py;
  //you can access the data
  double *data = (double *) PyArray_DATA(np_array);
  
 return Py_None;
}

The documentation for numpy c API

The reference manual for c python extention

CodePudding user response：

If the Python and C code are run in the same process, then the address you write from Python will be valid in C. I think you want the following:

test.py

import ctypes as ct
import numpy as np

matrix = np.array([1.1,2.2,3.3,4.4,5.5])

# use binary to write the address
with open('addr.bin','wb') as f:
    # type of pointer doesn't matter just need the address
    f.write(matrix.ctypes.data_as(ct.c_void_p))

# test function to receive the filename
dll = ct.CDLL('./test')
dll.func.argtypes = ct.c_char_p,
dll.func.restype = None

dll.func(b'addr.bin')

test.c

#include <stdio.h>

__declspec(dllexport)
void func(const char* file) {
    double* p;
    FILE* fp = fopen(file,"rb");  // read the pointer
    fread(&p, 1, sizeof(p), fp);
    fclose(fp);
    for(int i = 0; i < 5;   i)    // dump the elements
        printf("%lf\n", p[i]);
}

Output: