Currently all APIs related to Py_buffer are excluded from the limited API. It's not possible to use Py_buffer from a C extension with limited API nor is it possible to define heap types with buffer support using the stable ABI (Application Binary Interface).

The lack of Py_buffer support prevents prominent projects like NumPy or Pillow to use the limited API and produce abi3 binary wheel. To be fair it's not the only reason why these projects have not adopted the stable abi3 yet. Still Py_buffer is a necessary but not sufficient condition. Stable abi3 support would enable NumPy stack to build binary wheels that work with any Python version 3.11 <= and < 4.0.

The limited API excludes any C API that references Py_buffer:

• 8 PyBuffer_*() functions
• 21 PyBUF_* constants
• PyMemoryView_FromBuffer()
• PyObject_GetBuffer
• Py_bf_getbuffer / Py_bf_releasebuffer type slots for PyBufferProcs

It should not be terribly complicated to add Py_buffer to the stable API. All it takes are an opaque struct definition of Py_buffer, an allocate function, a free function and a bunch of getters and setters. The hard part is to figure out which getters and setters are needed and how many struct members must be exposed by getters and setters. I recommend to get feedback from NumPy, Pillow, and Cython devs first.

Prototype

typedef struct bufferinfo Py_buffer;

/* allocate a new Py_buffer object on the heap and initialize all members to
   NULL / 0
*/
Py_buffer*
PyBuffer_New()
{
    Py_buffer *view = PyMem_Calloc(1, sizeof(Py_buffer));
    if (view == NULL) {
        PyErr_NoMemory();
    }
    return view;
}

/* convenience function */
Py_buffer*
PyBuffer_NewEx(PyObject *obj, void *buf,  Py_ssize_t len, Py_ssize_t itemsize,
               int readonly, int ndim, char *format, Py_ssize_t *shape,
               Py_ssize_t *strides, Py_ssize_t *suboffsets, void *internal)
{
    ...
}

/* release and free buffer */
void
PyBuffer_Free(Py_buffer *view)
{
    if (view != NULL) {
        PyBuffer_Release(view);
        PyMem_Free(view);
    }
}

CodePudding user response：

Py_buffer.shape requires a Py_ssize_t* pointer. It's not convenient. For example, the array module uses:

static int
array_buffer_getbuf(arrayobject *self, Py_buffer *view, int flags)
{
    ...
    if ((flags & PyBUF_ND)==PyBUF_ND) {
        view->shape = &((PyVarObject*)self)->ob_size;
    }
    ...
    return 0;
}

IIRC shape, strides, and suboffsets are all arrays of ndims length.

We could optimize allocation if we would require users to specify the value of ndims and don't allow them to change the value afterwards. PyBuffer_New(int ndims) then would allocate view of size sizeof(Py_buffer) (3 * ndims * sizeof(Py_ssize_t *)). This would give us sufficient space to memcpy() shape, strides, and suboffsets arguments into memory after the Py_buffer struct.