I'm trying to understand how it works. I know that Python interpreter translates python source code to byte code representation for a virtual machine (Python interpreter is a virtual machine) and executes those instructions. Where exactly does a C/C code comes in here? Does this virtual machine (Python interpreter) can also compile and execute C/C code?

I don't even know exactly what are right questions to ask here, just want a good explanation of how.

My background: I programmed in Python for a long time (mostly analytics/ML) and I have some basic understanding of computer systems, C compilation process, memory and processor. But I am not even close to being an expert on it.

I just want good understanding, not so much practical tips on how to create a Python module in C.

Thank you, I really appreciate your help!

CodePudding user response：

It's all about a predictable entrypoint. The CPython reference interpreter (and other interpreters like PyPy that support C extensions of this sort), told to look for a module of a given name and finding a file matching the naming conventions for extension modules in one of the sys.path directories (e.g. for the spam module built for CPython 3.10 on x86-64 Linux, it would look for spam.cpython-310-x86_64-linux-gnu.so):

1. Uses the OS standard method for loading a dynamic library (aka shared object), e.g. LoadLibrary on Windows, dlopen on POSIX systems
2. Loads the entrypoint (using GetProcAddress on Windows, dlsym on POSIX) in it matching the specified naming convention, e.g. for the module named spam, it looks for a function named PyInit_spam following C name-mangling rules
3. Invokes that function, which is then wholly responsible for all other setup (calling PyModule_Create, performing any modifications to said module object, and returning it). The various APIs it invokes are what publish information for use by the user.

CodePudding user response：

CPython, the "standard" Python interpreter, is written in C. It provides an extension API in C, so extensions written in C or C can register themselves to be called like normal Python modules. The Python interpreter cannot compile C or C ; it is the extension writer's responsibility to compile the module. But Python can run arbitrary C and C code through the help of that API.