Implementing decorators in a Python C/C extension that can wrap/decorate functions in Python-CodePudding

Python decorators are a very "pythonic" solution to a lot of problems. Because of this, I'd like to include a pre-defined decorator in my C-extension that can decorate functions that are called in Python files that include my extension.

I can't seem to find anything in the CPython api documentation that describes how write decorators. Any help/direction would be appreciated.

Specifically, I'd like the Python code to look like the following:

import my_c_extension as m

@m.my_decorator(1)
def func():
    pass


func()

Where my_decorator would use the argument '1' to carry out some functionality (written in c inside my_c_extension) before func is called, and more functionality after func is called.

Thanks in advance!

CodePudding user response：

For problems like this, it's often helpful to first mock things up in Python. In your case, I expected there'd be two benefits:

If you do end up implementing the decorator entirely in an extension module, this will help you understand how it should work and what state will be held by which objects you'll be creating.
Often, you can simplify the problem by doing most of the work in Python.

For the sake of argument, let's say what you're really trying to do is use a C extension to help sample low-level CPU counters for performance tuning. Here would be a way to achieve that with a hybrid of Python and C. All the complicated decorator magic stays in Python while only the functionality that actually needs to be in C is in C.

def print_cpu_counter_info(counter_id):
    def wrapper(func):
        def wrapped(*args, **kwargs):
            before = my_c_extension.get_counter_value(counter_id)
            ret = func(*args, **kwargs)
            after = my_c_extension.get_counter_value(counter_id)
            print(f'counter {counter_id}: {before} -> {after}')
            return ret
        return wrapped
    return wrapper

@print_cpu_counter_info(L3_CACHE_LINE_MISS_COUNTER_ID)
def my_slow_func(...):
    ...

If that's not viable (too slow, more complicated than this, etc.), then you'll need to create extension objects in C that replicate the behavior of the wrapper and wrapped functions. It's definitely doable, but it'll take a bit of work, and it'll be harder to maintain. Expect to write hundreds to thousands of lines of C code to replicate what only took a few lines of Python code in the above example.