How to detect if `__init_subclass__` has been overridden in a subclass?-CodePudding

Normally in Python, it is possible to detect whether a method has been overridden in a subclass using the following technique:

>>> class Foo:
...     def mymethod(self): pass
...
>>> class Bar(Foo): pass
...
>>> Bar.mymethod is Foo.mymethod 
True

The expression Bar.mymethod is Foo.mymethod will evaluate to True if the method from Foo has not been overridden in Bar, but will evaluate to False if the method has been overridden in Bar. This technique works well with dunder methods inherited from object, as well as non-dunder methods:

>>> Bar.__new__ is Foo.__new__
True
>>> Bar.__eq__ is Foo.__eq__
True

We can formalise this logic in a function like so:

def method_has_been_overridden(superclass, subclass, method_name):
    """
    Return `True` if the method with the name `method_name`
    has been overridden in the subclass
    or an intermediate class in the method resolution order
    """
    if not issubclass(subclass, superclass):
        raise ValueError(
            "This function only makes sense if `subclass` is a subclass of `superclass`"
        )
    subclass_method = getattr(subclass, method_name)
    if not callable(method):
        raise ValueError(f"'{subclass.__name__}.{method_name}' is not a method")
    return subclass_method is not getattr(superclass, method_name, object())

However, this technique fails when it comes to two methods: __init_subclass__ and __subclasshook__:

>>> class Foo: pass
...
>>> class Bar(Foo): pass
...
>>> Bar.__init_subclass__ is Foo.__init_subclass__
False
>>> Bar.__subclasshook__ is Foo.__subclasshook__
False

And, for an even more perplexing example:

>>> type.__init_subclass__ is type.__init_subclass__
False

I have two questions:

Why does this technique fail with these methods, and these methods only? (I have not been able to find any other examples where this technique fails -- but if there are any, I'd love to know about them!)
Is there an alternative technique that can be used to detect if __init_subclass__ or __subclasshook__ have been defined in a subclass after being left undefined in the superclass?

CodePudding user response：

__init_subclass__ is special-cased to be a class method, whether you decorate it with classmethod or not. Just like Foo().mymethod returns a new method instance every time you access the attribute via a class instance, Foo.__init_subclass__ produces a new instance method every time you access the attribute via the class itself.

__subclasshook__, on the other hand, must be declared as a class method to work properly. It is not assumed to be a class method if you define it as a simple function/instance method.

CodePudding user response：

__init_subclass__ and __subclasshook__ are class methods. As you can see here:

>>> Bar.__sizeof__
<method '__sizeof__' of 'object' objects>
>>> Bar.__eq__
<slot wrapper '__eq__' of 'object' objects>
>>> Bar.__subclasshook__
<built-in method __subclasshook__ of type object at 0x000002D70AAC5340>
>>> Bar.__init_subclass__
<built-in method __init_subclass__ of type object at 0x000002D70AAC5340>
>>> Foo.__init_subclass__
<built-in method __init_subclass__ of type object at 0x000002D70AACAF70>
>>>

__init_subclass__ and __subclasshook__ refer to their classes that are different instances, Bar's hex is 0x000002D70AAC5340, and Foo's hex is 0x000002D70AACAF70.

As you can see in the documentation of __init_subclass__, it says:

classmethod object.__init_subclass__(cls)

It says "classmethod".

CodePudding user response：

__init_subclass__ is a special method, in that it is implicitly a classmethod even if you do not decorate it with @classmethod when defining it. However, the issue here does not arise from the fact that __init_subclass__ is a special method. Instead, there is a fundamental error in the technique you are using to detect whether a method has been overridden in a subclass: it won't work with any classmethods at all:

>>> class Foo:
...     def mymethod(self): pass
...     @classmethod
...     def my_classmethod(cls): pass
...
>>> class Bar(Foo): pass
...
>>> Bar.mymethod is Foo.mymethod
True
>>> Bar.my_classmethod is Foo.my_classmethod
False

This is because of how bound methods in Python work: methods, in Python, are descriptors.

Observe the equivalence of the following lines of code with respect to instance methods. The first (more normal) way of calling mymethod on the instance f is merely syntactic sugar for the second way of calling the method on the instance f:

>>> class Foo:
...     def mymethod(self):
...         print('Instance method')
... 
>>> f = Foo()
>>> f.mymethod()
Instance method
>>> Foo.__dict__['mymethod'].__get__(f, Foo)()
Instance method

Calling __get__ on the unbound method in Foo.__dict__ produces a new object each time; it is only by accessing the instance method on the class that we can test for identity, as you do in your question. However, with regards to classmethods, even accessing the method from the class will call __get__ on the method:

>>> class Foo:
...     @classmethod
...     def my_classmethod(cls):
...         print('Class method')
...
>>> Foo.my_classmethod is Foo.my_classmethod
False
>>> Foo.my_classmethod()
Class method
>>> Foo.__dict__['my_classmethod'].__get__(Foo, Foo)()
Class method

What about __new__?

Your question points out that your existing method works with __new__. That's strange -- we've just established that this method doesn't work with classmethods, and __new__ certainly looks like a classmethod. The first parameter of __new__ is named cls, after all! However, the Python documentation makes clear that this isn't the case at all:

object.__new__(cls[, ...])

Called to create a new instance of class cls. __new__() is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument.

It's a staticmethod, not a classmethod! Mystery solved.

A better way of detecting whether a subclass overrides a method from a superclass

The only sure-fire way of knowing for sure if a method has been overridden in a subclass is by traversing the __dict__ of each class in the method resolution order:

def method_has_been_overridden(superclass, subclass, method_name):
    """
    Return `True` if the method with the name `method_name`
    has been overridden in the subclass
    or an intermediate class in the method resolution order
    """
    if not issubclass(subclass, superclass):
        raise ValueError(
            "This function only makes sense if `subclass` is a subclass of `superclass`"
        )
    subclass_method = getattr(subclass, method_name)
    if not callable(subclass_method):
        raise ValueError(f"'{subclass.__name__}.{method_name}' is not a method")
    for cls in subclass.__mro__:
        if cls is superclass:
            return False
        if method_name in cls.__dict__:
            return True

This function can correctly determine whether or not __init_subclass__, or any other classmethod, has been overridden in a subclass:

>>> class Foo: pass
...
>>> class Bar(Foo): pass
...
>>> class Baz(Foo):
...     def __init_subclass__(cls, *args, **kwargs):
...         return super().__init_subclass__(*args, **kwargs)
>>> method_has_been_overridden(Foo, Bar, '__init_subclass__')
False
>>> method_has_been_overridden(Foo, Baz, '__init_subclass__')
True

Many thanks to @chepner and U12-Forward, whose excellent answers helped me figure this problem out.