I'm trying to bring together typing class attributes via generic types pointing at the current instance and inheritance. I ran into a problem that mypy seems to fail in narrowing down generic types bound by the class at creation to the same type as the current self. I want to know if it's a bug or misunderstanding on my side.
Before we start:
- python version: 3.10.6
- mypy version: 0.991
Consider a simple scenario of implementing a Node of a tree structure.
Naturally, such a node should have parent and children attributes.
These can be optional. But, when they are not None, they should be of the same class as Node.
Let me illustrate that below:
from __future__ import annotations
from collections import abc
class Node:
def __init__(
self, parent: Node | None = None, children: abc.Sequence[Node] | None = None
):
self.parent = parent
self.children = children
Now, if we are to inherit from Node without changing the __init__'s signature, we'd run into trouble: type of children doesn't reflect the inheritance:
from typing_extensions import reveal_type
class SpecialNode(Node):
...
sn = SpecialNode()
reveal_type(sn.children) # Union[Sequence[Node], None]
The correct way out (until PEP-673 is supported, see #11871) should be using generic types bound by Node (see this answer for instance):
import typing as t
T = t.TypeVar('T', bound='Node')
class Node:
def __init__(
self: T, parent: T | None = None, children: abc.Sequence[T] | None = None
):
self.parent = parent
self.children = children
However, inheriting from this class doesn't narrow the type T down to the subclass for the attributes:
sn = SpecialNode()
reveal_type(sn.children) # Union[Sequence[T`-1], None]
(It's not really clear to me at the moment what -1 stands for, but I assume T is still a type variable bound by Node)
The problem here is that if we were to use sn.children in a function accepting Sequence[SpecialNode], mypy would not see Sequence[T`-1] as an equivalent type and treat this attempt as an error.
def fn(xs: abc.Sequence[SpecialNode]) -> None:
...
if sn.children is not None:
fn(sn.children) # Argument 1 has incompatible type "Sequence[T]"
What I would want instead is for sn.children to have the type Union[Sequence[SpecialNode], None] or a type variable TypeVar('S', bound=SpecialNode).
One could also attempt to manually narrow down the T type to the current instance's type:
def is_type(x: t.Any, _type: t.Type[T]) -> t.TypeGuard[T]:
return isinstance(x, _type)
def is_seq_of(s: abc.Sequence[t.Any], _type: t.Type[T]) -> t.TypeGuard[abc.Sequence[T]]:
return all(isinstance(x, _type) for x in s)
class Node:
def __init__(
self: T, parent: T | None = None, children: abc.Sequence[T] | None = None
):
assert parent is None or is_type(parent, self.__class__)
assert children is None or is_seq_of(children, self.__class__)
self.parent = parent
self.children = children
reveal_type(self.parent) # Union[T`-1, None]
reveal_type(self.children) # Union[Sequence[T`-1], None`]
sn = SpecialNode()
reveal_type(sn.children) # Union[Sequence[T`-1], None]
However, the revealed type is still Union[Sequence[T`-1], None]; hence the type narrowing didn't work out.
CodePudding user response：
Let me begin with sane solution, that uses master branch of mypy where Self is already supported (it will be added in v 1.0 release).
from collections import abc
import sys
# You may omit this guard and pick the import for your version;
# typing_extensions import will be valid forever,
# but stl is preferrable in general
if sys.version_info < (3, 11):
from typing_extensions import Self
else:
from typing import Self
class Node:
def __init__(
self, parent: Self | None = None, children: abc.Sequence[Self] | None = None
):
self.parent = parent
self.children = children
class SpecialNode(Node):
...
sn = SpecialNode()
reveal_type(sn.children) # N: Revealed type is "Union[typing.Sequence[__main__.SpecialNode], None]"
Here's a playground link for this.
However, Self type is not strictly necessary. Your example fails, because T is bound only in __init__, but ends up in attribute annotation, being completely pointless. Check this issue (inspired by your question, though I've seen much simpler cases of the same problem on SO) for some context. You can write an ugly workaround. This shows that mypy actually has some knowledge about _parent and _children attributes (because we don't cast), but doesn't expose it without rain dances.
from collections import abc
import typing as t
_Self = t.TypeVar('_Self', bound='Node')
class Node:
# Here we require that constructr arguments are really of proper type,
# but hide _Self under properties later
def __init__(
self: _Self, parent: _Self | None = None, children: abc.Sequence[_Self] | None = None
):
self._parent = parent
self._children = children
@property
def parent(self: _Self) -> _Self | None:
return self._parent
@parent.setter
def parent(self: _Self, parent: _Self | None) -> None:
self._parent = parent
@parent.deletter
def parent(self) -> None:
del self._parent
@property
def children(self: _Self) -> abc.Sequence[_Self] | None:
return self._children
@children.setter
def children(self: _Self, children: abc.Sequence[_Self] | None) -> None:
self._children = children
@children.deletter
def children(self) -> None:
del self._children
class SpecialNode(Node):
...
sn = SpecialNode()
reveal_type(sn.children) # N: Revealed type is "Union[typing.Sequence[__main__.SpecialNode], None]"
reveal_type(sn.parent) # N: Revealed type is "Union[__main__.SpecialNode, None]"
sn.parent = SpecialNode()
sn.children = []
sn.parent = Node() # E: Incompatible types in assignment (expression has type "Node", variable has type "Optional[SpecialNode]") [assignment]
sn.children = [Node()] # E: List item 0 has incompatible type "Node"; expected "SpecialNode" [list-item]
Here's playground link for this.
Bonus: if you still have an opportunity to adjust the API, do it, please. There should be one - and preferably only one - way to do something. In your case children = [] and children = None means essentially the same - no children. If it doesn't make real difference (e.g. None meaning "not set yet"), then non-optional sequence would be saner.
Also (again, if mutating original list/set is not intended), I'd use children setter anyway to accept any abc.Iterable[_Self] and convert it to a list or tuple in setter - but this will kill mypy, unfortunately (very annoying issue with unsupported asymmetric properties).