mypy "Incompatible types" error when one @overload function calls another in __setitem_

I'm making a class that implements the MutableSequence contract and am having trouble type-hinting the getters & setters which take either a slice or a SupportsIndex instance as the index argument. In my application I have a _normalise_index function which turns negative relative indices and slices into positive ones by subtracting from len - e.g. collection[:-1] -> collection[:5], but mypy is not happy about my overloads...

Here is a simplified example:

from typing import Iterable, SupportsIndex, TypeVar, overload

T = TypeVar("T")


class Example:  # in my real code this is `class Example(typing.MutableSequence[T])`, but that's not required for this minrepro
    def __init__(self, iterable: Iterable[T]):
        self._lst = list(iterable)

    def __len__(self) -> int:
        return len(self._lst)

    @overload
    def _normalise_index(self, index: slice) -> slice: ...
    @overload
    def _normalise_index(self, index: SupportsIndex) -> int: ...

    def _normalise_index(self, index: slice | SupportsIndex) -> slice | int:
        "Convert -ve to  ve index - this makes sense for the real code..."
        if isinstance(index, slice):
            return slice(
                0 if not index.start else self._normalise_index(index.start),
                len(self) if not index.stop else self._normalise_index(index.stop),
                index.step or 1,
            )
        else:
            return len(self)   idx if (idx := int(index)) < 0 else idx

    @overload
    def __setitem__(self, unsafe_index: SupportsIndex, value: T) -> None: ...
    @overload
    def __setitem__(self, unsafe_index: slice, value: Iterable[T]) -> None: ...

    def __setitem__(  # line 34
        self, unsafe_index: SupportsIndex | slice, value: T | Iterable[T]
    ) -> None:
        normalised_index = self._normalise_index(unsafe_index)
        self._lst[normalised_index] = value  # line 38

... which gives 3 errors:

main.py:34: error: Overloaded function implementation does not accept all possible arguments of signature 2
main.py:38: error: Invalid index type "Union[int, slice]" for "List[T]"; expected type "SupportsIndex"
main.py:38: error: Incompatible types in assignment (expression has type "Union[T, Iterable[T]]", target has type "T")

View in online mypy playground

It looks like mypy is ignoring my overloads for __setitem__ so it thinks unsafe_index could be a slice or a SupportsIndex and then it is only matching one of the overloads for _normalise_index. No idea why...

CodePudding user response：

I'm not getting your first error on python 3.8. For the latter, you're hitting python's overload limitation. You can cast your types to what the overloads state:

    def __setitem__(
        self, unsafe_index: SupportsIndex | slice, value: T | Iterable[T]
    ) -> None:
        normalised_index = self._normalise_index(unsafe_index)
        if isinstance(normalised_index, slice):
            self._lst[normalised_index] = cast(Iterable[T], value)
        else:
            self._lst[normalised_index] = cast(T, value)

CodePudding user response：

I would recommend import Any from typing and use built-in method isinsthace to direct the operation of the __setitem__ implementation, so it could stop returning errors and you just have to add mechanisms to make it work your way:

from typing import Iterable, SupportsIndex, TypeVar, overload, Any

and

def __setitem__(self, unsafe_index: Any, value: Any) -> None:
    # check type
    if (isinstance(unsafe_index, SupportsIndex)):
        # Do some stuff ...
        normalised_index = self._normalise_index(unsafe_index)
        self._lst[normalised_index] = value  # line 38