Custom types that are wrappers on simple types on pydantic-CodePudding

I am trying to create custom types that are wrappers on simple types (like int, float and str) in order to perform value validation (i.e. reject values that are forbidden, invalid and/or undesired). For that, I'm using mypy and pydantic.

However, when I add those custom types into data objects and/or data models, they end up being setted as None.

I produced the MCVE below.

I already spent a few days in that issue trying many different approaches and even took a look into pydantic's source code at GitHub, but this is the best I could get so far.

from typing import Any, Callable, Generator, Type
from pydantic.dataclasses import dataclass
from pydantic import BaseModel
from pydantic.errors import PydanticValueError

def make_ge_validator(minimum: int) -> Callable[[int], None]:
    def validator(value: int) -> None:
        class GeError(PydanticValueError):
            msg_template: str = "Too small"
        if value < minimum: raise GeError
    return validator

class MinValueInt(int):
    minimum: int

    def __init__(self, value: int) -> None:
        super(int, value)

    def __new__(cls: Type["MinValueInt"], value: int) -> "MinValueInt":
        for v in cls.__get_validators__():
            v(value)
        return super().__new__(cls, value)

    @classmethod
    def __get_validators__(cls: Type["MinValueInt"]) -> Generator[Callable[[int], None], None, None]:
        yield make_ge_validator(cls.minimum)

class Positive(MinValueInt):
    minimum = 1

class LargeNumber(MinValueInt):
    minimum = 1000

@dataclass(frozen = True)
class SomeDataModel(BaseModel):
    a: int         # Works, but accepts any int value with no validation.
    b: Positive    # Don't works, gets None after being validated.
    c: LargeNumber # Don't works, gets None after being validated.

def testEverything() -> None:
    # LargeNumber and Positive can't be instantiated with bad values, just as intended. Yay!
    try:
        LargeNumber(5)
    except Exception as e:
        assert "Too small" in str(e)
    try:
        Positive(-1)
    except Exception as e:
        assert "Too small" in str(e)

    # On the other hand, they can be instantiated with good values.
    x: Positive = Positive(8)
    y: LargeNumber = LargeNumber(2002)
    assert y == 2002
    assert type(y) == LargeNumber
    assert int(y) == 2002
    assert type(int(y)) == int

    # If we just workaround or ignore mypy's check,
    # it validates the parameters and works as intended anyway. Yay!
    try:
        SomeDataModel(-555, -1, 2000) # type: ignore
    except Exception as e:
        assert "Too small" in str(e)
    try:
        SomeDataModel(-555, 1, 999) # type: ignore
    except Exception as e:
        assert "Too small" in str(e)

    # But now, it comes the trouble...
    d: SomeDataModel = SomeDataModel(-555, x, y)
    print(d) # Outputs a=-555 b=None c=None __pydantic_initialised__=True
    assert d.a == -555
    assert d.b == x # FAILS because b is None. Why?????
    assert d.c == y # Also fails if you remove the preceding line.

testEverything()

Just be aware that since this is a MVCE, some contextual information about "why the heck I want to do this" was lost. Also, I am aware of the XY problem and even if no one could solve this, I am able to work that around by doing validation the old way (i.e. defining a __init__ method with a bunch of ifs and not trusting pydantic too much).

CodePudding user response：

Validator callable should raise-or-return, because it is responsible for both converting to proper type (like int returns an integer, given a string) and checking input validity.

Code below should work. I have also removed __init__, improved typing for __new__ and __get_validators__ (so that subclass' __new__ returns subclass and not MinValueInt strictly, although it's not very important here).

#! /usr/bin/env python

from typing import Any, Callable, Iterator, TypeVar, Type
from pydantic import BaseModel
from pydantic.dataclasses import dataclass
from pydantic.errors import PydanticValueError


def make_ge_validator(minimum: int) -> Callable[[int], int]:
    
    def validator(value: int) -> int:
        class GeError(PydanticValueError):
            msg_template: str = "Too small"
        if value < minimum: raise GeError
        return value

    return validator


_SelfT = TypeVar('_SelfT', bound='MinValueInt')


class MinValueInt(int):
    minimum: int

    def __new__(cls: Type[_SelfT], value: int) -> _SelfT:
        for v in cls.__get_validators__():
            v(value)
        return super().__new__(cls, value)

    @classmethod
    def __get_validators__(cls: Type[_SelfT]) -> Iterator[Callable[[int], _SelfT]]:
        # To avoid type: ignore here, you can use classmethod
        # instead or pass `cls` to make_ge_validator too
        yield make_ge_validator(cls.minimum)  # type: ignore


class Positive(MinValueInt):
    minimum = 1


class LargeNumber(MinValueInt):
    minimum = 1000


@dataclass(frozen=True)
class SomeDataModel(BaseModel):
    a: int         # Works, but accepts any int value with no validation.
    b: Positive    # Don't works, gets None after being validated.
    c: LargeNumber # Don't works, gets None after being validated.