Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic v2 support #2972

Merged
merged 34 commits into from
Aug 7, 2023
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
ee45b97
work so far
thejaminator May 20, 2023
70c9423
add modelfield
thejaminator Jul 22, 2023
a66452b
Merge remote-tracking branch 'origin/main' into compat_folder
thejaminator Jul 22, 2023
fe52908
it works kinda
thejaminator Jul 22, 2023
356a5fb
fix missing type
thejaminator Jul 22, 2023
0c45267
fix default tesT
thejaminator Jul 22, 2023
37b8c2e
skip tests for constrained types
thejaminator Jul 22, 2023
f45d576
add tests pass?
thejaminator Jul 22, 2023
dc0b9dd
ruff
thejaminator Jul 22, 2023
1171abc
Revert "ruff"
thejaminator Jul 22, 2023
43dcf1d
make most of mypy happy
thejaminator Jul 22, 2023
969112b
make linters and mypy happy
thejaminator Jul 22, 2023
107ea3c
fix pydantic v1 import
thejaminator Jul 22, 2023
612a30c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2023
50d71fb
remove extra ruff
thejaminator Jul 22, 2023
e5673c1
fix tests for pydantic v1
thejaminator Jul 22, 2023
16c674d
remove weird stuff ruff added
thejaminator Jul 22, 2023
94d4844
add release file
thejaminator Jul 22, 2023
a116364
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2023
58e2f6f
bump pydantic in pyproject.toml
thejaminator Jul 22, 2023
7696efd
upgrade pydantic lock file
thejaminator Jul 22, 2023
79c41d2
remove unused
thejaminator Jul 22, 2023
3b3a392
fix mypy
thejaminator Jul 22, 2023
817c325
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2023
bfd7ac4
fix poetry
thejaminator Jul 23, 2023
9314609
remove implicit default test
thejaminator Jul 23, 2023
af3299e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 23, 2023
8cd57d9
force pydantic 1.10
thejaminator Jul 23, 2023
eec5c8e
Update noxfile.py
thejaminator Jul 24, 2023
f17ee94
Update tests/experimental/pydantic/schema/test_defaults.py
thejaminator Jul 24, 2023
5afee6b
rename v2_compat -> _compat
thejaminator Jul 24, 2023
db26d26
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2023
01182e2
ignore mypy windows tests
thejaminator Jul 25, 2023
d78d907
Merge branch 'main' into compat_folder
patrick91 Aug 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,9 @@ jobs:

- run: poetry install --with integrations
if: steps.setup-python.outputs.cache-hit != 'true'
# Since we are running all the integrations at once, we can't use
# pydantic v2. It is not compatible with starlette yet
- run: poetry run pip install pydantic==1.10

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errr otherwise starlette goes boom

# we use poetry directly instead of nox since we want to
# test all integrations at once on windows
Expand Down
Empty file added 2.0.0
Empty file.
7 changes: 7 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Release type: minor

Adds initial support for pydantic V2.

This is extremely experimental for wider initial testing.

We do not encourage using this in production systems yet.
2 changes: 1 addition & 1 deletion noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ def tests_integrations(session: Session, integration: str) -> None:

@session(python=["3.11"], name="Pydantic tests", tags=["tests"])
# TODO: add pydantic 2.0 here :)
thejaminator marked this conversation as resolved.
Show resolved Hide resolved
@nox.parametrize("pydantic", ["1.10"])
@nox.parametrize("pydantic", ["1.10", "2.0.3"])
def test_pydantic(session: Session, pydantic: str) -> None:
session.run_always("poetry", "install", external=True)

Expand Down
202 changes: 148 additions & 54 deletions poetry.lock

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ flask = {version = ">=1.1", optional = true}
opentelemetry-api = {version = "<2", optional = true}
opentelemetry-sdk = {version = "<2", optional = true}
chalice = {version = "^1.22", optional = true}
pydantic = {version = "<2", optional = true}
pydantic = {version = ">1.6.1", optional = true}
python-multipart = {version = ">=0.0.5,<0.0.7", optional = true}
sanic = {version = ">=20.12.2", optional = true}
aiohttp = {version = "^3.7.4.post0", optional = true}
Expand Down Expand Up @@ -112,7 +112,7 @@ channels = "^3.0.5"
Django = ">=3.2"
fastapi = {version = ">=0.65.0", optional = false}
flask = ">=1.1"
pydantic = {version = "<2", optional = false}
pydantic = {version = ">1.6.1", optional = false}
pytest-aiohttp = "^1.0.3"
Copy link
Contributor Author

@thejaminator thejaminator Jul 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set this back to >1.6.1 and update lock file (am I doing this right?)

pytest-django = {version = "^4.5"}
pytest-flask = {version = "^1.2.0"}
Expand Down
14 changes: 7 additions & 7 deletions strawberry/experimental/pydantic/error_type.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import dataclasses
import warnings
from typing import (
TYPE_CHECKING,
Any,
Callable,
List,
Expand All @@ -16,25 +15,26 @@
)

from pydantic import BaseModel
from pydantic.utils import lenient_issubclass

from strawberry.auto import StrawberryAuto
from strawberry.experimental.pydantic.utils import (
get_private_fields,
get_strawberry_type_from_model,
normalize_type,
)
from strawberry.experimental.pydantic.v2_compat import (
thejaminator marked this conversation as resolved.
Show resolved Hide resolved
CompatModelField,
get_model_fields,
lenient_issubclass,
)
from strawberry.object_type import _process_type, _wrap_dataclass
from strawberry.types.type_resolver import _get_fields
from strawberry.utils.typing import get_list_annotation, is_list

from .exceptions import MissingFieldsListError

if TYPE_CHECKING:
from pydantic.fields import ModelField


def get_type_for_field(field: ModelField) -> Union[Any, Type[None], Type[List]]:
def get_type_for_field(field: CompatModelField) -> Union[Any, Type[None], Type[List]]:
type_ = field.outer_type_
type_ = normalize_type(type_)
return field_type_to_type(type_)
Expand Down Expand Up @@ -72,7 +72,7 @@ def error_type(
all_fields: bool = False,
) -> Callable[..., Type]:
def wrap(cls: Type) -> Type:
model_fields = model.__fields__
model_fields = get_model_fields(model)
fields_set = set(fields) if fields else set()

if fields:
Expand Down
56 changes: 38 additions & 18 deletions strawberry/experimental/pydantic/fields.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,27 @@
import builtins
from decimal import Decimal
from typing import Any, List, Optional, Type
from typing import Any, List, Optional, Type, Union
from uuid import UUID

import pydantic
from pydantic import BaseModel
from pydantic.typing import get_args, get_origin, is_new_type, new_type_supertype
from pydantic.utils import lenient_issubclass

from strawberry.experimental.pydantic.exceptions import (
UnregisteredTypeException,
UnsupportedTypeError,
)
from strawberry.experimental.pydantic.v2_compat import (
IS_PYDANTIC_V2,
get_args,
get_origin,
is_new_type,
Comment on lines +11 to +13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have these on our typing module?

lenient_issubclass,
new_type_supertype,
)
from strawberry.types.types import StrawberryObjectDefinition

try:
from types import UnionType as TypingUnionType
from typing import GenericAlias as TypingGenericAlias # type: ignore
except ImportError:
import sys
Expand All @@ -25,6 +32,10 @@
TypingGenericAlias = ()
else:
raise
if sys.version_info < (3, 10):
Copy link

@Mark90 Mark90 Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, not sure if anyone will read this comment, but it is related to this change I believe.

On Python 3.9 I'm noticing some issues here when (indirectly) importing from this module:

from strawberry.experimental.pydantic.conversion_types import StrawberryTypeFromPydantic
/usr/local/lib/python3.9/site-packages/strawberry/experimental/pydantic/__init__.py:3: in <module>
    from .object_type import input, interface, type
/usr/local/lib/python3.9/site-packages/strawberry/experimental/pydantic/object_type.py:31: in <module>
    from strawberry.experimental.pydantic.fields import replace_types_recursively
/usr/local/lib/python3.9/site-packages/strawberry/experimental/pydantic/fields.py: in <module>
    from types import UnionType as TypingUnionType
E   ImportError: cannot import name 'UnionType' from 'types' (/usr/local/lib/python3.9/types.py)

On Python 3.10 and 3.11 it works fine.

From reading the import handling here I think it makes sense because it seems that this is what happens for different python versions;

Python 3.8 -> ImportError on UnionType

  • TypingGenericAlias = ()
  • TypingUnionType = ()

Python 3.9 -> ImportError on UnionType

  • Re-raise error

Python 3.10 -> (no import error)

  • TypingGenericAlias = typing.GenericAlias
  • TypingUnionType = types.UnionType

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mark90 this should be fixed now 😊

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great :) Thanks for notifying me!

TypingUnionType = ()
else:
raise

ATTR_TO_TYPE_MAP = {
"NoneStr": Optional[str],
Expand Down Expand Up @@ -70,23 +81,31 @@
"RedisDsn": str,
}


FIELDS_MAP = {
getattr(pydantic, field_name): type
for field_name, type in ATTR_TO_TYPE_MAP.items()
if hasattr(pydantic, field_name)
}
"""TODO:
Most of these fields are not supported by pydantic V2
"""
FIELDS_MAP = (
{
getattr(pydantic, field_name): type
for field_name, type in ATTR_TO_TYPE_MAP.items()
if hasattr(pydantic, field_name)
}
if not IS_PYDANTIC_V2
else {}
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think quite a few are still supported, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will investigate further in an issue / PR because i think pydantic v2's types are just normal annotated types that should work out of the box



def get_basic_type(type_: Any) -> Type[Any]:
if lenient_issubclass(type_, pydantic.ConstrainedInt):
return int
if lenient_issubclass(type_, pydantic.ConstrainedFloat):
return float
if lenient_issubclass(type_, pydantic.ConstrainedStr):
return str
if lenient_issubclass(type_, pydantic.ConstrainedList):
return List[get_basic_type(type_.item_type)] # type: ignore
if not IS_PYDANTIC_V2:
# only pydantic v1 has these
if lenient_issubclass(type_, pydantic.ConstrainedInt):
return int
if lenient_issubclass(type_, pydantic.ConstrainedFloat):
return float
if lenient_issubclass(type_, pydantic.ConstrainedStr):
return str
if lenient_issubclass(type_, pydantic.ConstrainedList):
return List[get_basic_type(type_.item_type)] # type: ignore

if type_ in FIELDS_MAP:
type_ = FIELDS_MAP.get(type_)
Expand Down Expand Up @@ -125,7 +144,8 @@ def replace_types_recursively(type_: Any, is_input: bool) -> Any:

if isinstance(replaced_type, TypingGenericAlias):
return TypingGenericAlias(origin, converted)

if isinstance(replaced_type, TypingUnionType):
return Union[converted]
replaced_type = replaced_type.copy_with(converted)

if isinstance(replaced_type, StrawberryObjectDefinition):
Expand Down
26 changes: 16 additions & 10 deletions strawberry/experimental/pydantic/object_type.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,28 +30,34 @@
get_default_factory_for_field,
get_private_fields,
)
from strawberry.experimental.pydantic.v2_compat import (
IS_PYDANTIC_V2,
CompatModelField,
get_model_fields,
)
from strawberry.field import StrawberryField
from strawberry.object_type import _process_type, _wrap_dataclass
from strawberry.types.type_resolver import _get_fields
from strawberry.utils.dataclasses import add_custom_init_fn

if TYPE_CHECKING:
from graphql import GraphQLResolveInfo
from pydantic.fields import ModelField


def get_type_for_field(field: ModelField, is_input: bool): # noqa: ANN201
def get_type_for_field(field: CompatModelField, is_input: bool): # noqa: ANN201
outer_type = field.outer_type_
replaced_type = replace_types_recursively(outer_type, is_input)
should_add_optional: bool = field.allow_none
if should_add_optional:
return Optional[replaced_type]
else:
return replaced_type
if not IS_PYDANTIC_V2:
# only pydantic v1 has this Optional logic
should_add_optional: bool = field.allow_none
if should_add_optional:
return Optional[replaced_type]

return replaced_type


def _build_dataclass_creation_fields(
field: ModelField,
field: CompatModelField,
is_input: bool,
existing_fields: Dict[str, StrawberryField],
auto_fields_set: Set[str],
Expand Down Expand Up @@ -85,7 +91,7 @@ def _build_dataclass_creation_fields(
default=dataclasses.MISSING,
default_factory=get_default_factory_for_field(field),
type_annotation=StrawberryAnnotation.from_annotation(field_type),
description=field.field_info.description,
description=field.description,
deprecation_reason=(
existing_field.deprecation_reason if existing_field else None
),
Expand Down Expand Up @@ -123,7 +129,7 @@ def type(
use_pydantic_alias: bool = True,
) -> Callable[..., Type[StrawberryTypeFromPydantic[PydanticModel]]]:
def wrap(cls: Any) -> Type[StrawberryTypeFromPydantic[PydanticModel]]:
model_fields = model.__fields__
model_fields = get_model_fields(model)
original_fields_set = set(fields) if fields else set()

if fields:
Expand Down
17 changes: 10 additions & 7 deletions strawberry/experimental/pydantic/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,17 @@
cast,
)

from pydantic.utils import smart_deepcopy

from strawberry.experimental.pydantic.exceptions import (
AutoFieldsNotInBaseModelError,
BothDefaultAndDefaultFactoryDefinedError,
UnregisteredTypeException,
)
from strawberry.experimental.pydantic.v2_compat import (
PYDANTIC_MISSING_TYPE,
CompatModelField,
get_model_fields,
smart_deepcopy,
)
from strawberry.private import is_private
from strawberry.unset import UNSET
from strawberry.utils.typing import (
Expand All @@ -32,7 +36,6 @@

if TYPE_CHECKING:
from pydantic import BaseModel
from pydantic.fields import ModelField
from pydantic.typing import NoArgAnyCallable


Expand Down Expand Up @@ -70,7 +73,7 @@ def to_tuple(self) -> Tuple[str, Type, dataclasses.Field]:


def get_default_factory_for_field(
field: ModelField,
field: CompatModelField,
) -> Union[NoArgAnyCallable, dataclasses._MISSING_TYPE]:
"""
Gets the default factory for a pydantic field.
Expand All @@ -83,10 +86,10 @@ def get_default_factory_for_field(
# replace dataclasses.MISSING with our own UNSET to make comparisons easier
default_factory = (
field.default_factory
if field.default_factory is not dataclasses.MISSING
if field.default_factory is not PYDANTIC_MISSING_TYPE
else UNSET
)
default = field.default if field.default is not dataclasses.MISSING else UNSET
default = field.default if field.default is not PYDANTIC_MISSING_TYPE else UNSET

has_factory = default_factory is not None and default_factory is not UNSET
has_default = default is not None and default is not UNSET
Expand Down Expand Up @@ -125,7 +128,7 @@ def ensure_all_auto_fields_in_pydantic(
model: Type[BaseModel], auto_fields: Set[str], cls_name: str
) -> Union[NoReturn, None]:
# Raise error if user defined a strawberry.auto field not present in the model
non_existing_fields = list(auto_fields - model.__fields__.keys())
non_existing_fields = list(auto_fields - get_model_fields(model).keys())

if non_existing_fields:
raise AutoFieldsNotInBaseModelError(
Expand Down
Loading