Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic v2 support #2972

Merged
merged 34 commits into from
Aug 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
ee45b97
work so far
thejaminator May 20, 2023
70c9423
add modelfield
thejaminator Jul 22, 2023
a66452b
Merge remote-tracking branch 'origin/main' into compat_folder
thejaminator Jul 22, 2023
fe52908
it works kinda
thejaminator Jul 22, 2023
356a5fb
fix missing type
thejaminator Jul 22, 2023
0c45267
fix default tesT
thejaminator Jul 22, 2023
37b8c2e
skip tests for constrained types
thejaminator Jul 22, 2023
f45d576
add tests pass?
thejaminator Jul 22, 2023
dc0b9dd
ruff
thejaminator Jul 22, 2023
1171abc
Revert "ruff"
thejaminator Jul 22, 2023
43dcf1d
make most of mypy happy
thejaminator Jul 22, 2023
969112b
make linters and mypy happy
thejaminator Jul 22, 2023
107ea3c
fix pydantic v1 import
thejaminator Jul 22, 2023
612a30c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2023
50d71fb
remove extra ruff
thejaminator Jul 22, 2023
e5673c1
fix tests for pydantic v1
thejaminator Jul 22, 2023
16c674d
remove weird stuff ruff added
thejaminator Jul 22, 2023
94d4844
add release file
thejaminator Jul 22, 2023
a116364
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2023
58e2f6f
bump pydantic in pyproject.toml
thejaminator Jul 22, 2023
7696efd
upgrade pydantic lock file
thejaminator Jul 22, 2023
79c41d2
remove unused
thejaminator Jul 22, 2023
3b3a392
fix mypy
thejaminator Jul 22, 2023
817c325
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 22, 2023
bfd7ac4
fix poetry
thejaminator Jul 23, 2023
9314609
remove implicit default test
thejaminator Jul 23, 2023
af3299e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 23, 2023
8cd57d9
force pydantic 1.10
thejaminator Jul 23, 2023
eec5c8e
Update noxfile.py
thejaminator Jul 24, 2023
f17ee94
Update tests/experimental/pydantic/schema/test_defaults.py
thejaminator Jul 24, 2023
5afee6b
rename v2_compat -> _compat
thejaminator Jul 24, 2023
db26d26
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2023
01182e2
ignore mypy windows tests
thejaminator Jul 25, 2023
d78d907
Merge branch 'main' into compat_folder
patrick91 Aug 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -157,12 +157,15 @@ jobs:

- run: poetry install --with integrations
if: steps.setup-python.outputs.cache-hit != 'true'
# Since we are running all the integrations at once, we can't use
# pydantic v2. It is not compatible with starlette yet
- run: poetry run pip install pydantic==1.10

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errr otherwise starlette goes boom

# we use poetry directly instead of nox since we want to
# test all integrations at once on windows
# but we want to exclude tests/mypy since we are using an old version of pydantic
- run: |
poetry run pytest --cov=. --cov-append --cov-report=xml -n auto --showlocals -vv

poetry run pytest --cov=. --cov-append --cov-report=xml -n auto --showlocals --ignore tests/mypy -vv
- name: coverage xml
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignores mypy for windows

run: coverage xml -i
if: ${{ always() }}
Expand Down
Empty file added 2.0.0
Empty file.
7 changes: 7 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Release type: minor

Adds initial support for pydantic V2.

This is extremely experimental for wider initial testing.

We do not encourage using this in production systems yet.
3 changes: 1 addition & 2 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,7 @@ def tests_integrations(session: Session, integration: str) -> None:


@session(python=["3.11"], name="Pydantic tests", tags=["tests"])
# TODO: add pydantic 2.0 here :)
@nox.parametrize("pydantic", ["1.10"])
@nox.parametrize("pydantic", ["1.10", "2.0.3"])
def test_pydantic(session: Session, pydantic: str) -> None:
session.run_always("poetry", "install", external=True)

Expand Down
288 changes: 191 additions & 97 deletions poetry.lock

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ flask = {version = ">=1.1", optional = true}
opentelemetry-api = {version = "<2", optional = true}
opentelemetry-sdk = {version = "<2", optional = true}
chalice = {version = "^1.22", optional = true}
pydantic = {version = "<2", optional = true}
pydantic = {version = ">1.6.1", optional = true}
python-multipart = {version = ">=0.0.5,<0.0.7", optional = true}
sanic = {version = ">=20.12.2", optional = true}
aiohttp = {version = "^3.7.4.post0", optional = true}
Expand Down Expand Up @@ -112,7 +112,7 @@ channels = "^3.0.5"
Django = ">=3.2"
fastapi = {version = ">=0.65.0", optional = false}
flask = ">=1.1"
pydantic = {version = "<2", optional = false}
pydantic = {version = ">1.6.1", optional = false}
pytest-aiohttp = "^1.0.3"
Copy link
Contributor Author

@thejaminator thejaminator Jul 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set this back to >1.6.1 and update lock file (am I doing this right?)

pytest-django = {version = "^4.5"}
pytest-flask = {version = "^1.2.0"}
Expand Down
104 changes: 104 additions & 0 deletions strawberry/experimental/pydantic/_compat.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
import dataclasses
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Callable, Dict, Optional, Type

import pydantic
from pydantic import BaseModel
from pydantic.version import VERSION as PYDANTIC_VERSION

if TYPE_CHECKING:
from pydantic.fields import FieldInfo

IS_PYDANTIC_V2: bool = PYDANTIC_VERSION.startswith("2.")
IS_PYDANTIC_V1: bool = not IS_PYDANTIC_V2


@dataclass
class CompatModelField:
name: str
type_: Any
outer_type_: Any
default: Any
default_factory: Optional[Callable[[], Any]]
required: bool
alias: Optional[str]
allow_none: bool
has_alias: bool
description: Optional[str]


if pydantic.VERSION[0] == "2":
from typing_extensions import get_args, get_origin

from pydantic._internal._typing_extra import is_new_type
from pydantic._internal._utils import lenient_issubclass, smart_deepcopy
from pydantic_core import PydanticUndefined

PYDANTIC_MISSING_TYPE = PydanticUndefined

def new_type_supertype(type_: Any) -> Any:
return type_.__supertype__

def get_model_fields(model: Type[BaseModel]) -> Dict[str, CompatModelField]:
field_info: dict[str, FieldInfo] = model.model_fields
new_fields = {}
# Convert it into CompatModelField
for name, field in field_info.items():
new_fields[name] = CompatModelField(
name=name,
type_=field.annotation,
outer_type_=field.annotation,
default=field.default,
default_factory=field.default_factory,
required=field.is_required(),
alias=field.alias,
# v2 doesn't have allow_none
allow_none=False,
has_alias=field is not None,
description=field.description,
)
return new_fields

else:
from pydantic.typing import ( # type: ignore[no-redef]
get_args,
get_origin,
is_new_type,
new_type_supertype,
)
from pydantic.utils import ( # type: ignore[no-redef]
lenient_issubclass,
smart_deepcopy,
)

PYDANTIC_MISSING_TYPE = dataclasses.MISSING # type: ignore[assignment]

def get_model_fields(model: Type[BaseModel]) -> Dict[str, CompatModelField]:
new_fields = {}
# Convert it into CompatModelField
for name, field in model.__fields__.items(): # type: ignore[attr-defined]
new_fields[name] = CompatModelField(
name=name,
type_=field.type_,
outer_type_=field.outer_type_,
default=field.default,
default_factory=field.default_factory,
required=field.required,
alias=field.alias,
allow_none=field.allow_none,
has_alias=field.has_alias,
description=field.field_info.description,
)
return new_fields


__all__ = [
"smart_deepcopy",
"lenient_issubclass",
"get_args",
"get_origin",
"is_new_type",
"new_type_supertype",
"get_model_fields",
"PYDANTIC_MISSING_TYPE",
]
14 changes: 7 additions & 7 deletions strawberry/experimental/pydantic/error_type.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
import dataclasses
import warnings
from typing import (
TYPE_CHECKING,
Any,
Callable,
List,
Expand All @@ -16,9 +15,13 @@
)

from pydantic import BaseModel
from pydantic.utils import lenient_issubclass

from strawberry.auto import StrawberryAuto
from strawberry.experimental.pydantic._compat import (
CompatModelField,
get_model_fields,
lenient_issubclass,
)
from strawberry.experimental.pydantic.utils import (
get_private_fields,
get_strawberry_type_from_model,
Expand All @@ -30,11 +33,8 @@

from .exceptions import MissingFieldsListError

if TYPE_CHECKING:
from pydantic.fields import ModelField


def get_type_for_field(field: ModelField) -> Union[Any, Type[None], Type[List]]:
def get_type_for_field(field: CompatModelField) -> Union[Any, Type[None], Type[List]]:
type_ = field.outer_type_
type_ = normalize_type(type_)
return field_type_to_type(type_)
Expand Down Expand Up @@ -72,7 +72,7 @@ def error_type(
all_fields: bool = False,
) -> Callable[..., Type]:
def wrap(cls: Type) -> Type:
model_fields = model.__fields__
model_fields = get_model_fields(model)
fields_set = set(fields) if fields else set()

if fields:
Expand Down
56 changes: 38 additions & 18 deletions strawberry/experimental/pydantic/fields.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,27 @@
import builtins
from decimal import Decimal
from typing import Any, List, Optional, Type
from typing import Any, List, Optional, Type, Union
from uuid import UUID

import pydantic
from pydantic import BaseModel
from pydantic.typing import get_args, get_origin, is_new_type, new_type_supertype
from pydantic.utils import lenient_issubclass

from strawberry.experimental.pydantic._compat import (
IS_PYDANTIC_V1,
get_args,
get_origin,
is_new_type,
Comment on lines +11 to +13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have these on our typing module?

lenient_issubclass,
new_type_supertype,
)
from strawberry.experimental.pydantic.exceptions import (
UnregisteredTypeException,
UnsupportedTypeError,
)
from strawberry.types.types import StrawberryObjectDefinition

try:
from types import UnionType as TypingUnionType
from typing import GenericAlias as TypingGenericAlias # type: ignore
except ImportError:
import sys
Expand All @@ -25,6 +32,10 @@
TypingGenericAlias = ()
else:
raise
if sys.version_info < (3, 10):
Copy link

@Mark90 Mark90 Aug 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, not sure if anyone will read this comment, but it is related to this change I believe.

On Python 3.9 I'm noticing some issues here when (indirectly) importing from this module:

from strawberry.experimental.pydantic.conversion_types import StrawberryTypeFromPydantic
/usr/local/lib/python3.9/site-packages/strawberry/experimental/pydantic/__init__.py:3: in <module>
    from .object_type import input, interface, type
/usr/local/lib/python3.9/site-packages/strawberry/experimental/pydantic/object_type.py:31: in <module>
    from strawberry.experimental.pydantic.fields import replace_types_recursively
/usr/local/lib/python3.9/site-packages/strawberry/experimental/pydantic/fields.py: in <module>
    from types import UnionType as TypingUnionType
E   ImportError: cannot import name 'UnionType' from 'types' (/usr/local/lib/python3.9/types.py)

On Python 3.10 and 3.11 it works fine.

From reading the import handling here I think it makes sense because it seems that this is what happens for different python versions;

Python 3.8 -> ImportError on UnionType

  • TypingGenericAlias = ()
  • TypingUnionType = ()

Python 3.9 -> ImportError on UnionType

  • Re-raise error

Python 3.10 -> (no import error)

  • TypingGenericAlias = typing.GenericAlias
  • TypingUnionType = types.UnionType

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mark90 this should be fixed now 😊

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great :) Thanks for notifying me!

TypingUnionType = ()
else:
raise

ATTR_TO_TYPE_MAP = {
"NoneStr": Optional[str],
Expand Down Expand Up @@ -70,23 +81,31 @@
"RedisDsn": str,
}


FIELDS_MAP = {
getattr(pydantic, field_name): type
for field_name, type in ATTR_TO_TYPE_MAP.items()
if hasattr(pydantic, field_name)
}
"""TODO:
Most of these fields are not supported by pydantic V2
"""
FIELDS_MAP = (
{
getattr(pydantic, field_name): type
for field_name, type in ATTR_TO_TYPE_MAP.items()
if hasattr(pydantic, field_name)
}
if IS_PYDANTIC_V1
else {}
)


def get_basic_type(type_: Any) -> Type[Any]:
if lenient_issubclass(type_, pydantic.ConstrainedInt):
return int
if lenient_issubclass(type_, pydantic.ConstrainedFloat):
return float
if lenient_issubclass(type_, pydantic.ConstrainedStr):
return str
if lenient_issubclass(type_, pydantic.ConstrainedList):
return List[get_basic_type(type_.item_type)] # type: ignore
if IS_PYDANTIC_V1:
# only pydantic v1 has these
if lenient_issubclass(type_, pydantic.ConstrainedInt):
return int
if lenient_issubclass(type_, pydantic.ConstrainedFloat):
return float
if lenient_issubclass(type_, pydantic.ConstrainedStr):
return str
if lenient_issubclass(type_, pydantic.ConstrainedList):
return List[get_basic_type(type_.item_type)] # type: ignore

if type_ in FIELDS_MAP:
type_ = FIELDS_MAP.get(type_)
Expand Down Expand Up @@ -125,7 +144,8 @@ def replace_types_recursively(type_: Any, is_input: bool) -> Any:

if isinstance(replaced_type, TypingGenericAlias):
return TypingGenericAlias(origin, converted)

if isinstance(replaced_type, TypingUnionType):
return Union[converted]
replaced_type = replaced_type.copy_with(converted)

if isinstance(replaced_type, StrawberryObjectDefinition):
Expand Down
26 changes: 16 additions & 10 deletions strawberry/experimental/pydantic/object_type.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@

from strawberry.annotation import StrawberryAnnotation
from strawberry.auto import StrawberryAuto
from strawberry.experimental.pydantic._compat import (
IS_PYDANTIC_V1,
CompatModelField,
get_model_fields,
)
from strawberry.experimental.pydantic.conversion import (
convert_pydantic_model_to_strawberry_class,
convert_strawberry_class_to_pydantic_model,
Expand All @@ -37,21 +42,22 @@

if TYPE_CHECKING:
from graphql import GraphQLResolveInfo
from pydantic.fields import ModelField


def get_type_for_field(field: ModelField, is_input: bool): # noqa: ANN201
def get_type_for_field(field: CompatModelField, is_input: bool): # noqa: ANN201
outer_type = field.outer_type_
replaced_type = replace_types_recursively(outer_type, is_input)
should_add_optional: bool = field.allow_none
if should_add_optional:
return Optional[replaced_type]
else:
return replaced_type
if IS_PYDANTIC_V1:
# only pydantic v1 has this Optional logic
should_add_optional: bool = field.allow_none
if should_add_optional:
return Optional[replaced_type]

return replaced_type


def _build_dataclass_creation_fields(
field: ModelField,
field: CompatModelField,
is_input: bool,
existing_fields: Dict[str, StrawberryField],
auto_fields_set: Set[str],
Expand Down Expand Up @@ -85,7 +91,7 @@ def _build_dataclass_creation_fields(
default=dataclasses.MISSING,
default_factory=get_default_factory_for_field(field),
type_annotation=StrawberryAnnotation.from_annotation(field_type),
description=field.field_info.description,
description=field.description,
deprecation_reason=(
existing_field.deprecation_reason if existing_field else None
),
Expand Down Expand Up @@ -123,7 +129,7 @@ def type(
use_pydantic_alias: bool = True,
) -> Callable[..., Type[StrawberryTypeFromPydantic[PydanticModel]]]:
def wrap(cls: Any) -> Type[StrawberryTypeFromPydantic[PydanticModel]]:
model_fields = model.__fields__
model_fields = get_model_fields(model)
original_fields_set = set(fields) if fields else set()

if fields:
Expand Down
Loading