|
| 1 | +# Using python-newtype with Pydantic |
| 2 | + |
| 3 | +This guide demonstrates how to effectively use python-newtype with Pydantic to create robust, type-safe data models with custom validation. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +When building APIs or data-intensive applications, you often need to ensure that your data adheres to specific validation rules. While Pydantic provides excellent validation capabilities, combining it with python-newtype enhances your data modeling capabilities in several important ways. |
| 8 | + |
| 9 | +Python-newtype enables you to create reusable, type-safe custom fields that maintain their type information throughout your entire codebase. This means you can define specialized types once and reuse them across multiple models, ensuring consistent behavior and validation. |
| 10 | + |
| 11 | +With python-newtype, you can add domain-specific validation logic directly to your custom types. This validation becomes an inherent part of the type itself, ensuring that any instance of that type always meets your business requirements. The validation is enforced at the type level, making it impossible to create invalid instances. |
| 12 | + |
| 13 | +The combination of python-newtype and Pydantic ensures consistent data handling across your application. Whether you're serializing to JSON, validating incoming data, or manipulating objects in your business logic, you can be confident that your data maintains its integrity and type safety throughout its lifecycle. |
| 14 | + |
| 15 | +## Basic Example |
| 16 | + |
| 17 | +Let's look at a practical example of creating a Person model with validated first and last names: |
| 18 | + |
| 19 | +```python |
| 20 | +from typing import Any |
| 21 | +from newtype import NewType |
| 22 | +from pydantic import BaseModel, ConfigDict, Field, GetCoreSchemaHandler |
| 23 | +from pydantic_core import core_schema, CoreSchema |
| 24 | + |
| 25 | +def is_valid_firstname_or_lastname(value: str): |
| 26 | + if not all(char.isalpha() for char in value): |
| 27 | + raise ValueError("Name must contain only alphabetic characters") |
| 28 | + if len(value) < 2 or len(value) > 32: |
| 29 | + raise ValueError("Name must be between 2 and 32 characters long") |
| 30 | + return |
| 31 | + |
| 32 | +class FirstName(NewType(str)): |
| 33 | + def __init__(self, value: str): |
| 34 | + super().__init__(value) |
| 35 | + is_valid_firstname_or_lastname(value) |
| 36 | + |
| 37 | +class LastName(NewType(str)): |
| 38 | + def __init__(self, value: str): |
| 39 | + super().__init__(value) |
| 40 | + is_valid_firstname_or_lastname(value) |
| 41 | + |
| 42 | +class Person(BaseModel): |
| 43 | + model_config = ConfigDict(arbitrary_types_allowed=True) |
| 44 | + |
| 45 | + first_name: FirstName = Field(..., description="The first name of the person") |
| 46 | + last_name: LastName = Field(..., description="The last name of the person") |
| 47 | +``` |
| 48 | + |
| 49 | +In this example: |
| 50 | +- We create custom `FirstName` and `LastName` types that inherit from `str` |
| 51 | +- Each type includes validation to ensure names contain only letters and are between 2-32 characters |
| 52 | +- We use these types in a Pydantic model with proper field descriptions |
| 53 | + |
| 54 | +## Advanced Validation with Pydantic Core Schema |
| 55 | + |
| 56 | +To provide even more robust validation and better JSON schema support, you can implement Pydantic's core schema: |
| 57 | + |
| 58 | +```python |
| 59 | +FirstOrLastNameSchema = lambda cls: core_schema.json_or_python_schema( |
| 60 | + json_schema=core_schema.str_schema(min_length=2, max_length=32, pattern="^[a-zA-Z]+$"), |
| 61 | + python_schema=core_schema.union_schema( |
| 62 | + [ |
| 63 | + core_schema.is_instance_schema(cls), |
| 64 | + core_schema.str_schema(min_length=2, max_length=32, pattern="^[a-zA-Z]+$"), |
| 65 | + ] |
| 66 | + ), |
| 67 | + serialization=core_schema.str_schema(), |
| 68 | +) |
| 69 | + |
| 70 | +class FirstName(NewType(str)): |
| 71 | + def __init__(self, value: str): |
| 72 | + super().__init__(value) |
| 73 | + is_valid_firstname_or_lastname(value) |
| 74 | + |
| 75 | + @classmethod |
| 76 | + def __get_pydantic_core_schema__( |
| 77 | + cls, |
| 78 | + _source_type: Any, |
| 79 | + _handler: GetCoreSchemaHandler, |
| 80 | + ) -> CoreSchema: |
| 81 | + return FirstOrLastNameSchema(cls) |
| 82 | +``` |
| 83 | + |
| 84 | +This setup provides: |
| 85 | +- JSON schema validation for API documentation |
| 86 | +- Runtime validation for both JSON and Python objects |
| 87 | +- Proper serialization behavior |
| 88 | + |
| 89 | +## Usage Examples |
| 90 | + |
| 91 | +### Basic Model Creation |
| 92 | + |
| 93 | +```python |
| 94 | +# Create a person with valid names |
| 95 | +person = Person(first_name=FirstName("John"), last_name=LastName("Doe")) |
| 96 | + |
| 97 | +# Access the values |
| 98 | +print(person.first_name) # Output: John |
| 99 | +print(person.last_name) # Output: Doe |
| 100 | +``` |
| 101 | + |
| 102 | +### JSON Serialization |
| 103 | + |
| 104 | +```python |
| 105 | +# Serialize to JSON |
| 106 | +json_data = person.model_dump_json() |
| 107 | +print(json_data) # Output: {"first_name":"John","last_name":"Doe"} |
| 108 | + |
| 109 | +# Deserialize from JSON |
| 110 | +person2 = Person.model_validate_json('{"first_name":"John","last_name":"Doe"}') |
| 111 | +assert person == person2 |
| 112 | +``` |
| 113 | + |
| 114 | +### Validation Examples |
| 115 | + |
| 116 | +```python |
| 117 | +# These will raise ValueError: |
| 118 | +try: |
| 119 | + Person(first_name="John1", last_name="Doe") # Contains number |
| 120 | +except ValueError: |
| 121 | + print("Invalid first name") |
| 122 | + |
| 123 | +try: |
| 124 | + Person(first_name="J", last_name="Doe") # Too short |
| 125 | +except ValueError: |
| 126 | + print("Name too short") |
| 127 | +``` |
| 128 | + |
| 129 | +### Type Safety with Method Chaining |
| 130 | + |
| 131 | +```python |
| 132 | +person = Person(first_name=FirstName("James"), last_name=LastName("Bond")) |
| 133 | +# Replace maintains type safety |
| 134 | +person.first_name = person.first_name.replace("James", "John") |
| 135 | +assert isinstance(person.first_name, FirstName) |
| 136 | +``` |
| 137 | + |
| 138 | +## Advantages |
| 139 | + |
| 140 | +**Type Safety and Correctness** |
| 141 | +Python-newtype ensures that string operations return properly typed instances, maintaining type safety throughout your codebase. Type hints work correctly in your IDE and static type checkers, providing excellent autocompletion and early error detection. This level of type safety helps catch potential issues during development rather than at runtime. |
| 142 | + |
| 143 | +**Robust Validation** |
| 144 | +Validation logic is centralized in the type definition itself, ensuring that validation rules are consistently applied whenever an instance is created. The integration with Pydantic adds another layer of validation, combining both runtime checks and schema validation. This multi-layered approach ensures that your data always meets your requirements. |
| 145 | + |
| 146 | +**Comprehensive Schema Generation** |
| 147 | +The combination generates proper JSON schemas that accurately reflect your data model's constraints and structure. This is particularly valuable when building APIs, as it provides accurate OpenAPI specifications and clear validation rules in the schema. Your API documentation automatically includes all the validation rules and constraints defined in your types. |
| 148 | + |
| 149 | +**Enhanced Maintainability** |
| 150 | +By creating reusable custom types, you establish a single source of truth for your validation rules and type definitions. This separation of concerns makes your code more maintainable and reduces duplication. When you need to modify validation rules or add new functionality, you only need to update the type definition in one place. |
| 151 | + |
| 152 | +**Powerful Flexibility** |
| 153 | +The system allows you to add custom methods to your types, seamlessly integrate with existing Pydantic models, and works perfectly with frameworks like FastAPI. This flexibility means you can gradually adopt the system in your codebase and extend it as needed. You can create domain-specific methods and behaviors while maintaining all the benefits of both python-newtype and Pydantic. |
| 154 | + |
| 155 | +## Schema Example |
| 156 | + |
| 157 | +The generated JSON schema for our Person model looks like this: |
| 158 | + |
| 159 | +```python |
| 160 | +schema = Person.model_json_schema() |
| 161 | +""" |
| 162 | +{ |
| 163 | + 'title': 'Person', |
| 164 | + 'type': 'object', |
| 165 | + 'properties': { |
| 166 | + 'first_name': { |
| 167 | + 'title': 'First Name', |
| 168 | + 'type': 'string', |
| 169 | + 'minLength': 2, |
| 170 | + 'maxLength': 32, |
| 171 | + 'pattern': '^[a-zA-Z]+$', |
| 172 | + 'description': 'The first name of the person' |
| 173 | + }, |
| 174 | + 'last_name': { |
| 175 | + 'title': 'Last Name', |
| 176 | + 'type': 'string', |
| 177 | + 'minLength': 2, |
| 178 | + 'maxLength': 32, |
| 179 | + 'pattern': '^[a-zA-Z]+$', |
| 180 | + 'description': 'The last name of the person' |
| 181 | + } |
| 182 | + }, |
| 183 | + 'required': ['first_name', 'last_name'] |
| 184 | +} |
| 185 | +""" |
| 186 | +``` |
| 187 | + |
| 188 | +## Best Practices |
| 189 | + |
| 190 | +**Simple and Clear Validation Logic** |
| 191 | +Keep your validation logic simple and focused. Complex validation rules should be broken down into separate, well-named functions that clearly express their purpose. Make validation rules explicit and easy to understand, and always provide descriptive error messages that help users quickly identify and fix issues. This approach makes your code more maintainable and helps other developers understand your validation requirements. |
| 192 | + |
| 193 | +**Comprehensive Type Hints** |
| 194 | +Always include proper type hints in your code when working with python-newtype and Pydantic. Use your NewType types in function signatures to take full advantage of static type checking. Let tools like mypy help you catch type-related errors early in development. Good type hints not only improve code quality but also provide better documentation and IDE support. |
| 195 | + |
| 196 | +**Thorough Documentation** |
| 197 | +Document your custom types thoroughly with clear docstrings that explain the validation rules and any special behavior. Include examples in your documentation showing how to use the types correctly. When adding custom methods to your types, document their purpose, parameters, and return types. Good documentation helps other developers understand how to use your types correctly and avoid common pitfalls. |
| 198 | + |
| 199 | +**Graceful Error Handling** |
| 200 | +Implement comprehensive error handling in your validation logic. Catch and handle validation errors appropriately, providing clear and helpful error messages that guide users toward the correct usage. Consider creating custom exception types for different categories of validation errors. This makes it easier to handle specific error cases and provide appropriate feedback to users. |
0 commit comments