Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[sdk] python 3.9 generics type annotations break compiling pipeline #11503

Open
racinmat opened this issue Jan 8, 2025 · 0 comments
Open

[sdk] python 3.9 generics type annotations break compiling pipeline #11503

racinmat opened this issue Jan 8, 2025 · 0 comments

Comments

@racinmat
Copy link

racinmat commented Jan 8, 2025

Environment

  • KFP version: none
  • KFP SDK version: 2.11.0
  • All dependencies version:
cachetools==5.5.0
certifi==2024.12.14
charset-normalizer==3.4.1
click==8.1.8
colorama==0.4.6
docstring_parser==0.16
google-api-core==2.24.0
google-auth==2.37.0
google-cloud-core==2.4.1
google-cloud-storage==2.19.0
google-crc32c==1.6.0
google-resumable-media==2.7.2
googleapis-common-protos==1.66.0
idna==3.10
kfp==2.11.0
kfp-pipeline-spec==0.6.0
kfp-server-api==2.3.0
kubernetes==30.1.0
oauthlib==3.2.2
proto-plus==1.25.0
protobuf==4.25.5
pyasn1==0.6.1
pyasn1_modules==0.4.1
python-dateutil==2.9.0.post0
PyYAML==6.0.2
requests==2.32.3
requests-oauthlib==2.0.0
requests-toolbelt==0.10.1
rsa==4.9
six==1.17.0
tabulate==0.9.0
urllib3==1.26.20
websocket-client==1.8.0

Steps to reproduce

# pipeline.py
from kfp import compiler, dsl

@dsl.component(base_image='python:3.12')
def comp(message: str, labels: dict[str, str]) -> str:
    print(message)
    print(labels)
    return message

@dsl.pipeline
def my_pipeline(message: str) -> str:
    """My ML pipeline."""
    return comp(message=message, labels={"version": "1.0.0"}).output

compiler.Compiler().compile(my_pipeline, package_path='pipeline.yaml')

run the following code like this:

python pipeline.py

and the output is

Traceback (most recent call last):
  File "C:\Projects\something\kfp_playground\pipeline.py", line 5, in <module>
    @dsl.component(base_image='python:3.12')
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\component_decorator.py", line 126, in component
    return component_factory.create_component_from_func(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\component_factory.py", line 589, in create_component_from_func
    component_spec = extract_component_interface(func)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\component_factory.py", line 451, in extract_component_interface
    name_to_input_spec, name_to_output_spec = get_name_to_specs(
                                              ^^^^^^^^^^^^^^^^^^
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\component_factory.py", line 302, in get_name_to_specs
    name, list(name_to_input_specs))] = make_input_spec(
                                        ^^^^^^^^^^^^^^^^
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\component_factory.py", line 406, in make_input_spec
    return structures.InputSpec(
           ^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 8, in __init__
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\structures.py", line 57, in __post_init__
    self._validate_type()
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\structures.py", line 131, in _validate_type
    type_utils.validate_bundled_artifact_type(self.type)
  File "C:\Projects\something\kfp_playground\.venv\Lib\site-packages\kfp\dsl\types\type_utils.py", line 544, in validate_bundled_artifact_type
    raise TypeError(
TypeError: Artifacts must have both a schema_title and a schema_version, separated by `@`. Got: dict[str, str]

Expected result

Successfully compile the pipeline.

This works:

from typing import Dict

from kfp import compiler, dsl

@dsl.component(base_image='python:3.12')
def comp(message: str, labels: Dict[str, str]) -> str:
    print(message)
    print(labels)
    return message

@dsl.pipeline
def my_pipeline(message: str) -> str:
    """My ML pipeline."""
    return comp(message=message, labels={"version": "1.0.0"}).output

compiler.Compiler().compile(my_pipeline, package_path='pipeline.yaml')

but it's ugly and the dict[str,str] is in python for over 4 years already.

Materials and Reference

[

---](https://peps.python.org/pep-0585/)

Impacted by this bug? Give it a 👍.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant