Missing typed Input and Output #30
Labels
enhancement
New feature or request
Comments
J-Loudet
added a commit
that referenced
this issue
Jun 4, 2024
This commit introduces a major (short of complete) rewrite of the Python bindings for Zenoh-Flow. The main motivation is to simplify the code base. This translates into less Python code and more Rust code. Which means that the Python bindings are more tightly coupled with Zenoh-Flow's code base which leaves less room for errors and misalignment. This rewrite is however not complete as mentioned in #30. This rewrite also introduces few improvements: - the bindings can now work in a Python virtual environment on macOS, - the log emitted in a Python node are now propagated to the Zenoh-Flow runtime and, thus, displayed. * 01-python.zfext: deleted and replaced with an example Zenoh-Flow configuration in the `examples` folder and an FAQ entry in the README. * Cargo.toml: - forced resolver to version 2, - removed no longer used `zenoh-flow-python-commons` crate, - removed `[profile.dev]` and `[profile.release]` sections, - added a `[workspace.package]` section, - removed the dependencies: - libloading - log - zenoh-flow - added or updated the dependencies: - anyhow - pyo3 - pyo3-asyncio - pyo3-pylogger - tracing - tracing-subscriber - zenoh-flow-nodes * NOTICE.md: removed as no longer appropriate. * README.md: - updated the content to reflect the new changes, - added a section "FAG and Troubleshooting". * examples/flow.yaml: example data flow that leverages a Zenoh built-in Source and a Python sink. * examples/sink.py: Sink node that prints the payload and the timestamp of the messages received by the Source. * examples/zenoh-flow-configuration.yaml: an example configuration for a Zenoh-Flow standalone daemon. * zenoh-flow-python-commons/Cargo.toml: deleted as no longer needed. * zenoh-flow-python-commons/src/lib.rs: deleted as no longer needed. * zenoh-flow-python-extension: deleted as no longer needed. * zenoh-flow-python-extension-plugin: deleted as no longer needed. * zenoh-flow-python-operator-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-operator-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-operator-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to load the `libpython` shared library. * zenoh-flow-python-sink-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-sink-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-sink-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the `libpython` shared library. * zenoh-flow-python-source-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-source-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-source-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the `libpython` shared library. * zenoh-flow-python/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - added the dependencies: - anyhow - pyo3-asyncio - serde_json - tracing - zenoh-flow-nodes - removed the dependency to zenoh-flow-python-commons as it no longer (need to) exists. - removed the features: - abi-py37: it is enabled at the workspace level, - default: we do not want to activate the extension-module feature by default. * zenoh-flow-python/README.md: deleted as it was redundant with the README at the root of the project. * zenoh-flow-python/examples/loader-config.yml: deleted as no longer relevant. * zenoh-flow-python/examples/operator.py: deleted (will be replaced). * zenoh-flow-python/examples/py-operator.yml: deleted, no longer necessary. * zenoh-flow-python/examples/py-pipeline.yml: deleted, replaced. * zenoh-flow-python/examples/py-sink.yml: deleted, no longer necessary. * zenoh-flow-python/examples/py-source.yml: deleted, no longer necessary. * zenoh-flow-python/examples/sink.py: deleted, replaced. * zenoh-flow-python/examples/source.py: deleted (will be replaced). * zenoh-flow-python/pyproject.toml: - updated the version of maturin used, - updated the `[project]` section, - added a `[tool.maturin]` section to only enable the "pyo3/extension-module" feature when building the Python package of Zenoh-Flow, - updated the urls. * zenoh-flow-python/requirements-dev.txt: - updated the version of maturin to reflect what is written in pyproject.toml, - removed wheel. * zenoh-flow-python/src/lib.rs: almost complete rewrite, leveraging the "newtype pattern" to write the bindings. * zenoh-flow-python/zenoh_flow/__init__.py: updated to reflect the new changes. * zenoh-flow-python/zenoh_flow/interfaces/operator.py: deleted, replaced by `nodes.py`. * zenoh-flow-python/zenoh_flow/interfaces/sink.py: deleted, replaced by `nodes.py` * zenoh-flow-python/zenoh_flow/interfaces/source.py: deleted, replaced by `nodes.py`. * zenoh-flow-python/zenoh_flow/types/__init__.py: deleted as no longer needed. * zenoh-flow-python/zenoh_flow/interfaces/__init__.py -> zenoh-flow-python/zenoh_flow_python/__init__.py * zenoh-flow-python/zenoh_flow_python/nodes.py: definition of Python "Protocol" that users must implement to create Zenoh-Flow nodes. * zenoh-flow-python/zenoh_flow_python/py.typed: empty file to indicate that we are using a stub file to define our types. * zenoh-flow-python/zenoh_flow_python/zenoh_flow_python.pyi: stub file exposing the types of our class, their methods and functions. Signed-off-by: Julien Loudet <[email protected]>
J-Loudet
added a commit
that referenced
this issue
Jun 4, 2024
This commit introduces a major (short of complete) rewrite of the Python bindings for Zenoh-Flow. The main motivation is to simplify the code base. This translates into less Python code and more Rust code. Which means that the Python bindings are more tightly coupled with Zenoh-Flow's code base which leaves less room for errors and misalignment. This rewrite is however not complete as mentioned in #30. This rewrite also introduces few improvements: - the bindings can now work in a Python virtual environment on macOS, - the log emitted in a Python node are now propagated to the Zenoh-Flow runtime and, thus, displayed. * 01-python.zfext: deleted and replaced with an example Zenoh-Flow configuration in the `examples` folder and an FAQ entry in the README. * Cargo.toml: - forced resolver to version 2, - removed no longer used `zenoh-flow-python-commons` crate, - removed `[profile.dev]` and `[profile.release]` sections, - added a `[workspace.package]` section, - removed the dependencies: - libloading - log - zenoh-flow - added or updated the dependencies: - anyhow - pyo3 - pyo3-asyncio - pyo3-pylogger - tracing - tracing-subscriber - zenoh-flow-nodes * NOTICE.md: removed as no longer appropriate. * README.md: - updated the content to reflect the new changes, - added a section "FAG and Troubleshooting". * examples/flow.yaml: example data flow that leverages a Zenoh built-in Source and a Python sink. * examples/sink.py: Sink node that prints the payload and the timestamp of the messages received by the Source. * examples/zenoh-flow-configuration.yaml: an example configuration for a Zenoh-Flow standalone daemon. * zenoh-flow-python-commons/Cargo.toml: deleted as no longer needed. * zenoh-flow-python-commons/src/lib.rs: deleted as no longer needed. * zenoh-flow-python-extension: deleted as no longer needed. * zenoh-flow-python-extension-plugin: deleted as no longer needed. * zenoh-flow-python-operator-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-operator-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-operator-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to load the `libpython` shared library. * zenoh-flow-python-sink-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-sink-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-sink-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the `libpython` shared library. * zenoh-flow-python-source-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-source-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-source-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the `libpython` shared library. * zenoh-flow-python/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - added the dependencies: - anyhow - pyo3-asyncio - serde_json - tracing - zenoh-flow-nodes - removed the dependency to zenoh-flow-python-commons as it no longer (need to) exists. - removed the features: - abi-py37: it is enabled at the workspace level, - default: we do not want to activate the extension-module feature by default. * zenoh-flow-python/README.md: deleted as it was redundant with the README at the root of the project. * zenoh-flow-python/examples/loader-config.yml: deleted as no longer relevant. * zenoh-flow-python/examples/operator.py: deleted (will be replaced). * zenoh-flow-python/examples/py-operator.yml: deleted, no longer necessary. * zenoh-flow-python/examples/py-pipeline.yml: deleted, replaced. * zenoh-flow-python/examples/py-sink.yml: deleted, no longer necessary. * zenoh-flow-python/examples/py-source.yml: deleted, no longer necessary. * zenoh-flow-python/examples/sink.py: deleted, replaced. * zenoh-flow-python/examples/source.py: deleted (will be replaced). * zenoh-flow-python/pyproject.toml: - updated the version of maturin used, - updated the `[project]` section, - added a `[tool.maturin]` section to only enable the "pyo3/extension-module" feature when building the Python package of Zenoh-Flow, - updated the urls. * zenoh-flow-python/requirements-dev.txt: - updated the version of maturin to reflect what is written in pyproject.toml, - removed wheel. * zenoh-flow-python/src/lib.rs: almost complete rewrite, leveraging the "newtype pattern" to write the bindings. * zenoh-flow-python/zenoh_flow/__init__.py: updated to reflect the new changes. * zenoh-flow-python/zenoh_flow/interfaces/operator.py: deleted, replaced by `nodes.py`. * zenoh-flow-python/zenoh_flow/interfaces/sink.py: deleted, replaced by `nodes.py` * zenoh-flow-python/zenoh_flow/interfaces/source.py: deleted, replaced by `nodes.py`. * zenoh-flow-python/zenoh_flow/types/__init__.py: deleted as no longer needed. * zenoh-flow-python/zenoh_flow/interfaces/__init__.py -> zenoh-flow-python/zenoh_flow_python/__init__.py * zenoh-flow-python/zenoh_flow_python/nodes.py: definition of Python "Protocol" that users must implement to create Zenoh-Flow nodes. * zenoh-flow-python/zenoh_flow_python/py.typed: empty file to indicate that we are using a stub file to define our types. * zenoh-flow-python/zenoh_flow_python/zenoh_flow_python.pyi: stub file exposing the types of our class, their methods and functions. Signed-off-by: Julien Loudet <[email protected]>
Merged
J-Loudet
added a commit
that referenced
this issue
Jun 4, 2024
This commit introduces a major (short of complete) rewrite of the Python bindings for Zenoh-Flow. The main motivation is to simplify the code base. This translates into less Python code and more Rust code. Which means that the Python bindings are more tightly coupled with Zenoh-Flow's code base which leaves less room for errors and misalignment. This rewrite is however not complete as mentioned in #30. This rewrite also introduces few improvements: - the bindings can now work in a Python virtual environment on macOS, - the log emitted in a Python node are now propagated to the Zenoh-Flow runtime and, thus, displayed. * 01-python.zfext: deleted and replaced with an example Zenoh-Flow configuration in the `examples` folder and an FAQ entry in the README. * Cargo.toml: - forced resolver to version 2, - removed no longer used `zenoh-flow-python-commons` crate, - removed `[profile.dev]` and `[profile.release]` sections, - added a `[workspace.package]` section, - removed the dependencies: - libloading - log - zenoh-flow - added or updated the dependencies: - anyhow - pyo3 - pyo3-asyncio - pyo3-pylogger - tracing - tracing-subscriber - zenoh-flow-nodes * NOTICE.md: removed as no longer appropriate. * README.md: - updated the content to reflect the new changes, - added a section "FAG and Troubleshooting". * examples/flow.yaml: example data flow that leverages a Zenoh built-in Source and a Python sink. * examples/sink.py: Sink node that prints the payload and the timestamp of the messages received by the Source. * examples/zenoh-flow-configuration.yaml: an example configuration for a Zenoh-Flow standalone daemon. * zenoh-flow-python-commons/Cargo.toml: deleted as no longer needed. * zenoh-flow-python-commons/src/lib.rs: deleted as no longer needed. * zenoh-flow-python-extension: deleted as no longer needed. * zenoh-flow-python-extension-plugin: deleted as no longer needed. * zenoh-flow-python-operator-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-operator-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-operator-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to load the `libpython` shared library. * zenoh-flow-python-sink-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-sink-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-sink-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the `libpython` shared library. * zenoh-flow-python-source-wrapper/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - removed the dependencies: - libloading - log - zenoh-flow-python-commons - added the dependencies: - tracing - tracing-subscriber - removed no longer needed build configuration, - removed `[package.metadata.deb]` section. * zenoh-flow-python-source-wrapper/build.rs: deleted as no longer needed. * zenoh-flow-python-source-wrapper/src/lib.rs: updated the code to the latest changes. In particular it is no longer needed to manually load the `libpython` shared library. * zenoh-flow-python/Cargo.toml: - added `[package]` section, - updated the `[dependencies]` section to point to the workspace, - added the dependencies: - anyhow - pyo3-asyncio - serde_json - tracing - zenoh-flow-nodes - removed the dependency to zenoh-flow-python-commons as it no longer (need to) exists. - removed the features: - abi-py37: it is enabled at the workspace level, - default: we do not want to activate the extension-module feature by default. * zenoh-flow-python/README.md: deleted as it was redundant with the README at the root of the project. * zenoh-flow-python/examples/loader-config.yml: deleted as no longer relevant. * zenoh-flow-python/examples/operator.py: deleted (will be replaced). * zenoh-flow-python/examples/py-operator.yml: deleted, no longer necessary. * zenoh-flow-python/examples/py-pipeline.yml: deleted, replaced. * zenoh-flow-python/examples/py-sink.yml: deleted, no longer necessary. * zenoh-flow-python/examples/py-source.yml: deleted, no longer necessary. * zenoh-flow-python/examples/sink.py: deleted, replaced. * zenoh-flow-python/examples/source.py: deleted (will be replaced). * zenoh-flow-python/pyproject.toml: - updated the version of maturin used, - updated the `[project]` section, - added a `[tool.maturin]` section to only enable the "pyo3/extension-module" feature when building the Python package of Zenoh-Flow, - updated the urls. * zenoh-flow-python/requirements-dev.txt: - updated the version of maturin to reflect what is written in pyproject.toml, - removed wheel. * zenoh-flow-python/src/lib.rs: almost complete rewrite, leveraging the "newtype pattern" to write the bindings. * zenoh-flow-python/zenoh_flow/__init__.py: updated to reflect the new changes. * zenoh-flow-python/zenoh_flow/interfaces/operator.py: deleted, replaced by `nodes.py`. * zenoh-flow-python/zenoh_flow/interfaces/sink.py: deleted, replaced by `nodes.py` * zenoh-flow-python/zenoh_flow/interfaces/source.py: deleted, replaced by `nodes.py`. * zenoh-flow-python/zenoh_flow/types/__init__.py: deleted as no longer needed. * zenoh-flow-python/zenoh_flow/interfaces/__init__.py -> zenoh-flow-python/zenoh_flow_python/__init__.py * zenoh-flow-python/zenoh_flow_python/nodes.py: definition of Python "Protocol" that users must implement to create Zenoh-Flow nodes. * zenoh-flow-python/zenoh_flow_python/py.typed: empty file to indicate that we are using a stub file to define our types. * zenoh-flow-python/zenoh_flow_python/zenoh_flow_python.pyi: stub file exposing the types of our class, their methods and functions. Signed-off-by: Julien Loudet <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Zenoh-Flow nodes written in Rust expose typed Input and Output: they take, at initialisation, an additional parameter, a function that performs the deserialisation / serialisation.
These typed Input and Output are not present in the current version of the Python bindings of Zenoh-Flow.
The text was updated successfully, but these errors were encountered: