-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
initial PoC implementation of UDPJobFactory #644
base: master
Are you sure you want to change the base?
Conversation
31d0c76
to
ff8b553
Compare
seems like pandas is handy to build foot guns maybe related to #641
@@ -13,8 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 | |||
- `MultiBackendJobManager`: Added `initialize_from_df(df)` (to `CsvJobDatabase` and `ParquetJobDatabase`) to initialize (and persist) the job database from a given DataFrame. | |||
Also added `create_job_db()` factory to easily create a job database from a given dataframe and its type guessed from filename extension. | |||
([#635](https://github.com/Open-EO/openeo-python-client/issues/635)) | |||
|
|||
|
|||
- `MultiBackendJobManager.run_jobs()` now returns a dictionary with counters/stats about various events during the job run ([#645](https://github.com/Open-EO/openeo-python-client/issues/645)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO Changelog entry
Batch job factory based on a parameterized process definition | ||
(e.g a user-defined process (UDP) or a remote process definition), | ||
to be used together with :py:class:`MultiBackendJobManager`. | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO add versionadded
*, | ||
namespace: Union[str, None] = None, | ||
parameter_defaults: Optional[dict] = None, | ||
parameter_column_map: Optional[dict] = None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO test for parameter_column_map feature?
@@ -886,3 +937,124 @@ def create_job_db(path: Union[str, Path], df: pd.DataFrame, *, on_exists: str = | |||
else: | |||
raise NotImplementedError(f"Initialization of {type(job_db)} is not supported.") | |||
return job_db | |||
|
|||
|
|||
class UDPJobFactory: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO: find better class name?
""" | ||
Batch job factory based on a parameterized process definition | ||
(e.g a user-defined process (UDP) or a remote process definition), | ||
to be used together with :py:class:`MultiBackendJobManager`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO: usage example here
for issue #604
self-review PR