Adds comet-ml plugin example #1702

thomasjpfan · 2024-07-03T03:10:41Z

This PR adds a comet-ml example.

To fully run the example, you need to use this ImageSpec, because the plugin is in a PR:

comet_ml_package = (
    "git+https://github.com/thomasjpfan/flytekit.git@"
    "d04d0129f99eb7e7edcffeae04d4f401f49be6d6#subdirectory=plugins/flytekit-comet-ml"
)

image = ImageSpec(
    builder="fast-builder",
    name="unionai",
    apt_packages=["git"],
    packages=[
        "torch==2.3.1",
        "comet-ml==3.43.2",
        "lightning==2.3.0",
        comet_ml_package,
        "unionai==0.1.43",
        "torchvision",
    ],
    registry="ghcr.io/thomasjpfan",
)

flyteorg/flytekit#2550

Signed-off-by: Thomas J. Fan <[email protected]>

Future-Outlier · 2024-07-17T17:33:37Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+        "comet-ml==3.43.2",
+        "lightning==2.3.0",
+        "flytekitplugins-comet-ml",
+        "unionai==0.1.43",


We don't need the unionai dependency, right?
Should we add this in Union's doc instead of Open Source's doc?

This was a mistake. unionai is not needed.

Future-Outlier · 2024-07-17T17:35:18Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+
+REGISTRY = "localhost:30000"
+image = ImageSpec(
+    name="unionai",


same as below, we should use ghcr.io/flyteorg

For ImageSpec, most users will not have push access to ghcr.io/flyteorg, so they will need to change the registry to run it.

Using localhost:30000 means I am optimizing for the sandbox use case. An alternative is to use os.getenv("REGISTRY", "localhost:30000"), and let it be an environment variable.

No problem, I like your idea

Signed-off-by: Thomas J. Fan <[email protected]>

Future-Outlier

LGTM

Future-Outlier · 2024-07-17T18:21:29Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+
+REGISTRY = "localhost:30000"
+image = ImageSpec(
+    name="unionai",


No problem, I like your idea

Future-Outlier · 2024-07-17T18:22:09Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+#     dynamic-log-links:
+#       - comet-ml-execution-id:
+#           displayName: Comet
+#           templateUris: {{ .taskConfig.host }}/{{ .taskConfig.workspace }}/{{ .taskConfig.project_name }}/{{ .executionName }}{{ .nodeId }}{{ .taskRetryAttempt }}{{ .taskConfig.link_suffix }}


We should add double quote here.

Future-Outlier · 2024-07-17T18:22:19Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+#           templateUris: {{ .taskConfig.host }}/{{ .taskConfig.workspace }}/{{ .taskConfig.project_name }}/{{ .executionName }}{{ .nodeId }}{{ .taskRetryAttempt }}{{ .taskConfig.link_suffix }}
+#       - comet-ml-custom-id:
+#           displayName: Comet
+#           templateUris: {{ .taskConfig.host }}/{{ .taskConfig.workspace }}/{{ .taskConfig.project_name }}/{{ .taskConfig.experiment_key }}


Signed-off-by: Thomas J. Fan <[email protected]>

neverett

Left a few suggestions, otherwise LGTM

neverett · 2024-07-23T20:49:59Z

examples/comet_ml_plugin/README.md

+.. tags:: Integration, Data, Metrics, Intermediate
+```
+
+Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models—from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms.


Suggested change

Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models—from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms.

Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms.

neverett · 2024-07-23T20:50:47Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+#
+# # Comet Example
+# Comet’s machine learning platform integrates with your existing infrastructure and
+# tools so you can manage, visualize, and optimize models—from training runs to


Suggested change

# tools so you can manage, visualize, and optimize models—from training runs to

# tools so you can manage, visualize, and optimize models from training runs to

neverett · 2024-07-23T20:51:04Z

examples/comet_ml_plugin/comet_ml_plugin/comet_ml_example.py

+
+# %% [markdown]
+# First, we specify the project and workspace that we will use with Comet's platform
+# Please update `PROJECT_NAME` and `WORKSPACE` to the value associated with your account.


Suggested change

# Please update `PROJECT_NAME` and `WORKSPACE` to the value associated with your account.

# Please update `PROJECT_NAME` and `WORKSPACE` to the values associated with your account.

Signed-off-by: Thomas J. Fan <[email protected]>

thomasjpfan · 2024-08-01T16:31:38Z

@neverett The comet-ml check is passing now: https://github.com/flyteorg/flytesnacks/actions/runs/10182511543/job/28165251450?pr=1702

There are a bunch other tests that are failing tho.

neverett · 2024-08-01T18:06:42Z

@thomasjpfan can you try merging master in and rerunning?

…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>

Signed-off-by: Thomas J. Fan <[email protected]>

…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>

Adds comet-ml plugin example

d6deff0

Signed-off-by: Thomas J. Fan <[email protected]>

thomasjpfan requested review from neverett and ppiegaze as code owners July 3, 2024 03:10

thomasjpfan mentioned this pull request Jul 3, 2024

Adds comet-ml plugin flyteorg/flytekit#2550

Merged

thomasjpfan added 2 commits July 2, 2024 23:12

Remove unneeded arguments

8d22818

Signed-off-by: Thomas J. Fan <[email protected]>

Use new comet-ml name

bcb5cff

Signed-off-by: Thomas J. Fan <[email protected]>

Future-Outlier reviewed Jul 17, 2024

View reviewed changes

thomasjpfan added 2 commits July 17, 2024 14:14

Use environment variable

0c90ad7

Signed-off-by: Thomas J. Fan <[email protected]>

Use double quotes

8adb29d

Signed-off-by: Thomas J. Fan <[email protected]>

Future-Outlier previously approved these changes Jul 17, 2024

View reviewed changes

Use flyte name

e54796e

Signed-off-by: Thomas J. Fan <[email protected]>

thomasjpfan dismissed Future-Outlier’s stale review via e54796e July 17, 2024 19:52

Future-Outlier previously approved these changes Jul 17, 2024

View reviewed changes

Use quotes

066bf9e

Signed-off-by: Thomas J. Fan <[email protected]>

thomasjpfan dismissed Future-Outlier’s stale review via 066bf9e July 18, 2024 19:50

neverett previously approved these changes Jul 23, 2024

View reviewed changes

thomasjpfan added 2 commits July 24, 2024 09:45

Add to toc

8dbc6bf

Signed-off-by: Thomas J. Fan <[email protected]>

Address comments

acbbe8a

Signed-off-by: Thomas J. Fan <[email protected]>

thomasjpfan dismissed neverett’s stale review via acbbe8a July 24, 2024 13:45

thomasjpfan added 2 commits July 25, 2024 10:53

Adds comet-ml dockerfile

82639e5

Signed-off-by: Thomas J. Fan <[email protected]>

Fixes docker image

bdb39f7

Signed-off-by: Thomas J. Fan <[email protected]>

thomasjpfan added 4 commits August 1, 2024 15:52

Merge remote-tracking branch 'upstream/master' into thomasjpfan/comet…

c16048b

…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>

Merge remote-tracking branch 'upstream/master' into thomasjpfan/comet…

87390bc

…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>

Fix merge

b61f903

Signed-off-by: Thomas J. Fan <[email protected]>

Merge remote-tracking branch 'upstream/master' into thomasjpfan/comet…

708cefc

…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds comet-ml plugin example #1702

Adds comet-ml plugin example #1702

thomasjpfan commented Jul 3, 2024 •

edited

Loading

Future-Outlier Jul 17, 2024

thomasjpfan Jul 17, 2024

Future-Outlier Jul 17, 2024

thomasjpfan Jul 17, 2024

Future-Outlier Jul 17, 2024

Future-Outlier left a comment

Future-Outlier Jul 17, 2024

Future-Outlier Jul 17, 2024

Future-Outlier Jul 17, 2024

neverett left a comment

neverett Jul 23, 2024

neverett Jul 23, 2024

neverett Jul 23, 2024

thomasjpfan commented Aug 1, 2024

neverett commented Aug 1, 2024

	Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models—from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms.
	Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms.

	# tools so you can manage, visualize, and optimize models—from training runs to
	# tools so you can manage, visualize, and optimize models from training runs to

	# Please update `PROJECT_NAME` and `WORKSPACE` to the value associated with your account.
	# Please update `PROJECT_NAME` and `WORKSPACE` to the values associated with your account.

Adds comet-ml plugin example #1702

Are you sure you want to change the base?

Adds comet-ml plugin example #1702

Conversation

thomasjpfan commented Jul 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Future-Outlier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neverett left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thomasjpfan commented Aug 1, 2024

neverett commented Aug 1, 2024

thomasjpfan commented Jul 3, 2024 •

edited

Loading