Support topology v2 #2598

andrewballantyne · 2024-03-13T20:58:20Z

https://issues.redhat.com/browse/RHOAIENG-2297

Description

Basically this will bring back the horizontal topology using DSPv2 data. It refactors a lot of what was in the sidebar and has some edge cases left. We may need to merge this as first implementation (after some cleanup) and then address reworking the artifact nodes to properly show up.

Screenshots

Pipeline

Run screen

Group nodes, not supported today (or for v1=>v2 parity)

Schedules

Gif

Notes:

[Bug] Seems the logs are not properly found -- will need to look into this (seems to be an issue more with auto pruning)
We don't have a good way to see what's going on -- maybe the above bug will lead to a better status update

Screen.Recording.2024-03-14.at.2.02.48.AM.mov

How Has This Been Tested?

You should be able to view any:

Pipeline Details
Run Details
Schedule Details

And get a topology graph. Statuses should reflect executions but have some quirks to them (see description).

Test Impact

WIP

Request review criteria:

Self checklist (all need to be checked):

The developer has manually tested the changes and verified that the changes work
Commits have been squashed into descriptive, self-contained units of work (e.g. 'WIP' and 'Implements feedback' style messages have been removed)
Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

Included any necessary screenshots or gifs if it was a UI change.
Included tags to the UX team if it was a UI/UX change (find relevant UX in the SMEs section).

After the PR is posted & before it merges:

The developer has tested their solution on a cluster by using the image produced by the PR to main

andrewballantyne · 2024-03-14T06:00:46Z

@yannnz bit of a work in progress still -- but this is where we are at with Topology. Group Nodes is probably the biggest gap. I know we are probably not showing artifact sidebars quite perfectly -- we can iterate on these.

andrewballantyne · 2024-03-14T06:05:53Z

@Gkrumbach07 @jpuzz0 feel free to start reviewing this... we may need to fix up some of it next week. I'll try to get more of it updated tomorrow.

andrewballantyne

State of things...

frontend/src/concepts/pipelines/content/pipelinesDetails/pipelineRun/runLogs/LogsTab.tsx

frontend/src/concepts/pipelines/content/pipelinesDetails/pipelineRun/PipelineRunDetails.tsx

andrewballantyne · 2024-03-14T06:10:26Z

frontend/src/concepts/pipelines/topology/__tests__/parseUtils.spec.ts

frontend/src/concepts/pipelines/topology/pipelineTaskTypes.ts

andrewballantyne · 2024-03-14T06:12:01Z

frontend/src/k8sTypes.ts

probably should delete all Tekton items -- but we may need it later for Edge -- so I'm torn tbh.

frontend/src/pages/ApplicationsPage.tsx

andrewballantyne · 2024-03-14T20:46:19Z

Seems pods are having some issues getting proper logs -- so I've started a thread with the backend team.

We should be down to a minor number of known issues. @Gkrumbach07 please take a look when you get a moment.

Gkrumbach07 · 2024-03-15T15:22:51Z

...d/src/concepts/pipelines/content/pipelinesDetails/pipelineRun/PipelineRunDrawerRightTabs.tsx

    // [PipelineRunNodeTabs.VISUALIZATIONS]: <>TBD 2</>,
    [PipelineRunNodeTabs.DETAILS]: <SelectedNodeDetailsTab task={task} />,
-    [PipelineRunNodeTabs.VOLUMES]: <SelectedNodeVolumeMountsTab task={task} />,
+    // [PipelineRunNodeTabs.VOLUMES]: <SelectedNodeVolumeMountsTab task={task} />,


why is volumes removed

I have not found a way to show them, if you have a Pipeline for them, let me know!

Found one, will implement again

#2598 (comment) yes, I am aware, fixing. They do weird spec things when it has volumes.

Gkrumbach07 · 2024-03-15T15:41:32Z

frontend/src/concepts/pipelines/topology/parseUtils.ts

+
+  const { task_details: taskDetails } = runDetails;
+
+  const thisTaskDetail = taskDetails.find(({ display_name: name }) => name === `${taskId}-driver`);


you only look at driver. I cannot get an error node to apear then. For some reason if a node has an error, it is not setting the state for the driver task, just the children ones

Damn, seems I might need to find a bunch of them -- I didn't at the start, but it doesn't show active states if I don't.

...d/src/concepts/pipelines/content/pipelinesDetails/pipelineRun/PipelineRunDrawerRightTabs.tsx

Gkrumbach07 · 2024-03-15T16:42:53Z

ran into this when looking at a run with volumes

Gkrumbach07 · 2024-03-15T18:55:33Z

i get this type of run graph

result yaml: https://gist.github.com/Gkrumbach07/c953f2175e5c97c05fc51391144cde27

andrewballantyne · 2024-03-15T19:45:15Z

/retest

Seems like Cypress is having some issues.

(got a TS error locally, fixed it -- not sure why I didn't see that on the cluster before Cypress)

andrewballantyne · 2024-03-15T20:27:20Z

Known gaps (cc @Gkrumbach07):

Missing tests for the topology utilities
Pipelines are still a little weird at run time, but work
Volumes are missing

Gkrumbach07 · 2024-03-15T20:29:42Z

This looks good to me. The known gaps are fine for now until we get the new topology rendering in.

/lgtm

andrewballantyne · 2024-03-15T20:35:53Z

/approve

openshift-ci · 2024-03-15T20:36:01Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewballantyne

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [andrewballantyne]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the do-not-merge/work-in-progress This PR is in WIP state label Mar 13, 2024

openshift-ci bot requested review from lucferbux and manaswinidas March 13, 2024 20:58

openshift-merge-robot added the needs-rebase PR needs to be rebased label Mar 13, 2024

andrewballantyne force-pushed the support-topology-v2 branch 4 times, most recently from 6d24e9e to 55287af Compare March 14, 2024 05:57

openshift-merge-robot removed the needs-rebase PR needs to be rebased label Mar 14, 2024

andrewballantyne requested review from Gkrumbach07 and jpuzz0 and removed request for lucferbux and manaswinidas March 14, 2024 06:05

andrewballantyne commented Mar 14, 2024

View reviewed changes

andrewballantyne force-pushed the support-topology-v2 branch 2 times, most recently from d426c6c to b80be31 Compare March 14, 2024 20:45

Gkrumbach07 reviewed Mar 15, 2024

View reviewed changes

...d/src/concepts/pipelines/content/pipelinesDetails/pipelineRun/PipelineRunDrawerRightTabs.tsx Show resolved Hide resolved

andrewballantyne force-pushed the support-topology-v2 branch from b80be31 to cfe9104 Compare March 15, 2024 16:15

andrewballantyne force-pushed the support-topology-v2 branch 3 times, most recently from 500a2f5 to 760c02b Compare March 15, 2024 18:49

andrewballantyne force-pushed the support-topology-v2 branch from 760c02b to 8bf3cdd Compare March 15, 2024 19:45

andrewballantyne force-pushed the support-topology-v2 branch from 8bf3cdd to b701e5e Compare March 15, 2024 20:17

Support rendering topology nodes under v2

eef0558

andrewballantyne force-pushed the support-topology-v2 branch from b701e5e to eef0558 Compare March 15, 2024 20:22

andrewballantyne changed the title ~~[WIP] Support topology v2~~ Support topology v2 Mar 15, 2024

openshift-ci bot removed the do-not-merge/work-in-progress This PR is in WIP state label Mar 15, 2024

openshift-ci bot assigned Gkrumbach07 Mar 15, 2024

openshift-ci bot added the lgtm label Mar 15, 2024

openshift-ci bot added the approved label Mar 15, 2024

openshift-merge-bot bot merged commit ac47429 into opendatahub-io:main Mar 15, 2024
6 checks passed

andrewballantyne deleted the support-topology-v2 branch June 26, 2024 18:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support topology v2 #2598

Support topology v2 #2598

andrewballantyne commented Mar 13, 2024 •

edited

Loading

andrewballantyne commented Mar 14, 2024

andrewballantyne commented Mar 14, 2024

andrewballantyne left a comment

andrewballantyne Mar 14, 2024

andrewballantyne Mar 14, 2024

andrewballantyne commented Mar 14, 2024

Gkrumbach07 Mar 15, 2024

andrewballantyne Mar 15, 2024

andrewballantyne Mar 15, 2024

andrewballantyne Mar 15, 2024

Gkrumbach07 Mar 15, 2024

andrewballantyne Mar 15, 2024

Gkrumbach07 commented Mar 15, 2024

Gkrumbach07 commented Mar 15, 2024

andrewballantyne commented Mar 15, 2024 •

edited

Loading

andrewballantyne commented Mar 15, 2024 •

edited

Loading

Gkrumbach07 commented Mar 15, 2024

andrewballantyne commented Mar 15, 2024

openshift-ci bot commented Mar 15, 2024


		const { task_details: taskDetails } = runDetails;

		const thisTaskDetail = taskDetails.find(({ display_name: name }) => name === `${taskId}-driver`);

Support topology v2 #2598

Support topology v2 #2598

Conversation

andrewballantyne commented Mar 13, 2024 • edited Loading

Description

Screenshots

Gif

How Has This Been Tested?

Test Impact

Request review criteria:

andrewballantyne commented Mar 14, 2024

andrewballantyne commented Mar 14, 2024

andrewballantyne left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andrewballantyne commented Mar 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Gkrumbach07 commented Mar 15, 2024

Gkrumbach07 commented Mar 15, 2024

andrewballantyne commented Mar 15, 2024 • edited Loading

andrewballantyne commented Mar 15, 2024 • edited Loading

Gkrumbach07 commented Mar 15, 2024

andrewballantyne commented Mar 15, 2024

openshift-ci bot commented Mar 15, 2024

andrewballantyne commented Mar 13, 2024 •

edited

Loading

andrewballantyne commented Mar 15, 2024 •

edited

Loading

andrewballantyne commented Mar 15, 2024 •

edited

Loading