Analytics on pipeline execution

[cmadjar]

Task

Create a dashboard of CONP pipeline executions on CONP datasets. This dashboard would track executions done in and outside of CBRAIN.

How

• Regularly upload Boutiques provenance from CBRAIN and any other execution platform.
• Pull Boutiques provenance records and present them in graphs

Who

@mandana-mazaheri for the provenance dashboard, liaise with @nbeck for provenance upload from CBRAIN.

Related issues (optional)

#63

[mandana-mazaheri]

Dear @cmadjar the deadline for dashboard of CONP pipeline executions on CONP datasets can be the end of March for me.

[cmadjar]

Thanks @mandana-mazaheri I updated the github project accordingly :-)

[mandana-mazaheri]

Thank you @cmadjar, also there will be a recommender system based on these provenance records which recommends pipelines and datasets given the other.
And I think the end of April is a reasonable deadline for that.
Please let me know if it's better to add that to this issue or create a new one?

[cmadjar]

@mandana-mazaheri Maybe for that case it is better to create a new one and link it to this issue. Could you create the issue with the details? Thank you!

[mandana-mazaheri]

Sure @cmadjar

[cmadjar]

@mandana-mazaheri here is a summary of what has been discussed today with Derek for the design of this feature.

The behaviour would similar to the way we implemented the link to the NEXUS search on the portal. In the "Tools & Pipelines" search page, add a link above the search table that will redirect to a new page showing the summary result table with which tool was run on which dataset and whether it was a successful run or not. That result table would have filters at the top to be able to filter through pipelines and datasets and see the status for those.

In addition, we could add in the "Tools & Pipelines" search a processing success rate next to the page views statistics. That number would be the percentage of successful run/total number of run.

[mandana-mazaheri]

Thank you @cmadjar for your comment

[github-actions]

This issue is stale because it has been open 5 months with no activity. Remove stale label or comment or this will be closed in 3 months.

[github-actions]

This issue was closed because it has been stalled for 3 months with no activity.