Model Serving Enhancements

[lucferbux]

Description

This issue is targeted to collect all the issues related to Model Serving, both ModelMesh, KServe and Custom Serving Runtimes.

Target Branch

main

Itemized Dev Issues

General UX Improvements

• Filter data connections without bucket name in deploy model modal #1604

General UI Improvements

• [Bug]: Inconsistent Model Path #2228

KServe UX Improvements

• [Bug]: KServe microcopy #2178
• [Feature Request]: Add tooltip to "Deploy model" buton disabled in Model Serving global view #2165
• [Feature Request]: Scroll to Models and model servers section when navigating from Model Serving view #2168
• [Bug]: Hide accelerators when they are not present in the SR selection #2170

KServe UI Improvements

• [Bug]: 403 error creating model server with restricted permissions #1776
• [Bug]: Accelerator enablement in kserve is not working #2244
• [Bug]: Performance issue in Model Serving Global #2177
• [Bug]: Inconsistent status handling for kserve #2172
• [Bug]: Cleanup caikit ootb runtime #2143
• [Bug]: Add spinner and change logic In Model Serving Global Page #2135
• [Bug]: Improve global model serving project selection #2137
• [Story]: Fix inconsistent header in Global Model Serving view #2105
• Unable to duplicate a serving runtime #1727
• Add dismissible notification when a model deployment fails #1584
• [Story]: Serving Runtime modal should not create a SerivceAccount when ServingRuntime has no auth #2051
• [Story]: Bring popovers to Kserve Modal #2097
• [Story]: Add test coverage for kserve migration #1980
• [Bug]: ServingRuntime for Kserve model is created regardless of error #2237
• [Bug]: Already deployed models cannot be seen anymore if the component gets disabled in DSC #2187

Improve Custom Serving Runtimes (templates)

• [Feature Request]: Support for template parameters #1260
• Add support to deploy other objects with Custom Serving Runtime Templates #1190

Routing

• Enable GRPC support for model serving routes #1149
• [Feature Request]: Enable Model Serving route per Project #1287

GPU Support

• [Feature Request]: Improve GPU detection in Model Serving #1313

Past improvements

• [Feature Request]: Revamp the labels of legacy model servers deployed #1395
• [Bug]: RoleBinding deletion API call fails when a user with Edit permission deletes the model server #1320
• [Bug]: Delay Model Serving Annotation until First Server #1310
• [Bug]: Model Errors are not refreshed or spaced properly #1216
• [Bug]: Deploying Models doesn't refresh for new connections #1215
• [Bug]: Model deployment: path not correctly trimmed when editing #1173
• [Bug]: Users with Edit permissions cannot create a model server runtime in a shared project #1259
• [Bug]: Rename "Configure Model Server" to "Add Model Server" #1357
• [Bug]: Custom serving runtimes can have arbitrarily long names #1394
• [Bug]: RoleBinding deletion API call fails when a user with Edit permission deletes the model server #1320
• Tests for Custom Serving Runtimes feature #1198
• [Feature Request]: Model Serving - Taints/Tolerations/Affinities/Etc #1261
• Warn users if they don't add token autorization with public route in Model Serving #1582
• Auto-expand deployed models column when a model is deployed #1605

[lucferbux]

We are gonna work on this for Model Serving v2, wait until we have #1810 in place

[lucferbux]

I've migrated some of the issues to #1810 the rest of them can be worked and implemented to main, some of these issues are blocked, such as #1149 and #1260 so we can work on them after we've finished kserve

[lucferbux]

Moved kserve enhancements from #1810

[dgutride]

Migrated to jira: https://issues.redhat.com/browse/RHOAIENG-330

[dgutride]

All subissues have been migrated - moving to closed.