-
Notifications
You must be signed in to change notification settings - Fork 82
Commit
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
*** Settings *** | ||
Documentation Collection of CLI tests to validate the model serving stack for Large Language Models (LLM). | ||
... These tests leverage on TGIS Standalone Serving Runtime | ||
Resource ../../../../Resources/OCP.resource | ||
Resource ../../../../Resources/CLI/ModelServing/llm.resource | ||
Resource ../../../../Resources/CLI/Minio.resource | ||
Library OpenShiftLibrary | ||
Suite Setup Suite Setup | ||
Suite Teardown RHOSi Teardown | ||
Test Tags KServe | ||
|
||
|
||
*** Variables *** | ||
${MODEL_S3_DIR}= flan-t5-small-hf | ||
${TEST_NS}= tgis-storages | ||
${TGIS_RUNTIME_NAME}= tgis-runtime | ||
@{SEARCH_METRICS}= tgi_ istio_ | ||
Check notice Code scanning / Robocop Variable '{{ name }}' is assigned but not used Note test
Variable '@{SEARCH_METRICS}' is assigned but not used
|
||
|
||
|
||
Check warning Code scanning / Robocop Trailing whitespace at the end of line Warning test
Trailing whitespace at the end of line
|
||
*** Test Cases *** | ||
Verify User Can Serve And Query A Model From Minio | ||
Check warning Code scanning / Robocop Test case '{{ test_name }}' is too long ({{ test_length }}/{{ allowed_length }}) Warning test
Test case 'Verify User Can Serve And Query A Model From Minio' is too long (32/20)
Check warning Code scanning / Robocop Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test
Test case 'Verify User Can Serve And Query A Model From Minio' has too many keywords inside (13/10)
|
||
[Documentation] Basic tests for preparing, deploying and querying a LLM model | ||
... using Kserve and Caikit+TGIS runtime | ||
[Tags] Tier1 RHOAIENG-3490 | ||
${minio_namespace}= Set Variable minio-models | ||
Deploy MinIO namespace=${minio_namespace} | ||
Wait For Pods To Be Ready label_selector=app=minio | ||
... namespace=${minio_namespace} | ||
${key} ${pw}= Get Minio Credentials namespace=${minio_namespace} | ||
Set Project And Runtime runtime=${TGIS_RUNTIME_NAME} namespace=${TEST_NS}-minio | ||
... access_key_id=${key} access_key=${pw} | ||
... endpoint=minio-service.${minio_namespace}.svc.cluster.local:9000 | ||
... verify_ssl=${FALSE} # temporary | ||
${test_namespace}= Set Variable ${TEST_NS}-minio | ||
${model_name}= Set Variable flan-t5-small-hf | ||
${models_names}= Create List ${model_name} | ||
Check notice Code scanning / Robocop Variable '{{ name }}' is assigned but not used Note test
Variable '${models_names}' is assigned but not used
Check notice Code scanning / Robocop {{ create_keyword }} can be replaced with VAR Note test
Create List can be replaced with VAR
|
||
${storage_uri}= Set Variable s3://models/${MODEL_S3_DIR}/ | ||
Compile Inference Service YAML isvc_name=${model_name} | ||
... sa_name=${DEFAULT_BUCKET_SA_NAME} | ||
... model_storage_uri=${storage_uri} | ||
... model_format=pytorch serving_runtime=${TGIS_RUNTIME_NAME} | ||
Deploy Model Via CLI isvc_filepath=${INFERENCESERVICE_FILLED_FILEPATH} | ||
... namespace=${test_namespace} | ||
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${model_name} | ||
... namespace=${test_namespace} | ||
Query Model Multiple Times model_name=${model_name} runtime=${TGIS_RUNTIME_NAME} | ||
... inference_type=all-tokens n_times=1 | ||
... namespace=${test_namespace} | ||
# Query Model Multiple Times model_name=${model_name} runtime=${TGIS_RUNTIME_NAME} | ||
# ... inference_type=streaming n_times=1 | ||
# ... namespace=${test_namespace} validate_response=${FALSE} | ||
# [Teardown] Clean Up Test Project test_ns=${test_namespace} | ||
# ... isvc_names=${models_names} wait_prj_deletion=${FALSE} | ||
|
||
|
||
*** Keywords *** | ||
Suite Setup | ||
[Documentation] | ||
Check warning Code scanning / Robocop Documentation of {{ block_name }} is empty Warning test
Documentation of 'Suite Setup' Keyword is empty
|
||
Skip If Component Is Not Enabled kserve | ||
# RHOSi Setup | ||
Load Expected Responses | ||
Run git clone https://github.com/IBM/text-generation-inference/ | ||
Check warning Code scanning / Robocop Missing trailing blank line at the end of file Warning test
Missing trailing blank line at the end of file
|