Skip to content

Commit

Permalink
add draft of minio test
Browse files Browse the repository at this point in the history
  • Loading branch information
bdattoma committed Feb 26, 2024
1 parent a8a4532 commit 1dcfe80
Showing 1 changed file with 62 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
*** Settings ***
Documentation Collection of CLI tests to validate the model serving stack for Large Language Models (LLM).
... These tests leverage on TGIS Standalone Serving Runtime
Resource ../../../../Resources/OCP.resource
Resource ../../../../Resources/CLI/ModelServing/llm.resource
Resource ../../../../Resources/CLI/Minio.resource
Library OpenShiftLibrary
Suite Setup Suite Setup
Suite Teardown RHOSi Teardown
Test Tags KServe


*** Variables ***
${MODEL_S3_DIR}= flan-t5-small-hf
${TEST_NS}= tgis-storages
${TGIS_RUNTIME_NAME}= tgis-runtime
@{SEARCH_METRICS}= tgi_ istio_

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '@{SEARCH_METRICS}' is assigned but not used


Check warning

Code scanning / Robocop

Trailing whitespace at the end of line Warning test

Trailing whitespace at the end of line
*** Test Cases ***
Verify User Can Serve And Query A Model From Minio

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' is too long ({{ test_length }}/{{ allowed_length }}) Warning test

Test case 'Verify User Can Serve And Query A Model From Minio' is too long (32/20)

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Verify User Can Serve And Query A Model From Minio' has too many keywords inside (13/10)
[Documentation] Basic tests for preparing, deploying and querying a LLM model
... using Kserve and Caikit+TGIS runtime
[Tags] Tier1 RHOAIENG-3490
${minio_namespace}= Set Variable minio-models
Deploy MinIO namespace=${minio_namespace}
Wait For Pods To Be Ready label_selector=app=minio
... namespace=${minio_namespace}
${key} ${pw}= Get Minio Credentials namespace=${minio_namespace}
Set Project And Runtime runtime=${TGIS_RUNTIME_NAME} namespace=${TEST_NS}-minio
... access_key_id=${key} access_key=${pw}
... endpoint=minio-service.${minio_namespace}.svc.cluster.local:9000
... verify_ssl=${FALSE} # temporary
${test_namespace}= Set Variable ${TEST_NS}-minio
${model_name}= Set Variable flan-t5-small-hf
${models_names}= Create List ${model_name}

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${models_names}' is assigned but not used

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create List can be replaced with VAR
${storage_uri}= Set Variable s3://models/${MODEL_S3_DIR}/
Compile Inference Service YAML isvc_name=${model_name}
... sa_name=${DEFAULT_BUCKET_SA_NAME}
... model_storage_uri=${storage_uri}
... model_format=pytorch serving_runtime=${TGIS_RUNTIME_NAME}
Deploy Model Via CLI isvc_filepath=${INFERENCESERVICE_FILLED_FILEPATH}
... namespace=${test_namespace}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${model_name}
... namespace=${test_namespace}
Query Model Multiple Times model_name=${model_name} runtime=${TGIS_RUNTIME_NAME}
... inference_type=all-tokens n_times=1
... namespace=${test_namespace}
# Query Model Multiple Times model_name=${model_name} runtime=${TGIS_RUNTIME_NAME}
# ... inference_type=streaming n_times=1
# ... namespace=${test_namespace} validate_response=${FALSE}
# [Teardown] Clean Up Test Project test_ns=${test_namespace}
# ... isvc_names=${models_names} wait_prj_deletion=${FALSE}


*** Keywords ***
Suite Setup
[Documentation]

Check warning

Code scanning / Robocop

Documentation of {{ block_name }} is empty Warning test

Documentation of 'Suite Setup' Keyword is empty
Skip If Component Is Not Enabled kserve
# RHOSi Setup
Load Expected Responses
Run git clone https://github.com/IBM/text-generation-inference/

Check warning

Code scanning / Robocop

Missing trailing blank line at the end of file Warning test

Missing trailing blank line at the end of file

0 comments on commit 1dcfe80

Please sign in to comment.