Add support for tgi metricesfor vllm #1513
Conversation
Signed-off-by: Milind Waykole <[email protected]>
|
Robot Results
|
|
|
| @@ -89,7 +103,7 @@ Verify Vllm Metrics Are Present | |||
| Set Suite Variable ${token} | |||
| Metrics Should Exist In UserWorkloadMonitoring ${thanos_url} ${token} ${SEARCH_METRICS} | |||
|
|
|||
| Verify Vllm Metrics Values Match Between UWM And Endpoint | |||
| Verify Vllm And tgi Metrics Values Match Between UWM And Endpoint | |||
There was a problem hiding this comment.
This does not work - Line 98/112 fetches the metrics to test for by looking at those that start with vllm:, so TGI metrics are not being tested at all here.
You will need to either change the logic of the Get Vllm Metrics And Values in Helpers.py or develop an equivalent keyword for TGI metrics
| Verify User Can Deploy A Model With Vllm Via CLI | ||
| Verify User Can Deploy A Model With Vllm And tgi Via CLI | ||
| [Documentation] Deploy a model (gpt2) using the vllm runtime and confirm that it's running | ||
| [Tags] Tier1 Sanity Resources-GPU RHOAIENG-6264 VLLM | ||
| ${rc} ${out}= Run And Return Rc And Output oc apply -f ${DL_POD_FILEPATH} |
There was a problem hiding this comment.
This test is not using tgi to deploy the model, it's using vLLM, please revert the name of this TC
| Verify Vllm And tgi Metrics Are Present | ||
| [Documentation] Confirm vLLM and tgi metrics are exposed in OpenShift metrics |
There was a problem hiding this comment.
IMHO you should create a separate test to validate that TGI metrics are present - as far as I understand you've already encountered the issue where only vLLM ones would get exposed, so it'd be better to separate the two and have clear reporting on which ones (if any) are failing to get exposed.
mwaykole
added
do not merge
No description provided.