Automation tests for pytorch model gRPC RHOAIENG-9053 for triton #1834
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
Fixed
Show fixed
Hide fixed
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
Fixed
Show resolved
Hide resolved
| @@ -151,6 +155,52 @@ | |||
| ... Clean All Models Of Current User | |||
| ... AND | |||
| ... Delete Serving Runtime Template From CLI displayed_name=triton-kserve-grpc | |||
|
|
|||
Check warning
Code scanning / Robocop
Trailing whitespace at the end of line Warning test
| @@ -151,6 +155,52 @@ | |||
| ... Clean All Models Of Current User | |||
| ... AND | |||
| ... Delete Serving Runtime Template From CLI displayed_name=triton-kserve-grpc | |||
|
|
|||
| Test PYTORCH Model Grpc Inference Via UI (Triton on Kserve) # robocop: off=too-long-test-case | |||
Check warning
Code scanning / Robocop
Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test
| ... token=${TRUE} | ||
| Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${PYTORCH_MODEL_LABEL} | ||
| ... namespace=${PRJ_TITLE} | ||
| ${EXPECTED_INFERENCE_GRPC_OUTPUT_PYTORCH}= Load Json File file_path=${EXPECTED_INFERENCE_GRPC_OUTPUT_FILE_PYTORCH} |
Check warning
Code scanning / Robocop
Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test
Robot Results
|
|
Verified with Jenkins Build 432 |
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
Outdated
Show resolved
Hide resolved
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
Outdated
Show resolved
Hide resolved
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
Outdated
Show resolved
Hide resolved
...odel_serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve.robot
Outdated
Show resolved
Hide resolved
| Log ${EXPECTED_INFERENCE_GRPC_OUTPUT_PYTORCH} | ||
| Open Model Serving Home Page | ||
| ${host_url}= Get Model Route Via UI model_name=${PYTORCH_MODEL_NAME} | ||
| ${host}= Evaluate re.search(r"${PATTERN}", r"${host_url}").group(1) re |
There was a problem hiding this comment.
please create a kw for this one given that it's been re-used.
One question, why do we need this? What's returning from "Get Model Route Via UI"? I don't remember we had issues in the past about this
There was a problem hiding this comment.
So, the Get Model Route Via UI keyword returns the whole URL. We just need the domain name, which is why we used a regex pattern. We will create it as a keyword
There was a problem hiding this comment.
IMO you should also report this to Dashboard team because the UI should report the right URL based on the protocol
rpancham
force-pushed
the
pytorch_grpc
branch
from
883acd5 to
f6ed6fc
Compare
rpancham
force-pushed
the
pytorch_grpc
branch
from
01c9586 to
4f5dd1f
Compare
|
Verified with Jenkins Build 465 |
| Log ${inference_output} | ||
| ${inference_output}= Evaluate json.dumps(${inference_output}) | ||
| Log ${inference_output} | ||
| ${result} ${list}= Inference Comparison ${EXPECTED_INFERENCE_GRPC_OUTPUT_PYTORCH} ${inference_output} |
There was a problem hiding this comment.
iirc this keyword does not fail directly, but you should add a check on the ${result}. Check how it is used in Verify Model Inference keyword.
|
|
Added Automation Test for RHOAIENG-11561