Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate Validate ServiceMeshControlPlaneAlreadyCreated Test Case #1762

Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
b8bbe5d
Automate Validate ServiceMeshControlPlaneAlreadyCreated Test Case
asanzgom Aug 30, 2024
601222a
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Aug 30, 2024
da00f98
Added RHODS_OSD_INSTALL_REPO variable
asanzgom Aug 30, 2024
3f99bc5
changed to relative path on smcp template
asanzgom Aug 30, 2024
3b085ac
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Aug 30, 2024
2296ab1
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 2, 2024
6c04140
removed olminstall repo
asanzgom Sep 2, 2024
7b4abdf
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 4, 2024
8f1f664
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 5, 2024
4cd24cb
Added Test Case teardown + restored -p & -n flags
asanzgom Sep 5, 2024
38b1dbf
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 5, 2024
c8cd1fa
Removed OPERATOR_NAME from Assign Vars According To Product
asanzgom Sep 5, 2024
6d36f21
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 11, 2024
cf1f1c5
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 11, 2024
7c48f75
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 11, 2024
4af25ad
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 12, 2024
4b76708
Added check to rebuild DSC if previously existed on cluster
asanzgom Sep 12, 2024
31205d5
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 17, 2024
cb4618e
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 17, 2024
8db0156
Added workaround to get catalogsource
asanzgom Sep 17, 2024
28dc83d
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 23, 2024
cc3559c
Added save component statuses and recreate DSC logic
asanzgom Sep 23, 2024
df37b0a
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 24, 2024
54e611f
changed to use Load Json string + renamed kw to applying devflags
asanzgom Sep 24, 2024
dec7e54
Renamed kw to Generate CustomManifest in DSC YAML
asanzgom Sep 24, 2024
17c986c
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 24, 2024
1be5592
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 24, 2024
1171fce
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 25, 2024
3474104
added final check for smcp + rename kws + modelregistry field update
asanzgom Sep 25, 2024
ac4fccf
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
asanzgom Sep 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions ods_ci/tasks/Resources/Files/smcp_template.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
kind: ServiceMeshControlPlane
apiVersion: maistra.io/v2
metadata:
name: basic
namespace: istio-system
spec:
version: v2.6
policy:
type: Istiod
telemetry:
type: Istiod
addons:
prometheus:
enabled: true
kiali:
enabled: true
grafana:
enabled: true
2 changes: 1 addition & 1 deletion ods_ci/tasks/Resources/RHODS_OLM/install/oc_install.robot
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@
Install RHODS In Self Managed Cluster Using CLI
[Documentation] Install rhods on self managed cluster using cli
[Arguments] ${cluster_type} ${image_url}
${return_code} Run and Watch Command cd ${EXECDIR}/${OLM_DIR} && ./setup.sh -t operator -u ${UPDATE_CHANNEL} -i ${image_url} -n ${OPERATOR_NAME} -p ${OPERATOR_NAMESPACE} timeout=20 min
CFSNM marked this conversation as resolved.
Show resolved Hide resolved
${return_code} Run and Watch Command cd ${EXECDIR}/${OLM_DIR} && ./setup.sh -t operator -u ${UPDATE_CHANNEL} -i ${image_url} timeout=20 min
Fixed Show fixed Hide fixed

Check warning

Code scanning / Robocop

The assignment sign is not consistent within the file. Expected '{{ expected_sign }}' but got '{{ actual_sign }}' instead Warning

The assignment sign is not consistent within the file. Expected ' =' but got '' instead
Should Be Equal As Integers ${return_code} 0 msg=Error detected while installing RHODS

Install RHODS In Managed Cluster Using CLI
Expand Down
2 changes: 1 addition & 1 deletion ods_ci/tests/Resources/ODS.robot
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,7 @@ Delete RHODS Config Map

Wait Until Operator Ready
[Documentation] Wait for operator to be available/ready
[Arguments] ${operator_name} ${namespace} ${wait_time}=2m
[Arguments] ${operator_name} ${namespace} ${wait_time}=10m
${rc} ${output}= Run And Return Rc And Output
... oc wait --timeout=${wait_time} --for condition=available -n ${namespace} deploy/${operator_name}
Should Be Equal "${rc}" "0" msg=${output}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,13 @@
${SERVICE_MESH_OPERATOR_DEPLOYMENT_NAME} istio-operator
${SERVICE_MESH_CR_NS} istio-system
${SERVICE_MESH_CR_NAME} data-science-smcp

${IS_PRESENT} 0
${IS_NOT_PRESENT} 1
${OLM_DIR} olm
${RHODS_OSD_INSTALL_REPO} https://gitlab.cee.redhat.com/data-hub/olminstall.git
Fixed Show fixed Hide fixed
asanzgom marked this conversation as resolved.
Show resolved Hide resolved
${INSTALL_TYPE} CLi

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${INSTALL_TYPE}' is assigned but not used
${TEST_ENV} PSI

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${TEST_ENV}' is assigned but not used
${IS_PRESENT} 0
${IS_NOT_PRESENT} 1
${MSSG_REGEX} denied the request: only one service mesh may be installed per project/namespace

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (124/120)


*** Test Cases ***
Expand All @@ -32,7 +36,6 @@

[Teardown] Set Service Mesh State To Managed And Wait For CR Ready ${SERVICE_MESH_CR_NAME} ${SERVICE_MESH_CR_NS} ${OPERATOR_NS}


Validate Service Mesh State Unmanaged
[Documentation] The purpose of this Test Case is to validate Service Mesh state 'Unmanaged'.
... The operator will not recreate/update the Service Mesh CR if removed or changed.
Expand All @@ -58,6 +61,28 @@
[Teardown] Set Service Mesh State To Managed And Wait For CR Ready
... ${SERVICE_MESH_CR_NAME} ${SERVICE_MESH_CR_NS} ${OPERATOR_NS}

Validate Service Mesh Control Plane Already Created

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Validate Service Mesh Control Plane Already Created' has too many keywords inside (11/10)
[Documentation] This Test Case validates that only one ServiceMeshControlPlane is allowed to be installed per project/namespace

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (134/120)
[Tags] Operator Tier3 RHOAIENG-2517
Fetch Image Url And Update Channel
Fetch Cluster Type By Domain
IF "${CLUSTER_TYPE}" == "selfmanaged"
Uninstall RHODS In Self Managed Cluster
Create Smcp From Template
Install RHODS In Self Managed Cluster Using CLI ${CLUSTER_TYPE} ${IMAGE_URL}
ELSE IF "${CLUSTER_TYPE}" == "managed"
Uninstall RHODS In OSD
Create Smcp From Template
Install RHODS In Managed Cluster Using CLI ${CLUSTER_TYPE} ${IMAGE_URL}
END
Operator Deployment Should Be Ready
# Go check the Operator logs for the error message: denied the request: only one service mesh may be installed per project/namespace

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (136/120)
Verify Pod Logs Contain Error
# Cleanup the SMCP
Delete Smcp
# Cleanup Olminstall dir
Cleanup Olm Install Dir

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a teardown method for the test just in case some step fails, so we are back to initial state, with ODH/RHOAI installed and working

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is the suite teardown
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Cesar, we need a teardown which restore the correct RHOAI status in the cluster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I can move it to the Suite Teardown

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added Test Case Teardown


*** Keywords ***
Suite Setup
Expand All @@ -70,3 +95,69 @@
Suite Teardown
[Documentation] Suite Teardown
RHOSi Teardown

Fetch Image Url And Update Channel

Check warning

Code scanning / Robocop

Keyword '{{ keyword_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Keyword 'Fetch Image Url And Update Channel' has too many keywords inside (12/10)
[Documentation] Fetch url for image and Update Channel
${rc} ${out}= Run And Return Rc And Output
... oc get catalogsource rhoai-catalog-dev --namespace openshift-marketplace -o jsonpath='{.spec.image}'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we test a catalog source that is not created by automations and the user includes manually another name, this line will fail.

Lets see if we can get a generic way to retrieve the used catalogsource, maybe by tags or by image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try with some label selector

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added workaround to get CatalogSource from subscription

Should Be Equal As Integers ${rc} 0
Set Global Variable ${IMAGE_URL} ${out}
Fixed Show fixed Hide fixed

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Global Variable can be replaced with VAR
${rc} ${out}= Run And Return Rc And Output
... oc get subscription rhoai-operator-dev --namespace redhat-ods-operator -o jsonpath='{.spec.channel}'
Should Be Equal As Integers ${rc} 0
Set Global Variable ${UPDATE_CHANNEL} ${out}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Global Variable can be replaced with VAR
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same here, the subscription can have a different name if we dont use automations and we deploy the catalog source by hand


Fetch Cluster Type By Domain
[Documentation] This Keyword outputs the kind of cluster depending on the console URL domain
${matches}= Get Regexp Matches ${OCP_CONSOLE_URL} rh-ods
${domain}= Get From List ${matches} 0
IF "${domain}" == "rh-ods"
Set Global Variable ${CLUSTER_TYPE} selfmanaged

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Global Variable can be replaced with VAR
ELSE
Set Global Variable ${CLUSTER_TYPE} managed

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Global Variable can be replaced with VAR
END

Create Smcp From Template
[Documentation] Create a default ServiceMeshControlPlane from a template
${file_path}= Set Variable ~/work/ods-ci/ods_ci/tasks/Resources/Files/
${return_code}= Run And Return Rc oc apply -f ${file_path}smcp_template.yml
Should Be Equal As Integers ${return_code} 0

Delete Smcp
[Documentation] Delete the ServiceMeshControlPlane already created
Run and Watch Command

Check warning

Code scanning / Robocop

Keyword name '{{ keyword_name }}' does not follow case convention Warning test

Keyword name 'Run and Watch Command' does not follow case convention
... oc delete ServiceMeshControlPlane basic --namespace ${SERVICE_MESH_CR_NS} --force --grace-period=0

Operator Deployment Should Be Ready
[Documentation] Loop until the Operator deployment is ready
${rc}= Set Variable 1
TRY
WHILE ${rc} != 0 limit=10m

Check notice

Code scanning / Robocop

'{{ block_name }}' condition can be simplified Note test

'WHILE' condition can be simplified
Sleep 5s
${rc} ${output}= Run And Return Rc And Output
... oc wait --for condition=available -n ${OPERATOR_NAMESPACE} deploy/${OPERATOR_DEPLOYMENT_NAME}
END
EXCEPT WHILE loop was aborted type=start
Fail msg=Operator deployment did not get ready
END

Verify Pod Logs Contain Error
[Documentation] Checks whether there is a SMCP related error on the Pod Logs
${pod_name}= Get Pod Name ${OPERATOR_NAMESPACE} ${OPERATOR_LABEL_SELECTOR}
${pod_logs}= Oc Get Pod Logs
... name=${pod_name}
... namespace=${OPERATOR_NAMESPACE}
... container=${OPERATOR_POD_CONTAINER_NAME}
${match_list}= Get Regexp Matches ${pod_logs} ${MSSG_REGEX}
${entry_msg}= Remove Duplicates ${match_list}
${length}= Get Length ${entry_msg}
IF ${length} == ${0}
FAIL Pod ${pod_name} logs should contain message ${MSSG_REGEX}.
ELSE
Log message=Pod ${pod_name} logs contain error message '${MSSG_REGEX}' level=INFO
END

Cleanup Olm Install Dir
[Documentation] Clean up olm install repo
${return_code}= Run and Watch Command rm -vRf ${EXECDIR}/${OLM_DIR} timeout=10 min

Check warning

Code scanning / Robocop

Keyword name '{{ keyword_name }}' does not follow case convention Warning test

Keyword name 'Run and Watch Command' does not follow case convention
Should Be Equal As Integers ${return_code} 0 msg=Error while cleaning up olminstall directory
Loading