forked from red-hat-data-services/ods-ci
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'master' into feature/Validate_ServiceMeshControlPlaneAl…
…readyCreated
- Loading branch information
Showing
36 changed files
with
346 additions
and
225 deletions.
There are no files selected for viewing
115 changes: 115 additions & 0 deletions
115
ods_ci/tests/Resources/CLI/DataSciencePipelines/DataSciencePipelinesBackend.resource
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,115 @@ | ||
| *** Settings *** | ||
| Documentation Collection of keywords to interact with Data Science Pipelines via CLI | ||
| Library OperatingSystem | ||
| Resource ../../../Resources/OCP.resource | ||
|
|
||
|
|
||
| *** Variables *** | ||
| ${DSPA_PATH}= tests/Resources/Files/pipeline-samples/v2/dspa | ||
|
|
||
|
|
||
| *** Keywords *** | ||
| Create Pipeline Server | ||
| [Documentation] Creates a pipeline server providing object storage and database information | ||
| ... Note: currently, only some of the parameters are used. In the future this keyword will be | ||
| ... enhanced to use them all | ||
| [Arguments] ${namespace} | ||
| ... ${object_storage_access_key} ${object_storage_secret_key} | ||
| # ... ${object_storage_endpoint} ${object_storage_region} | ||
| # ... ${object_storage_bucket_name} | ||
| # ... ${database_host}=${EMPTY} ${database_port}=3306 | ||
| # ... ${database_username}=${EMPTY} ${database_password}=${EMPTY} | ||
| # ... ${database_db_name}=${EMPTY} | ||
| ... ${dsp_version}=v2 | ||
|
|
||
| Create Secret With Pipelines Object Storage Information namespace=${namespace} | ||
| ... object_storage_access_key=${object_storage_access_key} | ||
| ... object_storage_secret_key=${object_storage_secret_key} | ||
|
|
||
| # Process DSPA Template to create pipeline server | ||
| ${TEMPLATE_PARAMETERS}= Set Variable -p DSP_VERSION=${dsp_version} | ||
| Run oc process -f ${DSPA_PATH}/dspa-template.yaml ${TEMPLATE_PARAMETERS} | oc apply -n ${namespace} -f - | ||
|
|
||
| # robocop: disable:line-too-long | ||
| Create PipelineServer Using Custom DSPA | ||
| [Documentation] Install and verifies that DataSciencePipelinesApplication CRD is installed and working | ||
| [Arguments] ${namespace} ${dspa_file}=data-science-pipelines-sample.yaml ${assert_install}=True | ||
|
|
||
| Run oc apply -f "${DSPA_PATH}/${dspa_file}" -n ${namespace} | ||
| IF ${assert_install}==True | ||
| ${generation_value} Run oc get datasciencepipelinesapplications -n ${namespace} -o json | jq '.items[0].metadata.generation' | ||
| Should Be True ${generation_value} == 2 DataSciencePipelinesApplication created | ||
| END | ||
|
|
||
| Verify Pipeline Server Deployments # robocop: disable | ||
| [Documentation] Verifies the correct deployment of DS Pipelines in the rhods namespace | ||
| [Arguments] ${namespace} | ||
|
|
||
| @{all_pods}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=component=data-science-pipelines | ||
| Run Keyword And Continue On Failure Length Should Be ${all_pods} 7 | ||
|
|
||
| @{pipeline_api_server}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=ds-pipeline-dspa | ||
| ${containerNames}= Create List oauth-proxy ds-pipeline-api-server | ||
| Verify Deployment ${pipeline_api_server} 1 2 ${containerNames} | ||
|
|
||
| @{pipeline_metadata_envoy}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=ds-pipeline-metadata-envoy-dspa | ||
| ${containerNames}= Create List container oauth-proxy | ||
| Verify Deployment ${pipeline_metadata_envoy} 1 2 ${containerNames} | ||
|
|
||
| @{pipeline_metadata_grpc}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=ds-pipeline-metadata-grpc-dspa | ||
| ${containerNames}= Create List container | ||
| Verify Deployment ${pipeline_metadata_grpc} 1 1 ${containerNames} | ||
|
|
||
| @{pipeline_persistenceagent}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=ds-pipeline-persistenceagent-dspa | ||
| ${containerNames}= Create List ds-pipeline-persistenceagent | ||
| Verify Deployment ${pipeline_persistenceagent} 1 1 ${containerNames} | ||
|
|
||
| @{pipeline_scheduledworkflow}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=ds-pipeline-scheduledworkflow-dspa | ||
| ${containerNames}= Create List ds-pipeline-scheduledworkflow | ||
| Verify Deployment ${pipeline_scheduledworkflow} 1 1 ${containerNames} | ||
|
|
||
| @{pipeline_workflow_controller}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=ds-pipeline-workflow-controller-dspa | ||
| ${containerNames}= Create List ds-pipeline-workflow-controller | ||
| Verify Deployment ${pipeline_workflow_controller} 1 1 ${containerNames} | ||
|
|
||
| @{mariadb}= Oc Get kind=Pod namespace=${namespace} | ||
| ... label_selector=app=mariadb-dspa | ||
| ${containerNames}= Create List mariadb | ||
| Verify Deployment ${mariadb} 1 1 ${containerNames} | ||
|
|
||
| Wait Until Pipeline Server Is Deployed | ||
| [Documentation] Waits until all the expected pods of the pipeline server | ||
| ... are running | ||
| [Arguments] ${namespace} | ||
| Wait Until Keyword Succeeds 10 times 10s | ||
| ... Verify Pipeline Server Deployments namespace=${namespace} | ||
|
|
||
| Wait Until Pipeline Server Is Deleted | ||
| [Documentation] Waits until all pipeline server pods are deleted | ||
| [Arguments] ${namespace} | ||
| # robocop: off=expression-can-be-simplified | ||
| FOR ${_} IN RANGE 0 30 | ||
| ${pod_count}= Run oc get pods -n ${namespace} -l component=data-science-pipelines | wc -l | ||
| IF ${pod_count}==0 BREAK | ||
| Sleep 1s | ||
| END | ||
|
|
||
| # robocop: disable:line-too-long | ||
| Create Pipelines ConfigMap With Custom Pip Index Url And Trusted Host | ||
| [Documentation] Creates a Configmap (ds-pipeline-custom-env-vars) in the project, | ||
| ... storing the values for pip_index_url and pip_trusted_host | ||
| [Arguments] ${namespace} | ||
| Run oc create configmap ds-pipeline-custom-env-vars -n ${namespace} --from-literal=pip_index_url=${PIP_INDEX_URL} --from-literal=pip_trusted_host=${PIP_TRUSTED_HOST} | ||
|
|
||
| Create Secret With Pipelines Object Storage Information | ||
| [Documentation] Creates a secret needed to create a pipeline server containing the object storage credentials | ||
| [Arguments] ${namespace} ${object_storage_access_key} ${object_storage_secret_key} | ||
| Run oc create secret generic dashboard-dspa-secret -n ${namespace} --from-literal=AWS_ACCESS_KEY_ID=${object_storage_access_key} --from-literal=AWS_SECRET_ACCESS_KEY=${object_storage_secret_key} | ||
| Run oc label secret dashboard-dspa-secret -n ${namespace} opendatahub.io/dashboard=true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
39 changes: 39 additions & 0 deletions
39
ods_ci/tests/Resources/Files/pipeline-samples/v2/dspa/dspa-template.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| kind: Template | ||
| apiVersion: template.openshift.io/v1 | ||
| metadata: | ||
| name: dspa-template | ||
| objects: | ||
| - apiVersion: datasciencepipelinesapplications.opendatahub.io/v1alpha1 | ||
| kind: DataSciencePipelinesApplication | ||
| metadata: | ||
| name: dspa | ||
| spec: | ||
| dspVersion: ${DSP_VERSION} | ||
| objectStorage: | ||
| disableHealthCheck: false | ||
| enableExternalRoute: false | ||
| externalStorage: | ||
| basePath: '' | ||
| bucket: ${OBJECT_STORAGE_BUCKET} | ||
| host: ${OBJECT_STORAGE_HOST} | ||
| port: '' | ||
| region: ${OBJECT_STORAGE_REGION} | ||
| s3CredentialsSecret: | ||
| accessKey: AWS_ACCESS_KEY_ID | ||
| secretKey: AWS_SECRET_ACCESS_KEY | ||
| secretName: dashboard-dspa-secret | ||
| scheme: https | ||
| podToPodTLS: true | ||
| parameters: | ||
| - description: Kubeflow Pipelines Version | ||
| value: "v2" | ||
| name: DSP_VERSION | ||
| - description: Object Storage Bucket Name | ||
| value: "ods-ci-ds-pipelines" | ||
| name: OBJECT_STORAGE_BUCKET | ||
| - description: Object Storage Host | ||
| value: "s3.amazonaws.com" | ||
| name: OBJECT_STORAGE_HOST | ||
| - description: Object Storage Region | ||
| value: "us-east-1" | ||
| name: OBJECT_STORAGE_REGION |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.