Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDCICD-1338: custom task to generate OLM catalog #1170

Merged
merged 1 commit into from
Jul 31, 2024

Conversation

jbpratt
Copy link
Contributor

@jbpratt jbpratt commented Jul 17, 2024

Add a new custom task to generate the OLM catalog based on a bundle image. This creates the Dockerfile and generates the catalog contents by adding a single bundle and a single channel entry for that bundle to the catalog with a skip range to the latest version. OLM can then be treated in such a way that it will always upgrade to the latest version available in the catalog.

following the documentation1 for creating raw FBCs

PR #1153 will be used to generate the bundle that is fed into this task as a parameter. SRE-P operators are maintained without actively versioning or managing upgrade graphs; the latest version is always the version that should be running.

task output
❯ opc pipeline start --use-param-defaults --showlog --workspace name=workspace,volumeClaimTemplateFile=tkn/workspace-template.yaml catalog-dynamic-build
PipelineRun started: catalog-dynamic-build-run-sldr4
Waiting for logs to be available...
[init : init] Build Initialize: quay.io/redhat-user-workloads/oeo-cicada-tenant/osd-example-operator/catalog:on-pr-test
[init : init]
[init : init] Determine if Image Already Exists

[clone-repository : clone] {"level":"info","ts":1721221123.8397663,"caller":"git/git.go:176","msg":"Successfully cloned https://github.com/openshift/osd-example-operator.git @ 587fc22f55c819208f91d719a40527a8bf913142 (grafted, HEAD) in path /workspace/output/source"}
[clone-repository : clone] {"level":"info","ts":1721221123.8739703,"caller":"git/git.go:215","msg":"Successfully initialized and updated submodules in path /workspace/output/source"}

[clone-repository : symlink-check] Running symlink check

[generate-catalog : generate-olm-catalog] + mkdir -p catalog/
[generate-catalog : generate-olm-catalog] + opm generate dockerfile catalog/ --binary-image=registry.redhat.io/openshift4/ose-operator-registry-rhel9:v4.16
[generate-catalog : generate-olm-catalog] + opm init osd-example-operator --default-channel=stable --description=./README.md --output=yaml
[generate-catalog : generate-olm-catalog] + opm render quay.io/redhat-user-workloads/oeo-cicada-tenant/osd-example-operator/bundle@sha256:570c3e3808236d003d0430f035f7c108200c7adc3927e49919aea830c6ed95d2 --output=yaml
[generate-catalog : generate-olm-catalog] + cat
[generate-catalog : generate-olm-catalog] + cat catalog/operator.yaml
[generate-catalog : generate-olm-catalog] ---
[generate-catalog : generate-olm-catalog] defaultChannel: stable
[generate-catalog : generate-olm-catalog] description: "# osd-example-operator\n\nThis repository serves as a test bed for the
[generate-catalog : generate-olm-catalog]   SD-CICD team to build tooling and\nsupport operators with minimal impact on other
[generate-catalog : generate-olm-catalog]   teams\n\n## Complete SOP on operator test harness\n\nhttps://github.com/openshift/ops-sop/blob/master/v4/howto/osde2e/operator-test-harnesses.md\n\n##
[generate-catalog : generate-olm-catalog]   Locally Running Test Harness\n- Run `make e2e-harness-build`  to make sure harness
[generate-catalog : generate-olm-catalog]   builds ok\n- Deploy your new version of operator in a test cluster\n- Ensure e2e
[generate-catalog : generate-olm-catalog]   test scenarios run green on a test cluster using one of the methods below\n\n###
[generate-catalog : generate-olm-catalog]   Using ginkgo\n1. create stage rosa cluster\n2. install ginkgo executable\n3. get
[generate-catalog : generate-olm-catalog]   kubeadmin credentials from your cluster using\n```\nocm get /api/clusters_mgmt/v1/clusters/$CLUSTER_ID/credentials
[generate-catalog : generate-olm-catalog]   | jq -r .kubeconfig > /<path-to>/kubeconfig\n```\n4. Run harness using\n```\nOCM_ENVIRONMENT=stage
[generate-catalog : generate-olm-catalog]   KUBECONFIG=/<path-to>/kubeconfig  ./<path-to>/bin/ginkgo  --tags=osde2e -v \n```\n5.
[generate-catalog : generate-olm-catalog]   This will show test results, but also one execution error due to reporting configs.
[generate-catalog : generate-olm-catalog]   You can ignore this, or get rid of this, by temporarily removing the `suiteConfig`
[generate-catalog : generate-olm-catalog]   and `reporterConfig` arguments from `RunSpecs()` function in `osde2e/<operator-name_>test_harness_runner_test.go`
[generate-catalog : generate-olm-catalog]   file\n\n\n### Using osde2e\n\n1. Publish a docker image for the test harness from
[generate-catalog : generate-olm-catalog]   operator repo using\n   ```\n   HARNESS_IMAGE_REPOSITORY=<your quay HARNESS_IMAGE_REPOSITORY>
[generate-catalog : generate-olm-catalog]   \ HARNESS_IMAGE_NAME=<your quay HARNESS_IMAGE_NAME> make e2e-image-build-push\n
[generate-catalog : generate-olm-catalog]   \  ```\n1. Create a stage rosa cluster\n1. Clone osde2e: `git clone [email protected]:openshift/osde2e.git`\n1.
[generate-catalog : generate-olm-catalog]   Build osde2e executable: `make build`\n1. Run osde2e\n\n  ```bash\n  #!/usr/bin/env
[generate-catalog : generate-olm-catalog]   bash\n  OCM_TOKEN=\"[OCM token here]\" \\ \n  CLUSTER_ID=\"[cluster id here]\" \\\n
[generate-catalog : generate-olm-catalog]   \ AWS_ACCESS_KEY_ID=\"[aws access key here]\" \\\n  AWS_SECRET_ACCESS_KEY=\"[aws
[generate-catalog : generate-olm-catalog]   access secret here]\" \\\n  TEST_HARNESSES=\"quay.io/$HARNESS_IMAGE_REPOSITORY/$HARNESS_IMAGE_NAME\"
[generate-catalog : generate-olm-catalog]   \\\n#  Save results in specific local dir \n  REPORT_DIR=\"[path to local report
[generate-catalog : generate-olm-catalog]   directory]\" \\\n#  OR in s3\n  LOG_BUCKET=\"[name of the s3 bucket to upload log
[generate-catalog : generate-olm-catalog]   files to]\" \\\n  ./out/osde2e test \\\n  --configs rosa,stage,sts,test-harness
[generate-catalog : generate-olm-catalog]   \\\n  --skip-must-gather \\\n  --skip-destroy-cluster \\\n  --skip-health-check
[generate-catalog : generate-olm-catalog]   \n"
[generate-catalog : generate-olm-catalog] name: osd-example-operator
[generate-catalog : generate-olm-catalog] schema: olm.package
[generate-catalog : generate-olm-catalog] ---
[generate-catalog : generate-olm-catalog] image: quay.io/redhat-user-workloads/oeo-cicada-tenant/osd-example-operator/bundle@sha256:570c3e3808236d003d0430f035f7c108200c7adc3927e49919aea830c6ed95d2
[generate-catalog : generate-olm-catalog] name: osd-example-operator.v4.16.0-dc7b3d1
[generate-catalog : generate-olm-catalog] package: osd-example-operator
[generate-catalog : generate-olm-catalog] properties:
[generate-catalog : generate-olm-catalog] - type: olm.gvk
[generate-catalog : generate-olm-catalog]   value:
[generate-catalog : generate-olm-catalog]     group: managed.openshift.io
[generate-catalog : generate-olm-catalog]     kind: Example
[generate-catalog : generate-olm-catalog]     version: v1alpha1
[generate-catalog : generate-olm-catalog] - type: olm.package
[generate-catalog : generate-olm-catalog]   value:
[generate-catalog : generate-olm-catalog]     packageName: osd-example-operator
[generate-catalog : generate-olm-catalog]     version: 4.16.0-dc7b3d1
[generate-catalog : generate-olm-catalog] - type: olm.csv.metadata
[generate-catalog : generate-olm-catalog]   value:
[generate-catalog : generate-olm-catalog]     annotations:
[generate-catalog : generate-olm-catalog]       alm-examples: '[]'
[generate-catalog : generate-olm-catalog]       capabilities: Basic Install
[generate-catalog : generate-olm-catalog]       createdAt: "2024-07-16T12:29:27Z"
[generate-catalog : generate-olm-catalog]       operators.operatorframework.io/builder: operator-sdk-v1.35.0
[generate-catalog : generate-olm-catalog]       operators.operatorframework.io/project_layout: unknown
[generate-catalog : generate-olm-catalog]     apiServiceDefinitions: {}
[generate-catalog : generate-olm-catalog]     crdDescriptions:
[generate-catalog : generate-olm-catalog]       owned:
[generate-catalog : generate-olm-catalog]       - kind: Example
[generate-catalog : generate-olm-catalog]         name: examples.managed.openshift.io
[generate-catalog : generate-olm-catalog]         version: v1alpha1
[generate-catalog : generate-olm-catalog]     description: Osd Example Operator description. TODO.
[generate-catalog : generate-olm-catalog]     displayName: Osd Example Operator
[generate-catalog : generate-olm-catalog]     installModes:
[generate-catalog : generate-olm-catalog]     - supported: false
[generate-catalog : generate-olm-catalog]       type: OwnNamespace
[generate-catalog : generate-olm-catalog]     - supported: false
[generate-catalog : generate-olm-catalog]       type: SingleNamespace
[generate-catalog : generate-olm-catalog]     - supported: false
[generate-catalog : generate-olm-catalog]       type: MultiNamespace
[generate-catalog : generate-olm-catalog]     - supported: true
[generate-catalog : generate-olm-catalog]       type: AllNamespaces
[generate-catalog : generate-olm-catalog]     keywords:
[generate-catalog : generate-olm-catalog]     - osd-example-operator
[generate-catalog : generate-olm-catalog]     links:
[generate-catalog : generate-olm-catalog]     - name: Osd Example Operator
[generate-catalog : generate-olm-catalog]       url: https://osd-example-operator.domain
[generate-catalog : generate-olm-catalog]     maintainers:
[generate-catalog : generate-olm-catalog]     - email: [email protected]
[generate-catalog : generate-olm-catalog]       name: Maintainer Name
[generate-catalog : generate-olm-catalog]     maturity: alpha
[generate-catalog : generate-olm-catalog]     provider:
[generate-catalog : generate-olm-catalog]       name: Provider Name
[generate-catalog : generate-olm-catalog]       url: https://your.domain
[generate-catalog : generate-olm-catalog] relatedImages:
[generate-catalog : generate-olm-catalog] - image: quay.io/redhat-user-workloads/oeo-cicada-tenant/osd-example-operator/bundle@sha256:570c3e3808236d003d0430f035f7c108200c7adc3927e49919aea830c6ed95d2
[generate-catalog : generate-olm-catalog]   name: ""
[generate-catalog : generate-olm-catalog] - image: quay.io/redhat-user-workloads/oeo-cicada-tenant/osd-example-operator/osd-example-operator-main@sha256:7c1dc71157fe2e8dc7718eb17ee53516400bf4f60c6f2690a71921404a0ac62a
[generate-catalog : generate-olm-catalog]   name: ""
[generate-catalog : generate-olm-catalog] schema: olm.bundle
[generate-catalog : generate-olm-catalog] ---
[generate-catalog : generate-olm-catalog] schema: olm.channel
[generate-catalog : generate-olm-catalog] package: osd-example-operator
[generate-catalog : generate-olm-catalog] name: stable
[generate-catalog : generate-olm-catalog] entries:
[generate-catalog : generate-olm-catalog]   - name: osd-example-operator.v4.16.0-dc7b3d1
[generate-catalog : generate-olm-catalog]     skipRange: ">=0.0.1 <v4.16.0-dc7b3d1"
[generate-catalog : generate-olm-catalog] + opm validate catalog

[build-container : build] Adding the entitlement to the build
[build-container : build] STEP 1/7: FROM registry.redhat.io/openshift4/ose-operator-registry-rhel9:v4.16
[build-container : build] Trying to pull registry.redhat.io/openshift4/ose-operator-registry-rhel9:v4.16...

@arewm arewm requested a review from a team July 17, 2024 23:42
@jbpratt jbpratt force-pushed the SDCICD-1338 branch 3 times, most recently from 425ed90 to e2c8fdb Compare July 22, 2024 23:12
@jbpratt jbpratt requested a review from arewm July 22, 2024 23:12
@tkdchen
Copy link
Contributor

tkdchen commented Jul 24, 2024

/ok-to-test

@jbpratt
Copy link
Contributor Author

jbpratt commented Jul 26, 2024

/retest

Copy link
Contributor

@chmeliik chmeliik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't have much experience with operators, but the task does look reasonable.

I'm just wondering - the combo of this task and #1153 generates a lot of additional source files. Should we try to include them in source containers (https://github.com/konflux-ci/build-definitions/tree/main/task/source-build/0.1)?

On the other hand, a source repo + a pipeline that uses those two tasks should generate the same files every time (hopefully?) so I'd say we can treat that as intermediate build-system artifacts and don't need to distribute the sources

Copy link
Member

@arewm arewm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I thought I had reviewed this a while ago, but everything is pending. Apologies!

@jbpratt jbpratt requested a review from arewm July 26, 2024 16:28
@jbpratt jbpratt force-pushed the SDCICD-1338 branch 2 times, most recently from 105ee52 to 0072218 Compare July 29, 2024 09:27
Add a new custom task to generate the OLM catalog based on a bundle
image. This creates the Dockerfile and generates the catalog contents by
adding a single bundle and a single channel entry for that bundle to the
catalog with a skip range to the latest version. OLM can then be treated
in such a way that it will always upgrade to the latest version
available in the catalog.

following the documentation[1] for creating raw FBCs

[1]: https://olm.operatorframework.io/docs/tasks/creating-a-catalog/#catalog-creation-with-raw-file-based-catalogs

PR konflux-ci#1153 will be used to generate the bundle that is fed into this task
as a parameter. SRE-P operators are maintained without actively
versioning or managing upgrade graphs; the latest version is always the
version that should be running.

Signed-off-by: Brady Pratt <[email protected]>
Copy link
Contributor

@chmeliik chmeliik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbpratt
Copy link
Contributor Author

jbpratt commented Jul 31, 2024

@chmeliik resolved! This should be ready to go 🚀

@chmeliik chmeliik added this pull request to the merge queue Jul 31, 2024
Merged via the queue into konflux-ci:main with commit c22e710 Jul 31, 2024
9 checks passed
@jbpratt jbpratt deleted the SDCICD-1338 branch July 31, 2024 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants