From 190596209fe583ce77391f1888d08df4f7f2b148 Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Thu, 13 Jun 2024 15:52:00 -0400 Subject: [PATCH 1/6] Move steps for migrating code to csi-operator to ocp enhancementsn --- .../storage/csi-driver-operator-merge.md | 64 ++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-) diff --git a/enhancements/storage/csi-driver-operator-merge.md b/enhancements/storage/csi-driver-operator-merge.md index e607cda448..bfd1f69ec6 100644 --- a/enhancements/storage/csi-driver-operator-merge.md +++ b/enhancements/storage/csi-driver-operator-merge.md @@ -469,6 +469,68 @@ N/A Same as today. +## Process of moving operators to csi-operator monorepo + +We have come with following flow for moving operators from their own repository into csi-operator mono repo. + +### Avoid .gitignore related footguns + +Remove any .gitignore entries in the source repository that would match a directory / file that we need. For example, `azure-disk-csi-driver-operator` in .gitignore matched `cmd/azure-disk-csi-driver-operator` directory that we really need not to be ignored. See https://github.com/openshift/csi-operator/pull/110, where we had to fix after merge to csi-operator. + +### Move existing code into csi-operator repository + +Using git-subtree move your existing operator code to https://github.com/openshift/csi-operator/tree/master/legacy + +``` +git subtree add --prefix legacy/azure-disk-csi-driver-operator https://github.com/openshift/azure-disk-csi-driver-operator.git master --squash +git subtree push --prefix legacy/azure-disk-csi-driver-operator https://github.com/openshift/azure-disk-csi-driver-operator.git master +``` + +### Add Dockerfiles for building images from new location + +Place a `Dockerfile.` and `Dockerfile..test` at top of csi-operator tree and make sure that you are able to build an image of the operator from csi-operator repository. + +### Update openshift/release to build image from new location +Make a PR to openshift/release repository to build the operator from csi-operator. For example - https://github.com/openshift/release/pull/46233. + +1. Update also `storage-conf-csi--commands.sh`, the test manifest will be at a different location. +2. Make sure that rehearse jobs for both older versions of operator and newer versions of operator pass. + +### Change ocp-build-data repository to ship image from new location + +Make a PR to [ocp-build-data](https://github.com/openshift-eng/ocp-build-data) repository to change location of the image etc - https://github.com/openshift-eng/ocp-build-data/pull/4148 + +1. Notice the `cachito` line in the PR - we need to build with the vendor from legacy/ directory. + +2. Ask ART for a scratch build. Make sure you can install a cluster with that build. + +``` +oc adm release new \ +--from-release=registry.ci.openshift.org/ocp/release:4.15.0-0.nightly.XYZ \ +azure-disk-csi-driver-operator= \ +--to-image=quay.io/jsafrane/scratch:release1 \ +--name=4.15.0-0.nightly.jsafrane.1 + +oc adm release extract --command openshift-install quay.io/jsafrane/scratch:release1 +``` + +### Co-ordinating merges in ocp-build-data and release repository + +Both PRs in openshift/release and ocp-build-data must be merged +/- at the same time. There is a robot that syncs some data from ocp-build-data to openshift/release and actually breaks things when these two repos use different source repository to build images. + +### Enjoy the build from csi-operator repository + +After aforementioned changes, your new operator should be able to be built from csi-operator repo and everything should work. + +### Moving operator to new structure in csi-operator + +So in previous section we merely copied existing code from operator’s own repository into `csi-operator` repository. We did not change anything. + +But once your operator has been changed to conform to new code in csi-operator repo, You need to perform following additional steps: + +1. Make sure that `Dockerfile.` at top of the `csi-operator` tree refers to new location of code and not older `legacy/` location.See example of existing Dockerfiles. +2. After your changes to `csi-operator` are merged, you should remove the old location from cachito - https://github.com/openshift-eng/ocp-build-data/pull/4219 + ## Implementation History Major milestones in the life cycle of a proposal should be tracked in `Implementation @@ -548,4 +610,4 @@ Advantages: ## Infrastructure Needed [optional] -N/A (other than the usual CI + QE) \ No newline at end of file +N/A (other than the usual CI + QE) From d0883f9c7a30ebecd6c73b9ed5a194a58e6b0ae2 Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Mon, 1 Jul 2024 11:59:57 -0400 Subject: [PATCH 2/6] Add a note about ocp-build-data changes to be merged same time as csi-operato --- enhancements/storage/csi-driver-operator-merge.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/enhancements/storage/csi-driver-operator-merge.md b/enhancements/storage/csi-driver-operator-merge.md index bfd1f69ec6..fb4316e309 100644 --- a/enhancements/storage/csi-driver-operator-merge.md +++ b/enhancements/storage/csi-driver-operator-merge.md @@ -531,6 +531,8 @@ But once your operator has been changed to conform to new code in csi-operator r 1. Make sure that `Dockerfile.` at top of the `csi-operator` tree refers to new location of code and not older `legacy/` location.See example of existing Dockerfiles. 2. After your changes to `csi-operator` are merged, you should remove the old location from cachito - https://github.com/openshift-eng/ocp-build-data/pull/4219 +Please note the changes to `ocp-build-data`should be merged almost same time as changes into `csi-operator`'s Dockerfile are merged, otherwise we risk builds from breaking. + ## Implementation History Major milestones in the life cycle of a proposal should be tracked in `Implementation From e7d5e020cf8d2ccd9cdf7d7884efdc06229270c5 Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Mon, 1 Jul 2024 12:21:46 -0400 Subject: [PATCH 3/6] Add steps for post merge cleanup --- enhancements/storage/csi-driver-operator-merge.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/enhancements/storage/csi-driver-operator-merge.md b/enhancements/storage/csi-driver-operator-merge.md index fb4316e309..bd4b24ad59 100644 --- a/enhancements/storage/csi-driver-operator-merge.md +++ b/enhancements/storage/csi-driver-operator-merge.md @@ -533,6 +533,16 @@ But once your operator has been changed to conform to new code in csi-operator r Please note the changes to `ocp-build-data`should be merged almost same time as changes into `csi-operator`'s Dockerfile are merged, otherwise we risk builds from breaking. +### Post migration changes + +Once migration is complete, we should perform following post migration steps to ensure that we are not left over with legacy stuff: + +1. Mark existing `openshift/--operator` repository as deprecated. +2. Ensure that we have test manifest available in `test/e2e2` directory. +3. Make changes into `release` repository so as it longer relies on anything from `legacy` directory. See - https://github.com/openshift/release/pull/49655 for example. +4. Remove code from `vendor/legacy` in `csi-operator` repository. + + ## Implementation History Major milestones in the life cycle of a proposal should be tracked in `Implementation From 4e0d5bd9993a6fad403eb4c8e41a5a7caa62a7a4 Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Tue, 2 Jul 2024 15:02:35 -0400 Subject: [PATCH 4/6] Update enhancements/storage/csi-driver-operator-merge.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Roman Bednář --- enhancements/storage/csi-driver-operator-merge.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/enhancements/storage/csi-driver-operator-merge.md b/enhancements/storage/csi-driver-operator-merge.md index bd4b24ad59..af1d470807 100644 --- a/enhancements/storage/csi-driver-operator-merge.md +++ b/enhancements/storage/csi-driver-operator-merge.md @@ -538,7 +538,7 @@ Please note the changes to `ocp-build-data`should be merged almost same time as Once migration is complete, we should perform following post migration steps to ensure that we are not left over with legacy stuff: 1. Mark existing `openshift/--operator` repository as deprecated. -2. Ensure that we have test manifest available in `test/e2e2` directory. +2. Ensure that we have test manifest available in `test/e2e` directory. 3. Make changes into `release` repository so as it longer relies on anything from `legacy` directory. See - https://github.com/openshift/release/pull/49655 for example. 4. Remove code from `vendor/legacy` in `csi-operator` repository. From 75ef5474b783085718f1a02b9c75e791cb2d07cc Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Tue, 2 Jul 2024 15:42:11 -0400 Subject: [PATCH 5/6] Update enhancements/storage/csi-driver-operator-merge.md Co-authored-by: Jonathan Dobson --- enhancements/storage/csi-driver-operator-merge.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/enhancements/storage/csi-driver-operator-merge.md b/enhancements/storage/csi-driver-operator-merge.md index af1d470807..3ec64e04c4 100644 --- a/enhancements/storage/csi-driver-operator-merge.md +++ b/enhancements/storage/csi-driver-operator-merge.md @@ -528,7 +528,7 @@ So in previous section we merely copied existing code from operator’s own repo But once your operator has been changed to conform to new code in csi-operator repo, You need to perform following additional steps: -1. Make sure that `Dockerfile.` at top of the `csi-operator` tree refers to new location of code and not older `legacy/` location.See example of existing Dockerfiles. +1. Make sure that `Dockerfile.` at top of the `csi-operator` tree refers to new location of code and not older `legacy/` location. See example of existing Dockerfiles. 2. After your changes to `csi-operator` are merged, you should remove the old location from cachito - https://github.com/openshift-eng/ocp-build-data/pull/4219 Please note the changes to `ocp-build-data`should be merged almost same time as changes into `csi-operator`'s Dockerfile are merged, otherwise we risk builds from breaking. From 9660e8c029786f59ce54b56f5e2dbfb133f558ad Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Tue, 2 Jul 2024 15:42:51 -0400 Subject: [PATCH 6/6] Add additional docs for OLM based operators --- enhancements/storage/csi-driver-operator-merge.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/enhancements/storage/csi-driver-operator-merge.md b/enhancements/storage/csi-driver-operator-merge.md index 3ec64e04c4..038a5848af 100644 --- a/enhancements/storage/csi-driver-operator-merge.md +++ b/enhancements/storage/csi-driver-operator-merge.md @@ -488,7 +488,7 @@ git subtree push --prefix legacy/azure-disk-csi-driver-operator https://github.c ### Add Dockerfiles for building images from new location -Place a `Dockerfile.` and `Dockerfile..test` at top of csi-operator tree and make sure that you are able to build an image of the operator from csi-operator repository. +Place a `Dockerfile.` at top of csi-operator tree and make sure that you are able to build an image of the operator from csi-operator repository. ### Update openshift/release to build image from new location Make a PR to openshift/release repository to build the operator from csi-operator. For example - https://github.com/openshift/release/pull/46233. @@ -514,6 +514,8 @@ azure-disk-csi-driver-operator= \ oc adm release extract --command openshift-install quay.io/jsafrane/scratch:release1 ``` +This step is only applicable for CVO based operators and not OLM based operators. For OLM based operator - either an image can be built locally and deployed using your personal index image or you can ask ART team for a scratch image when you open `ocp-build-data` PR and proceed to include that image in your personal index image. + ### Co-ordinating merges in ocp-build-data and release repository Both PRs in openshift/release and ocp-build-data must be merged +/- at the same time. There is a robot that syncs some data from ocp-build-data to openshift/release and actually breaks things when these two repos use different source repository to build images.