-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This change augments the e2e test suite to simulate the operator's deployment using OLM. The setup consists of the following components: - A docker/distribution container registry, running in docker outside of any Kubernetes cluster. - A KinD cluster that is configured to resolve image refs which use "localhost" as the image registry domain. - Installing OLM on the KinD cluster. Once set up, the operator and its associated OLM bundle are built and pushed to the local container registry. Next, an OLM catalog is built based on the catalog published in operatorhub.io. The catalog is what allows OLM to find the Tekton operator that Shipwright depends on, and is likewise pushed to the local container registry. Building the operator, bundle, and catalog with a fully on-cluster registry is problematic for several reasons: - Not all tools can push to the on-cluster registry in this fashion - Mainfests need to be rewritten to reference the on-cluster DNS name for the registry - The catalog source needs to be pullable within the cluster. The test runs as follows: - Create a namespace to run the operator under test - Create a CatalogSource using the catalog containing the operator under test. - Create an OperatorGroup which allows AllNamespace operators to be installed in the given namespace. - Create a Subscription to install the Shipwright operator and its associated Tekton operator. - Verify that the shipwright operator deploys successfully. Contributor documentation has also been updated so that developers can run this process using make commands on their Kubernetes cluster of choice. See also: - https://kind.sigs.k8s.io/docs/user/local-registry/ - https://olm.operatorframework.io/docs/tasks/creating-a-catalog/ - https://olm.operatorframework.io/docs/tasks/make-catalog-available-on-cluster/ - https://olm.operatorframework.io/docs/tasks/install-operator-with-olm/ - https://olm.operatorframework.io/docs/advanced-tasks/operator-scoping-with-operatorgroups/
- Loading branch information
1 parent
c3bbdd9
commit 0285982
Showing
15 changed files
with
391 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -50,7 +50,7 @@ jobs: | |
| go-version: [1.15.x] | ||
| os: [ubuntu-latest] | ||
| kubernetes: | ||
| - v1.19.7 | ||
| - v1.20.7 | ||
| max-parallel: 2 | ||
| runs-on: ${{ matrix.os }} | ||
| steps: | ||
|
|
@@ -60,5 +60,35 @@ jobs: | |
| go-version: ${{ matrix.go-version }} | ||
| - name: Check out code | ||
| uses: actions/checkout@v2 | ||
| - name: Build Image | ||
| run: make ko-publish IMAGE_REPO=ko.local | ||
| - name: Install kubectl | ||
| uses: azure/setup-kubectl@v1 | ||
| with: | ||
| version: ${{ matrix.kubernetes }} | ||
| - name: Deploy KinD Local Container Registry | ||
| run: make deploy-kind-registry | ||
| - name: Create KinD cluster | ||
| uses: helm/[email protected] | ||
| with: | ||
| version: v0.11.1 | ||
| node_image: kindest/node:${{ matrix.kubernetes }} | ||
| cluster_name: kind | ||
| config: test/kind/config.yaml | ||
| wait: 120s | ||
| - name: Verify KinD cluster | ||
| run: make verify-kind | ||
| - name: Install KinD post-actions | ||
| run: make deploy-kind-registry-post | ||
| - name: Install OLM | ||
| run: make install-olm | ||
| # Builds the operator and makes the image readable in the KinD cluster | ||
| - name: Build Operator Image | ||
| run: | | ||
| make ko-publish IMAGE_REPO=localhost:5000 | ||
| - name: Build Operator Bundle | ||
| run: | | ||
| make bundle-push IMAGE_REPO=localhost:5000 | ||
| - name: Build Catalog Source | ||
| run: | | ||
| make catalog-push IMAGE_REPO=localhost:5000 | ||
| - name: Run Operator with Catalog | ||
| run: make catalog-run IMAGE_REPO=localhost:5000 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| labels: | ||
| app: shipwright-operator | ||
| name: system | ||
| --- | ||
| apiVersion: operators.coreos.com/v1alpha1 | ||
| kind: CatalogSource | ||
| metadata: | ||
| name: operator | ||
| namespace: system | ||
| spec: | ||
| sourceType: grpc | ||
| image: catalog-source:latest | ||
| displayName: Shipwright Operator Catalog | ||
| publisher: The Shipwright Contributors | ||
| updateStrategy: | ||
| registryPoll: | ||
| interval: 10m |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| namespace: shipwright-operator | ||
| namePrefix: shipwright- | ||
|
|
||
| resources: | ||
| - catalog_source.yaml | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| namespace: shipwright-operator | ||
| namePrefix: shipwright- | ||
|
|
||
| resources: | ||
| - subscription.yaml | ||
| - operator_group.yaml | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| apiVersion: operators.coreos.com/v1alpha2 | ||
| kind: OperatorGroup | ||
| metadata: | ||
| name: operator | ||
| namespace: system | ||
| spec: {} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| apiVersion: operators.coreos.com/v1alpha1 | ||
| kind: Subscription | ||
| metadata: | ||
| name: operator | ||
| namespace: system | ||
| spec: | ||
| channel: alpha | ||
| name: shipwright-operator | ||
| source: shipwright-operator | ||
| sourceNamespace: shipwright-operator | ||
| installPlanApproval: Automatic | ||
| startingCSV: shipwright-operator.v0.0.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # OLM Development | ||
|
|
||
| The Shipwright operator is meant to be deployed on a cluster using the | ||
| [Operator Lifecycle Manager](https://olm.operatorframework.io/) (OLM). | ||
| OLM provides mechanisms to support over the air upgrades and automatically deploy related operators | ||
| that are packaged in operator [catalogs](https://olm.operatorframework.io/). | ||
| Additional steps need to be taken to ensure the operator can be deployed with OLM. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| * Ensure you have access to a Kubernetes cluster via `kubectl` with cluster admin permissions. | ||
| * Install Go version 1.15 or higher. | ||
| * Install OLM on your cluster. This can be done using the `make install-olm` command. | ||
| * Ability to push to a container registry that is accessible inside your Kubernetes cluster. | ||
|
|
||
| ## Step 1: Push the operator image to a registry | ||
|
|
||
| Run `make ko-publish IMAGE_REPO=<your-registry>`, pushing to a container registry that is accessible inside your Kubernetes cluster. | ||
| Using `ko.local` or `kind.local` is not recommended, as this will not push the resulting image to a container registry. | ||
|
|
||
| If you are using [KinD](https://kind.sigs.k8s.io/), follow the instructions on how to configure a | ||
| [local registry](https://kind.sigs.k8s.io/docs/user/local-registry/). | ||
| This will let you use `localhost:<port>` as your container registry. | ||
|
|
||
| ## Step 2: Build and push the operator bundle | ||
|
|
||
| Next, run `make bundle-push IMAGE_REPO=<your-registry>`. | ||
| This will push the [operator bundle](https://olm.operatorframework.io/docs/tasks/creating-operator-bundle/) | ||
| to the container registry. | ||
| An operator bundle is an OCI aritfact that tells OLM how to deploy your operator. | ||
| Be sure to use the same container registry for the `IMAGE_REPO` argument. | ||
|
|
||
| ## Step 3: Build and push an operator catalog | ||
|
|
||
| Next, run `make catalog-push IMAGE_REPO=<your-registry>`. | ||
| This will build and push an [operator catalog](https://olm.operatorframework.io/docs/tasks/creating-a-catalog/), | ||
| which packages your test operator bundle with the other operators available on [operatorhub.io](https://operatorhub.io). | ||
| As in step 2, be sure to use the same container registry for the `IMAGE_REPO` argument. | ||
|
|
||
| ## Step 4: Deploy the operator using the catalog image | ||
|
|
||
| Finally, deploy the operator using `make catalog-run IMAGE_REPO=<your-registry>`, using the same | ||
| value for `IMAGE_REPO` as in the previous steps. | ||
| This will run a script that does the following: | ||
|
|
||
| 1. Creates a custom [CatalogSource](https://olm.operatorframework.io/docs/tasks/make-catalog-available-on-cluster/) | ||
| and [OperatorGroup](https://olm.operatorframework.io/docs/advanced-tasks/operator-scoping-with-operatorgroups/), | ||
| which allows the operators in step 3's catalog to be installed anywhere on the clsuter. | ||
| 2. Creates a [Subscription](https://olm.operatorframework.io/docs/tasks/install-operator-with-olm/), | ||
| which instructs OLM to install the operator and any dependent operators. | ||
| 3. Checks that the operator has successfully been installed and rolled out. | ||
|
|
||
| Once the script completes, the Shipwright and Tekton operators will be installed on the cluster. | ||
|
|
||
| _Note:_ | ||
|
|
||
| Scripts in `hack` folder may require `sed` (GNU), therefore in platforms other than Linux you may have it with a different name. For instance, on macOS it's usually named `gsed`, in this case provide the `SED_BIN` make variable with the alternative name. | ||
|
|
||
| ```bash | ||
| $ make catalog-run SED_BIN=gsed ... | ||
| ``` |
Oops, something went wrong.