deploy: c8bf40b

blog/2021-12-31-medical-diagnosis/index.html

@@ -3631,11 +3631,11 @@
 Sync-Waves and ArgoCD/OpenShift GitOps Resource Hooks The way that resource hooks are designed to work is by giving ordering hints, so that ArgoCD knows what order to apply resources in. The mechanism is described in the ArgoCD upstream docs here. When sync-waves are in use, all resouces in the same sync-wave have to be "healthy" before resources in the numerically next sync-wave are synced. This mechanism gives us a way of having ArgocD help us enforce order with objects that it manages.
 Solution 1: Sync-Waves for Subscriptions in clusterGroup The Validated Patterns framework now allows Kubernetes annotations to be added directly to subscription objects in the clusterGroup. ArgoCD uses annotations for Resource Hooks. The clustergoup chart now passes any annotations attached to subscriptions through to the subscription object(s) that the clustergroup chart creates. For example:
 openshift-virtualization: name: kubevirt-hyperconverged namespace: openshift-cnv channel: stable annotations: argocd.argoproj.io/sync-wave: "10" openshift-data-foundation: name: odf-operator namespace: openshift-storage annotations: argocd.argoproj.io/sync-wave: "5" will result in a subscription object that includes the annotations:
-apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: annotations: argocd.argoproj.io/sync-wave: "10" kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"operators.coreos.com/v1alpha1","kind":"Subscription","metadata":{"annotations":{"argocd.argoproj.io/sync-wave":"10"},"labels":{"app.kubernetes.io/instance":"ansible-edge-gitops-hub"},"name":"kubevirt-hyperconverged","namespace":"openshift-cnv"},"spec":{"channel":"stable","installPlanApproval":"Automatic","name":"kubevirt-hyperconverged","source":"redhat-operators","sourceNamespace":"openshift-marketplace"}} creationTimestamp: "2024-11-07T14:24:31Z" generation: 1 labels: app.kubernetes.io/instance: ansible-edge-gitops-hub operators.coreos.com/kubevirt-hyperconverged.openshift-cnv: "" name: kubevirt-hyperconverged namespace: openshift-cnv resourceVersion: "46763" uid: e9b3892c-9383-41ca-9e8f-ae7be82f012f spec: channel: stable installPlanApproval: Automatic name: kubevirt-hyperconverged source: redhat-operators sourceNamespace: openshift-marketplace apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: annotations: argocd.argoproj.io/sync-wave: "5" kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"operators.coreos.com/v1alpha1","kind":"Subscription","metadata":{"annotations":{"argocd.argoproj.io/sync-wave":"5"},"labels":{"app.kubernetes.io/instance":"ansible-edge-gitops-hub"},"name":"odf-operator","namespace":"openshift-storage"},"spec":{"installPlanApproval":"Automatic","name":"odf-operator","source":"redhat-operators","sourceNamespace":"openshift-marketplace"}} creationTimestamp: "2024-11-07T14:21:12Z" generation: 1 labels: app.kubernetes.io/instance: ansible-edge-gitops-hub operators.coreos.com/odf-operator.openshift-storage: "" name: odf-operator namespace: openshift-storage resourceVersion: "56652" uid: 2d9f026f-50e6-4fc1-ad11-8a6a2a636017 spec: installPlanApproval: Automatic name: odf-operator source: redhat-operators sourceNamespace: openshift-marketplace With this configuration, any objects created with sync-waves lower than "10" must be healthy before the objects in sync-wave "10" sync. In particular, the odf-operator subscription must be healthy before the kubevirt-hyperconverged subscription will sync. Similarly, if we defined objects with higher sync-waves than "10", all the resources with sync-waves higher than "10" will wait until the resources in "10" are healthy. If the subscriptions in question wait until their components are healthy before reporting they are healthy themselves, this might be all you need to do. In the case of this particular issue, it was not enough. But because all sequencing in ArgoCD requires the use of sync-wave annotations, adding the annotation to the subscription object will be necessary for using the other solutions.
+apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: annotations: argocd.argoproj.io/sync-wave: "10" labels: app.kubernetes.io/instance: ansible-edge-gitops-hub operators.coreos.com/kubevirt-hyperconverged.openshift-cnv: "" name: kubevirt-hyperconverged namespace: openshift-cnv spec: channel: stable installPlanApproval: Automatic name: kubevirt-hyperconverged source: redhat-operators sourceNamespace: openshift-marketplace apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: annotations: argocd.argoproj.io/sync-wave: "5" labels: app.kubernetes.io/instance: ansible-edge-gitops-hub operators.coreos.com/odf-operator.openshift-storage: "" name: odf-operator namespace: openshift-storage spec: installPlanApproval: Automatic name: odf-operator source: redhat-operators sourceNamespace: openshift-marketplace With this configuration, any objects created with sync-waves lower than "10" must be healthy before the objects in sync-wave "10" sync. In particular, the odf-operator subscription must be healthy before the kubevirt-hyperconverged subscription will sync. Similarly, if we defined objects with higher sync-waves than "10", all the resources with sync-waves higher than "10" will wait until the resources in "10" are healthy. If the subscriptions in question wait until their components are healthy before reporting they are healthy themselves, this might be all you need to do. In the case of this particular issue, it was not enough. But because all sequencing in ArgoCD requires the use of sync-wave annotations, adding the annotation to the subscription object will be necessary for using the other solutions.
 Solution 2: The sequenceJob attribute for Subscriptions in clusterGroup In this situation, we have a subscription that installs an operator, but it is not enough for just the subscriptions to be in sync-waves. This is because the subscriptions install operators, and it is the action of the operators themselves that we have to sequence. In many of these kinds of situations, we can sequence the action by looking for the existence of a single resource. The new sequenceJob construct in subscriptions allows for this kind of relationship by creating a Job at the same sync-wave precedence as the subscription, and looking for the existence of a single arbitrary resource in an arbitrary namespace. The Job then waits for that resource to appear, and when it does, it will be seen as "healthy" and will allow future sync-waves to proceed.
 In this example, the ODF operator needs to have created a storageclass so that the OCP-Virt operators can use it as virtualization storage. If it does not find the kind of storage it wants, it will use the default storageclass instead, which may lead to inconsistencies in behavior. We can have the Validated Patterns framework create a mostly boilerplate job to look for the needed resource this way:
 openshift-virtualization: name: kubevirt-hyperconverged namespace: openshift-cnv channel: stable annotations: argocd.argoproj.io/sync-wave: "10" openshift-data-foundation: name: odf-operator namespace: openshift-storage sequenceJob: resourceType: sc resourceName: ocs-storagecluster-ceph-rbd annotations: argocd.argoproj.io/sync-wave: "5" Note the addition of the sequenceJob section in the odf-operator subscription block. This structure will result in the following Job being created alongside the subscriptions:
-apiVersion: batch/v1 kind: Job metadata: annotations: argocd.argoproj.io/hook: Sync argocd.argoproj.io/sync-wave: "5" kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{"argocd.argoproj.io/hook":"Sync","argocd.argoproj.io/sync-wave":"5"},"labels":{"app.kubernetes.io/instance":"ansible-edge-gitops-hub"},"name":"odf-operator-sequencejob","namespace":"openshift-operators"},"spec":{"completions":1,"parallelism":1,"template":{"spec":{"containers":[{"command":["/bin/bash","-c","while [ 1 ];\\ndo\\n oc get sc ocs-storagecluster-ceph-rbd \\u0026\\u0026 break\\n echo \\"sc ocs-storagecluster-ceph-rbd not found, waiting...\\"\\n sleep 5\\ndone\\necho \\"sc ocs-storagecluster-ceph-rbd found, exiting...\\"\\nexit 0\\n"],"image":"quay.io/hybridcloudpatterns/imperative-container:v1","name":"odf-operator-sequencejob"}],"restartPolicy":"OnFailure"}}}} creationTimestamp: "2024-11-07T16:27:26Z" generation: 1 labels: app.kubernetes.io/instance: ansible-edge-gitops-hub name: odf-operator-sequencejob namespace: openshift-operators resourceVersion: "201283" uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a spec: backoffLimit: 6 completionMode: NonIndexed completions: 1 manualSelector: false parallelism: 1 podReplacementPolicy: TerminatingOrFailed selector: matchLabels: batch.kubernetes.io/controller-uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a suspend: false template: metadata: creationTimestamp: null labels: batch.kubernetes.io/controller-uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a batch.kubernetes.io/job-name: odf-operator-sequencejob controller-uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a job-name: odf-operator-sequencejob spec: containers: - command: - /bin/bash - -c - | while [ 1 ]; do oc get sc ocs-storagecluster-ceph-rbd && break echo "sc ocs-storagecluster-ceph-rbd not found, waiting..." sleep 5 done echo "sc ocs-storagecluster-ceph-rbd found, exiting..." exit 0 image: quay.io/hybridcloudpatterns/imperative-container:v1 imagePullPolicy: IfNotPresent name: odf-operator-sequencejob resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst restartPolicy: OnFailure schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 Since the job is created in sync-wave "5" (which it inherits from the subscription it is attached to by default, though you can specify a different sync-wave if you prefer), this job must complete before sync-wave "10" starts. So the storageclass ocs-storagecluster-ceph-rbd must exist before OCP-Virt starts deploying, ensuring that it will be able to "see" and use that storageclass as its default virtualization storage class.
+apiVersion: batch/v1 kind: Job metadata: annotations: argocd.argoproj.io/hook: Sync argocd.argoproj.io/sync-wave: "5" labels: app.kubernetes.io/instance: ansible-edge-gitops-hub name: odf-operator-sequencejob namespace: openshift-operators spec: backoffLimit: 6 completionMode: NonIndexed completions: 1 manualSelector: false parallelism: 1 podReplacementPolicy: TerminatingOrFailed selector: matchLabels: batch.kubernetes.io/controller-uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a suspend: false template: metadata: creationTimestamp: null labels: batch.kubernetes.io/controller-uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a batch.kubernetes.io/job-name: odf-operator-sequencejob controller-uid: 3084075d-bc1f-4e23-b44d-a13c5d184a6a job-name: odf-operator-sequencejob spec: containers: - command: - /bin/bash - -c - | while [ 1 ]; do oc get sc ocs-storagecluster-ceph-rbd && break echo "sc ocs-storagecluster-ceph-rbd not found, waiting..." sleep 5 done echo "sc ocs-storagecluster-ceph-rbd found, exiting..." exit 0 image: quay.io/hybridcloudpatterns/imperative-container:v1 imagePullPolicy: IfNotPresent name: odf-operator-sequencejob resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst restartPolicy: OnFailure schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 Since the job is created in sync-wave "5" (which it inherits from the subscription it is attached to by default, though you can specify a different sync-wave if you prefer), this job must complete before sync-wave "10" starts. So the storageclass ocs-storagecluster-ceph-rbd must exist before OCP-Virt starts deploying, ensuring that it will be able to "see" and use that storageclass as its default virtualization storage class.
 Each subscription is permitted one sequenceJob. Each sequenceJob may have the following attributes:
 syncWave: Defaults to the subscription’s syncwave from annotations.
 resourceType: Resource kind for the resource to watch for.