Argo-CD and CRD management

[object88]

I am confused about some of the text in the README for Argo-CD, around the management of Argo-CD's custom resource definitions.

Here there is text instructing the user to upgrade the CRDs using kubectl.

But the notes for 5.2.0 say that the CRDs are now (by default) managed by the Helm chart.

I would guess that going forward after 5.2.0 (unless crds.install is false?), the helm chart would manage the CRDs. But if that's the case, then I am not sure why the comment about helm being able to upgrade CRDs. Unless that is referring to CRDs that live in <chart>/crds, and not in <charts>/templates as the Argo-CD ones now do.

I am sure that a good part of this question is just my not having a 100% clear picture of what Helm is up to. Thanks for any elaboration.

[pdrastil]

Hi,
since version 5.2.0 CRDs should be managed by Helm as a template and user should do the adoption of current resources before upgrading. Versions prior chart v5.2.0 required kubectl apply approach. We should update documentation in this regard.

[lucasrod16]

I have been working on migrating the CustomResourceDefinitions from chart/crds to templates/crds, and have opted to utilize a helm hook job that executes the kubectl commands as noted here to adopt existing CRDs.

This note in this section of the chart README stood out to me:

Helm cannot upgrade custom resource definitions in the /crds folder by design. Starting with 5.2.0, the CRDs have been moved to /templates to address this design decision.

There's a thorough breakdown of this problem found here. From my understanding, it seems that Helm cannot upgrade CRDs even if managed as a Helm resource in the templates/ directory, for various reasons stated in the document linked.

I am curious if there is any more information and/or context on the decision to move them to be managed as Helm resources.

Thanks in advance

[object88]

Great; thank you. I'll try to remember to submit a doc PR later this week with the clarification.

[ismailyenigul]

Hi,
I applied
kubectl apply -k "https://github.com/argoproj/argo-cd/manifests/crds?ref=v2.6.7"
before upgrading helm chart from "4.10.0" -> "5.27.1"
but I still got

helm_release.argo: Modifying... [id=argocd]
helm_release.argo: Still modifying... [id=argocd, 10s elapsed]
helm_release.argo: Still modifying... [id=argocd, 20s elapsed]
helm_release.argo: Still modifying... [id=argocd, 30s elapsed]
helm_release.argo: Still modifying... [id=argocd, 40s elapsed]
╷
│ Error: rendered manifests contain a resource that already exists. Unable to continue with update: CustomResourceDefinition 
"applications.argoproj.io" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata;
 label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing 
key "meta.helm.sh/release-name": must be set to "argocd"; annotation validation error: missing key "meta.helm.sh/release-
namespace": must be set to "argocd"

If I delete CRDS to let helm chart create it, all running apps will be deleted then all running pods will be destroyed.
Is there anyway to manage this upgrade without any downtime?
Do I have to add helm labels to all crds too manually?

Thanks

[ismailyenigul]

I found solution by adding above annotations to crds file by patching in kustomization.yaml
im my case, argocd is deployed on argocd namespace and release-name argocd

helm list -A                                                                        1674ms
NAME                      	NAMESPACE  	REVISION	UPDATED                             	STATUS  	CHART                             	APP VERSION
argocd                    	argocd     	3       	2023-03-30 11:17:01.167384 +0300 +03	deployed	argo-cd-5.27.1 

here is updated version of kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - application-crd.yaml
  - appproject-crd.yaml
  - applicationset-crd.yaml

patches:
  - target:
      kind: CustomResourceDefinition
    patch: |
      apiVersion: apiextensions.k8s.io/v1
      kind: CustomResourceDefinition
      metadata:
        name: will-be-ignored
        annotations:
          meta.helm.sh/release-name: argocd  <--change-this value with your release name
          meta.helm.sh/release-namespace: argocd <--change-this value with your namespace
        labels: 
          app.kubernetes.io/managed-by: Helm

Steps (to upgrade CRDS from 2.4 to 2.6 in my case)

$ git clone https://github.com/argoproj/argo-cd
$ git checkout v2.6.7
$ cd manifests/crds/
$ vi kustomization.yaml # update as I described above
$ kubectl apply -k "." 

[ivneetsinghwahi03]

Thanks @ismailyenigul