The update from v1beta1 to v1 broke our old deployer images

[maelvls]

I suspect that the CRD upgrade from v1beta1 to v1 of app-crd.yaml in #562 broke our old deployer images that had an misplaced field info (it was under descriptor instead of spec).

This mistake wasn't picked up by the validation, and we did not discover that it wasn't valid until a few days ago.

apiVersion: app.k8s.io/v1beta1
kind: Application
spec:
  descriptor:
    ...
    info: []                          # Field not in the right spot.

The valid Application should have been:

  apiVersion: app.k8s.io/v1beta1
  kind: Application
  spec:
    descriptor:
      ...
    info: []

The error looks like this:

error: error validating "/data/resources.yaml": error validating data:
ValidationError(Application.spec.descriptor): unknown field "info" in
io.k8s.app.v1beta1.Application.spec.descriptor; if you choose to ignore
these errors, turn validation off with --validate=false

My hypothesis is that when the CRD was moved to v1 by default, the conversion webhook that converts old v1beta1 CustomResourceDefinitions to v1 started complaining about this misplaced info field.

Reproduce the issue

I was able to reproduce this bug with Kubernetes 1.19, but it should also appear with other Kubernetes versions. First, create a Kubernetes cluster on GKE

gcloud container clusters create test --zone=europe-west2-b --num-nodes=2

Now, let's use the old v1beta1 and deploy an application that has a misplaced info field. You can check that the field is misplaced using the command:

docker run --rm --entrypoint=cat gcr.io/jetstack-public/jetstack-secure-for-cert-manager/deployer:1.3.1-gcm.0 /data/chart/chart.tar.gz | tar xz -C /tmp/ chart/templates/application.yaml
cat /tmp/chart/templates/application.yaml

Now, let's install the old version v1beta1 of the CRD, and install the application:

kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/marketplace-k8s-app-tools/b724392636c4ce5c2e14be20a516b647a31cac42/crd/app-crd.yaml
kubectl create ns test
mpdev install --deployer=marketplace.gcr.io/jetstack-public/jetstack-secure-for-cert-manager/deployer:1.3.1-gcm.0 \
    --parameters='{"name": "test", "namespace": "test"}'

The application is installed with no problem.

Now, let us install the new v1 CRD instead, and deploy the application again:

kubectl delete ns test
kubectl delete crds applications.app.k8s.io
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/marketplace-k8s-app-tools/1919eebe79ea1465c72377c3f50808cc276b812f/crd/app-crd.yaml
kubectl create ns test
mpdev install --deployer=marketplace.gcr.io/jetstack-public/jetstack-secure-for-cert-manager/deployer:1.3.1-gcm.0 \
    --parameters='{"name": "test", "namespace": "test"}'

This time, the following error appears:

error: error validating "/data/resources.yaml": error validating data:
[ValidationError(Application.spec): unknown field "addOwnerRef" in
io.k8s.app.v1beta1.Application.spec,
ValidationError(Application.spec.descriptor): unknown field "info" in
io.k8s.app.v1beta1.Application.spec.descriptor]; if you choose to ignore these
errors, turn validation off with --validate=false

Questions:

1. Is there a way to subscribe to some changelog for the changes made to the CRD, mpdev and testrunner? I would expect this changelog to contain a "breaking change" section.
2. What should I do with the "broken" deployers for old versions? Should I re-issue a new deployer with the fix?

cc @gibbleyg