MCO-820: Add boot images update opt-in mechanism

[djoshy]

In addition to the feature gate that will be added by #1646, we'd also like to add a secondary opt-in knob for the user to toggle the boot image update mechanism. We expect that some users would want to keep their boot images static and this will serve as the primary switch when this feature leaves tech preview.

This PR introduces a new struct in types_machineconfiguration.go, ManagedBootImages which encloses an array of MachineManager objects. A MachineManager object contains the resource type of the machine management object that is being opted-in, the API group of that object and a union discriminant object of the type MachineManagerSelector. This object MachineManagerSelector encloses:

• The union discriminator, Mode, can be set to three values : All, Partial and None.
• Partial This is a label selector that will be used by objects to opt-in. When the Mode is set to Partial mode, all machinesets in the selector list would be considered enrolled for updates. For all other values of Mode, this selector does not exist.

Here is a YAML snippet of what this config could look like

managedBootImages:
  machineManagers:
  - resource: machinesets
    apiGroup: cluster.x-k8s.io
    selection:
      mode: Partial
      partial:
        matchLabels: {}
  - resource: machinesets
    apiGroup: machine.openshift.io
    selection:
       mode: All

The above YAML would provide a label selector for enrolling Cluster API MachineSets and all Machine API MachineSets. Only one machineManager per unique pair of resource/APIGroup will be accepted. This is to avoid conflicting instructions provided by multiple machineManagers.

We may also add future configuration points to ManagedBootImages. This PR also adds a new degrade condition to the MachineConfigPool CRD. This will be used by the new subcontroller to signal a failure in updating boot images.

More details about this feature can be found in enhancement #1496. Comments welcome!

[openshift-ci]

Hello @djoshy! Some important instructions when contributing to openshift/api:
API design plays an important part in the user experience of OpenShift and as such API PRs are subject to a high level of scrutiny to ensure they follow our best practices. If you haven't already done so, please review the OpenShift API Conventions and ensure that your proposed changes are compliant. Following these conventions will help expedite the api review process for your PR.

[djoshy]

So with this new approach, we still have have three modes:

• Enabled - this will run the feature on all machinesets
• Disabled - this will turn off the feature completely
• CustomConfig - this will run the feature on machinesets that match with the selector enclosed within the ManagedBootImageConfigCustomization struct

We could then add any future configurations as new fields in the ManagedBootImageConfigCustomization object.
In the event that we do add future configurations, Enabled will have to assume certain defaults for them. I think this is ok, as long as we document these assumptions/defaults so users will know what to expect.

[JoelSpeed]

Thinking about the shape of the union, I'm wondering if CustomConfig is one level too many. What would go alongside a MachineSetSelector? Whatever it is, it would have to work in tandem with the selector. An exclusions list perhaps?

Either way, I think Custom is too broad. We do have the option of adding more enum values to the discriminator later, so perhaps for now having the selector is a better discriminator.

Or perhaps we go for something like:

managedBootImages:
  managedImages: None | Partial | All
  partial:
    machineSetSelector: ...
   controlPlaneMachineSetSelector: ... <- a potential future addition?

If we take that route, I guess then machineSetSelector doesn't need to be required, but we should include a validation that partial contains at least one non-empty property using the MinItems validation

[djoshy]

I've updated the struct to follow the shape mentioned in the comment here: #1672 (comment)
It does clean up a lot of weird naming issues, thanks @JoelSpeed !

[djoshy]

/test e2e-gcp

/test e2e-azure

[JoelSpeed]

Few formatting things and a question on one of the validations to work out

[djoshy]

Updated to:

• render nicely in kubectl explain
• removed the featureset aware validation rule

[JoelSpeed]

Ok, from an API perspective I'm happy with this now, I would like to take a final pass over the enhancement before we merge as well

@yuqi-zhang I believe you've been reviewing the EP, are you happy with the shape of this API?

[djoshy]

/test e2e-upgrade

[yuqi-zhang]

Looked over the managedbootimages commit again, and the current API lgtm!

[djoshy]

Updated based on discussion from the enhancement:

• removed the MachineConfigPool degrade type since we are switching to an alert
• enclosed the MachineSet label selector within a struct so we have flexibility to expand in the future

[openshift-ci-robot]

@djoshy: This pull request references MCO-820 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

In addition to the feature gate that will be added by #1646, we'd also like to add a secondary opt-in knob for the user to toggle the boot image update mechanism. We expect that some users would want to keep their boot images static and this will serve as the primary switch when this feature leaves tech preview.

This PR introduces a new struct in types_machineconfiguration.go, ManagedBootImages which encloses an array of MachineManager objects. A MachineManager object contains the resource type of the machine management object that is being opted-in, the API group of that object and a union discriminant object of the type MachineManagerSelector. This object MachineManagerSelector encloses:

• The union discriminator, Mode, can be set to three values : All, Partial and None.
• Partial This is a label selector that will be used by objects to opt-in. When the Mode is set to Partial mode, all machinesets in the selector list would be considered enrolled for updates. For all other values of Mode, this selector does not exist.

Here is a YAML snippet of what this config could look like

managedBootImages:
 machineManagers:
 - resource: machinesets
   apiGroup: cluster.x-k8s.io
   selection:
     mode: Partial
     partial:
       matchLabels: {}
 - resource: machinesets
   apiGroup: machine.openshift.io
   selection:
      mode: All

The above YAML would provide a label selector for enrolling Cluster API MachineSets and all Machine API MachineSets. Only one machineManager per unique pair of resource/APIGroup will be accepted. This is to avoid conflicting instructions provided by multiple machineManagers.

We may also add future configuration points to ManagedBootImages. This PR also adds a new degrade condition to the MachineConfigPool CRD. This will be used by the new subcontroller to signal a failure in updating boot images.

More details about this feature can be found in enhancement #1496. Comments welcome!

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[JoelSpeed]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djoshy, JoelSpeed, yuqi-zhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JoelSpeed]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[JoelSpeed]

/override ci/prow/verify-crd-schema

Introduction fo techpreview/custom versions of CRDs has caused some pre-existing failures that cannot be fixed

[openshift-ci]

@JoelSpeed: Overrode contexts on behalf of JoelSpeed: ci/prow/verify-crd-schema

In response to this:

/override ci/prow/verify-crd-schema

Introduction fo techpreview/custom versions of CRDs has caused some pre-existing failures that cannot be fixed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[djoshy]

/retest-required

[openshift-ci]

@djoshy: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-bot]

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-cluster-config-api-container-v4.16.0-202402281710.p0.g217662e.assembly.stream.el9 for distgit ose-cluster-config-api.
All builds following this will include this PR.