How do I add more exempted namespaces for pod security

[stevefan1999-personal]

I have this patch.yaml:

cluster:
  apiServer:
    admissionControl:
    - name: PodSecurity
      configuration:
        exemptions:
          namespaces:
          - openelb-system

Then I ran talosctl patch machineconfig --patch "@patch.yaml" --talosconfig=talosconfig, then the cluster stopped working. Why? Am I doing this wrong?

[smira]

You need to add some details on what went wrong exactly.

[stevefan1999-personal]

Uh the apiserver, controller-manager and scheduler just stopped working, and I couldn't find more diagnostic log that are useful from the console

[stevefan1999-personal]

just FYI the admission-control YAML looks like this after patching:

metadata:
    namespace: controlplane
    type: AdmissionControlConfigs.kubernetes.talos.dev
    id: admission-control
    version: 3
    owner: k8s.ControlPlaneAdmissionControlController
    phase: running
    created: 2023-12-08T11:00:54Z
    updated: 2023-12-08T11:12:45Z
spec:
    config:
        - name: PodSecurity
          configuration:
            apiVersion: pod-security.admission.config.k8s.io/v1alpha1
            defaults:
                audit: restricted
                audit-version: latest
                enforce: baseline
                enforce-version: latest
                warn: restricted
                warn-version: latest
            exemptions:
                namespaces:
                    - kube-system
                    - openelb-system
                    - openelb-system
                runtimeClasses: []
                usernames: []
            kind: PodSecurityConfiguration

[smira]

well, you patched it twice, I'm not sure if api-server gets mad about it. You can use talosctl c -k and talosctl logs -k to see the logs of each component and figure out the problem.

[jaroslawhartman]

I can confirm, in case of duplicates the Control Plane cannot be connected anymore. Spent a few long hours troubleshooting this....

[paulcsiki]

@jaroslawhartman I am experiencing the same problem, how did you remove the duplicate? I've managed to repair the array contents using this command:
talosctl patch machineconfig --patch '[{"op": "replace", "path": "/cluster/apiServer/admissionControl/0/configuration/exemptions/namespaces", "value": ["kube-system"]}]' --nodes 192.168.10.21,192.168.10.22,192.168.10.23

[smira]

you can always use talosctl edit mc to edit the machine config directly instead of patching to revert back to a known good version.

another option is to implement a flow where machine configuration (base) and patches applied on top of it are managed outside of Talos, and talosctl apply-config is used to send the patched machine configuration to Talos. This way reverting a bad patch is the only thing that needs to be done.