Building image as rootless in RKE2 with profile cis

[luysantana]

Hello.

I have a RKE2 cluster with profile: cis enabled, that way I've to set the security context in all containers that run in cluster:

securityContext:
  allowPrivilegeEscalation: false
  runAsNonRoot: true
  runAsUser: 1000
  seccompProfile:
    type: RuntimeDefault
  capabilities:
    drop:
    - ALL

I'm using Tekton as CI tool, the pipeline has all tasks configured like this and the only step isn't work is the image build with buildah.

My buildah tekton task is:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: buildah
spec:
  params:
  - name: IMAGE
    description: Reference of the image buildah will produce.
  - name: BUILDER_IMAGE
    description: The location of the buildah builder image.
    default: quay.io/buildah/stable:v1
  **- name: STORAGE_DRIVER
    description: Set buildah storage driver
    default: vfs**
  - name: CONTEXT
    description: Path to the directory to use as context.
    default: .
  - name: TLSVERIFY
    description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)
    default: "false"
  - name: FORMAT
    description: "The format of the built container, oci or docker"
    default: oci
  - name: BUILD_EXTRA_ARGS
    description: Extra parameters passed for the build command when building images.
    default: ""
  - name: PUSH_EXTRA_ARGS
    description: Extra parameters passed for the push command when pushing images.
    default: ""
  - name: SKIP_PUSH
    description: Skip pushing the built image
    default: "false"
  results:
  - name: DOCKERFILE_PATH
    description: "Path to the Dockerfile to build."
  workspaces:
  - name: source
  steps:
    # some steps before ...
  - image: myregistry/buildah-rootless:1.0.2
    name: build-and-push
    securityContext:
      allowPrivilegeEscalation: false
      runAsNonRoot: true
      runAsUser: 1000
      seccompProfile:
        type: RuntimeDefault
      capabilities:
        drop:
        - ALL
    workingDir: $(workspaces.source.path)
    env:
    - name: username
      valueFrom:
        secretKeyRef:
          name: tekton-harbor-basic-auth
          key: username
    - name: password
      valueFrom:
        secretKeyRef:
          name: tekton-harbor-basic-auth
          key: password
    script: |
      echo "[INFO] Executando buildah bud"
      
      export BUILDAH_ISOLATION=chroot
      #export STORAGE_DRIVER=overlay

      buildah bud --storage-driver=$(params.STORAGE_DRIVER) $(params.BUILD_EXTRA_ARGS) \
        --format=$(params.FORMAT) --tls-verify=$(params.TLSVERIFY) --no-cache \
        -f $(cat $(results.DOCKERFILE_PATH.path)) -t $(params.IMAGE) $(params.CONTEXT)

      [[ "$(params.SKIP_PUSH)" == "true" ]] && echo "[INFO] Pulando push e finalizando Task" && exit 0

      echo "[INFO] Executando buildah push"
            
      buildah login -u ${username} -p ${password} {{ .Values.registry.url }} --tls-verify=$(params.TLSVERIFY)

      buildah --storage-driver=$(params.STORAGE_DRIVER) push \
        $(params.PUSH_EXTRA_ARGS) --tls-verify=$(params.TLSVERIFY) \
        --creds=${username}:${password} $(params.IMAGE) 

      echo "[INFO] Removendo imagem gerada locamente, pois a mesma já foi enviada para o registry"
      buildah --storage-driver=$(params.STORAGE_DRIVER) rmi \
      $(params.IMAGE)

      echo "[INFO] Listando todas as imagens presentes no storage local do buildah"
      buildah --storage-driver=$(params.STORAGE_DRIVER) images

This rootless image is based on this doc that I've found in other articles too.

The error I have now is:

2024-11-21T09:45:43.811199551-03:00 [INFO] Executando buildah bud
2024-11-21T09:45:43.839215941-03:00 Error during unshare(CLONE_NEWUSER): Operation not permitted
2024-11-21T09:45:43.839604336-03:00 time="2024-11-21T12:45:43Z" level=error msg="parsing PID "": strconv.Atoi: parsing "": invalid syntax"
2024-11-21T09:45:43.839611999-03:00 time="2024-11-21T12:45:43Z" level=error msg="(Unable to determine exit status)"

But if I put capabilities, I have:

  - lastTransitionTime: '2024-11-21T12:13:36Z'
    message: 'pods "ranchd81faf7131d3eb0246bd4c2d7664e823675645d8-pod" is forbidden: violates PodSecurity "restricted:latest": unrestricted capabilities (container "step-build-and-push" must not include "CAP_SETGID", "CAP_SETUID" in securityContext.capabilities.add)'
    reason: PodAdmissionFailed
    status: 'False'
    type: Succeeded

I have tried with overlay and vfs, set and removed runAsUser: 1000, with official buildah image, nothing has working in this cluster and I need to run this cluster with profile: cis enabled.

What can I do?

[nalind]

When started as an unprivileged UID, buildah needs to be able to create a user namespace with unshare(), and set mappings in it with the help of the the newuidmap and newgidmap helpers. It currently can't function otherwise.