Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: docker-kind-support #428

Merged
merged 57 commits into from
Aug 22, 2023
Merged

Feat: docker-kind-support #428

merged 57 commits into from
Aug 22, 2023

Conversation

rauerhans
Copy link
Contributor

@rauerhans rauerhans commented Aug 11, 2023

Summary

This adds support to run docker containers and/or a kind cluster from within a plural cloud-cli kubectl exec session. The dind container needed for that is running in a rootfull, but unprviliged sidecar to the plural cloud-cli container on a sysbox node:

apiVersion: v1 
kind: Pod 
metadata: 
  name: plural-cli-dind-sidecar
  namespace: sysbox
  annotations:
    io.kubernetes.cri-o.userns-mode: "auto:size=65536"
spec: 
  runtimeClassName: sysbox-runc
  tolerations:
    - key: plural.sh/sysbox
      operator: Exists
    - key: plural.sh/capacityType
      operator: Exists
  restartPolicy: Never
  containers: 
    - name: plural-cli
      image: ghcr.io/pluralsh/plural-cli-cloud:pr-428
      command: ["sh", "-c", "while true; do sleep 3600; done"]
      resources: 
        requests: 
          cpu: 10m 
          memory: 256Mi 
      env: 
      - name: DOCKER_HOST 
        value: tcp://localhost:2375 
      volumeMounts: 
        - name: docker
          mountPath: /var/lib/docker 
    - name: sysbox-dind
      image: ghcr.io/pluralsh/plural-dind:pr-428
      resources: 
        requests: 
          cpu: 20m 
          memory: 512Mi 
      securityContext: 
        privileged: false 
      volumeMounts: 
        - name: docker
          mountPath: /var/lib/docker 
  volumes: 
    - name: docker 
      emptyDir: {}

!Make sure to only run this on the sysbox nodes!

Test:

plural@plural-cli-dind-sidecar:~$ whoami
plural
plural@plural-cli-dind-sidecar:~$ kind create cluster
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.27.3) 🖼
 ✓ Preparing nodes 📦  
 ✓ Writing configuration 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a nice day! 👋
plural@plural-cli-dind-sidecar:~$ kubectl get nodes
NAME                 STATUS   ROLES           AGE   VERSION
kind-control-plane   Ready    control-plane   72s   v1.27.3
plural@plural-cli-dind-sidecar:~$ kubectl get pods --all-namespaces
NAMESPACE            NAME                                         READY   STATUS    RESTARTS   AGE
kube-system          coredns-5d78c9869d-2nqh6                     1/1     Running   0          80s
kube-system          coredns-5d78c9869d-vjfdb                     1/1     Running   0          79s
kube-system          etcd-kind-control-plane                      1/1     Running   0          89s
kube-system          kindnet-gdgs8                                1/1     Running   0          80s
kube-system          kube-apiserver-kind-control-plane            1/1     Running   0          90s
kube-system          kube-controller-manager-kind-control-plane   1/1     Running   0          89s
kube-system          kube-proxy-qk4mr                             1/1     Running   0          80s
kube-system          kube-scheduler-kind-control-plane            1/1     Running   0          93s
local-path-storage   local-path-provisioner-6bc4bddd6b-bmkf6      1/1     Running   0          80s
plural@plural-cli-dind-sidecar:~$ kind delete cluster
Deleting cluster "kind" ...
Deleted nodes: ["kind-control-plane"]

Labels

Test Plan

Checklist

  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • I have added relevant labels to this PR to help with categorization for release notes.

@rauerhans rauerhans added the enhancement New feature or request label Aug 11, 2023
@rauerhans rauerhans changed the title Feat/docker kind support Feat/docker-kind-support Aug 11, 2023
@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@rauerhans rauerhans changed the title Feat/docker-kind-support Feat: docker-kind-support Aug 21, 2023
@rauerhans rauerhans marked this pull request as ready for review August 21, 2023 16:57
@rauerhans rauerhans merged commit b79724d into main Aug 22, 2023
michaeljguarino pushed a commit that referenced this pull request Aug 28, 2024
* add docker and kind

* fix, 24.0.5 not signed yet (?)

* install s6

* need the dockerfiles in the context

* s6 services config for docker

* docker as nonroot

* rename

* fix typo

* fix iptables issue

* add modprobe

* use rootless install script

* add env vars

* fix

* fix

* fix

* rm debug code

* try with fuse-overlayfs

* test

* test dockerfile systemd

* fix build

* add missing iproute2

* pod yamls

* jammy test with plural login

* update image

* sysbox priv

* add user/contents.d

* test podman build

* this at least works with --privileged

* rm dockerd s6 service, clogs logs

* try on sysbox runc

* shouldn't matter, but let's see

* unset mount program, be specific about defaults

* test with new storage.conf

* simplify

* for some reason this does make a difference

* try with vfs

* makes it worse

* fix perm on volume

* add back mknod

* force newer version of podman, crun dep is ancient

* trigger build

* build with docker-ce-cli only

* dind sidecar approach

* clean up

* create dind dockerfile with systemd conf

* add make target

* dont ignore systemd files

* add dind build gh wf step

* rm trest image dockerfile

* rm debug yamnls

* clean up

* clean up, not needed with correct unit file

* rm dependencies of podman

* clean up

* rm get-docker.sh was need for debugging

* update yaml

* delete sidecar yaml example, not needed in repo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request roadmap
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants