[cinder-csi-plugin] Node plugin image with utils

[sergelogvinov]

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug
/kind feature

What happened:

The cinder-csi-plugin image runs as root in privileged mode (not a secrets) on each node.
And this image includes many packages inside ~126Mb.
CSI-Node-Plugin does not use those packages. Such as python, perl, edit, swapon, bash...

GCP-CSI Plugin https://github.com/nberlee/gcp-compute-persistent-disk-csi-driver/blob/master/Dockerfile#L46 shrinks the image, removes most of the utils/libraries.

So, what do you think? How we can repeat this idea?
Reuse the image: use gke.gcr.io/gcp-compute-persistent-disk-csi-driver, and replace csi plugin by openstack version.
Or make our own scripts, which does like the same?

Thanks.

What you expected to happen:

How to reproduce it:

Anything else we need to know?:

Environment:

• openstack-cloud-controller-manager(or other related binary) version:
• OpenStack version:
• Others:

[mdbooth]

There's a hint about the requirements of this image in the Dockerfile:

cloud-provider-openstack/Dockerfile

Lines 111 to 112 in 4e04ca9

	# Install e4fsprogs for format
	RUN clean-install btrfs-progs e2fsprogs mount udev xfsprogs

We should try to validate all those requirements and move to a much smaller image.

Incidentally, although the git blame on that will have my name on it I was just copying what was there before! I don't have the context here.

[sergelogvinov]

yep, this script does not do anything special

apt-get update
apt-get install -y --no-install-recommends $@
apt-get clean -y
rm -rf \
   /var/cache/debconf/* \
   /var/lib/apt/lists/* \
   /var/log/* \
   /tmp/* \
   /var/tmp/*

ok, i will try to make clean process...

And one question about btrfs-progs is it really need here?

[jichenjc]

And one question about btrfs-progs is it really need here?

don't remember clearly but seems it's introduced because some file system operations are needed in cinder csi operations..
e.g ext2/btrfs might need different utils

[sergelogvinov]

What do you think if we will use gcr.io/distroless/base

It does not have shell at all...
The image registry.k8s.io/build-image/debian-base:bullseye-v1.4.3 has many insecure tools/utils inside...

[mdbooth]

How much test coverage do we have here? Will the conformance tests give us confidence to make this change?

[sergelogvinov]

Most of the dependencies come from https://github.com/kubernetes/mount-utils.
It does not have official list of external utils.
So, I've searched the code and made it.

The first test (the same that google does) - check the result, all utils have dependent libs.
But, in my opinion, it does not enough.

Do you have any ideas?

[jichenjc]

@sergelogvinov do you have a sizing compare based on your current PR?

How much test coverage do we have here

conformance test might be enough , but I knew we have several issues reported by users which is not in conformance before on this area..

[sergelogvinov]

• registry.k8s.io/provider-os/cinder-csi-plugin:v1.27.0-alpha.0-31-g35ce5a0c - 198MB - master
  trivy scan - Total: 142 (UNKNOWN: 0, LOW: 77, MEDIUM: 20, HIGH: 40, CRITICAL: 5)
• registry.k8s.io/provider-os/cinder-csi-plugin:v1.27.0-alpha.0-32-gcd000d54 - 118MB - PR
  trivy scan - Total: 66 (UNKNOWN: 0, LOW: 53, MEDIUM: 6, HIGH: 6, CRITICAL: 1)
• registry.k8s.io/provider-os/cinder-csi-plugin:v1.27.0-alpha.0-32-gcd000d54-dirty - 75.9MB (distroless/base image)
  trivy scan - Total: 17 (UNKNOWN: 0, LOW: 11, MEDIUM: 4, HIGH: 2, CRITICAL: 0)

Second layer (file system utils)

/dest/lib/x86_64-linux-gnu/libe2p.so.2
/dest/lib/x86_64-linux-gnu/liblzo2.so.2
/dest/lib/x86_64-linux-gnu/libc.so.6
/dest/lib/x86_64-linux-gnu/librt.so.1
/dest/lib/x86_64-linux-gnu/libz.so.1
/dest/lib/x86_64-linux-gnu/libcom_err.so.2
/dest/lib/x86_64-linux-gnu/libpthread.so.0
/dest/lib/x86_64-linux-gnu/libselinux.so.1
/dest/lib/x86_64-linux-gnu/libdl.so.2
/dest/lib/x86_64-linux-gnu/liblzma.so.5
/dest/lib/x86_64-linux-gnu/libext2fs.so.2
/dest/lib/udev
/dest/lib/udev/rules.d
/dest/lib/udev/rules.d/80-net-setup-link.rules
/dest/lib/udev/rules.d/60-persistent-storage-tape.rules
/dest/lib/udev/rules.d/60-block.rules
/dest/lib/udev/rules.d/60-input-id.rules
/dest/lib/udev/rules.d/70-mouse.rules
/dest/lib/udev/rules.d/60-serial.rules
/dest/lib/udev/rules.d/60-persistent-input.rules
/dest/lib/udev/rules.d/80-drivers.rules
/dest/lib/udev/rules.d/73-special-net-names.rules
/dest/lib/udev/rules.d/70-power-switch.rules
/dest/lib/udev/rules.d/60-evdev.rules
/dest/lib/udev/rules.d/75-net-description.rules
/dest/lib/udev/rules.d/96-e2scrub.rules
/dest/lib/udev/rules.d/60-cdrom_id.rules
/dest/lib/udev/rules.d/60-sensor.rules
/dest/lib/udev/rules.d/60-persistent-storage.rules
/dest/lib/udev/rules.d/70-touchpad.rules
/dest/lib/udev/rules.d/95-dm-notify.rules
/dest/lib/udev/rules.d/80-debian-compat.rules
/dest/lib/udev/rules.d/70-joystick.rules
/dest/lib/udev/rules.d/78-sound-card.rules
/dest/lib/udev/rules.d/60-fido-id.rules
/dest/lib/udev/rules.d/64-btrfs.rules
/dest/lib/udev/rules.d/64-btrfs-dm.rules
/dest/lib/udev/rules.d/60-autosuspend.rules
/dest/lib/udev/rules.d/60-persistent-alsa.rules
/dest/lib/udev/rules.d/50-firmware.rules
/dest/lib/udev/rules.d/75-probe_mtd.rules
/dest/lib/udev/rules.d/60-drm.rules
/dest/lib/udev/rules.d/60-persistent-v4l.rules
/dest/lib/udev/rules.d/55-dm.rules
/dest/lib/udev/rules.d/60-persistent-storage-dm.rules
/dest/lib/udev/rules.d/50-udev-default.rules
/dest/sbin
/dest/sbin/fsck.cramfs
/dest/sbin/mkfs.ext4
/dest/sbin/e2label
/dest/sbin/mkfs.xfs
/dest/sbin/fsck
/dest/sbin/dumpe2fs
/dest/sbin/mkfs.bfs
/dest/sbin/xfs_repair
/dest/sbin/fsck.ext3
/dest/sbin/fsck.ext2
/dest/sbin/mkfs
/dest/sbin/blkid
/dest/sbin/blockdev
/dest/sbin/e2image
/dest/sbin/e2scrub
/dest/sbin/fsck.btrfs
/dest/sbin/mkfs.cramfs
/dest/sbin/e2scrub_all
/dest/sbin/resize2fs
/dest/sbin/mkfs.ext3
/dest/sbin/mkfs.minix
/dest/sbin/fsck.xfs
/dest/sbin/e2undo
/dest/sbin/fsck.minix
/dest/sbin/fsck.ext4
/dest/sbin/e2mmpstatus
/dest/sbin/mke2fs
/dest/sbin/e2fsck
/dest/sbin/mkfs.ext2
/dest/sbin/mkfs.btrfs
/dest/etc
/dest/etc/mke2fs.conf
/dest/bin
/dest/bin/btrfsck
/dest/bin/umount
/dest/bin/btrfstune
/dest/bin/btrfs-select-super
/dest/bin/btrfs-convert
/dest/bin/btrfs-map-logical
/dest/bin/btrfs-find-root
/dest/bin/mount
/dest/bin/findmnt
/dest/bin/btrfs-image
/dest/bin/btrfs
/dest/bin/udevadm
/dest/usr
/dest/usr/lib
/dest/usr/lib/x86_64-linux-gnu
/dest/usr/lib/x86_64-linux-gnu/libsmartcols.so.1
/dest/usr/lib/x86_64-linux-gnu/libuuid.so.1
/dest/usr/lib/x86_64-linux-gnu/libpcre2-8.so.0
/dest/usr/lib/x86_64-linux-gnu/libblkid.so.1
/dest/usr/lib/x86_64-linux-gnu/libinih.so.1
/dest/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
/dest/usr/lib/x86_64-linux-gnu/libzstd.so.1
/dest/usr/lib/x86_64-linux-gnu/libmount.so.1
/dest/usr/lib/x86_64-linux-gnu/libacl.so.1
/dest/usr/lib/x86_64-linux-gnu/libkmod.so.2
/dest/usr/lib/x86_64-linux-gnu/libudev.so.1
/dest/usr/sbin
/dest/usr/sbin/xfs_estimate
/dest/usr/sbin/xfs_mkfile
/dest/usr/sbin/xfs_admin
/dest/usr/sbin/xfs_ncheck
/dest/usr/sbin/xfs_quota
/dest/usr/sbin/xfs_metadump
/dest/usr/sbin/xfs_growfs
/dest/usr/sbin/xfs_db
/dest/usr/sbin/xfs_fsr
/dest/usr/sbin/xfs_freeze
/dest/usr/sbin/xfs_scrub_all
/dest/usr/sbin/xfs_io
/dest/usr/sbin/xfs_copy
/dest/usr/sbin/xfs_logprint
/dest/usr/sbin/xfs_spaceman
/dest/usr/sbin/xfs_scrub
/dest/usr/sbin/xfs_info
/dest/usr/sbin/xfs_rtcp
/dest/usr/sbin/xfs_bmap
/dest/usr/sbin/xfs_mdrestore

[mdbooth]

Most of the dependencies come from https://github.com/kubernetes/mount-utils. It does not have official list of external utils. So, I've searched the code and made it.

The first test (the same that google does) - check the result, all utils have dependent libs. But, in my opinion, it does not enough.

Do you have any ideas?

I just commented about this on your PR. Could you add a long-form comment to your script explaining how you created both lists?

I wonder if it's also worth creating an issue against mount-utils (presumably in k/k?) to ask them to maintain the list and perhaps even a script or image.

[sergelogvinov]

I've also added the csi-deps-check.sh file, it runs all programs which I found in go modules.

can be used as make build-local-image-cinder-csi-plugin-utils-check
Can you give me advice on the best place to use it as a test stage?

I wonder if it's also worth creating an issue against mount-utils (presumably in k/k?) to ask them to maintain the list and perhaps even a script or image.

Yep, this is a good idea... kubernetes/mount-utils#13

I prefer distroless images, but tools we definitely should get from debian distro.