checking for exclusive access and found busy partitions

[alrf]

Bug

Partitions in use on /dev/sda, Error: checking for exclusive access to /dev/sda

May 10 13:00:39  install-to-disk.sh[22377]: Executing coreos-installer with the following options: install -i /opt/openshift/master.ign /dev/sda
May 10 13:00:39  install-to-disk.sh[22407]: Downloading Fedora CoreOS stable x86_64 metal image (raw.xz) and signature
May 10 13:00:40  install-to-disk.sh[22407]: Partitions in use on /dev/sda:
May 10 13:00:40  install-to-disk.sh[22407]:     /dev/sda3 mounted on /boot
May 10 13:00:40  install-to-disk.sh[22407]:     /dev/sda4 mounted on /sysroot
May 10 13:00:40  install-to-disk.sh[22407]: Error: checking for exclusive access to /dev/sda
May 10 13:00:40  install-to-disk.sh[22407]: Caused by:
May 10 13:00:40  install-to-disk.sh[22407]:     found busy partitions
May 10 13:00:40  systemd[1]: install-to-disk.service: Main process exited, code=exited, status=1/FAILURE
May 10 13:00:40  systemd[1]: install-to-disk.service: Failed with result 'exit-code'.

Target Operating System Version

Bare-metal server with 2 disks.

coreos-installer Version

v0.21

Expected Behavior

It should install single node OKD instance.

Actual Behavior

Installation fails on Error: checking for exclusive access

Reproduction Steps

In install-config.yaml:

bootstrapInPlace:
  installationDisk: /dev/sda

Installation:

curl -L https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20231101.3.0/x86_64/fedora-coreos-39.20231101.3.0-metal.x86_64.raw.xz -o fcos-raw.xz
coreos-installer install -i bootstrap-in-place-for-live-iso.ign -f fcos-raw.xz --insecure /dev/sda
mdadm --stop /dev/md/* 2> /dev/null
mdadm --remove /dev/md* 2> /dev/null
mdadm --zero-superblock /dev/sda /dev/sdb 2> /dev/null
dd if=/dev/zero of=/dev/sda bs=1 count=64 seek=446 conv=notrunc 2> /dev/null
dd if=/dev/zero of=/dev/sdb bs=1 count=64 seek=446 conv=notrunc 2> /dev/null
wipefs -fa /dev/sd*
coreos-installer install -f fcos-live.iso --insecure /dev/sda

I also tried to install on /dev/sdb as: coreos-installer install -f fcos-live.iso --insecure /dev/sdb, but it fails with /dev/sda error again:

May 10 13:00:40  install-to-disk.sh[22407]: Partitions in use on /dev/sda:
May 10 13:00:40  install-to-disk.sh[22407]:     /dev/sda3 mounted on /boot
May 10 13:00:40  install-to-disk.sh[22407]:     /dev/sda4 mounted on /sysroot
May 10 13:00:40  install-to-disk.sh[22407]: Error: checking for exclusive access to /dev/sda
May 10 13:00:40  install-to-disk.sh[22407]: Caused by:
May 10 13:00:40  install-to-disk.sh[22407]:     found busy partitions

P.S. before every installation that I tried, I run those wipefs and dd commands, i.e. lsblk shows nothing on both disks:

root@rescue ~ # lsblk -f
NAME  FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
loop0 ext2   1.0         829dea1e-3d91-47e9-bc8f-a7a4eab805f4
sda
sdb

Other Information

# cat /proc/mdstat
Personalities :
unused devices: <none>
# vgdisplay
# lvdisplay
# pvs
# pvdisplay
# ll /dev/sd*
brw-rw----. 1 root disk 8,  0 May 10 13:08 /dev/sda
brw-rw----. 1 root disk 8,  1 May 10 13:08 /dev/sda1
brw-rw----. 1 root disk 8,  2 May 10 13:08 /dev/sda2
brw-rw----. 1 root disk 8,  3 May 10 13:08 /dev/sda3
brw-rw----. 1 root disk 8,  4 May 10 13:08 /dev/sda4
brw-rw----. 1 root disk 8, 16 May 10 12:41 /dev/sdb
# lsblk -f
NAME   FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda
├─sda1
├─sda2 vfat   FAT16 EFI-SYSTEM D234-D8A2
├─sda3 ext4   1.0   boot       22458e48-d327-4629-8f7e-f78e01be05e5  121.2M    59% /boot
└─sda4 xfs          root       cb871e6c-2942-4653-9293-0b268ff48e7f  220.3G     5% /var/lib/containers/storage/overlay
                                                                                   /var
                                                                                   /sysroot/ostree/deploy/fedora-coreos/var
                                                                                   /usr
                                                                                   /etc
                                                                                   /
                                                                                   /sysroot
sdb

[jlebon]

Hmm, the lsblk output in the "Reproduction Steps" show nothing on /dev/sda, but the lsblk output in "Other Information" does.

Are you sure you're booting into the ISO and not a stale FCOS installation on one of the disks?

[alrf]

Hmm, the lsblk output in the "Reproduction Steps" show nothing on /dev/sda, but the lsblk output in "Other Information" does.

Are you sure you're booting into the ISO and not a stale FCOS installation on one of the disks?

Yes, I'm sure. Seems "Random order of disks" is a well known issue:
https://forums.debian.net/viewtopic.php?t=158005
https://access.redhat.com/discussions/6004221
In my case the system booted in rescue mode (Debian-11), I get everything ready to install FCOS, then I reboot the system to start the installation and the drive letters change.

[jlebon]

Can you provide the full journal output?

[alrf]

Can you provide the full journal output?

Unfortunately, not right now since the test server has been decommissioned.