Missing documentation on BYOI images creation

[NightTsarina]

Hi,

For a customer who wants to deploy a big number of dedicated servers on OVH I am researching automating the process, and I am interested in taking advantage of the BYOI feature. Sadly, I have not been able to find any documentation on how to create those images, or even what are the technical specifications.

I would like to be able to control exactly how the servers are installed, ideally including disk partitioning. But I have no clue how to even start.

Thank you.

[OVH-Team-Guides]

Hello @NightTsarina, have you read this documentation https://docs.ovh.com/gb/en/dedicated/bringyourownimage/ ?
There is a sub-part about API usage that may be helpful for you to work on automation.
Feel free to ask our community for help on https://community.ovh.com/en/.
Best,
The OVHcloud Guides Team

[NightTsarina]

Hello @NightTsarina, have you read this documentation https://docs.ovh.com/gb/en/dedicated/bringyourownimage/ ? There is a sub-part about API usage that may be helpful for you to work on automation. Feel free to ask our community for help on https://community.ovh.com/en/. Best, The OVHcloud Guides Team

Hi!

Yes, I have checked that doc and also the community pages, but it does not say anything about how to create the image. I know how to upload and use the image, but where do I find an image to use?

[OVH-Team-Guides]

Hi @NightTsarina
The purpose of the bring-your-own-image feature is to let customers use their own image, usually custom ones or images provided on distributions websites such as Ubuntu, Red Hat, etc. We don't provide any tool to create or customize an image.
I'm sorry the documentation team won't be able to give you more guidance. I suggest you create either a support ticket or a new thread on the community pages, so that the community can help you with your use case.
Best,
The OVHcloud Guides Team

[OVH-Team-Guides]

If your request is to find the ISO of the OS you want to install and generate a custom image in qcow2 format, we currently don't have a documentation to address it but we're thinking about it.
Best,
The OVHcloud Guides Team

[NightTsarina]

Right, that is the problem. I have an ISO that has an interactive installer, but that won't work here. I can convert that to qcow2, but that won't work, because it requires user intervention.. So what I would like is to know what does the OVH system expect from that image and how is it deployed into the server... I have not been able to find any info at all about that.

Maybe you are just running the qcow2 image inside QEMU, and assuming it will automatically install the system? That'd be great, but I'd need to know at least a bit about the runtime environment and how I can pass parameters or configuration to it, so I can create the scripts to drive the install..

On the other hand, the fact that you seem to be managing partitions independently tells me that maybe you are just DD-ing the image onto the harddrive, so then what I need to prepare is just a snapshot of a running system instead of an installer...

These are the kind of questions that I think need to be answered in the documentation so sysadmins can build on top of your great service.

Hope this clarifies my request. Thanks again

[sbraz]

Hi,
Maybe the documentation is lacking an introductory sentence. BYOI expects an image of your disk containing the OS.

On the other hand, the fact that you seem to be managing partitions independently

We do not touch partitions apart from the configdrive one which we create if your image doesn't contain one. Expanding partitions and filesystems should be handled by cloud-init which you are strongly encouraged to use.

maybe you are just DD-ing the image onto the harddrive, so then what I need to prepare is just a snapshot of a running system instead of an installer...

Yes, that's the general idea. Either you provide a raw image of your disk that will get copied bit-for-bit to the first disk of the server, or you provide a qcow2, in which case it will be converted to raw and then copied to the disk.

You could start by creating a qcow2 from an installed server with qemu-img convert -f raw -O qcow2 -c /dev/sda image.qcow2 . If you don't want to use cloud-init at first, make sure your network configuration will work with DHCP and varying interfaces names.

[NightTsarina]

Hi, Maybe the documentation is lacking an introductory sentence. BYOI expects an image of your disk containing the OS.

Ah, I see, this is a key piece of information I was missing!

We do not touch partitions apart from the configdrive one which we create if your image doesn't contain one. Expanding partitions and filesystems should be handled by cloud-init which you are strongly encouraged to use.

maybe you are just DD-ing the image onto the harddrive, so then what I need to prepare is just a snapshot of a running system instead of an installer...

Yes, that's the general idea. Either you provide a raw image of your disk that will get copied bit-for-bit to the first disk of the server, or you provide a qcow2, in which case it will be converted to raw and then copied to the disk.

Gotcha. I suppose then there is no way to configure arbitrary partitioning schemes before the image is copied? I found the OVH partitioning definition a bit restricted. For example, I usually create only 2 partitions on each drive: one for /boot and another for luks encryption + LVM, and then create multiple LVMs with different sizes depending on configuration.

You could start by creating a qcow2 from an installed server with qemu-img convert -f raw -O qcow2 -c /dev/sda image.qcow2 . If you don't want to use cloud-init at first, make sure your network configuration will work with DHCP and varying interfaces names.

Excellent, thanks for this information! I think just adding this paragraph to the documentation would be a huge help for OVH customers.

[sbraz]

I suppose then there is no way to configure arbitrary partitioning schemes before the image is copied? I found the OVH partitioning definition a bit restricted. For example, I usually create only 2 partitions on each drive: one for /boot and another for luks encryption + LVM, and then create multiple LVMs with different sizes depending on configuration.

The partitions present on your image are left untouched when it is copied to disk. If you want to change that at boot, cloud-init can do it.

I think just adding this paragraph to the documentation would be a huge help for OVH customers.

Yeah I will discuss this with my colleagues. You're also welcome to make a PR modifying https://github.com/ovh/docs/blob/develop/pages/cloud/dedicated/bring-your-own-image/guide.en-gb.md if you'd like.

[bgabler]

Is there any documentation about getting both hard drives to show during BYOI install? We're trying to use a Virtuozzo image which has a GUI installer, but the installer only sees one disk vs both.

Basically, we need to use Virtuozzo with RAID-1 and previously we had to do an install once, then reboot into VNC to break the raid and run it again, but that's no longer working and takes way too much t ime.

What would be ideal is to boot into the image and see both drives available so we can create the proper software raid during install.

What would be really ideal is a way for us to do this install once, then make an image of that completed install to reuse on all other OVH installs since the disks will be the same size going forward for us.

For example, after we do an install where we could only see one disk:

NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1      259:0    0  1.8T  0 disk
nvme1n1      259:1    0  1.8T  0 disk
├─nvme1n1p1  259:2    0    1G  0 part
├─nvme1n1p2  259:3    0  200M  0 part
├─nvme1n1p3  259:4    0 62.8G  0 part
└─nvme1n1p4  259:5    0  1.7T  0 part
  ├─vhs-vz   251:0    0  1.6T  0 lvm
  └─vhs-root 251:1    0   64G  0 lvm

Is there a way we can use cloud-init before running the GUI installer to see both nvme0n1 and nvme1n1?

Example of an install we did get to work after a few hours (ran our GUI installer on to one disk, then rebooted into VNC and redid the install as it now showed both disks):

NAME           MAJ:MIN    RM   SIZE RO TYPE  MOUNTPOINT
nvme0n1        259:0       0 894.3G  0 disk
├─nvme0n1p1    259:1       0     1G  0 part
│ └─md127        9:127     0  1022M  0 raid1 /boot
├─nvme0n1p2    259:2       0   200M  0 part
│ └─md125        9:125     0   200M  0 raid1 /boot/efi
├─nvme0n1p3    259:3       0    32G  0 part  [SWAP]
├─nvme0n1p4    259:4       0    32G  0 part
│ └─md126        9:126     0    32G  0 raid1 /
└─nvme0n1p5    259:5       0 829.1G  0 part
  └─md124        9:124     0   829G  0 raid1 /vz
nvme1n1        259:6       0 894.3G  0 disk
├─nvme1n1p1    259:7       0     1G  0 part
│ └─md127        9:127     0  1022M  0 raid1 /boot
├─nvme1n1p2    259:8       0   200M  0 part
│ └─md125        9:125     0   200M  0 raid1 /boot/efi
├─nvme1n1p3    259:9       0    32G  0 part
│ └─md126        9:126     0    32G  0 raid1 /
└─nvme1n1p4    259:10      0 829.1G  0 part
  └─md124        9:124     0   829G  0 raid1 /vz
ploop38854     182:621664  0    10G  0 disk
└─ploop38854p1 182:621665  0    10G  0 part  /vz/pfcache

[sbraz]

I spoke to @bgabler directly, but here are the key points:

• BYOI itself doesn't support software RAID. If you want to use software RAID, the idea is to
  • install the OS in a virtual machine / dedicated server using software RAID
  • create an image from one of the disks used by the md array
  • install this image with BYOI
  • in post-installation, add the second disk (and third if relevant, etc.) to the RAID array
• In the coming weeks, we will add a new feature called Bring Your Own Linux which makes things easier, in which files are copied from the QCOW2 to a customer-defined partitioning scheme (created by our installer).

[bgabler]

I spoke to @bgabler directly, but here are the key points:

• BYOI itself doesn't support software RAID. If you want to use software RAID, the idea is to

  • install the OS in a virtual machine / dedicated server using software RAID
  • create an image from one of the disks used by the md array
  • install this image with BYOI
  • in post-installation, add the second disk (and third if relevant, etc.) to the RAID array

• In the coming weeks, we will add a new feature called Bring Your Own Linux which makes things easier, in which files are copied from the QCOW2 to a customer-defined partitioning scheme (created by our installer).

Shoutout to @sbraz, after one year we were finally able to come up with a proper solution for deploying Virtuozzo Hybrid Server on OVH.

Thank you again!