-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provisioning: Add Hetzner #654
Open
travier
wants to merge
1
commit into
coreos:main
Choose a base branch
from
travier:main-prov-hetzner
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,183 @@ | ||
= Provisioning Fedora CoreOS on Hetzner | ||
|
||
This guide shows how to provision new Fedora CoreOS (FCOS) nodes on Hetzner. | ||
Fedora CoreOS is currently not available as an option in the operating system selection on Hetzner. | ||
Fedora CoreOS images for Hetzner are currently not directly available for download either. | ||
Thus you must first download a Fedora CoreOS RAW disk image, then convert it to an Hetzner image locally and finally create a snapshot from it in your Hetzner account using the https://github.com/apricote/hcloud-upload-image[hcloud-upload-image] tool. | ||
|
||
IMPORTANT: Support for Fedora CoreOS on Hetzner is considered emerging, in that it does not yet offer an optimized user experience and relies on tools not officially supported by Hetzner. | ||
See https://github.com/coreos/fedora-coreos-tracker/issues/1324[issue #1324] for more details. | ||
|
||
IMPORTANT: The https://github.com/apricote/hcloud-upload-image[hcloud-upload-image] tool is not an official Hetzner Cloud product and Hetzner Cloud does not provide support for it. | ||
Alternatively, you can also use the official https://github.com/hetznercloud/packer-plugin-hcloud[packer-plugin-hcloud] to install the image via `coreos-installer`. | ||
|
||
IMPORTANT: In order to create a snapshot, the https://github.com/apricote/hcloud-upload-image[hcloud-upload-image] tool will provision a small server and boot it in rescue mode. | ||
As this server is short lived, the cost should be very limited. | ||
The resulting snapshots are charged per GB per month. | ||
See https://docs.hetzner.com/cloud/servers/backups-snapshots/overview/[Backups/Snapshots] in the Hetzner Cloud documentation. | ||
You may delete this snapshot once the server has been provisioned. | ||
|
||
== Prerequisites | ||
|
||
Before provisioning an FCOS machine, you must have an Ignition configuration file containing your customizations. | ||
If you do not have one, see xref:producing-ign.adoc[Producing an Ignition File]. | ||
|
||
NOTE: Fedora CoreOS has a default `core` user that can be used to explore the OS. | ||
If you want to use it, finalize its xref:authentication.adoc[configuration] by providing e.g. an SSH key. | ||
|
||
If you do not want to use Ignition to get started, you can make use of the https://coreos.github.io/afterburn/platforms/[Afterburn support] and only configure SSH keys. | ||
|
||
You also need to have access to a Hetzner account. | ||
The examples below use the https://github.com/hetznercloud/cli[hcloud] command-line tool, the https://github.com/apricote/hcloud-upload-image[hcloud-upload-image] tool and https://stedolan.github.io/jq/[jq] as a command-line JSON processor. | ||
|
||
== Creating a snapshot | ||
|
||
Fedora CoreOS is designed to be updated automatically, with different schedules per stream. | ||
|
||
. Once you have picked the relevant stream, download the latest RAW image from the https://fedoraproject.org/coreos/download/?stream=stable#baremetal[download page] or with podman (see https://coreos.github.io/coreos-installer/cmd/download/[documentation] for options): | ||
+ | ||
[source, bash] | ||
---- | ||
arch="x86_64" # or aarch64 | ||
podman run --security-opt label=disable --pull=always --rm -v .:/data -w /data \ | ||
quay.io/coreos/coreos-installer:release download -s stable -p metal -f raw.xz -a "${arch}" --decompress | ||
---- | ||
+ | ||
Note this is just using `coreos-installer` as a tool to download the RAW disk image. | ||
+ | ||
NOTE: Both x86_64 and aarch64 architectures are supported on Hetzner. | ||
|
||
. Copy paste the following Bash script into a file name `convert-image.sh`: | ||
+ | ||
.QEMU to Hetzner image conversion script | ||
[source, bash] | ||
---- | ||
#!/bin/bash | ||
|
||
set -euo pipefail | ||
|
||
if [[ ${#} -ne 3 ]]; then | ||
echo "Usage: <source image> <dest image> <platform>" | ||
echo "" | ||
echo "Example:" | ||
echo "./$(basename "${0}") fedora-coreos-40.20240616.3.0-{metal,hetzner}.x86_64.raw hetzner" | ||
exit 1 | ||
fi | ||
|
||
source="${1}" | ||
dest="${2}" | ||
platform="${3}" | ||
|
||
if [[ ! -f "${source}" ]]; then | ||
echo "Source image ${source} does not exists" | ||
exit 1 | ||
fi | ||
|
||
if [[ -f "${dest}" ]]; then | ||
echo "Destination image ${dest} already exists" | ||
exit 1 | ||
fi | ||
|
||
if [[ -z "$(command -v guestfish)" ]]; then | ||
echo "Could not find 'guestfish' command" | ||
exit 1 | ||
fi | ||
|
||
cp --reflink=auto "${source}" "${dest}" | ||
guestfish -a "${dest}" <<EOF | ||
run | ||
mount /dev/sda3 / | ||
download /loader/entries/ostree-1.conf tmp.loader.entries.ostree-1.conf | ||
<! sed -i "s/ignition.platform.id=metal/ignition.platform.id=${platform}/" tmp.loader.entries.ostree-1.conf | ||
upload tmp.loader.entries.ostree-1.conf /loader/entries/ostree-1.conf | ||
EOF | ||
|
||
rm -v ./tmp.loader.entries.ostree-1.conf | ||
|
||
echo "Done" | ||
---- | ||
+ | ||
. Make sure that you have `guestfish` installed on your system and convert the metal image to an Hetzner one: | ||
+ | ||
[source, bash, subs="attributes"] | ||
---- | ||
source_image"fedora-coreos-{stable-version}-metal.x86_64.raw" | ||
image_name="fedora-coreos-{stable-version}-hetzner.x86_64.raw" | ||
chmod a+x ./convert-image.sh | ||
./convert-image.sh "${source_image}" "${image_name}" hetzner | ||
travier marked this conversation as resolved.
Show resolved
Hide resolved
|
||
---- | ||
+ | ||
. Compress the image using `bzip2`: | ||
+ | ||
[source, bash] | ||
---- | ||
bzip2 --compress --fast "${image_name}" | ||
---- | ||
+ | ||
. Use the `hcloud-upload-image` to create a snapshot out of your image: | ||
+ | ||
[source, bash] | ||
---- | ||
export HCLOUD_TOKEN="<your token>" | ||
hcloud-upload-image upload \ | ||
--architecture x86 \ | ||
--compression bz2 \ | ||
--image-path "${image_name}.bz2" \ | ||
--labels os=fedora-coreos,channel=stable \ | ||
--description "Fedora CoreOS (stable, x86_64)" | ||
---- | ||
+ | ||
. Wait for the process to complete and validate that you have a snapshot: | ||
+ | ||
[source, bash] | ||
---- | ||
hcloud image list --type=snapshot --selector=os=fedora-coreos | ||
---- | ||
|
||
== Launching a server | ||
|
||
. If you don't already have an SSH key uploaded to Hetzner, you may upload one: | ||
+ | ||
.Example uploading an SSH key to Hetzner | ||
[source, bash] | ||
---- | ||
ssh_pubkey="ssh-ed25519 ..." | ||
ssh_key_name="fedora-coreos-hetzner" | ||
hcloud ssh-key create --name "${ssh_key_name}" --public-key "${ssh_pubkey}" | ||
---- | ||
+ | ||
. Launch a server. Your Ignition configuration can be passed to the VM as its user data, or you can skip passing user data if you just want SSH access. | ||
This provides an easy way to test out FCOS without first creating an Ignition config. | ||
+ | ||
.Example launching FCOS on Hetzner using an Ignition configuration file and SSH key | ||
[source, bash] | ||
---- | ||
image_id="$(hcloud image list \ | ||
--type=snapshot \ | ||
--selector=os=fedora-coreos \ | ||
--output json \ | ||
| jq -r '.[0].id')" | ||
ssh_key_name="fedora-coreos-hetzner" # See: hcloud ssh-key list | ||
datacenter="fsn1-dc14" # See: hcloud datacenter list | ||
type="cx22" # See: hcloud server-type list | ||
name="fedora-coreos-test" | ||
ignition_config="./config.ign" | ||
hcloud server create \ | ||
--name "${name}" \ | ||
--type "${type}" \ | ||
--datacenter "${datacenter}" \ | ||
--image "${image_id}" \ | ||
--ssh-key "${ssh_key_name}" \ | ||
--user-data-from-file "${ignition_config}" | ||
---- | ||
+ | ||
NOTE: While the Hetzner documentation and website mentions `cloud-init` and "cloud config", FCOS does not support cloud-init. | ||
It accepts only Ignition configuration files. | ||
|
||
. You now should be able to SSH into the instance using the associated IP address. | ||
+ | ||
.Example connecting | ||
[source, bash] | ||
---- | ||
ssh core@"$(hcloud server ip "${name}")" | ||
---- |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FWIW, as per https://github.com/apricote/hcloud-upload-image?tab=readme-ov-file#support-disclaimer, this isn't an official tool. I feel like our docs should stick with tools available in Fedora or from the cloud provider directly. At the very least, it seems worth calling this out in a WARNING.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two official alternatives:
hcloud
CLI and some shell scriptpacker-plugin-hcloud
. Example templates:I personally found both of these options annoying and not very user friendly, so I wrote the CLI/Go Lib to improve on this.
hcloud-upload-image
will be deprecated in favor of anhcloud
subcommand if an official endpoint for uploading images is added to the API.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would say this entire page is "best-effort" given that we don't produce images. But I'm also fine adding a warning specifically for this tool.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not go the packer route because it depends on packer and it's no longer open source.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should also be mentioned that all variants ultimately create a temporary server that will be billed at the end of the month (even though it shouldn't be that much money since Hetzner charges by minutes used; however, unknown servers unexpectedly appearing on the bill might confuse users). It should also be mentioned that this will create a snapshot that also costs money depending on the image size (0.0110 € per GB per month plus VAT); permanently if not deleted manually.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added an "important" note about both of those topics at the top.