Issues with Bootstrapping Talos Linux Cluster on Apple Silicon (M3 Max) using QEMU #9773
OmidZamani
started this conversation in
General
Replies: 3 comments 1 reply
-
|
Here's your text formatted in Markdown for clarity and readability: # start-node-2.sh
#!/bin/bash
qemu-img create -f qcow2 disk-2.qcow2 20G
qemu-system-aarch64 \
-M virt,highmem=on,gic-version=3 \
-cpu cortex-a72 \
-smp 2 \
-m 2048M \
-accel hvf \
-bios /opt/homebrew/share/qemu/edk2-aarch64-code.fd \
-device virtio-scsi-pci \
-device scsi-cd,drive=cd0 \
-drive if=none,id=cd0,format=raw,media=cdrom,readonly=on,file=../metal-arm64.iso \
-device scsi-hd,drive=hd0 \
-drive if=none,id=hd0,format=qcow2,file=disk-2.qcow2 \
-device virtio-net-pci,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::2222-:22,hostfwd=tcp::51000-:50000 \
-d unimp,guest_errors \
-nographic \
-monitor none \
-serial stdio# start-node-3.sh
#!/bin/bash
qemu-img create -f qcow2 disk-3.qcow2 20G
qemu-system-aarch64 \
-M virt,highmem=on,gic-version=3 \
-cpu cortex-a72 \
-smp 2 \
-m 2048M \
-accel hvf \
-bios /opt/homebrew/share/qemu/edk2-aarch64-code.fd \
-device virtio-scsi-pci \
-device scsi-cd,drive=cd0 \
-drive if=none,id=cd0,format=raw,media=cdrom,readonly=on,file=../metal-arm64.iso \
-device scsi-hd,drive=hd0 \
-drive if=none,id=hd0,format=qcow2,file=disk-3.qcow2 \
-device virtio-net-pci,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::2223-:22,hostfwd=tcp::52000-:50000 \
-d unimp,guest_errors \
-nographic \
-monitor none \
-serial stdioExpected Output Node-1 |
Beta Was this translation helpful? Give feedback.
0 replies
-
Expected Output Node 2Expected Output Node 3 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Thanks for detailed report, but let's split it up a bit. As the machine boots up in the maintenance mode, it doesn't matter what the contents of machine configuration is, or what is the in the Once Talos has machine configuration, it will use mutual TLS and it will start checking |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I just verified that the cert fingerprint check works correctly, so your issues are either related to the wrong fingerprint being used, or trying to connect to a wrong VM. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Issues with Bootstrapping Talos Linux Cluster on Apple Silicon (M3 Max) using QEMU
System Information
/opt/homebrew/bin/qemu-system-x86_64Goal
Attempting to create a 3-node Talos Linux cluster running on QEMU/KVM with:
Environment Setup
Node Configurations
Control Plane Node (start-node-1.sh):
#!/bin/bash qemu-img create -f qcow2 disk-1.qcow2 20G qemu-system-aarch64 \ -M virt,highmem=on,gic-version=3 \ -cpu cortex-a72 \ -smp 2 \ -m 2048M \ -accel hvf \ -bios /opt/homebrew/share/qemu/edk2-aarch64-code.fd \ -device virtio-scsi-pci \ -device scsi-cd,drive=cd0 \ -drive if=none,id=cd0,format=raw,media=cdrom,readonly=on,file=../metal-arm64.iso \ -device scsi-hd,drive=hd0 \ -drive if=none,id=hd0,format=qcow2,file=disk-1.qcow2 \ -device virtio-net-pci,netdev=net0 \ -netdev user,id=net0,hostfwd=tcp::2221-:22,hostfwd=tcp::50000-:50000,hostfwd=tcp::6443-:6443 \ -d unimp,guest_errors \ -nographic \ -monitor none \ -serial stdioAttempted Steps and Issues Encountered
Initial Cluster Configuration Attempts
First configuration attempt:
Output:
Initial attempt to apply configuration:
Error received:
Second configuration attempt with certificate SANs:
Certificate fingerprint challenges:
talosctl --talosconfig talosconfig apply-config --insecure --nodes 127.0.0.1:50000 --file controlplane.yaml --cert-fingerprint 'YbA2pBTOfmf0lFdmnbrKTNHqAaYtMAaiOSlDMCoPBAU='Error:
Attempts with IP Address Instead of Localhost:
talosctl --talosconfig talosconfig apply-config --insecure --nodes 10.0.2.15 --file controlplane.yaml --cert-fingerprint 'tAn9jcORuAkGwgdFW5fbV/hOI35EBkDrnOT/fl7foHc='Error:
Final Configuration Attempt with Modified Endpoint:
Bootstrap Attempts:
Error:
Key Issues Identified
Certificate and Authentication Issues:
Network Configuration Challenges:
Bootstrap Process:
Environment-Specific Notes
VM Network Configuration:
10.0.2.15IP address in its own network namespace50000,50001,50002for API6443is forwarded for the control planePort Forwarding Setup:
2221:22,50000:50000,6443:64432222:22,50001:500002223:22,50002:50000Attempted Solutions
Certificate Configuration:
127.0.0.1,10.0.2.15,localhost)Network Configuration:
Bootstrap Process:
Questions and Considerations
System Logs and Errors
Notable errors from node startup logs:
Worker Node Configurations
Worker Node 1 (start-node-2.sh):
Worker Node 2 (start-node-3.sh):
Multiple Fingerprint Attempts
We observed the certificate fingerprint changing during node startup:
tAn9jcORuAkGwgdFW5fbV/hOI35EBkDrnOT/fl7foHc=6NXHenl9s2przPy5xAkjg9J1H3GPlfTDxH2ZW0elkac=byKnK5mKeQUuDmse8Pvp3tSpbdpEfdZWuv12gnlnwhY=Each attempt with a new fingerprint resulted in similar authentication failures.
Environment Limitations
QEMU/ARM64 Specific:
Network Constraints:
10.0.2.15) in its own namespaceBeta Was this translation helpful? Give feedback.
All reactions