Safely updating nebula binaries

[DavidMSCode]

I couldn't find any info in the docs or when searching here for "update," so I may as well be the one to ask. How do I update Nebula to a new version on every host without risking a connection drop?

As an example, I have the following hosts at multiple sites. Most hosts I can get to on any given day, but there is one where it's far enough away that messing up this update would be a real pain. Every host is currently running Nebula 1.7.2 (except Windows which has been running a forked version with RIO) and I would like to update to 1.9.4

Site 1 (Physically accessible):
  Host 1: (Linux)
  Host 2: (Windows)

Site 2 (Virtual Server: No physical accessibility. Can SSH through known address)
  Lighthouse (Linux)

Site 3 (Can sometimes get to physically)
  Host 3 (Linux)

Roaming Hosts
  Host 4 (Mac OS)
  Host 5 (iOS)

My ideal scenario would be connecting to each host from my laptop using the overlay network and replacing the binaries and restarting the nebula service/daemon on each host without causing a disconnect. I am okay with a temporary outage while each service restarts.

Each linux host is managing the Nebula service through systemctl
The Mac OS host service is managed by launchctl
Windows has nebula register itself as a service using "nebula -service install"

Can I just delete the old binaries and put the new ones in their place telling each host to restart their Nebula service while Nebula is running? If this works, does it work on every OS?

I'm going to just give it a whirl on the hosts that I can physically get to since I can fix any issue today, but it would be nice to have this info written down somewhere. I imagine the safest thing to do would be to setup a shell script for each host that downloads the latest version, unzips it, stops the nebula service, replaces the binary, and restarts the service.

[DavidMSCode]

Performed two different updates successfully.

Mac OS host
I was surprised to find that Mac OS lets you just overwrite a file in use. Which is what I did. Launchctl saw the service interruption and restarted the service after a second. Even though I'm physically at my Mac laptop host I wanted to see what would happen while remotely conencted. So, I sshed in and when I ran the command to copy nebula over there was a hang while launchctl recovered. My specific steps were

cd ~/temp
mkdir nebula
cd nebula
wget https://github.com/slackhq/nebula/releases/download/v1.9.4/nebula-darwin.zip
unzip nebula-darwin.zip
sudo cp nebula /usr/local/bin
cd ..
rm -rf nebula

It would probably be better to stop and start the service before and after replacing the binary file but oh well.

Linux Host
I specifically made a shell script here cause Linux specifically stops you from overwriting a file in use. I assume I could force the copy but writing a quick shell script was easy enough. It assumes that the service controling nebula is called "nebula.service" and that the binary is being stored in /usr/local/bin.

mkdir -p ~/temp/nebula
cd ~/temp/nebula
touch update.sh

I then wrote the following shell script in update.sh

#!/bin/bash
if [ ! -f ./nebula ]
then
    echo "New nebula binary not found, aborting update."
    exit
fi
echo "Updating Nebula"
echo "Stopping Nebula Service"
systemctl stop nebula
echo "Deleting Nebula Binary"
rm /usr/local/bin/nebula
echo "Copying Nebula Binary to /usr/local/bin"
cp nebula /usr/local/bin
echo "Starting Nebula Service"
systemctl start nebula

Set the script to be executable, download the 1.9.4 nebula version, extract the tarball and run the script as su.

chmod +x update.sh
wget https://github.com/slackhq/nebula/releases/download/v1.9.4/nebula-linux-amd64.tar.gz
tar -xf nebula-linux-amd64.tar.gz
sudo ./update.sh