2023-11-25 - Simple rolling reboot script

(outdated, superseded by proxmox rolling upgrade script)

This is the reboot function from a basic script I use to perform a rolling reboot of a cluster. The way you’d use it is by looping through a list of nodes, and calling the script like ./roll.sh ${NODE}.

#!/bin/bash
# roll.sh
set -euxo pipefail
NODE=$1
ssh ${NODE} reboot
# the following blocks until the reboot begins, and then loops until we can authenticate
ssh ${NODE} dmesg -w || until ssh ${NODE} whoami 
do
        sleep 1s
done

If you can safely reboot more than one node at a time in your cluster, you can use threading to call the script more than once. Exercise left to the reader, but I probably will eventually enhance this myself with a loop running up to MAX_OFFLINE_NODES threads at once.

If my content has been helpful to you, please consider buying me a coffee or