Resilience

It is essential that Smart Home systems are reliable.

Increasing Reliability of a single MisterHouse instance

The following techniques increase the reliability of a single MisterHouse installation;

• MisterHouse Watchdog
• Use a hardware watchdog e.g. The Raspberry Pi Watchdog
• Use SSD's instead of SD cards
• Have a backup/recovery strategy

With the widespread availability of very inexpensive, powerful computers like the Raspberry Pi, it is now viable to build in redundancy.

Simple Redundant design

This design has two identical Raspberry Pis with identical software, one with a hostname of Master and the other Slave'. It uses the hostname` in scripts and MisterHouse to decide how to behave.

The Slave machine runs a script run by crontab every few minutes to ping the Master and the Master runs the watchdog script to monitor itself e.g.

THISHOST=$(hostname -s)
case $THISHOST in
    Master )   
      #echo "Starting watchcron"     
      /home/pi/mh/scripts/watchcron.sh;;
   
    Slave )  
      #echo "Starting launchslave"       
      perl /home/pi/mh/scripts/launchslave.pl ;;
    * )               
      echo "ERROR: Invalid hostname"
      exit 1 ;;
esac

The Master watchcron.sh looks like this and restarts Misterhouse if the watchdog file is older than 5 minutes:

#!/bin/sh
WATCHDOGFILE=$HOME/mh/data/watchdog
# if watchdog hasn't been touched in 5 minutes, restart MisterHouse
MAXDIFFERENCE=300
watchdog=`stat --format=%Y $WATCHDOGFILE`;
now=`date +%s`;
difference=$(( $now - $watchdog))

if [ $difference -gt $MAXDIFFERENCE ]; then
 echo "Misterhouse needs restart"
 echo `date` Restarting Misterhouse >> $HOME/mh/data/logs/watchcron.log
 touch $WATCHDOGFILE
 sudo systemctl start misterhouse

fi

The launchslave.pl script does the following

Ping the masters http port
If the Master ping succeeds but MisterHouse is already running on the Slave, stop MisterHouse on the Slave and email what happened
If the Master ping succeeds copy over the mh_temp.saved_states so that if we need to satrt the Slave, it has a recent status
If the master ping failed and MisterHouse is not running on the Slave, start MisterHouse on the Slave and email what happened