Setting up NRPE daemon

Source code: http://nagios.org/

To install, add the edge testing repository, then run:

apk add nrpe

Set up config file to bind to local IP, only allow needed hosts to connect (in our case, our redundant Nagios servers). Add a definition for a check command to /etc/nrpe.cfg, for example:

command[check_routes]=/usr/bin/check_routes.sh

Create the above script, and populate (code below needs a cleanup, but was done more as proof-of-concept):

##!/bin/ash
##
## Script to check whether routes to branches are being received properly
NUMROUTES=`ip route | grep -n  | awk -F ':' '{print $1}' | tail -n 1`
if [[ $NUMROUTES > 80 ]];
then echo "OK: $NUMROUTES routes in routing table" && exit 0;
elif [[ $NUMROUTES < 80 ]] && [[ $NUMROUTES > 15 ]];
then echo "WARNING: $NUMROUTES routes in routing table" && exit 1;
elif $NUMROUTES = "" ;
then echo "WARNING: No routing information received" && exit 1;
else echo "CRITICAL: $NUMROUTES routes in routing table" && exit 2;
fi

Restart NRPE. Allow port 5666 through Shorewall (in /etc/shorewall/rules) through to monitoring hosts. On the monitoring host, run the following command to test:

root#/usr/local/nagios/libexec/check_nrpe -H 10.14.8.3 -p 5666 -c check_routes

You should get output like:

OK: 173 routes in routing table

If you are having trouble, enable debugging in /etc/nrpe.cfg, and check /var/log/messages for errors. Most likely error(s) has to do with permissions of what you are trying to execute. Remember that scripts are executed as nagios user, not root, so keep the "fancy tricks" to a minimum.