Nagios escalations are used to submit check results to remote systems.
The ufficial way to escalate betewen 2 nagios installations is using NSCA . NSCA is a deamon running on the remote nagios server, it listens on a TCP port and receives check results submitted from the local nagios (“escalated”) using send_nsca command to a passive check on the remote one.
Sometimes you do not wish to have and extra daemon on your remote server or , maybe due to firewall policies, you can only use http protocol to escalate a check result.
We have a “remote” nagios installation which is used by all sysadmins and a “local” installation only used by network people.
On both servers we use Centos 5 as OS and nagios from RPMForge as nagios versions.
What I needed was a quick and dirty way to escalate some “local” mayor network problems to a single remote passive check on the “remote” nagios to let everyone know about the problem.
“Local” Nagios setup
On the “local” nagios I created a new notification command with the following bash script:
#!/bin/bash
# notify nagios event using http
# /usr/lib/nagios/plugins/eventhandlers/notify-http
# this script uses 5 params:
hostname=$1
servicedesc=$2
servicestate=$3
output=$4
servicestatetype=$5
# exit if not in HARD state ( occurs after 2 SOFT states )
if [ "$servicestatetype" != "HARD" ]
then
exit 0
fi
#returncode must be numeric
case $servicestate in
"OK" )
returncode=0 ;;
"WARNING" )
returncode=1 ;;
"CRITICAL" )
returncode=2 ;;
"UNKNOWN" )
returncode=3 ;;
esac
#new output is servicedesc + old output + state
output="$servicestate $servicedesc $output"
# new service desc is remotepassivecheckname
servicedesc="remotepassivecheckname $hostname"
#hostname is remote nagios
hostname="remotenagiosname"
#urlencode output
output=$(echo -n "$output" | \
/usr/bin/perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');
/usr/bin/wget -q \
"http://remoteserver/cgi-bin/passv.pl?hostname=$hostname&servicedesc=$servicedesc&returncode=$returncode&output=$output" \
-O /dev/null 2>&1
The script uses some nagios standard macros as GET parameters to a remote perl cgi.
It escaltes to the remote installation only when the check gets into HARD state
It is registered in /etc/nagios/objects/commands.cfg with the following lines:
define command{
command_name http-notify
command_line /usr/lib/nagios/plugins/eventhandlers/notify-http "$HOSTNAME$" "$SERVICEDESC$" "$SERVICESTATE$" "$SERVICEOUT
PUT$" "$SERVICESTATETYPE$"
}
Every check tha thas to be escalated just needs the line
event_handler http-notify
into its service definition.
“Remote” nagios setup
On the remote instance I just created a properly named passive check linked to the remote nagios host.
The passive check is notified by the following cgi which uses the submit_check_result command.
#!/usr/bin/perl
# /var/www/cgi-bin/passv.pl
# submits a remote check to a local passive check
# using submit_check_result
use CGI;
$query = new CGI;
$hostname=$query->url_param('hostname');
$servicedesc=$query->url_param('servicedesc');
$returncode=$query->url_param('returncode);
$output=$query->url_param('output');
if ($esitochk =~ /CRITICAL/) {$esitochk=2;}
$myoutput=`/usr/lib/nagios/plugins/eventhandlers/submit_check_result $hostname $servicedesc $returncode $output`;
print "Content-type: text/html\n\n\n";
print "myoutput $myoutput";
I hope these suggestions will be useful to anyone , please let me know what you think about it.
