Installing nagios with nginx in amazon EC2 or CentOS6

Nagios is a great monitoring system which is also distributed as rpm package in some repositories.

All  rpms , as far as I know , are preconfigured to be used with apache as a webserver.

I recently been using nginx  on some linux servers in Amazon VPC  because it’s very efficient and uses resources quite lightly, allowing more users for the same server.

I’d like to share my configuration to use amazon stock nginx, php-fpm and nagios rpm packages from amazon repositories, this should be also very easyly portable on CentOS/ RHEL 6 with epel and maybe rpmforge repos.

The tricky part in the installation is that the latest nagios uses both php and CGIs , they are usually handled by nginx as FastCGI applications.

There are a lot of possible implementations, I choose php-fpm (with php 5.4) for php pages and a perl fastcgi-wrapper.pl script I found mentioned here, here you can download the script.

First of all I  yum installed  nginx, php54-fpm , all php54* packages I needed, nagios, nagios-plugins-all , perl-FCGI

I configured php-fpm to use a unix socket in  /etc/php-fpm.d/www.conf   with :

listen = /tmp/phpfpm.sock

while my fastcgi wrapper in /usr/local/sbin/fastcgi-wrapper.pl  contains an IP socket:

        $socket = FCGI::OpenSocket( "127.0.0.1:8999", 10 ); #use IP sockets

You can start php-fpm as a service with service php-fpm start

While you can control the fastcgi-wrapper.pl script with upstart by placing the file   /etc/init/fastcgi-wrapper.conf  with this content

start on runlevel [2345]
stop on runlevel [016]
respawn

exec /usr/bin/perl /usr/local/sbin/fastcgi-wrapper.pl

and issueing initctl start fastcgi-wrapper .

This is the nginx virtualhost in /etc/nginx/conf.d/mynagios.mydomain.conf

server {

        server_name mynagios.mydomain.com;
        root /usr/share/nagios/html; 

        error_log  /var/log/nginx/nagios.error.log ;
        access_log  /var/log/nginx/nagios.access.log  main;

        expires 31d;
        index index.php index.html;

  auth_basic            "Restricted";
  auth_basic_user_file  /etc/nginx/htpasswd;

        location /nagios/stylesheets {
                alias /usr/share/nagios/html/stylesheets;
        }

        location /nagios/js {
                alias /usr/share/nagios/html/js;
        }

        location /nagios/images {
                alias /usr/share/nagios/html/images;         
        }

 location ~ \.cgi$ {
        root /usr/lib64/nagios/cgi-bin;

        rewrite ^/cgi-bin/(.*)$ /$1;
        include /etc/nginx/fastcgi_params;
        fastcgi_param AUTH_USER $remote_user;
        fastcgi_param REMOTE_USER $remote_user;
        fastcgi_param SCRIPT_FILENAME /usr/lib64/$fastcgi_script_name;
        fastcgi_pass 127.0.0.1:8999;
        }

        location ~ \.php$ {
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_pass unix:/tmp/phpfpm.sock;
        }

}

As you can see the whole virtualhost is behind basic authentication.

Please let me know what you think of this, is there any better way to achieve the same?

Cheers
G.

Nagios Escalations using http and cgi

Nagios escalations are used to submit check results to remote systems.
The ufficial way to escalate betewen 2 nagios installations is using NSCA . NSCA is a deamon running on the remote nagios server, it listens on a TCP port and receives check results submitted from the local nagios (“escalated”) using send_nsca command to a passive check on the remote one.

Sometimes you do not wish to have and extra daemon on your remote server or , maybe due to firewall policies, you can only use http protocol to escalate a check result.

We have a “remote” nagios installation which is used by all sysadmins and a “local” installation only used by network people.

On both servers we use Centos 5 as OS and nagios from RPMForge as nagios versions.

What I needed was a quick and dirty way to escalate some “local” mayor network problems to a single remote passive check on the “remote” nagios to let everyone know about the problem.

“Local” Nagios setup

On the “local” nagios I created a new notification command with the following bash script:

#!/bin/bash
# notify nagios event using http
# /usr/lib/nagios/plugins/eventhandlers/notify-http 

# this script uses 5 params:
hostname=$1
servicedesc=$2
servicestate=$3
output=$4
servicestatetype=$5

# exit if not in HARD state ( occurs after 2 SOFT states )
if [ "$servicestatetype" != "HARD" ]
then
exit 0
fi

#returncode must be numeric
case $servicestate in
 "OK" )
 returncode=0 ;;
 "WARNING" )
 returncode=1 ;;
 "CRITICAL" )
 returncode=2 ;;
 "UNKNOWN" )
 returncode=3 ;;
esac

#new output is servicedesc + old output + state
output="$servicestate $servicedesc $output"
# new service desc is remotepassivecheckname
servicedesc="remotepassivecheckname $hostname"

#hostname is remote nagios
hostname="remotenagiosname"

#urlencode output
output=$(echo -n "$output" | \
/usr/bin/perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');

/usr/bin/wget -q \
"http://remoteserver/cgi-bin/passv.pl?hostname=$hostname&servicedesc=$servicedesc&returncode=$returncode&output=$output" \
-O /dev/null 2>&1

The script uses some nagios standard macros as GET parameters to a remote perl cgi.

It escaltes to the remote installation only when the check gets into HARD state

It is registered in /etc/nagios/objects/commands.cfg with the following lines:

define command{
 command_name    http-notify
 command_line    /usr/lib/nagios/plugins/eventhandlers/notify-http "$HOSTNAME$" "$SERVICEDESC$" "$SERVICESTATE$" "$SERVICEOUT
PUT$" "$SERVICESTATETYPE$"
 }

Every check tha thas to be escalated just needs the line

event_handler           http-notify

into its service definition.

“Remote” nagios setup

On the remote instance I just created a properly named passive check linked to the remote nagios host.

The passive check is notified by the following cgi which uses the submit_check_result command.

#!/usr/bin/perl
# /var/www/cgi-bin/passv.pl
# submits a remote check to a local passive check
# using submit_check_result

use CGI;
$query = new CGI;
$hostname=$query->url_param('hostname');
$servicedesc=$query->url_param('servicedesc');
$returncode=$query->url_param('returncode);
$output=$query->url_param('output');
if ($esitochk =~ /CRITICAL/) {$esitochk=2;}
$myoutput=`/usr/lib/nagios/plugins/eventhandlers/submit_check_result $hostname $servicedesc $returncode $output`;
print "Content-type: text/html\n\n\n";
print "myoutput $myoutput";

I hope these suggestions will be useful to anyone , please let me know what you think about it.

porting phpfreechat from Mysql to Oracle

I’ve been recently playing with the porting to Oracle of a really nice ajax php chat application :  phpfreechat or PfC for short.
This application makes very easy to integrate new types of persistance containers because it abstracts them in class implementations that provide just 3 methods that have to be adapted to the new type of container, and you just need a single table in the database.

I used pear DB classes to get access to the database.

Of course, php has to be compiled with oci extensions that require at least an instant client installation.

I used oracle express edition as DB server on my development server, it is free (as in beer) for development use, and very light on resources.

I had to convert datatypes from Mysql to Oracle and I confess I used phpbb3 oracle sql schema as an inspiration.

Oracle has no autoincrement types so you have to use sequence (+ triggers).

As far as I know, there is no timestamp type (seconds from the epoch)  as well  so I had to improvise some PL/SQL.

Another issue that I had to struggle with is timezones : UTC vs your local time, be careful to calculate time always in the same way.

Now we have a prototype, that is currently in trunk.
It certainly needs some testing and some advice from skilled oracle users, please try it out and let me know any issue, suggestion, idea, insults….

The quickest way to test it is : download the latest stable pfc, get oracle.class.php , drop it in the containers dir , create database and parameters as explained in php file.

useful apache modules for your reverse proxy

There are a couple of reasons why you might need to reverse proxy a web site:  security, high availability, etc.

My favorite software to implement a reverse proxy is ( of course ) apache.

Some modules that allow this possibility are mod_proxy and mod_rewrite, usually you can use both of them for the more flexible results.

Mod_proxy lets you to publish internal http,ftp and AJP (= tomcat webapps) sites while mod_rewrite can be a godsend in rewriting urls , for example in SEO friendly urls or to redirect users from moved applications.
Mod_rewrite has also proxying capabilities but the latest mod_proxy has better features like caching and load balancing.

All this modules cannot change the content of the proxyed pages while sometimes this could be needed, you might have a proxied application that contains hardcoded absolute URLs which are different from you reverse proxy.

The most used module to change some strings in the output of your reverse proxy is mod_proxy_html.
This module allows you to intercept and substitute or add strings in the html produced by the proxyed application, it can be used together with mod_proxy to provide a more functional reverse proxy.

Sometimes mod_proxy_html power is still insufficient, that might happen because it fails to understand the html markup or scripting produced by the proxyed application.
In these cases I found another module that will be part of the next apache release and is currently in trunk: mod_sed.

Beware that mod_sed is still in its infancy (I would not recommend it in a big web site of a major italian financial institution) but I found it very handy when other solutions couldn’t globally replace some patterns.

One final suggestion is mod_security .
This great piece of software in an application level firewall: it (tries to) filter any malicious request before fowarding the request to the proxyed application.
Sometimes you just do not feel very comfortable about the security of some major php application or you do not trust blindly the latest microsoft technology du jour, this little friend could be some extra layer of security you were looking for…

That’s it for now, please let me know what do you think or your revproxying experiences

PS : ` Despite the tons of examples and docs, mod_rewrite is voodoo. Damned cool voodoo, but still voodoo. ”

— Brian Moore
bem@news.cmc.net (quoted from mod_rewrite docs)