Installing nagios with nginx in amazon EC2 or CentOS6

November 10, 2013

Nagios is a great monitoring system which is also distributed as rpm package in some repositories.

All  rpms , as far as I know , are preconfigured to be used with apache as a webserver.

I recently been using nginx  on some linux servers in Amazon VPC  because it’s very efficient and uses resources quite lightly, allowing more users for the same server.

I’d like to share my configuration to use amazon stock nginx, php-fpm and nagios rpm packages from amazon repositories, this should be also very easyly portable on CentOS/ RHEL 6 with epel and maybe rpmforge repos.

The tricky part in the installation is that the latest nagios uses both php and CGIs , they are usually handled by nginx as FastCGI applications.

There are a lot of possible implementations, I choose php-fpm (with php 5.4) for php pages and a perl fastcgi-wrapper.pl script I found mentioned here, here you can download the script.

First of all I  yum installed  nginx, php54-fpm , all php54* packages I needed, nagios, nagios-plugins-all , perl-FCGI

I configured php-fpm to use a unix socket in  /etc/php-fpm.d/www.conf   with :

listen = /tmp/phpfpm.sock

while my fastcgi wrapper in /usr/local/sbin/fastcgi-wrapper.pl  contains an IP socket:

        $socket = FCGI::OpenSocket( "127.0.0.1:8999", 10 ); #use IP sockets

You can start php-fpm as a service with service php-fpm start

While you can control the fastcgi-wrapper.pl script with upstart by placing the file   /etc/init/fastcgi-wrapper.conf  with this content

start on runlevel [2345]
stop on runlevel [016]
respawn

exec /usr/bin/perl /usr/local/sbin/fastcgi-wrapper.pl

and issueing initctl start fastcgi-wrapper .

This is the nginx virtualhost in /etc/nginx/conf.d/mynagios.mydomain.conf

server {

        server_name mynagios.mydomain.com;
        root /usr/share/nagios/html; 

        error_log  /var/log/nginx/nagios.error.log ;
        access_log  /var/log/nginx/nagios.access.log  main;

        expires 31d;
        index index.php index.html;

  auth_basic            "Restricted";
  auth_basic_user_file  /etc/nginx/htpasswd;

        location /nagios/stylesheets {
                alias /usr/share/nagios/html/stylesheets;
        }

        location /nagios/js {
                alias /usr/share/nagios/html/js;
        }

        location /nagios/images {
                alias /usr/share/nagios/html/images;         
        }

 location ~ \.cgi$ {
        root /usr/lib64/nagios/cgi-bin;

        rewrite ^/cgi-bin/(.*)$ /$1;
        include /etc/nginx/fastcgi_params;
        fastcgi_param AUTH_USER $remote_user;
        fastcgi_param REMOTE_USER $remote_user;
        fastcgi_param SCRIPT_FILENAME /usr/lib64/$fastcgi_script_name;
        fastcgi_pass 127.0.0.1:8999;
        }

        location ~ \.php$ {
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_intercept_errors on;
                fastcgi_pass unix:/tmp/phpfpm.sock;
        }

}

As you can see the whole virtualhost is behind basic authentication.

Please let me know what you think of this, is there any better way to achieve the same?

Cheers
G.

Nagios Escalations using http and cgi

December 26, 2009

Nagios escalations are used to submit check results to remote systems.
The ufficial way to escalate betewen 2 nagios installations is using NSCA . NSCA is a deamon running on the remote nagios server, it listens on a TCP port and receives check results submitted from the local nagios (“escalated”) using send_nsca command to a passive check on the remote one.

Sometimes you do not wish to have and extra daemon on your remote server or , maybe due to firewall policies, you can only use http protocol to escalate a check result.

We have a “remote” nagios installation which is used by all sysadmins and a “local” installation only used by network people.

On both servers we use Centos 5 as OS and nagios from RPMForge as nagios versions.

What I needed was a quick and dirty way to escalate some “local” mayor network problems to a single remote passive check on the “remote” nagios to let everyone know about the problem.

“Local” Nagios setup

On the “local” nagios I created a new notification command with the following bash script:

#!/bin/bash
# notify nagios event using http
# /usr/lib/nagios/plugins/eventhandlers/notify-http 

# this script uses 5 params:
hostname=$1
servicedesc=$2
servicestate=$3
output=$4
servicestatetype=$5

# exit if not in HARD state ( occurs after 2 SOFT states )
if [ "$servicestatetype" != "HARD" ]
then
exit 0
fi

#returncode must be numeric
case $servicestate in
 "OK" )
 returncode=0 ;;
 "WARNING" )
 returncode=1 ;;
 "CRITICAL" )
 returncode=2 ;;
 "UNKNOWN" )
 returncode=3 ;;
esac

#new output is servicedesc + old output + state
output="$servicestate $servicedesc $output"
# new service desc is remotepassivecheckname
servicedesc="remotepassivecheckname $hostname"

#hostname is remote nagios
hostname="remotenagiosname"

#urlencode output
output=$(echo -n "$output" | \
/usr/bin/perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg');

/usr/bin/wget -q \
"http://remoteserver/cgi-bin/passv.pl?hostname=$hostname&servicedesc=$servicedesc&returncode=$returncode&output=$output" \
-O /dev/null 2>&1

The script uses some nagios standard macros as GET parameters to a remote perl cgi.

It escaltes to the remote installation only when the check gets into HARD state

It is registered in /etc/nagios/objects/commands.cfg with the following lines:

define command{
 command_name    http-notify
 command_line    /usr/lib/nagios/plugins/eventhandlers/notify-http "$HOSTNAME$" "$SERVICEDESC$" "$SERVICESTATE$" "$SERVICEOUT
PUT$" "$SERVICESTATETYPE$"
 }

Every check tha thas to be escalated just needs the line

event_handler           http-notify

into its service definition.

“Remote” nagios setup

On the remote instance I just created a properly named passive check linked to the remote nagios host.

The passive check is notified by the following cgi which uses the submit_check_result command.

#!/usr/bin/perl
# /var/www/cgi-bin/passv.pl
# submits a remote check to a local passive check
# using submit_check_result

use CGI;
$query = new CGI;
$hostname=$query->url_param('hostname');
$servicedesc=$query->url_param('servicedesc');
$returncode=$query->url_param('returncode);
$output=$query->url_param('output');
if ($esitochk =~ /CRITICAL/) {$esitochk=2;}
$myoutput=`/usr/lib/nagios/plugins/eventhandlers/submit_check_result $hostname $servicedesc $returncode $output`;
print "Content-type: text/html\n\n\n";
print "myoutput $myoutput";

I hope these suggestions will be useful to anyone , please let me know what you think about it.

porting phpfreechat from Mysql to Oracle

October 3, 2009

I’ve been recently playing with the porting to Oracle of a really nice ajax php chat application :  phpfreechat or PfC for short.
This application makes very easy to integrate new types of persistance containers because it abstracts them in class implementations that provide just 3 methods that have to be adapted to the new type of container, and you just need a single table in the database.

I used pear DB classes to get access to the database.

Of course, php has to be compiled with oci extensions that require at least an instant client installation.

I used oracle express edition as DB server on my development server, it is free (as in beer) for development use, and very light on resources.

I had to convert datatypes from Mysql to Oracle and I confess I used phpbb3 oracle sql schema as an inspiration.

Oracle has no autoincrement types so you have to use sequence (+ triggers).

As far as I know, there is no timestamp type (seconds from the epoch)  as well  so I had to improvise some PL/SQL.

Another issue that I had to struggle with is timezones : UTC vs your local time, be careful to calculate time always in the same way.

Now we have a prototype, that is currently in trunk.
It certainly needs some testing and some advice from skilled oracle users, please try it out and let me know any issue, suggestion, idea, insults….

The quickest way to test it is : download the latest stable pfc, get oracle.class.php , drop it in the containers dir , create database and parameters as explained in php file.

useful apache modules for your reverse proxy

September 19, 2009

There are a couple of reasons why you might need to reverse proxy a web site:  security, high availability, etc.

My favorite software to implement a reverse proxy is ( of course ) apache.

Some modules that allow this possibility are mod_proxy and mod_rewrite, usually you can use both of them for the more flexible results.

Mod_proxy lets you to publish internal http,ftp and AJP (= tomcat webapps) sites while mod_rewrite can be a godsend in rewriting urls , for example in SEO friendly urls or to redirect users from moved applications.
Mod_rewrite has also proxying capabilities but the latest mod_proxy has better features like caching and load balancing.

All this modules cannot change the content of the proxyed pages while sometimes this could be needed, you might have a proxied application that contains hardcoded absolute URLs which are different from you reverse proxy.

The most used module to change some strings in the output of your reverse proxy is mod_proxy_html.
This module allows you to intercept and substitute or add strings in the html produced by the proxyed application, it can be used together with mod_proxy to provide a more functional reverse proxy.

Sometimes mod_proxy_html power is still insufficient, that might happen because it fails to understand the html markup or scripting produced by the proxyed application.
In these cases I found another module that will be part of the next apache release and is currently in trunk: mod_sed.

Beware that mod_sed is still in its infancy (I would not recommend it in a big web site of a major italian financial institution) but I found it very handy when other solutions couldn’t globally replace some patterns.

One final suggestion is mod_security .
This great piece of software in an application level firewall: it (tries to) filter any malicious request before fowarding the request to the proxyed application.
Sometimes you just do not feel very comfortable about the security of some major php application or you do not trust blindly the latest microsoft technology du jour, this little friend could be some extra layer of security you were looking for…

That’s it for now, please let me know what do you think or your revproxying experiences

PS : ` Despite the tons of examples and docs, mod_rewrite is voodoo. Damned cool voodoo, but still voodoo. ”

– Brian Moore
bem@news.cmc.net (quoted from mod_rewrite docs)

php + owa 2003 calendar fun

September 13, 2009

Hi,
I’ve been recently playing with WEBDAV integration between OWA 2003 ( Microsoft Exchange Outlook Web Access) and php.
Looks like Troy Wolf has made an excellent work creating some classes to provide WEBDAV methods , xml parsing and also Exchange 2003 integration examples.

I had specific needs in calendar integration and I found some good extra work provided by Ryan Hadley.

Looks like he made an excellent implementation of exchange calendar to ics format conversion.
The small bit I still needed was the creation of appointments using WEBDAV, like it is explained here.
After applying all my google-fu I still could not find any php examples  so I had to adapt visual basic and other dialects to php….

this is a working example:

<?php

// Modify the path to this class file as needed.
require_once("class_http.php");

// Change these values for your Exchange Server.
$exchange_server = "http://exchange";
$admin_username = "exchadministrator";
$admin_password = "password";
$partecipants="user1,user2";
$exchange_username="user1";
// We use Troy's http class object to send the XML-formatted WebDAV request
// to the Exchange Server and to receive the response from the Exchange Server.
// The response is also XML-formatted.
$h = new http();
$h->headers["Content-Type"] = 'text/xml; charset="UTF-8"';
$h->headers["Depth"] = "0";
$h->headers["Translate"] = "f";

// Build the XML request.
// This section must be against the left margin.

$golemreq= <<<END
<?xml version="1.0"?>
<g:propertyupdate xmlns:g="DAV:"
xmlns:e="http://schemas.microsoft.com/exchange/"
xmlns:mapi="http://schemas.microsoft.com/mapi/"
xmlns:mapit="http://schemas.microsoft.com/mapi/proptag/" xmlns:x="xml:"
xmlns:cal="urn:schemas:calendar:"
xmlns:dt="urn:uuid:c2f41010-65b3-11d1-a29f-00aa00c14882/"
xmlns:header="urn:schemas:mailheader:" xmlns:mail="urn:schemas:httpmail:">
<g:set>
<g:prop>
<g:contentclass>urn:content-classes:appointment</g:contentclass>
<e:outlookmessageclass>IPM.Appointment</e:outlookmessageclass>

END;

$strMailInfo = "<mail:subject>Test Appointment Subject</mail:subject>\n";
$strMailInfo .= "<mail:htmldescription>Lets meet here</mail:htmldescription>\n";

$strCalInfo = "<cal:location>meetappt Location</cal:location>\n";
$strCalInfo .="<cal:dtstart dt:dt=\"dateTime.tz\">2009-11-18T23:00:00.000Z</cal:dtstart>\n";
$strCalInfo .="<cal:dtend dt:dt=\"dateTime.tz\">2009-11-18T23:30:00.000Z</cal:dtend>\n";
$strCalInfo .="<cal:instancetype dt:dt=\"int\">0</cal:instancetype>\n";
$strCalInfo .="<cal:busystatus>BUSY</cal:busystatus>\n";
$strCalInfo .="<cal:meetingstatus>CONFIRMED</cal:meetingstatus>\n";
$strCalInfo .="<cal:alldayevent dt:dt=\"boolean\">0</cal:alldayevent>\n";
$strCalInfo .="<cal:responserequested dt:dt=\"boolean\">1</cal:responserequested>\n";
$strCalInfo .="<cal:reminderoffset dt:dt=\"int\">900</cal:reminderoffset>\n";

$strHeaderInfo = "<header:to>".$partecipants."</header:to>\n";

$golemreq .= $strMailInfo . $strCalInfo . $strHeaderInfo ;
$golemreq .= "<mapi:finvited dt:dt=\"boolean\">1</mapi:finvited>\n";
$golemreq .= "</g:prop>\n</g:set>\n";
$golemreq .= "</g:propertyupdate>";

$h->xmlrequest = $golemreq;

// The http object's 'fetch' method does the work of sending and receiving the
// request. We use the WebDAV PROPPATCH method to create or update Exchange items.
//$url = $exchange_server."/public/Company%20Contacts/john%20doe.EML";
$url = $exchange_server."/exchange/$exchange_username/Calendar/testapp.eml";
if (!$h->fetch($url, 0, null, $admin_username, $admin_password, "PROPPATCH")) {
 echo "<h2>There is a problem with the http request!</h2>";
 echo $h->log;
 exit();
}

// You can print out the response to help troubleshoot.
echo "debug \n";
echo "<pre>".$h->header."</pre><hr />\n";
echo "<pre>".$h->body."</pre><hr />\n";

?>

It took a while to get the proper xml namespaces so I hope that might be of some use to someone else…
Btw the administrator account needs explicit permission to write into the user calendar, beware also that the calendar is created at the first time the user writes something into it and the name is dependant on the language used by the calendar ( “Calendar” is just for english users).

I am afraid all this will not work on recent ( exchange 2007 and over) versions because there is a new (and  maybe saner) web services interface to exchange.
Please let me know what you think….

Remotely Install VNC on windows server from linux box

September 12, 2009

Have you ever needed to remotely access a windows box with a borked terminal server and that you would not/could not rather reboot?

That’s exactly what happened me a couple of days ago….

My idea was to remotely install and start a vnc server.

After some googling I managed to fish out this and this

Which is a good start,  the problem is  the windows  server was reachable through ssh on a centos box ( port forwarding terminal server port tcp 3389 inside the ssh connection).
So I had to get the ” Complete set of executables, no installer” version from tightvnc

Unpack it, and get the files (WinVNC.exe, VNCHooks.dll) on the centos box.
Then I copied the files to win server on the ADMIN$ share using:

smbclient //10.0.0.5/ADMIN$ -U administrator

I managed then to get a remote prompt using

winexe -U WINDOWSDOMAIN/Administrator%password' //10.0.0.5 "cmd.exe"

BTW winexe is great , watch out that on centos/RHEL linux the static compiled version is not compatible with the OS libc…

I had to compile it , untarring sources and with a :

autogen.sh
configure
make proto bin/winexe

then, on the windows prompt  you need to install the service with

winvnc.exe -service 

please note that the service won’t allow logins until you set a password, what I did is exporting a vnc config from another windows machine and merge it into the registry:

this one for example sets the password to   secret

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\ORL\WinVNC3\Default]
"SocketConnect"=dword:00000001
"AutoPortSelect"=dword:00000001
"InputsEnabled"=dword:00000001
"LocalInputsDisabled"=dword:00000000
"IdleTimeout"=dword:00000000
"LocalInputsPriorityTime"=dword:00000003
"QuerySetting"=dword:00000002
"QueryTimeout"=dword:0000001e
"QueryAccept"=dword:00000000
"QueryAllowNoPass"=dword:00000000
"LockSetting"=dword:00000000
"RemoveWallpaper"=dword:00000001
"BlankScreen"=dword:00000000
"EnableFileTransfers"=dword:00000001
"Password"=hex:2e,2d,bf,57,6e,b0,6c,9e
"PasswordViewOnly"=hex:2e,2d,bf,57,6e,b0,6c,9e
"PollUnderCursor"=dword:00000000
"PollForeground"=dword:00000001
"PollFullScreen"=dword:00000000
"OnlyPollConsole"=dword:00000001
"OnlyPollOnEvent"=dword:00000000
"PollingCycle"=dword:0000012c
"DontSetHooks"=dword:00000000
"DontUseDriver"=dword:00000000
"DriverDirectAccess"=dword:00000001
"LocalInputsPriority"=dword:00000000

The idea was to copy it on the windows box in a vnc.reg file and then use:

regedit /s vnc.reg

finally you can remotely start the service in the windows prompt using:

net start winvnc

If you forward your local port 5900 inside the ssh connection on the remote windows machine and connect with a vnc client to localhost you will magically reach the remote server.

PS:

I noticed that on windows, the remote forwarding trick for terminal server connections using localhost is not possible anymore with the new mstsc.exe client , remember to keep an old version around.

I have 5.0.2195.6674 version which still works, 6.0.6001.18000 from windows xp sp3 it is not, or please check out this

Wow that was my first post, geeky enough isn’t it?



Follow

Get every new post delivered to your Inbox.

Join 274 other followers