@LSPEMeetup made available the video on justin.tv; unfortunately the quality of the video/sound is not the best; you can find it here.

The daemons noitd and stratcond are written in C, and the database used is postgressql, while the web interface is written in php. We will need to install a few dependencies to be able to compile noitd/stratcond:

Dependencies

apt-get install autoconf build-essential \
zlib1g-dev uuid-dev libpcre3-dev libssl-dev libpq-dev \
libxslt-dev libapr1-dev libaprutil1-dev xsltproc \
libncurses5-dev python libssh2-1-dev libsnmp-dev \
sun-java6-jdk

In case you don’t have already installed svn, we will need that also (to download the sources):
apt-get install subversion

Compiling

Next we will download the source and compile it. I used the trunk version bellow, but you might want use a stable tag (Urskek is the latest when writing this):
cd /usr/local/src/
svn co https://labs.omniti.com/reconnoiter/trunk reconnoiter
cd reconnoiter
autoconf
./configure
make
make install

Database

We need to instal PostgreSQL 8.4 and this is not available in lenny; the simplest way we can do this is to use the backports package. To enable the backports sources add this line
deb http://www.backports.org/debian lenny-backports main contrib non-free
to your /etc/apt/sources.list and run apt-get update to refresh your apt indexes. And finally add the backports key into apt:
apt-get install debian-backports-keyring

Now we can install postgres 8.4 using:
apt-get -t lenny-backports install postgresql

Next we’ll need to edit /etc/postgresql/8.4/main/pg_hba.conf and replace this line
local all all identwith
local all all trust
and restart postgres:
/etc/init.d/postgresql-8.4 restart
to allow local access without password.

Next we can install the dbs, using:
su postgres
cd /usr/local/src/reconnoiter/sql
psql
\i scaffolding.sql
\q

DB crontab

You can find a sample crontab in the sql folder that needs to be customized for our particular setup (/opt/psql835/bin -> /usr/bin/), and it should look like this:

su postgres
crontab -l
# m h  dom mon dow   command
#rollup jobs
* * * * * /usr/bin/psql -d reconnoiter -U stratcon -c "select stratcon.rollup_metric_numeric(rollup) from metric_numeric_rollup_config order by seconds asc;" >/tmp/rollup.log 2>&1
#cleanup jobs
01 00 * * * /usr/bin/psql -d reconnoiter -U reconnoiter -c "select stratcon.archive_part_maint('noit.metric_numeric_archive', 'whence', 'day', 7);" 1>/dev/null
01 00 * * * /usr/bin/psql -d reconnoiter -U reconnoiter -c "select stratcon.archive_part_maint('noit.metric_text_archive', 'whence', 'day', 7);" 1>/dev/null
01 00 * * * /usr/bin/psql -d reconnoiter -U reconnoiter -c "select stratcon.archive_part_maint('noit.check_status_archive', 'whence', 'day', 7);" 1>/dev/null
01 00 * * * /usr/bin/psql -d reconnoiter -U reconnoiter -c "select stratcon.archive_part_maint('noit.check_archive', 'whence', 'day', 7);" 1>/dev/null
01 00 * * * /usr/bin/psql -d reconnoiter -U reconnoiter -c "delete from prism.saved_graphs where saved = false and last_update/dev/null

!!! I would suggest to run the daily crons manually once as they create the needed tables and in case you don’t have them you will receive various errors (I know I lost a lot of time trying to figure out why those were not there). This should be just like:
/usr/bin/psql -d reconnoiter -U reconnoiter -c "select stratcon.archive_part_maint('noit.metric_text_archive', 'whence', 'day', 7);"
and so on for each of the 5 daily crons from above.

keys

For a production install you will obviously create some real certs, but in my case where I was just testing this out the test certs build by the installer are just fine:
cd /usr/local/src/reconnoiter/test
cp test-*.crt test-*.key /usr/local/etc/

In dealing with the configuration files uuid’s you will find useful the uuidgen program from the uuid-runtime package:
apt-get install uuid-runtime
that will generate a new uuid on each run:
uuidgen
22426aa0-db34-437d-a891-e759ffbdb494

noitd

We will start from the sample configuration:
mv /usr/local/etc/noit.conf.sample /usr/local/etc/noit.conf
and edit/remove it as needed. Be sure to edit the sslconfig part with the keys you copied above. For more details on the config file look on the wiki.

Finally run noitd and see if all is good:
/usr/local/sbin/noitd -c /usr/local/etc/noit.conf -D
If there are no errors stop it, and run it in the background (without -D):
/usr/local/sbin/noitd -c /usr/local/etc/noit.conf

stratcon

For stratcon we are going to use a similar approach:
mv /usr/local/etc/stratcon.conf.sample /usr/local/etc/stratcon.conf
and then customize the config based on our needs. Set the keys in sslconfig section and the db details

    <dbconfig>
      <host>localhost</host>
      <dbname>reconnoiter</dbname>
      <user>stratcon</user>
      <password>unguessable</password>
    </dbconfig>    

the password is “stratcon” or whatever you set in scaffolding.sql if you customized it (a good idea ).
and finally run it with debuging in the foreground:
/usr/local/sbin/stratcond -c /usr/local/etc/stratcon.conf -D
and if all is good (you did ran the db creation task, right?) you can start it in the background without -D and move to the final step:
/usr/local/sbin/stratcond -c /usr/local/etc/stratcon.conf

Web interface

Finally the last step is to configure the web interface. For this we need to be sure we have apache/php installed and the pgsql module:
apt-get install apache2 libapache2-mod-php5 php5-pgsql
and to enable the rewrite module
a2enmod rewrite
Next we would copy the web files ui to some place like /var/www/ui and create a vhost for reconnoiter inside /etc/apache2/sites-available:

<VirtualHost *:80>
  DocumentRoot /var/www/ui/web/htdocs

  <Directory "/">
      Options None
      AllowOverride None
      Order allow,deny
      Deny from all
  </Directory>
  <FilesMatch "^\.ht">
      Order allow,deny
      Deny from all
      Satisfy All
  </FilesMatch>
  <Directory "/var/www/ui/web/htdocs/">
      php_value include_path /var/www/ui/web/lib
      php_value short_open_tag off
      Options FollowSymLinks Indexes
      AllowOverride All
      Order deny,allow
      Allow from all
  </Directory>

  LogLevel warn
  LogFormat "%h %l %u %t \"%r\" %>s %b" common

  ErrorLog /var/www/ui/web/logs/error_log
  CustomLog /var/www/ui/web/logs/access_log common

  AddType application/x-compress .Z
  AddType application/x-gzip .gz .tgz
  AddType application/x-httpd-php .php
  DefaultType text/plain

  RewriteEngine On
  RewriteRule ^(/data/.+)$ http://localhost:80$1 [P,L,QSA]
</VirtualHost>

enable the config:
a2ensite reconnoiter
and restart apache to enable it: /etc/init.d/apache2 restart

After this hopefully you can open the web interface and start adding some metrics. My goal was to test this with collectd and if there is interest I could describe the details on how to make reconnoiter work with collectd in another post. let me know…

All I had to do was to change from regular 32-bit counters to 64-bit counters and increase the maximum value to 1M. I have done that, but to my surprise the graph stopped working completely for the respective interface. So I had to dig into it, and debug the problem. I was able to see that the cacti poller snmp query was failing and returning:
Error in packet
Reason: (noSuchName) There is no such variable name in this MIB.
Failed object: IF-MIB::ifHCInOctets.10124

That was strange, as I knew for sure a similar cisco switch works just fine with 64-bit counters (btw this was a C2960G). Running the snmp query by hand, like:
snmpget -c <comunity> -v 2c <ip> .1.3.6.1.2.1.31.1.1.1.6.10124
IF-MIB::ifHCInOctets.10124 = Counter64: 1741880465459
was working just fine and returning the counter. Finally after wasting some time I realized what was the problem: the host device was configured in cacti to use SNMPv1, and I only had to change this to SNMPv2 to make it work as expected. Hopefully this will help others in similar situations…

Conclusion: if your 64-bit snmp counters are not working as expected, be sure to check if you are using SNMPv2 or not. That might be your problem, as SNMPv1 doesn’t support that.

edit: lib/database.php and replace PConnect with Connect

if ($cnn_id->PConnect($hostport,$user,$pass,$db_name)) {

Again, this is just for the web interface and if you don’t have any problems or particular reasons to change this, it is better to leave it as it is. For example I had to do this on a server where mysql had problems with persistent connections as those were affecting the proper functionality of the site.

1. Centreon / Oreon – http://www.centreon.com/
Centreon is a network, system, applicative supervision and monitoring tool, it is based upon the most effective Open Source monitoring engine : Nagios. Centreon provides a new frontend and new functionalities to Nagios.

2. Fruity – http://fruity.sourceforge.net/
Fruity is a web based application (PHP & MySQL) to make Nagios configuration easier. It provides a logical flow of your hosts with services and creates relationships between objects. It stores it’s configuration internally into a database and when you are satisfied with your configuration, you can then export it to nagios’s configuration files.

3. Monarch – GroundWork Monitor Architect – http://sourceforge.net/projects/monarch
GroundWork Monitor Architect (Monarch) is a full-featured, easy-to-use configuration system for use with Nagios. Monarch consists of a set of tools that allow a user to easily configure and maintain Nagios, or GroundWork Monitor, without editing the Nagios configuration files directly.

4. NagiosQL – http://www.nagiosql.org
NagiosQL is a web based administration tool for Nagios 2.x. It helps you to easy build a complex configuration with all options, manage them and use them. NagiosQL needs a webserver with php, MySQL and file access to the nagios configuration files.

5. NagiosWeb – http://sourceforge.net/projects/nagiosweb/
A PHP/MySQL based front-end for Nagios 2.x setup. Complete with the ability to add/remove hosts, hostgroups, contacts, contactgroups, services, timeperiods and commands.

6. NCPL – http://sourceforge.net/projects/ncpl/
NCPL is a webbased solution written in PERL for configuring Nagios. NCPL gives you centralized control over remote Nagios systems from a central server and permits to edit and install new config files. It uses MySQL to store configuration data.

7. Nagios Config – http://nagiosconfig.com/
Nagios Config is a FREE web front end to Nagios, written in PHP.

8. NagMIN – http://nagmin.sourceforge.net/
NagMIN is a WebMIN module that provides centralized, integrated, web-based management of popular Open Source monitoring tools to help create a holistic network monitoring environment.

9. nag2web – http://nag2web.de/
A PHP/MySQL web front-end for Nagios 2.x setup. With the ability to add/remove/edit the config items. The web front-end for NAGIOS has the ability to test the config. We want to build a simple possibility to find and show all the features in NAGIOS.

10. NaWui – http://www.stigma.dyndns.org/NaWui
NaWui stands for Nagios Administration Web User Interface.It is a Web frontend written in PHP to manage Nagios configurations and Database records. Database connectivity is supported using plugins, which simplifies the way records are stored.

Source, where you can find many nagios interesting stuff: http://www.nagiosexchange.org/

ps: If you think I have missed a similar project and you want it added to this page, please feel free to use the contact form or the comments to let me know.