Adding Cacti Logs to Splunk

This basically the same process as adding the nagios logs, but I’ll put it up anyway.

1. Click Manager in the upper right-hand corner of Splunk Web.
2. Under System configurations, click Data Inputs.
3. Click Files and directories.
4. Click New to add an input
I choose Monitor a file or directory
6. Specify the path to the file:
With my setup it is:
/Library/WebServer/Documents/cacti/log/cacti.log
7. Under Host Heading
I choose constant value
8. Under Source Type
I choose Automatic
9. Click Save

Thats it now your cacti logs show up in splunk.

Checking the OSSEC Processes from Nagios

There are several OSSEC processes that are running at once. So I’ll add some simple process checking to nagios to make sure I know they are running. One note is that if you haven’t enabled the ossec-csyslogd to run (I did that to talk to splunk) then you won’t need that one. So here we go, its easy.

1. Add it to your commands (Note I’m adding a new command for this instead of using the command already there so I can pass some different information, plus I like to keep all my modifications separate to make things easier to reproduce on other boxes)

vi /opt/local/etc/nagios/objects/commands.cfg

# ‘check_ossec_procs’ command definition
define command{
command_name check_ossec_procs
command_line $USER1$/check_procs -c $ARG1$ -C ARG2$
}

2. Add it to your localhost
vi /opt/local/etc/nagios/objects/localhost.cfg

define service{
use generic-service
host_name localhost
service_description OSSEC csyslogd
check_command check_ossec_procs!1:1!ossec-csyslogd
}

define service{
use generic-service
host_name localhost
service_description OSSEC maild
check_command check_ossec_procs!1:1!ossec-maild
}

define service{
use generic-service
host_name localhost
service_description OSSEC execd
check_command check_ossec_procs!1:1!ossec-execd
}

define service{
use generic-service
host_name localhost
service_description OSSEC analysisd
check_command check_ossec_procs!1:1!ossec-analysisd
}

define service{
use generic-service
host_name localhost
service_description OSSEC logcollector
check_command check_ossec_procs!1:1!ossec-logcollector
}

define service{
use generic-service
host_name localhost
service_description OSSEC monitord
check_command check_ossec_procs!1:1!ossec-monitord
}

Now just reload nagios and you should be able to tell if you ossec process is there or not.

Adding OSSEC Alerts to Splunk

Next up I want to add my OSSEC Alerts to Splunk. This is slightly more complicated then adding the nagios logs, but well document. The main part of this comes from the OSSEC Wiki Here. And the rest from the forums. But I’ll put it all here for my reference.

1. Edit your ossec.conf (If you’ve installed it like I have its located at: /var/ossec/etc/ossec.conf)
add the following block:
<syslog_output>
<server>172.10.2.3</server>
<port>10002</port>
</syslog_output>

2. Enable syslog_output module and restart OSSEC:
#/var/ossec/bin/ossec-control enable client-syslog
#/var/ossec/bin/ossec-control restart

On restart you’ll see ossec-csyslogd starting up.

On the Splunk Side
1. Goto Manager
2. Goto Data Inputs
3. On UDP click Add New
4. On my setup the UPD Port is 10002
5. Set sourcetype is Manual
6. Source type is ossec
7. Save

Now since things have changed in Splunk 4 the rest of the wiki entry doesn’t help. But there is more information on this Forum Thread.

So download that file and extract it to your splunk directory as stated. Restart splunk and bingo your OSSEC alerts plus lots of nice menu options to access that data.

Adding Nagios Logs to Splunk

Now that we have all these systems working correctly under OSX its time to start making them work together a little.

First up I want to add the nagios logs to splunk. This is very easy, you can get this off the splunk site here. But I’ll recap exactly what I did for my setup here.

1. Click Manager in the upper right-hand corner of Splunk Web.
2. Under System configurations, click Data Inputs.
3. Click Files and directories.
4. Click New to add an input
I choose Monitor a file or directory
6. Specify the path to the file:
With my setup it is:
/opt/local/var/nagios/nagios.log
7. Under Host Heading
I choose constant value
8. Under Source Type
I choose Automatic
9. Click Save

Thats it now your nagios logs show up in splunk. Pretty easy stuff.

Custom Weather Notifications with Growl

Last night I download Prowl on my iPhone and setup my growl to work with it. It’s very cool stuff together, i’ve been using growl forever.

Anyway tonight I was reading in this thread in the prowl forum where one poster is using growl notifications for weather. Not just any weather but really local weather. Now if you live in or around a big town, most weather apps are pretty accurate for you area. But when you live out in the sticks like I do, they are only close most of the time.

Anyway tonight I set up this excellent pair of perl scripts as outlined here from IBM: Develop your own weahter maps and alerts. Which is a very cool script that will allow you pinpoint your location. I used Photoshop to create the base map from the layers. Once followed all the instructions, some things are not exactly clear at first, but if your familiar with perl reading the code sorts it all out. I setup the notify scripts to send the messages to growl via the growlnotify command.

Now once that was all setup I created a simply bash script that would delete the old Radar overlay, pull the current Radar overlay needed and run the perl weather scripts. I then stuck that script in my crontab. So if I’m at my computer I get notified and if i’m away from my computer i get a push notification to my phone. Very cool stuff. Of course I could just look outside to see if it is raining 🙂

Splunk on OSX

Another tool that I like use is Splunk. Now we use a different set of tools for log monitoring and management at work, but I enjoy using splunk at home.

The good thing about Splunk on OSX is that they provide you with a .dmg to download and .pkg to install. Takes longer to download than to install. Once the install is done just start it up and log in.

OSSEC on OSX

Next up for reinstall is OSSEC. OSSEC is an Open Source Host-basted Intrusion Detection System. I also had this installed before the i reinstalled OSX.

To install OSSEC just follow the default instructions and everything works out just fine. Note, you’ll have to start this manual after each reboot, I’m sure there is a way to add it to autostart, but I haven’t gotten there yet.

To install the OSSEC-WUI follow the instructions up to the point before running the setup.sh script, it will not work on OSX (client anyway, not sure about server). All you need to do to get it working is first change the permission on the whole folder and files to _www. Then you need to add the _www user to the ossec group. That is done with the following command:
sudo dscl . -append /Groups/ossec GroupMembership _www

Thats it now its up and running and you have a nice interface for it.