ELK Stack
Combining Elastic search, Logstash, and Kibana (ELK) gives you an easy way to analyze logged data. The typical pipeline goes like this. 
- Some tasks output logs into files.
 
- Logstash monitors those files.
 
- Logstash translates those logs into records.
 
- Logstash saves the records into elastic search.
 
- Elastic search indexes the logged records. 
 
- Kibana allows you to query the elastic search engine
 
It is a powerful combination, so I wanted to give it a try. 
I followed the steps outlined in http://aarvik.dk/a-bit-on-elasticsearch-logstash-kibana-the-elk-stack/ . You can obviously do the same, but here are my notes on the process (I did this on my Macbook Air). 
Get the files
curl -OL https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.1.0.tar.gz
curl -OL https://download.elasticsearch.org/logstash/logstash/logstash-1.4.0.tar.gz
curl -OL https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
It is amazing that all of this is completely free to download and try out. 
Testing each of the downloads
# Start your (elastic search) engines ... 
elasticsearch-1.3.4/bin/elasticsearch
curl -X GET localhost:9200
curl -X POST localhost:9200/person/1 -d '{ "info" : {"height" : 2, "width" : 20 } }'
# Testing logstash with command line arguments:
logstash-1.4.0/bin/logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'
# or run with a config file
logstash-1.4.0/bin/logstash -f ./logstash.conf
I used the same logstash config file listed in the reference page I noted above. However, I did change one of the inputs to read from /var/log/system.log .  At this point, I just wanted it get input from a file I knew was geting updates. 
Setting up a local web development server - apache
In order to use kibana, you need a web server. Luckily apache is pretty much ready to use on Macbooks. I followed steps outline in http://ole.michelsen.dk/blog/setup-local-web-server-apache-php-macos-x-mavericks.html. Update The setup is slightly different for Yosemite.
I followed that site to get my server started, until my browser said "It works". Then I configured kibana to use my elastic search engine. To do this, update kibana/config.js to have this line: 
elasticsearch: "http://localhost:9200",
This replaces --> 
elasticsearch: "http://"+window.location.hostname+":9200",
This works because everything is on one machine. Finally, I moved the kibana folder over to be read. 
mkdir ~/Sites
cp -r kibana-3.0.0 ~/Sites/
sudo apachectl start
You should now be able to visit your kibana dashboard at http://localhost/~username/kibana
Conclusion
Getting the ELK stack up and running was really easy. Hopefully I can think of a way to put this cool technology to good use. 
Resources I used ...
- http://aarvik.dk/a-bit-on-elasticsearch-logstash-kibana-the-elk-stack/
 
- http://ole.michelsen.dk/blog/setup-local-web-server-apache-php-macos-x-mavericks.html