What Do You Do with Your Logfiles?

Dear Lazyweb,

We've built this Splunk thing, which is essentially a clustered search engine for temporal data, usually logfiles (*). Lots and lots and lots of logfiles. Thing is, everyone manages their technical infrastructure a bit differently and it's hard to figure out what's OK and what's not in terms of how to get at the log contents. Can you clue us in as to how you manage yours at a very high level? E.g.,

• Do you roll your important logfiles to a single location? If so, how are you transporting the logfiles?
• Do you do it in batches or real-time?
• Do you just leave your logfiles on the boxes they were created on?
• How often do you roll them?
• Are there some you care about more than others?
• How many machines do you roll logfiles on?

Thanks...

* - Or other time stamped data...

Update: if you're realizing that your logfile management is more accident than design, some good resources are:

• SAGE has an excellent booklet titled Building a Logging Infrastructure by Abe Singer and Tina Bird that is only available to members.
• Tina Bird's Top Ten syslog Signs that You've Been Hacked
• The Frequently Discussed Topics page at loganalysis.org

Technorati Tags: Splunk

Comments

[Dennis Au]

I have a batch file that zips logfiles on a daily basis on each machine which is then copied to a central system for archival purposes.