Subscribe
RSS Feed

Archive

←2008→

Months
Jan	Feb	Mar
Apr	May	Jun
Jul	Aug	Sep
Oct	Nov	Dec

Homepage

Sat, 23 Feb 2008

Delta: A shell script command line utility for processing log files

For security every web master knows you need to review your log files. A lot of us use automation to make this task possible. You need to regularly review the content of your logs. So how do you efficiently do this?

You could use a specialty program designed for that one task. Or, if you're like me you'd rather make your own. I wanted a program to extract new content but be generic enough it could be used as a tool for other tasks. I built 'delta'.

It's simple in operation. You give it the name of a file and it outputs the content of that file that hasn't been previously seen before. The first time you use it you get the complete content of the file. It records how much of the file was processed. The second time it starts where it left off at and outputs any content appended to the file since the last run. Delta records the file position of the end of the file. On subsequent runs it reads content after the previous position.

The C++ source for the program link. It should compile fine under Linux/Unix/OSX etc.

A compiled version for windows link

Some examples showing usage:

• EMail new web server log entries to yourself: delta /var/log/apache/error.log |sendmail me@myserver.com
• Search new web server log entries for worm attacks: delta /var/log/apache/access.log |grep dll |sendmail me@myserver.com

If you find the program useful an email or a contribution would be a fine gesture.

Jay Sprenkle