Logsurfer Project

Overview

The Logsurfer program was designed to monitor any text-based logfiles on a system in real-time. The large amount of log information collected (like all messages handled by the syslog-daemon or logfiles from your information services FTP, WWW etc.) makes it nearly impossible to check your logs manually to find any unusual activity. You need a program to do this for you. If you don't want to use a script that checks your logs in certain time intervals (like once a day) then you might be interested in the programs like swatch or logsurfer.

The first try: swatch

The swatch package (written by Todd Atkins, Stanford) was the first well known program to monitor logfiles. It uses a configuration file to build and start a PERL script that monitors a logfile. For every incoming message line it compares the line against a set of regular expressions (from the configuration file) and takes an action based on this line. Possible actions are: echo this line, ring the bell, execute external programs, pipe the message into an external program, send it via mail to other people or use the write command to forward the message to some users.

The idea is simple and effective but has some major problems:

  • It requires perl
  • It will work only with one messageline at a time (problems with logmessages, that are more than one line)
  • Only very basic dynamic rules (is able to ignore repeated messages if the input file has timestamps)

Logsurfer

Due to the limited possibilities of swatch (especially the limitation on single lines and the missing possibility to parse substrings of the message as arguments to external programs) the logsurfer program was developed. Just to list some of the important things:

  • Works on any textfile (or text from standard input)
  • Matching of lines is done by two regular expression (logline must match the first expression but must not match the optional second regular expression). So you are able to specify exceptions.
  • Uses contexts (collection of messages) instead single lines
  • Flexible but easy configuration
  • Timeouts and resource limits included
  • Handles "shifting" of logfiles (just send a -HUP signal to close and reopen the logfile after you have moved the old one to another place and created a new one)
  • Dynamic rules can change the actions associated with logmessages (something might happen that makes you interested in messages you would usually drop)
  • Multiple reactions on one logline possible
  • Portable written C-code (uses GNU regex library and autoconfigure)
  • ...

Example Messages

Messages detected by swatch

somehost ftpd[14311]: cmd failure - not logged in

This maybe an interesting message but this single line isn't enough to make a decision.

Messages detected by logsurfer

somehost tcpd-ftpd[14311]: connect from host.some.where
somehost ftpd[14311]: connection from *@host.some.where [42.42.42.42]
somehost ftpd[14311]: USER anonynous
somehost ftpd[14311]: PASS password
somehost ftpd[14311]: failed login from host.some.where [42.42.42.42], anonynous
somehost ftpd[14311]: SYST
somehost ftpd[14311]: PORT
somehost ftpd[14311]: cmd failure - not logged in
somehost ftpd[14311]: LIST
somehost ftpd[14311]: QUIT
somehost ftpd[14311]: FTP session closed

Now you can easily see, that the user has misspelled the loginname "anonymous" and has tried to get a directory listing (ftp command "dir"). He has recognized the error and logged off. So you can forget this message...

Documentation

Currently the following documentation is available:

Some people have not installed TeX on their systems and may not be able to read the documentation of the GNU regex library (with explanation of the special regex chars for POSIX_EGREP that are used by the logsurfer). Here are some versions as DVI and PostScript file available:

Availability

The program has been released on April, 9th 1996. The latest version (v1.5a) is available via anonymous ftp from ftp://ftp.cert.dfn.de/pub/tools/audit/logsurfer/

FAQ

Currently I haven't received much questions. As a result there is currently no FAQ. This might change in the future. (Ha! Isn't that sentence better than the boring "under construction" phrase?)

Authors

  • Swatch was developed and is maintained by Todd Atkins.
  • The logsurfer was written by Wolfgang Ley and Uwe Ellerman.

Last modified: 28 April 2004