[Os-sim-devel] Dealing with syslog rotation

[Brian L. <br...@br...>]

I was looking at the ParserLog.py code for the agent, and it currently
has no way of telling when the log file is rotated. I found a chunk of
code that stats the file, and would reopen it, but I think it will
suffer from a race condition if the log file is closed and then a bunch
of log data is written to it again, before it stats it again to realize
the log file was closed and then rotated again. Not to mention that it
reads a block, but I am sure one could split on newlines. Has anyone
else looked at this problem?

Snippet from ParserLog.py that reads log file.
ParserLog.py


        while 1:

            # is plugin enabled?
            if not self._plugin.getboolean("config", "enable"):

                # wait until plugin is enabled
                while not self._plugin.getboolean("config", "enable"):
                    time.sleep(1)

                # plugin is now enabled, skip events generated on
                # 'disable' state, so move to the end of file
                fd.seek(0, 2)

            self._thresholding()

            where = fd.tell()
            line = fd.readline()

            if not line: # EOF reached
                time.sleep(1)
                fd.seek(where)
            else:

                # this could make a lot of noise...
                # logger.debug('Line read: %s' % (line))

                for rule in self.rules:
#                    logger.info("Trying rule: [%s]" % (rule.name))
                    rule.feed(line)
                    if rule.match():
                        logger.debug("Matched rule: [%s]" % (rule.name))
                        event = rule.generate_event()
                        if event is not None:
                            self.send_message(event)

                            # one rule matched, no need to check more
                            break

        fd.close()

Code, that could detect closing of syslog file. Reads chunks too. 

#!/usr/bin/python

import os
import time
import sys

def process_next_part(part):
	print part

lastsize=os.path.getsize('/var/log/syslog')
f=open('/var/log/syslog')

while 1:
    size=os.path.getsize('/var/log/syslog')
    if size>lastsize:
        f.seek(lastsize)
        part = f.read(size-lastsize)  # reads chunks, not lines!
        process_next_part(part)
	lastsize = size
    elif size==lastsize:
        time.sleep(0.2) # seconds to wait
    elif size<lastsize:
        print >>sys.stderr, "/var/log/syslog got truncated"
        f.close()
	f=open('/var/log/syslog')
	f.seek(0)
	part = f.read(size)  # reads chunks, not lines!
        lastsize=size


-- 
Brian Lavender
http://www.brie.com/brian/