Re: [Peep-develop] missing lines in logparser

[xavier r. <xa...@ha...>]

On Thu, Sep 05, 2002 at 10:41:35AM -0600, Collin Starkweather wrote:
|tor, 2002-09-05 kl. 08:40 skrev xavier renaut:
|> 
|> hello,
|> 
|> is it possible that logparser does not print everyline it parses ?
|> 
|> [Thu Sep  5 10:37:51 2002] Checking [@400000003d776c472a513aa4 tcpserver: status: 2/60] ...
|> [Thu Sep  5 10:37:51 2002] Checking group [sudo] against group and excluded group lists ...
|> [Thu Sep  5 10:37:51 2002] Checking group [incoming] against group and excluded group lists ...
|> [Thu Sep  5 10:37:51 2002]      Trying to match regex [sudo] for event [sudo]
|> [Thu Sep  5 10:37:51 2002]      Trying to match regex [ok] for event [incoming]
|> [Thu Sep  5 10:37:59 2002] Checking [@400000003d776c4f1fb56c8c tcpserver: status: 2/60] ...
|> [Thu Sep  5 10:37:59 2002] Checking group [sudo] against group and excluded group lists ...
|> [Thu Sep  5 10:37:59 2002] Checking group [incoming] against group and excluded group lists ...
|> [Thu Sep  5 10:37:59 2002]      Trying to match regex [sudo] for event [sudo]
|> [Thu Sep  5 10:37:59 2002]      Trying to match regex [ok] for event [incoming]
|> 
|> 
|> in my log file, there is few lines between thoses 2. (that is, around 4 lines)
|> is that fine ?
|> 
|> bye
|
|Curious.  It shouldn't be dropping any lines, though the debugging
|statements should reflect all the lines.  
|
|I use the File::Tail module to get the tails.  I'll have to do some
|stress testing to what's going on.
|


i've been playing with it a bit : 


i=1;while (( ( i += 1 ) <= 100000 )); do echo $i; sleep 0.1;done > /tmp/test

so the numbers here are tenth of seconds.

[Tue Sep 10 09:28:40 2002] Checking [1359] ...
[Tue Sep 10 09:28:40 2002] Checking group [sudo] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] Checking group [procmail] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] Checking group [inrx] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] Checking group [incoming] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] [incoming] will be ignored:  Host [ibis] is not in the host pool [inet-cou2].
[Tue Sep 10 09:28:40 2002] Checking group [pop] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] [pop] will be ignored:  Host [ibis] is not in the host pool [inet-cou2].
[Tue Sep 10 09:28:40 2002] Checking group [nrpe] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] [nrpe] will be ignored:  Host [ibis] is not in the host pool [inet-cou2].
[Tue Sep 10 09:28:40 2002]      Trying to match regex [sudo] for event [sudo]
[Tue Sep 10 09:28:40 2002]      Trying to match regex [D=procmail] for event [procmail]
[Tue Sep 10 09:28:40 2002]      Trying to match regex [LASTFOLDER=IN.rx] for event [inrx]
[Tue Sep 10 09:28:40 2002] Checking [1383] ...
[Tue Sep 10 09:28:40 2002] Checking group [sudo] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] Checking group [procmail] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] Checking group [inrx] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] Checking group [incoming] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] [incoming] will be ignored:  Host [ibis] is not in the host pool [inet-cou2].
[Tue Sep 10 09:28:40 2002] Checking group [pop] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] [pop] will be ignored:  Host [ibis] is not in the host pool [inet-cou2].
[Tue Sep 10 09:28:40 2002] Checking group [nrpe] against group and excluded group lists ...
[Tue Sep 10 09:28:40 2002] [nrpe] will be ignored:  Host [ibis] is not in the host pool [inet-cou2].
[Tue Sep 10 09:28:40 2002]      Trying to match regex [sudo] for event [sudo]
[Tue Sep 10 09:28:40 2002]      Trying to match regex [D=procmail] for event [procmail]
[Tue Sep 10 09:28:40 2002]      Trying to match regex [LASTFOLDER=IN.rx] for event [inrx]
[Tue Sep 10 09:28:40 2002] Checking [1391] ...



as you can see, it's missing a lot of them. 

any idea on how to fix that problem ?  

are you able to reproduce it ?
(i was on the 2 machines i tried)


thanks.

ps : i think file tail is fine : 

#  i=0;while (( ( i += 1 ) <= 100000 )); do echo $i; sleep 0.01;done > /tmp/test


/tmp/test.pl :

 use File::Tail;

$i=1;
         $file=File::Tail->new("/tmp/test");
         while (defined($line=$file->read)) {
             print $i++." $line";
         }



gives the expected result.

-- 
xavier


(needless to say, that's hard to do something with peep with that problem)