webadmin-devel

RE: [webmin-devel] Problem opening maillog

[Larry G. <Li...@IS...>]

Hi Jamie,

Thanks for the response and help!


> -----Original Message-----
> From: Jamie Cameron
> 
> Larry Gilson wrote:
> > I hope I can both articulate the problem and that this is 
> > just something I am overlooking.
> > I have a report library for a module.  In a specific section, 
> > it parses /var/log looking for maillog*.  I have no problem 
> > accessing the files and reading them.  However, after copying the 
> 
> > file to an array, the arrayappears to have no input record 
> > separators.  The foreach control structure
> 
> You can read a whole file into an array with code like :
> 
> open(FILE, $file);
> @lines = <FILE>;
> close(FILE);

Yes, that is essentially what I do:

foreach $file (@files) {
   open (FH, "$path/$file");
   @wholefile = <FH>;
   close(FH);

The file is read without a problem.  When it comes to iterating through the
file though, it only makes one pass.  I have also attached output so you can
see the problem.  I actually forced it to read my custom procmaillog and I
do not see the same problem.  While I get 0 for output, it reads each line
of the file.  For some reason, reading maillog with 'foreach $line
(@wholefile) { }' only makes one pass when run _within_ reports-lib.pl.  It
is like the entire file was read into one array element.  However when
mail-stats.pl, is run from reports-lib.pl with the following code, it works
fine.

   $mailstats = `/usr/libexec/webmin/merlin/mail-stats.pl`;
   print $mailstats;


> > that should iterate through each line of the log only makes one 
> > pass.  The odd thing is that I can take the exact script, make it 
> > stand alone and use a print statement that utilizing backtick 
> > assignment from the same locationand it will iterate through the 
> > file and provide the appropriate output.  Additionally, I can 
> > call the stand alone script from the shell and it provides the 
> > appropriate output.  I am not using any external modules.
> > This simply opens the file, parses it, does some math, and 
> > 
> > outputs the result.  Am I doing something wrong?  Can anyone help
> > me with this?
>
> Would it be possible for you to post the actual code to the list? 
> Or at least the segment that is having trouble .. That would make 
> it easier to track down the problem.

Most definitely.  Please see the attached.


P.S. If you are wondering why the high scan times, I forgot to remove
Osirusoft and Orbs checks in SA after I recompiled and updated.  Doh!



Happy Holidays!

Regards,
Larry

RE: [webmin-devel] Problem opening maillog

[Jamie C. <jca...@we...>]

Larry Gilson <Li...@IS...> wrote ..
> Hi Jamie,
> 
> Thanks for the response and help!
> 
> 
> > -----Original Message-----
> > From: Jamie Cameron
> > 
> > Larry Gilson wrote:
> > > I hope I can both articulate the problem and that this is 
> > > just something I am overlooking.
> > > I have a report library for a module.  In a specific section, 
> > > it parses /var/log looking for maillog*.  I have no problem 
> > > accessing the files and reading them.  However, after copying the 
> > 
> > > file to an array, the arrayappears to have no input record 
> > > separators.  The foreach control structure
> > 
> > You can read a whole file into an array with code like :
> > 
> > open(FILE, $file);
> > @lines = <FILE>;
> > close(FILE);
> 
> Yes, that is essentially what I do:
> 
> foreach $file (@files) {
>    open (FH, "$path/$file");
>    @wholefile = <FH>;
>    close(FH);
> 
> The file is read without a problem.  When it comes to iterating through
> the
> file though, it only makes one pass.  I have also attached output so you
> can
> see the problem.  I actually forced it to read my custom procmaillog and
> I
> do not see the same problem.  While I get 0 for output, it reads each line
> of the file.  For some reason, reading maillog with 'foreach $line
> (@wholefile) { }' only makes one pass when run _within_ reports-lib.pl.
> It
> is like the entire file was read into one array element.  However when
> mail-stats.pl, is run from reports-lib.pl with the following code, it works
> fine.
> 
>    $mailstats = `/usr/libexec/webmin/merlin/mail-stats.pl`;
>    print $mailstats;
> 
> 
> > > that should iterate through each line of the log only makes one 
> > > pass.  The odd thing is that I can take the exact script, make it 
> > > stand alone and use a print statement that utilizing backtick 
> > > assignment from the same locationand it will iterate through the 
> > > file and provide the appropriate output.  Additionally, I can 
> > > call the stand alone script from the shell and it provides the 
> > > appropriate output.  I am not using any external modules.
> > > This simply opens the file, parses it, does some math, and 
> > > 
> > > outputs the result.  Am I doing something wrong?  Can anyone help
> > > me with this?
> >
> > Would it be possible for you to post the actual code to the list? 
> > Or at least the segment that is having trouble .. That would make 
> > it easier to track down the problem.
> 
> Most definitely.  Please see the attached.
> 
> 
> P.S. If you are wondering why the high scan times, I forgot to remove
> Osirusoft and Orbs checks in SA after I recompiled and updated.  Doh!

Are you perhaps setting the $/ variable somewhere in your code? If set to undef, perl will read a whole file at once with code like

$data = <FH>;

or the whole file into the first element of the array with code like :

@data = <FH>;

 - Jamie

RE: [webmin-devel] Problem opening maillog

[Larry G. <Li...@IS...>]

> -----Original Message-----
> From: Jamie Cameron
> 
> Are you perhaps setting the $/ variable somewhere in your 
> code? If set to undef, perl will read a whole file at once 
> with code like
> 
> $data = <FH>;
> 
> or the whole file into the first element of the array with code
> like :
> 
> @data = <FH>;

No.  I can actually force this code to read my procmaillog without a
problem.  It is only the maillog that I am having a problem with.  I also
have code like the following in reports-lib.pl and it works without a
problem.


   #
   #### Week-to-Date Merlin/Procmail Log Report
   #
      elsif ($match_exp eq "procmaillog") {
         open(PLOG, $logfile);
         while (<PLOG>) {
            $output = 1;
            print &html_escape($_);
         }
         close (PLOG);
      }
   #


I also tried to use a while loop instead of a foreach but I get the same
results - as I would suspect.

Thanks Jamie,
Larry

Re: [webmin-devel] Problem opening maillog

[Jamie C. <jca...@we...>]

Larry Gilson wrote:
> 
>>-----Original Message-----
>>From: Jamie Cameron
>>
>>Are you perhaps setting the $/ variable somewhere in your 
>>code? If set to undef, perl will read a whole file at once 
>>with code like
>>
>>$data = <FH>;
>>
>>or the whole file into the first element of the array with code
>>like :
>>
>>@data = <FH>;
> 
> 
> No.  I can actually force this code to read my procmaillog without a
> problem.  It is only the maillog that I am having a problem with.  I also
> have code like the following in reports-lib.pl and it works without a
> problem.
> 
> 
>    #
>    #### Week-to-Date Merlin/Procmail Log Report
>    #
>       elsif ($match_exp eq "procmaillog") {
>          open(PLOG, $logfile);
>          while (<PLOG>) {
>             $output = 1;
>             print &html_escape($_);
>          }
>          close (PLOG);
>       }
>    #
> 
> 
> I also tried to use a while loop instead of a foreach but I get the same
> results - as I would suspect.

So this problem only happens for a single log file (mailog), and works 
for procmaillog without the code being changed? Perhaps one of those 
files has lines ending with \r\n, while the other ends with just \n

  - Jamie

RE: [webmin-devel] Problem opening maillog

[Larry G. <Li...@IS...>]

Hi Jamie,

> -----Original Message-----
> From: Jamie Cameron
> 
> > I also tried to use a while loop instead of a foreach but I get 
> > the same results - as I would suspect.
> 
> So this problem only happens for a single log file (mailog), 
> and works for procmaillog without the code being changed? Perhaps
> one of those files has lines ending with \r\n, while the other ends
> with just \n

I would believe that.  I am, however, confused as I have a problem running
the code I supplied within reports-lib.pl but when I place that code in an
external script (mail-stats.pl) and call that external script from
reports-lib.pl, it works.  Is it possible that Webmin has the $/ variable
set somewhere external to my code?  How would I be able to track that?

Interesting, I tried the following:
egrep 'spamd.*\r.*seconds' /var/log/maillog.1

Dec 21 06:48:30 sisi spamd[5045]: identified spam (16.1/6.0) for merlin:91
in 2.5 seconds, 1085 bytes.
Dec 21 07:44:41 sisi spamd[5260]: clean message (-4.5/6.0) for merlin:91 in
3.4 seconds, 3741 bytes.
Dec 21 07:48:20 sisi spamd[5295]: identified spam (15.1/6.0) for merlin:91
in 2.8 seconds, 962 bytes.
Dec 21 08:09:10 sisi spamd[5560]: clean message (-6.2/6.0) for merlin:91 in
4.0 seconds, 5034 bytes.

It would seem that the \r is creating a problem but I just don't get where
the $/ variable could be set as I am not setting it.


Thanks for the help!

Regards,
Larry

RE: [webmin-devel] Problem opening maillog

[Larry G. <Li...@IS...>]

Hi Jamie,

> -----Original Message-----
> From: Jamie Cameron
> 
> > I also tried to use a while loop instead of a foreach but I get 
> > the same results - as I would suspect.
> 
> So this problem only happens for a single log file (mailog), 
> and works for procmaillog without the code being changed? Perhaps
> one of those files has lines ending with \r\n, while the other ends
> with just \n

Ok, I am the stupid one.  I did reset the $/ variable.  Since I have a
custom procmaillog and I have custom reports, I put a custom record
separator between the headers so I can treat the entire header as a record.
I set it to something that would never match in maillog which is why maillog
would never match in the foreach loop.  I feel stupid for missing this but
at least I am coming clean.  All I needed to do is reset $/ equal to \n and
I am in business.

Sorry for wasting your time!

--Larry