Re: [Perfparse-users] Upgrade issues with Perfparse 0.103.1

[Yves M. <yme...@li...>]

> Hi all,
>
> Yesterday, I upgraded from Perfparse 0.100.6 to 0.103.1. It was set up =
to use
> --with-file-perfdata and I kept it that way to save time since it works=
 well. I have a
> cron job that runs perfparse.sh every 15 minutes, leaving the servicepe=
rf.log intact. A
> cron job also runs perfparse.sh once just before midnight with --delete=
 that tells it to
> restart nagios and delete the serviceperf.log file.
>
> When I upgraded, I created a new perfparse.sh script to mimic this beha=
viour. It works
> well, but I ran into several issues that I wanted to share with the dev=
elopment group.
>
> 1) perfparse-log2db --show-config doesn't mention anything about
> Service_Log_Position_Mark_File. The docs mention it, but it doesn't see=
m to use it. It
> does use Service_Log_Position_Mark_Path.

The doc will be changed. The option is Service_Log_Position_Mark_Path tha=
t is for a
directory. Maybe I should rename it (again :)  Service_Log_Position_Mark_=
Dir ?
That directory contains the position mark files. There is one position ma=
rk file for one
servicelog file. If you use the same servicelog file, you have only one p=
osition mark
file.
This is interesting for those who have more than one service log file and=
 need one
position mark file per service log file.

If you reset the service log file but still use the position mark file, p=
erfparse will
detect that the file changed and will restart from the beginning. But if =
you do that,
maybe you should disable this feature ?

> 2) The mark file that was created looked like this:
>
> -rw-r--r--  1 root   root           19 Nov  3 09:30 perfdata_log.servic=
eperf.log.mark
> -rw-r--r--  1 root   root           17 Nov  2 23:56 perfdata_log.servic=
eperf.log1.mark
>
> I didn't know if the filename was supposed to be that complicated.
> Service_Log_Position_Mark_File would have been useful here.

No :)
But Service_Log_Position_Mark_File_Prefix or Service_Log_Position_Mark_Fi=
le_Suffix yes.
This can be done easily. What do you suggest ? (I prefer a suffix because=
 the directory
is already some kind of prefix if you want to use something else than som=
ewhere/var/ :)

> 3) I found an issue where the serviceperf.log file is renamed to servic=
eperf.log1 in the
> perfparse.sh script when nagios is restarted and the serviceperf.log is=
 deleted. This
> causes the creation of the second mark file. But the next day, the midn=
ight run sees the
> previous mark file and uses that to process the newer serviceperf.log f=
ile. A lot of
> records were processed a second time. The solution was to remove the ma=
rk file along
> with the serviceperf.log file at nagios restart time.

With one position mark file created for each different service log file n=
ame, you can
understand what happened.

But I want to be sure that it is not a bug in perfparse, but some problem=
 in your
script. When you move the serviceperf.log file to serviceperf.log1 (and r=
eboot nagios),
what does that file contain ? All the data for the day ?
If yes, you need to also move the position mark file because perfparse wi=
ll read
perfdata_log.serviceperf.log1.mark for serviceperf.log1. If it finds anyt=
hing else
(nothing or some wrong position mark file), it will start from the beginn=
ing, and in
your case, record duplicate data.

If I'm not too far from the truth, is there something that we can do in p=
erfparse to
prevent that, or is it only the documentation that need to be updated ?

> 4) The file perfparse.log.log was created in the var directory. I'm not=
 sure where this
> is coming from. Seems to only include some errors:

Why perfparse.log.log ? Well, this is a minor bug. I fix that right now a=
nd 0.104 will
be correct :)

> # cat perfparse.log.log
> 2004/11/02 15:39:53 [log_reader.c:181   28998 ] Could not open
> '/usr/local/nagios/var/perfdata_log.serviceperf.log1.mark'

Well, this is not an error if you run perfparse on serviceperf.log1 for t=
he first time.
This is not an error if you deleted perfdata_log.serviceperf.log1.mark.
This is an error if you expect the file to exist and perfparse to read it=
 and if
perfparse cannot.

Perfparse cannot know when it runs the first time or if you deleted that =
file yourself,
and cannot tell if being unable to read the file is an error or not. So I=
 decided to
write a message in the log but not stop the program.

If you have a better idea, tell me, because I'm conscious that such a mes=
sage does not
have a clear meaning :)

>
> 5) Here are the contents of my perfparse.cfg file:

Run perfparse-log2db -c perfparse.cfg --show_config and you will know the=
 options that
perfparse really uses and the ones that are obsolete or unused for perfpa=
rse-log2db.
Do that with all the perfparse binaries that you use, because they don't =
output the same
thing (they don't use the same options)

Yves

> #
> # PerfParse Config file.
> #
> # In all cases, variables used as:
> #
> # 1. From program default defined by configure.
> # 2. From this file.
> # 3. From variables passed to programs.
> #
>
> # Perfparsed server management
>
> Server_Port =3D                     "1976"
>
> Use_Storage_Socket_Output =3D       "no"
> Storage_Socket_Output_Port =3D      "1974"
> Storage_Socket_Output_Host_Name =3D "localhost"
>
> # Database management and authentication
>
> Use_Storage_Mysql =3D               "yes"
> No_Raw_Data =3D                     "no"
> No_Bin_Data =3D                     "no"
>
> DB_User =3D                         "nagios"
> DB_Name =3D                         "nagios"
> DB_Pass =3D                         "nagios"
> DB_Host =3D                         "127.0.0.1"
>
> # Performance Data Log Files ("-" for stdin)
> # Drop file       - For lines of log rejected
> # Lock file       - For only one perfparse running at the same time
> # Nagios lock     - Nagios file containing its PID (Necessary to delete=
 log files)
> # Output Log File - For logging all the data that perfparse reads
>
> Delete_Host_Log =3D                 "no"
> Delete_Service_Log =3D              "no"
>
> Host_Log =3D                        "/usr/local/nagios/var/hostperf.log=
"
> Service_Log =3D                     "/usr/local/nagios/var/serviceperf.=
log"
> Service_Log_Position_Mark_File =3D  "serviceperf.pos"
> Service_Log_Position_Mark_Path =3D  "/usr/local/nagios/var"
> Service_Log_Save_Position =3D       "yes"
>
> Drop_File =3D                       "/tmp/perfparse.drop"
> Drop_File_Rotate =3D                "no"
>
> Error_Log =3D                       "/usr/local/nagios/var/perfparse.lo=
g"
> Error_Log_Rotate =3D                "no"
>
> Lock_File =3D                       "/usr/local/nagios/var/perfparse.lo=
ck"
> Nagios_Lock =3D                     "/usr/local/nagios/var/nagios.lock"
>
> Output_Log_File =3D                 "no"
> Output_Log_Rotate =3D               "no"
> Output_Log_Filename =3D             "/usr/local/nagios/var/perfparse_ou=
tput_log"
>
> # Reporting options
> # Do not display a report on exit of perfparse
> # Do not display completion status of perfparse
>
> Do_Report =3D                       "no"
> Show_Status_Bar =3D                 "no"
>
> # Misc
> Dummy_Hostname =3D                  "dummy"
>
> # CGI management
> # Default user permissions in the CGI. Accepted values are (ro|rw|hide)
>
> Default_user_permissions_Policy =3D      "rw"
> Default_user_permissions_Host_groups =3D "rw"
>
>
> -------------------
> James Ferrill
> IT Analyst II
> NextiraOne LLC
> Phone: 713-307-4029
> Email: jam...@ne...
>
>


--=20
- Homepage    - http://ymettier.free.fr - http://www.logicacmg.com -
- GPG key     - http://ymettier.free.fr/gpg.txt                    -
- Maitretarot - http://www.nongnu.org/maitretarot/                 -
- Perfparse   - http://perfparse.sf.net/                           -