PeerGuardian

Hi,

I like the proposal but I have a few additional suggestions. In order for
the GUI(s) to parse the log entries easily, I believe that they all should
have a common basic format. I'm thinking something like:
*Date* || *Type || Event

**The "II" characters could be something different but from my experience
this sequence is never found in any blocklist descriptions. If you have
something else to propose do so but make sure it's a bit unusual.

**
**For example(General): *

Oct 25 12:15:48 || General || PeerGuardian Ascii: 550190 entries loaded
Oct 25 12:15:50 || General || 378203 entries merged
Oct 25 12:15:50 || General || Connected to system bus.
Oct 25 12:15:50 || General || Started
Oct 25 12:15:50 || General || Blocklist has 171987 entries
Oct 25 12:15:50 || General || NFQUEUE: binding to queue 92

*Adding comas to the hit entries would be useful too:*

Oct 25 12:21:35 || IN || 65.55.207.135:34874, 192.168.178.20:8803, TCP,
Microsoft Corp
Oct 25 12:22:55 || OUT || 192.168.178.20:35884, 79.176.17.1:18142,TCP, Slot
Blocker

*Last but not least, merge entries should definitely have a type so that the
GUI can safely ignore them:*

Oct 25 12:26:30 || Merge || 222.255.74.0-222.255.223.255
222.255.74.16-222.255.74.23
Oct 25 12:26:30 || Merge || 222.255.74.48-222.255.74.55
222.255.74.64-222.255.74.71
Oct 25 12:26:30 || Merge || 222.255.76.0-222.255.76.63
222.255.76.96-222.255.76.127

That's all for now, I would like to know what you think.


On Sun, Oct 25, 2009 at 2:53 PM, jre-phoenix <
jre-phoenix@...> wrote:

> Hi
>
> I'm just discussing the logging with Cade and wanted to make a
> proposal. First I'll summarize the current status (I hope I got it
> right), then I'll show my proposal.
>
> ==========================================================
>
> Currently we have logging to syslog and dbus. In syslog we have:
>
>
> 1.) General information:
> Oct 25 12:15:48 bastard pgld: PeerGuardian Ascii: 550190 entries loaded
> Oct 25 12:15:50 bastard pgld: 378203 entries merged
> Oct 25 12:15:50 bastard pgld: Connected to system bus.
> Oct 25 12:15:50 bastard pgld: Started
> Oct 25 12:15:50 bastard pgld: Blocklist has 171987 entries
> Oct 25 12:15:50 bastard pgld: NFQUEUE: binding to queue 92
>
>
> 2.) Hit logging:
> Oct 25 12:21:35 bastard pgld: Blocked IN: Bogon, hits: 1, SRC:
> 192.168.178.1
> Oct 25 12:22:55 bastard pgld: Blocked OUT: Bogon, hits: 1, DST:
> 239.255.255.250
>
> You can disable hit logging with "--no-syslog" and "--no-dbus".
>
>
> 3.) Merge information:
> Oct 25 12:41:58 bastard pgld: Merging ranges:
> 222.255.74.0-222.255.223.255 222.255.74.16-222.255.74.23
> 222.255.74.48-222.255.74.55 222.255.74.64-222.255.74.71
> 222.255.76.0-222.255.76.63 222.255.76.96-222.255.76.127
> 222.255.76.160-222.255.76.191 222.255.77.16-222.255.77.31
> 222.255.77.32-222.255.77.47 222.255.77.64-222.255.77.79
> 222.255.78.56-222.255.78.63 222.255.78.120-222.255.78.127
> 222.255.78.144-222.255.78.151 222.255.79.28-222.255.79.31
> 222.255.79.36-222.255.79.39 222.255.79.40-222.255.79.43
> 222.255.79.76-222.255.79.79 222.255.79.96-222.255.79.99
> 222.255.79.100-222.255.79.103 222.255.79.128-222.255.79.131
> 222.255.83.160-222.255.83.175 222.255.96.0-222.255.98.255
> 222.255.160.0-222.255.160.255 into 222.255.74.0-222.255.223.255
> Oct 25 12:41:58 bastard pgld: Merging ranges:
> 222.255.228.0-222.255.255.255 223.0.0.0-255.255.255.254 into
> 222.255.228.0-255.255.255.254
>
> Merge information is nearly zero for single lists like ipfilter.dat,
> but with the current multiple blocklist setup 6 MB!
>
>
> ==========================================================
>
>
> Now the proposal:
>
>
> Log targets:
> ============
>
> Per default do not log.
> Enable logging to syslog with option "--syslog|-s"
> Enable logging to logfile with option "--logfile|-l LOGFILE""
> Enable logging to dbus with option "--dbus|-b"
>
> So I'd like to add the logfile option, and change from the "disable
> hit logging" approach to an "enable log target" approach.
>
> Note: Personally I see no need for output to STDOUT. But I always run
> pgld in daemon mode.
>
>
> Verbosity level:
> ================
>
> If logging is disabled (see above), do not log at all.
> Except errors, log them to syslog.
>
> If logging to one or several targets is enabled:
> - default: log general information
> - -v: also log hits
> - -vv: also log merge information
>
> Note: I see no need for different verbosity levels per log target.
>
>
> Hit logging:
> ============
>
> Change this to format
> CHAIN SRC:PORT DEST:PORT PROTO | DESC
>
> IN: 65.55.207.135:34874 192.168.178.20:8803 TCP | Microsoft Corp
> OUT: 192.168.178.20:35884 79.176.17.1:18142 TCP | Slot Blocker
>
> Note: In a low memory mode (for routers and NAS) at least the
> description would be omitted. Perhaps this should be reduced even
> more.
>
>
> Greets
> jre
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Peerguardian-devel mailing list
> Peerguardian-devel@...
> https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
>



-- 
"The man who trades freedom for security does not deserve nor will he ever
receive either."
-Benjamin Franklin

The GUI can get its information via dbus now, so it's no more
necessary to parse the logfile. But I'm not really sure if pgld logs
just whole lines to dbus, or several separate fields per entry. In the
latter case we don´t even need separators like "||" or ",".
Further I´m not sure about timestamps. AFAIK in syslog the date gets
there from syslogd. I see no need for a timestamp for the live
notification via dbus, but I think in the logfile it would be nice.

Otherwise I´m absolutely fine with your suggestions, there definitely
should be the same logic in all logs. Perhaps add another "Error",
next to "General", "IN", "OUT", "FWD", "Merge".

BTW, I found some "|" occurences in the blocklists, but indeed no "||".

syslog will log like so:
Oct 25 11:21:10 hostname pgld:

So we can just look for pgld: in the logs (syslog, messages, daemon.log)
The Merged lines have "Merged ranges" in it - and I have taken those
lines out of log since it blow my logs up way way way to much for the
info they provided.
Also I would like to get pglcmd at some point to pre-merge the lists
so pgld starts quicker.

I would also like to get pglcmd to log to syslog if it doesn't already
since pgld logs to syslog.

For commas I don't think we need them.  Everything is space separated
on the hit format.
Oct 25 10:52:25 hostname pgld:  IN: 216.208.255.131:57088
96.3.153.138:0         UDP  | PUBLIC WORKS GOV. SERV. CANADA
Change the "|" that separates the IP info from the desc if needed but
your format is:
Oct 25 10:52:25 hostname pgld:  CHAIN: SRC:PORT DEST:PORT PROTO | DESC
Using whitespace (like awk default) you can get all the info needed.
IMO commas will make the lines messier.
As will all the extra pipes.

For the "General" lines again I don't think we need to add "General"
The lines will have the standard syslog format and then the messages
which you have to parse either way.

Also IMO the logging is a total mess in pgld with no standard anything
and it also logs WAY to much extra info.
I will work on getting it cleaned up and then maybe get the discussion
going on tuning it up.

I am also trying to pull using buf1, buf2 etc out.  No need for extra
buffers for mem and buffer overflow reasons when you can use the info
directly.


On Mon, Oct 26, 2009 at 7:27 AM, Jim <jimaras@...> wrote:
> Hi,
>
> I like the proposal but I have a few additional suggestions. In order for
> the GUI(s) to parse the log entries easily, I believe that they all should
> have a common basic format. I'm thinking something like:
> Date || Type || Event
>
> The "II" characters could be something different but from my experience this
> sequence is never found in any blocklist descriptions. If you have something
> else to propose do so but make sure it's a bit unusual.
>
>
> For example(General):
>
> Oct 25 12:15:48 || General || PeerGuardian Ascii: 550190 entries loaded
> Oct 25 12:15:50 || General || 378203 entries merged
> Oct 25 12:15:50 || General || Connected to system bus.
> Oct 25 12:15:50 || General || Started
> Oct 25 12:15:50 || General || Blocklist has 171987 entries
> Oct 25 12:15:50 || General || NFQUEUE: binding to queue 92
>
> Adding comas to the hit entries would be useful too:
>
> Oct 25 12:21:35 || IN || 65.55.207.135:34874, 192.168.178.20:8803, TCP,
> Microsoft Corp
> Oct 25 12:22:55 || OUT || 192.168.178.20:35884, 79.176.17.1:18142,TCP, Slot
> Blocker
>
> Last but not least, merge entries should definitely have a type so that the
> GUI can safely ignore them:
>
> Oct 25 12:26:30 || Merge || 222.255.74.0-222.255.223.255
> 222.255.74.16-222.255.74.23
> Oct 25 12:26:30 || Merge || 222.255.74.48-222.255.74.55
> 222.255.74.64-222.255.74.71
> Oct 25 12:26:30 || Merge || 222.255.76.0-222.255.76.63
> 222.255.76.96-222.255.76.127
>
> That's all for now, I would like to know what you think.
>
>
> On Sun, Oct 25, 2009 at 2:53 PM, jre-phoenix
> <jre-phoenix@...> wrote:
>>
>> Hi
>>
>> I'm just discussing the logging with Cade and wanted to make a
>> proposal. First I'll summarize the current status (I hope I got it
>> right), then I'll show my proposal.
>>
>> ==========================================================
>>
>> Currently we have logging to syslog and dbus. In syslog we have:
>>
>>
>> 1.) General information:
>> Oct 25 12:15:48 bastard pgld: PeerGuardian Ascii: 550190 entries loaded
>> Oct 25 12:15:50 bastard pgld: 378203 entries merged
>> Oct 25 12:15:50 bastard pgld: Connected to system bus.
>> Oct 25 12:15:50 bastard pgld: Started
>> Oct 25 12:15:50 bastard pgld: Blocklist has 171987 entries
>> Oct 25 12:15:50 bastard pgld: NFQUEUE: binding to queue 92
>>
>>
>> 2.) Hit logging:
>> Oct 25 12:21:35 bastard pgld: Blocked IN: Bogon, hits: 1, SRC:
>> 192.168.178.1
>> Oct 25 12:22:55 bastard pgld: Blocked OUT: Bogon, hits: 1, DST:
>> 239.255.255.250
>>
>> You can disable hit logging with "--no-syslog" and "--no-dbus".
>>
>>
>> 3.) Merge information:
>> Oct 25 12:41:58 bastard pgld: Merging ranges:
>> 222.255.74.0-222.255.223.255 222.255.74.16-222.255.74.23
>> 222.255.74.48-222.255.74.55 222.255.74.64-222.255.74.71
>> 222.255.76.0-222.255.76.63 222.255.76.96-222.255.76.127
>> 222.255.76.160-222.255.76.191 222.255.77.16-222.255.77.31
>> 222.255.77.32-222.255.77.47 222.255.77.64-222.255.77.79
>> 222.255.78.56-222.255.78.63 222.255.78.120-222.255.78.127
>> 222.255.78.144-222.255.78.151 222.255.79.28-222.255.79.31
>> 222.255.79.36-222.255.79.39 222.255.79.40-222.255.79.43
>> 222.255.79.76-222.255.79.79 222.255.79.96-222.255.79.99
>> 222.255.79.100-222.255.79.103 222.255.79.128-222.255.79.131
>> 222.255.83.160-222.255.83.175 222.255.96.0-222.255.98.255
>> 222.255.160.0-222.255.160.255 into 222.255.74.0-222.255.223.255
>> Oct 25 12:41:58 bastard pgld: Merging ranges:
>> 222.255.228.0-222.255.255.255 223.0.0.0-255.255.255.254 into
>> 222.255.228.0-255.255.255.254
>>
>> Merge information is nearly zero for single lists like ipfilter.dat,
>> but with the current multiple blocklist setup 6 MB!
>>
>>
>> ==========================================================
>>
>>
>> Now the proposal:
>>
>>
>> Log targets:
>> ============
>>
>> Per default do not log.
>> Enable logging to syslog with option "--syslog|-s"
>> Enable logging to logfile with option "--logfile|-l LOGFILE""
>> Enable logging to dbus with option "--dbus|-b"
>>
>> So I'd like to add the logfile option, and change from the "disable
>> hit logging" approach to an "enable log target" approach.
>>
>> Note: Personally I see no need for output to STDOUT. But I always run
>> pgld in daemon mode.
>>
>>
>> Verbosity level:
>> ================
>>
>> If logging is disabled (see above), do not log at all.
>> Except errors, log them to syslog.
>>
>> If logging to one or several targets is enabled:
>> - default: log general information
>> - -v: also log hits
>> - -vv: also log merge information
>>
>> Note: I see no need for different verbosity levels per log target.
>>
>>
>> Hit logging:
>> ============
>>
>> Change this to format
>> CHAIN SRC:PORT DEST:PORT PROTO | DESC
>>
>> IN: 65.55.207.135:34874 192.168.178.20:8803 TCP | Microsoft Corp
>> OUT: 192.168.178.20:35884 79.176.17.1:18142 TCP | Slot Blocker
>>
>> Note: In a low memory mode (for routers and NAS) at least the
>> description would be omitted. Perhaps this should be reduced even
>> more.
>>
>>
>> Greets
>> jre
>>
>>
>> ------------------------------------------------------------------------------
>> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>> is the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>> http://p.sf.net/sfu/devconference
>> _______________________________________________
>> Peerguardian-devel mailing list
>> Peerguardian-devel@...
>> https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
>
>
>
> --
> "The man who trades freedom for security does not deserve nor will he ever
> receive either."
> -Benjamin Franklin
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Peerguardian-devel mailing list
> Peerguardian-devel@...
> https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
>
>



-- 
Cade

Hi,

I have some questions.

To begin with, will the date always be in this format? I think that I had
this problem with moblock. Sometimes the date format was different on
different computers and as a result parsing the log was a problem.

Moreover, I strongly believe that we should use a double pipe instead of
just one. I don't think it will be that messy and it will prevent some
strange descriptions containing pipes from causing any problems.

As for the entries refering to an error, please make sure they have the word
*error* somewhere in them so that I can mark them as such.

To sum up, from what I understand the format will be:
*mmm dd hh:mm:ss hostname pgld: General_Entry_Here*

or

*mmm dd hh:mm:ss hostname pgld: IN/OUT/FWD: ...*

Am I right? If this is always the case, parsing the log would not be too
difficult. What about dbus entries however? How are they formated?

On Mon, Oct 26, 2009 at 5:17 PM, Cade <cade.robinson@...> wrote:

> syslog will log like so:
> Oct 25 11:21:10 hostname pgld:
>
> So we can just look for pgld: in the logs (syslog, messages, daemon.log)
> The Merged lines have "Merged ranges" in it - and I have taken those
> lines out of log since it blow my logs up way way way to much for the
> info they provided.
> Also I would like to get pglcmd at some point to pre-merge the lists
> so pgld starts quicker.
>
> I would also like to get pglcmd to log to syslog if it doesn't already
> since pgld logs to syslog.
>
> For commas I don't think we need them.  Everything is space separated
> on the hit format.
> Oct 25 10:52:25 hostname pgld:  IN: 216.208.255.131:57088
> 96.3.153.138:0         UDP  | PUBLIC WORKS GOV. SERV. CANADA
> Change the "|" that separates the IP info from the desc if needed but
> your format is:
> Oct 25 10:52:25 hostname pgld:  CHAIN: SRC:PORT DEST:PORT PROTO | DESC
> Using whitespace (like awk default) you can get all the info needed.
> IMO commas will make the lines messier.
> As will all the extra pipes.
>
> For the "General" lines again I don't think we need to add "General"
> The lines will have the standard syslog format and then the messages
> which you have to parse either way.
>
> Also IMO the logging is a total mess in pgld with no standard anything
> and it also logs WAY to much extra info.
> I will work on getting it cleaned up and then maybe get the discussion
> going on tuning it up.
>
> I am also trying to pull using buf1, buf2 etc out.  No need for extra
> buffers for mem and buffer overflow reasons when you can use the info
> directly.
>
>
> On Mon, Oct 26, 2009 at 7:27 AM, Jim <jimaras@...> wrote:
> > Hi,
> >
> > I like the proposal but I have a few additional suggestions. In order for
> > the GUI(s) to parse the log entries easily, I believe that they all
> should
> > have a common basic format. I'm thinking something like:
> > Date || Type || Event
> >
> > The "II" characters could be something different but from my experience
> this
> > sequence is never found in any blocklist descriptions. If you have
> something
> > else to propose do so but make sure it's a bit unusual.
> >
> >
> > For example(General):
> >
> > Oct 25 12:15:48 || General || PeerGuardian Ascii: 550190 entries loaded
> > Oct 25 12:15:50 || General || 378203 entries merged
> > Oct 25 12:15:50 || General || Connected to system bus.
> > Oct 25 12:15:50 || General || Started
> > Oct 25 12:15:50 || General || Blocklist has 171987 entries
> > Oct 25 12:15:50 || General || NFQUEUE: binding to queue 92
> >
> > Adding comas to the hit entries would be useful too:
> >
> > Oct 25 12:21:35 || IN || 65.55.207.135:34874, 192.168.178.20:8803, TCP,
> > Microsoft Corp
> > Oct 25 12:22:55 || OUT || 192.168.178.20:35884, 79.176.17.1:18142,TCP,
> Slot
> > Blocker
> >
> > Last but not least, merge entries should definitely have a type so that
> the
> > GUI can safely ignore them:
> >
> > Oct 25 12:26:30 || Merge || 222.255.74.0-222.255.223.255
> > 222.255.74.16-222.255.74.23
> > Oct 25 12:26:30 || Merge || 222.255.74.48-222.255.74.55
> > 222.255.74.64-222.255.74.71
> > Oct 25 12:26:30 || Merge || 222.255.76.0-222.255.76.63
> > 222.255.76.96-222.255.76.127
> >
> > That's all for now, I would like to know what you think.
> >
> >
> > On Sun, Oct 25, 2009 at 2:53 PM, jre-phoenix
> > <jre-phoenix@...> wrote:
> >>
> >> Hi
> >>
> >> I'm just discussing the logging with Cade and wanted to make a
> >> proposal. First I'll summarize the current status (I hope I got it
> >> right), then I'll show my proposal.
> >>
> >> ==========================================================
> >>
> >> Currently we have logging to syslog and dbus. In syslog we have:
> >>
> >>
> >> 1.) General information:
> >> Oct 25 12:15:48 bastard pgld: PeerGuardian Ascii: 550190 entries loaded
> >> Oct 25 12:15:50 bastard pgld: 378203 entries merged
> >> Oct 25 12:15:50 bastard pgld: Connected to system bus.
> >> Oct 25 12:15:50 bastard pgld: Started
> >> Oct 25 12:15:50 bastard pgld: Blocklist has 171987 entries
> >> Oct 25 12:15:50 bastard pgld: NFQUEUE: binding to queue 92
> >>
> >>
> >> 2.) Hit logging:
> >> Oct 25 12:21:35 bastard pgld: Blocked IN: Bogon, hits: 1, SRC:
> >> 192.168.178.1
> >> Oct 25 12:22:55 bastard pgld: Blocked OUT: Bogon, hits: 1, DST:
> >> 239.255.255.250
> >>
> >> You can disable hit logging with "--no-syslog" and "--no-dbus".
> >>
> >>
> >> 3.) Merge information:
> >> Oct 25 12:41:58 bastard pgld: Merging ranges:
> >> 222.255.74.0-222.255.223.255 222.255.74.16-222.255.74.23
> >> 222.255.74.48-222.255.74.55 222.255.74.64-222.255.74.71
> >> 222.255.76.0-222.255.76.63 222.255.76.96-222.255.76.127
> >> 222.255.76.160-222.255.76.191 222.255.77.16-222.255.77.31
> >> 222.255.77.32-222.255.77.47 222.255.77.64-222.255.77.79
> >> 222.255.78.56-222.255.78.63 222.255.78.120-222.255.78.127
> >> 222.255.78.144-222.255.78.151 222.255.79.28-222.255.79.31
> >> 222.255.79.36-222.255.79.39 222.255.79.40-222.255.79.43
> >> 222.255.79.76-222.255.79.79 222.255.79.96-222.255.79.99
> >> 222.255.79.100-222.255.79.103 222.255.79.128-222.255.79.131
> >> 222.255.83.160-222.255.83.175 222.255.96.0-222.255.98.255
> >> 222.255.160.0-222.255.160.255 into 222.255.74.0-222.255.223.255
> >> Oct 25 12:41:58 bastard pgld: Merging ranges:
> >> 222.255.228.0-222.255.255.255 223.0.0.0-255.255.255.254 into
> >> 222.255.228.0-255.255.255.254
> >>
> >> Merge information is nearly zero for single lists like ipfilter.dat,
> >> but with the current multiple blocklist setup 6 MB!
> >>
> >>
> >> ==========================================================
> >>
> >>
> >> Now the proposal:
> >>
> >>
> >> Log targets:
> >> ============
> >>
> >> Per default do not log.
> >> Enable logging to syslog with option "--syslog|-s"
> >> Enable logging to logfile with option "--logfile|-l LOGFILE""
> >> Enable logging to dbus with option "--dbus|-b"
> >>
> >> So I'd like to add the logfile option, and change from the "disable
> >> hit logging" approach to an "enable log target" approach.
> >>
> >> Note: Personally I see no need for output to STDOUT. But I always run
> >> pgld in daemon mode.
> >>
> >>
> >> Verbosity level:
> >> ================
> >>
> >> If logging is disabled (see above), do not log at all.
> >> Except errors, log them to syslog.
> >>
> >> If logging to one or several targets is enabled:
> >> - default: log general information
> >> - -v: also log hits
> >> - -vv: also log merge information
> >>
> >> Note: I see no need for different verbosity levels per log target.
> >>
> >>
> >> Hit logging:
> >> ============
> >>
> >> Change this to format
> >> CHAIN SRC:PORT DEST:PORT PROTO | DESC
> >>
> >> IN: 65.55.207.135:34874 192.168.178.20:8803 TCP | Microsoft Corp
> >> OUT: 192.168.178.20:35884 79.176.17.1:18142 TCP | Slot Blocker
> >>
> >> Note: In a low memory mode (for routers and NAS) at least the
> >> description would be omitted. Perhaps this should be reduced even
> >> more.
> >>
> >>
> >> Greets
> >> jre
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> >> is the only developer event you need to attend this year. Jumpstart your
> >> developing skills, take BlackBerry mobile applications to market and
> stay
> >> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> >> http://p.sf.net/sfu/devconference
> >> _______________________________________________
> >> Peerguardian-devel mailing list
> >> Peerguardian-devel@...
> >> https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
> >
> >
> >
> > --
> > "The man who trades freedom for security does not deserve nor will he
> ever
> > receive either."
> > -Benjamin Franklin
> >
> >
> ------------------------------------------------------------------------------
> > Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> > is the only developer event you need to attend this year. Jumpstart your
> > developing skills, take BlackBerry mobile applications to market and stay
> > ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> > http://p.sf.net/sfu/devconference
> > _______________________________________________
> > Peerguardian-devel mailing list
> > Peerguardian-devel@...
> > https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
> >
> >
>
>
>
> --
> Cade
>



-- 
"The man who trades freedom for security does not deserve nor will he ever
receive either."
-Benjamin Franklin

> To begin with, will the date always be in this format? I think that I had
> this problem with moblock. Sometimes the date format was different on
> different computers and as a result parsing the log was a problem.

I think Cade can give a clear answer here.

> Moreover, I strongly believe that we should use a double pipe instead of
> just one. I don't think it will be that messy and it will prevent some
> strange descriptions containing pipes from causing any problems.

Currently "everything between first pipe and end of line" is the
description. But if you think this can cause problems, then we can
easily change that, no problem.

> As for the entries refering to an error, please make sure they have the word
> *error* somewhere in them so that I can mark them as such.
>
> To sum up, from what I understand the format will be:
> *mmm dd hh:mm:ss hostname pgld: General_Entry_Here*
>
> or
>
> *mmm dd hh:mm:ss hostname pgld: IN/OUT/FWD: ...*

Again for Cade, best give the concrete line of code.

> Am I right? If this is always the case, parsing the log would not be too
> difficult. What about dbus entries however? How are they formated?

But what I really want to say: pgld now has three output options,
where ALL output goes:
- log to logfile
- log to syslog
- log to dbus
>From all that I know, dbus is THE interface to exchange things between
e.g. a daemon and a GUI. So I think the GUI should only listen to
dbus, but not parse the logfiles. Unfortunately I don´t know dbus
really yet. But I will work on understanding it, and I suggest you do
so, too.
Even more unfortunately dbus output is broken currently with our
recent push, but we will definitely reenable it.
Maybe you don´t even have to parse dbus log messages for "Error", if
it is possible to just get the importance of the log message (in the
code we already have LOG_INFO, LOG_WARNING and LOG_ERR)

BTW, one point for dbus is, that it solves the problem with too big
logfiles (mobloquer was nearly dead if you had a 16MB logfile).

Oh, I forgot: I will add logging to dbus support to pglcmd, too. If I
find out ho to do it. Currently I´m working on log to syslog support
for pglcmd.

On Thu, Oct 29, 2009 at 11:07 PM, jre-phoenix
<jre-phoenix@...> wrote:
>> To begin with, will the date always be in this format? I think that I had
>> this problem with moblock. Sometimes the date format was different on
>> different computers and as a result parsing the log was a problem.
>
> I think Cade can give a clear answer here.
>
>> Moreover, I strongly believe that we should use a double pipe instead of
>> just one. I don't think it will be that messy and it will prevent some
>> strange descriptions containing pipes from causing any problems.
>
> Currently "everything between first pipe and end of line" is the
> description. But if you think this can cause problems, then we can
> easily change that, no problem.
>
>> As for the entries refering to an error, please make sure they have the word
>> *error* somewhere in them so that I can mark them as such.
>>
>> To sum up, from what I understand the format will be:
>> *mmm dd hh:mm:ss hostname pgld: General_Entry_Here*
>>
>> or
>>
>> *mmm dd hh:mm:ss hostname pgld: IN/OUT/FWD: ...*
>
> Again for Cade, best give the concrete line of code.
>
>> Am I right? If this is always the case, parsing the log would not be too
>> difficult. What about dbus entries however? How are they formated?
>
> But what I really want to say: pgld now has three output options,
> where ALL output goes:
> - log to logfile
> - log to syslog
> - log to dbus
> From all that I know, dbus is THE interface to exchange things between
> e.g. a daemon and a GUI. So I think the GUI should only listen to
> dbus, but not parse the logfiles. Unfortunately I don´t know dbus
> really yet. But I will work on understanding it, and I suggest you do
> so, too.
> Even more unfortunately dbus output is broken currently with our
> recent push, but we will definitely reenable it.
> Maybe you don´t even have to parse dbus log messages for "Error", if
> it is possible to just get the importance of the log message (in the
> code we already have LOG_INFO, LOG_WARNING and LOG_ERR)
>
> BTW, one point for dbus is, that it solves the problem with too big
> logfiles (mobloquer was nearly dead if you had a 16MB logfile).
>

On Thu, 2009-10-29 at 23:07 +0100, jre-phoenix wrote:
> > To begin with, will the date always be in this format? I think that I had
> > this problem with moblock. Sometimes the date format was different on
> > different computers and as a result parsing the log was a problem.
> 
> I think Cade can give a clear answer here.

If logging to syslog date and format of the line is handled by the
syslog daemon.
For me the format is 
"Oct 30 09:45:01 hostname daemon: MESSAGE"
The date is probably locale dependent so that could change depending on
locale.

When logging to a log file I tried to mimic the date and time of syslog.
"%b %e %X"

%b is replaced by the locale's abbreviated month name.
%e is replaced by the day of the month as a decimal number [1,31]; a
single digit is preceded by a space.
%X is replaced by the locale's appropriate time representation.

Again locale will affect the format of the fields.
I didn't put "hostname" and "daemon:" in the log file because I figured
that would be obvious since you would only write to your own host and
the file name of the log would tell you the daemon.

For dbus the format was "HH:MM:SS" but the plan is to get the "do_log"
function to do all the logging - not have several logging calls to
several different places.
So once I (or someone that knows it better) can get dbus working right
the dbus messages can/will have the same time format as log file
logging. 

> 
> > Moreover, I strongly believe that we should use a double pipe instead of
> > just one. I don't think it will be that messy and it will prevent some
> > strange descriptions containing pipes from causing any problems.
> 
> Currently "everything between first pipe and end of line" is the
> description. But if you think this can cause problems, then we can
> easily change that, no problem.

I can add double pipe between PROTO and DESC easily enough.

> 
> > As for the entries refering to an error, please make sure they have the word
> > *error* somewhere in them so that I can mark them as such.

Yes errors need a bit of work yet.  Originally pgld was only coded to
log to syslog. I added logging to logfile.
When using syslog a priority was set on the messages.
LOG_ERR is set on error.
That doesn't actually log a word "error" at all - it just told syslog
where to put  the message if you had something setup where *.err
messages went to a different syslog logfile.

Also most of them log to stderr so they don't even go to syslog since
most of the errors are fatal to starting the daemon.

> >
> > To sum up, from what I understand the format will be:
> > *mmm dd hh:mm:ss hostname pgld: General_Entry_Here*
> >
> > or
> >
> > *mmm dd hh:mm:ss hostname pgld: IN/OUT/FWD: ...*
> 
> Again for Cade, best give the concrete line of code.
Once dbus is working does the GUI really should care about format or
looking in syslog or a logfile.  Everything should be provided to it via
dbus messages, that is if I understand dbus.
Wouldn't the GUI just collect the message and spit it out the way it got
it?

> 
> > Am I right? If this is always the case, parsing the log would not be too
> > difficult. What about dbus entries however? How are they formated?
> 
> But what I really want to say: pgld now has three output options,
> where ALL output goes:
> - log to logfile
> - log to syslog
> - log to dbus
> From all that I know, dbus is THE interface to exchange things between
> e.g. a daemon and a GUI. So I think the GUI should only listen to
> dbus, but not parse the logfiles. Unfortunately I don´t know dbus
> really yet. But I will work on understanding it, and I suggest you do
> so, too.
> Even more unfortunately dbus output is broken currently with our
> recent push, but we will definitely reenable it.
> Maybe you don´t even have to parse dbus log messages for "Error", if
> it is possible to just get the importance of the log message (in the
> code we already have LOG_INFO, LOG_WARNING and LOG_ERR)
> 
> BTW, one point for dbus is, that it solves the problem with too big
> logfiles (mobloquer was nearly dead if you had a 16MB logfile).