Hi,
I have some questions.
To begin with, will the date always be in this format? I think that I had
this problem with moblock. Sometimes the date format was different on
different computers and as a result parsing the log was a problem.
Moreover, I strongly believe that we should use a double pipe instead of
just one. I don't think it will be that messy and it will prevent some
strange descriptions containing pipes from causing any problems.
As for the entries refering to an error, please make sure they have the word
*error* somewhere in them so that I can mark them as such.
To sum up, from what I understand the format will be:
*mmm dd hh:mm:ss hostname pgld: General_Entry_Here*
or
*mmm dd hh:mm:ss hostname pgld: IN/OUT/FWD: ...*
Am I right? If this is always the case, parsing the log would not be too
difficult. What about dbus entries however? How are they formated?
On Mon, Oct 26, 2009 at 5:17 PM, Cade <cade.robinson@...> wrote:
> syslog will log like so:
> Oct 25 11:21:10 hostname pgld:
>
> So we can just look for pgld: in the logs (syslog, messages, daemon.log)
> The Merged lines have "Merged ranges" in it - and I have taken those
> lines out of log since it blow my logs up way way way to much for the
> info they provided.
> Also I would like to get pglcmd at some point to pre-merge the lists
> so pgld starts quicker.
>
> I would also like to get pglcmd to log to syslog if it doesn't already
> since pgld logs to syslog.
>
> For commas I don't think we need them. Everything is space separated
> on the hit format.
> Oct 25 10:52:25 hostname pgld: IN: 216.208.255.131:57088
> 96.3.153.138:0 UDP | PUBLIC WORKS GOV. SERV. CANADA
> Change the "|" that separates the IP info from the desc if needed but
> your format is:
> Oct 25 10:52:25 hostname pgld: CHAIN: SRC:PORT DEST:PORT PROTO | DESC
> Using whitespace (like awk default) you can get all the info needed.
> IMO commas will make the lines messier.
> As will all the extra pipes.
>
> For the "General" lines again I don't think we need to add "General"
> The lines will have the standard syslog format and then the messages
> which you have to parse either way.
>
> Also IMO the logging is a total mess in pgld with no standard anything
> and it also logs WAY to much extra info.
> I will work on getting it cleaned up and then maybe get the discussion
> going on tuning it up.
>
> I am also trying to pull using buf1, buf2 etc out. No need for extra
> buffers for mem and buffer overflow reasons when you can use the info
> directly.
>
>
> On Mon, Oct 26, 2009 at 7:27 AM, Jim <jimaras@...> wrote:
> > Hi,
> >
> > I like the proposal but I have a few additional suggestions. In order for
> > the GUI(s) to parse the log entries easily, I believe that they all
> should
> > have a common basic format. I'm thinking something like:
> > Date || Type || Event
> >
> > The "II" characters could be something different but from my experience
> this
> > sequence is never found in any blocklist descriptions. If you have
> something
> > else to propose do so but make sure it's a bit unusual.
> >
> >
> > For example(General):
> >
> > Oct 25 12:15:48 || General || PeerGuardian Ascii: 550190 entries loaded
> > Oct 25 12:15:50 || General || 378203 entries merged
> > Oct 25 12:15:50 || General || Connected to system bus.
> > Oct 25 12:15:50 || General || Started
> > Oct 25 12:15:50 || General || Blocklist has 171987 entries
> > Oct 25 12:15:50 || General || NFQUEUE: binding to queue 92
> >
> > Adding comas to the hit entries would be useful too:
> >
> > Oct 25 12:21:35 || IN || 65.55.207.135:34874, 192.168.178.20:8803, TCP,
> > Microsoft Corp
> > Oct 25 12:22:55 || OUT || 192.168.178.20:35884, 79.176.17.1:18142,TCP,
> Slot
> > Blocker
> >
> > Last but not least, merge entries should definitely have a type so that
> the
> > GUI can safely ignore them:
> >
> > Oct 25 12:26:30 || Merge || 222.255.74.0-222.255.223.255
> > 222.255.74.16-222.255.74.23
> > Oct 25 12:26:30 || Merge || 222.255.74.48-222.255.74.55
> > 222.255.74.64-222.255.74.71
> > Oct 25 12:26:30 || Merge || 222.255.76.0-222.255.76.63
> > 222.255.76.96-222.255.76.127
> >
> > That's all for now, I would like to know what you think.
> >
> >
> > On Sun, Oct 25, 2009 at 2:53 PM, jre-phoenix
> > <jre-phoenix@...> wrote:
> >>
> >> Hi
> >>
> >> I'm just discussing the logging with Cade and wanted to make a
> >> proposal. First I'll summarize the current status (I hope I got it
> >> right), then I'll show my proposal.
> >>
> >> ==========================================================
> >>
> >> Currently we have logging to syslog and dbus. In syslog we have:
> >>
> >>
> >> 1.) General information:
> >> Oct 25 12:15:48 bastard pgld: PeerGuardian Ascii: 550190 entries loaded
> >> Oct 25 12:15:50 bastard pgld: 378203 entries merged
> >> Oct 25 12:15:50 bastard pgld: Connected to system bus.
> >> Oct 25 12:15:50 bastard pgld: Started
> >> Oct 25 12:15:50 bastard pgld: Blocklist has 171987 entries
> >> Oct 25 12:15:50 bastard pgld: NFQUEUE: binding to queue 92
> >>
> >>
> >> 2.) Hit logging:
> >> Oct 25 12:21:35 bastard pgld: Blocked IN: Bogon, hits: 1, SRC:
> >> 192.168.178.1
> >> Oct 25 12:22:55 bastard pgld: Blocked OUT: Bogon, hits: 1, DST:
> >> 239.255.255.250
> >>
> >> You can disable hit logging with "--no-syslog" and "--no-dbus".
> >>
> >>
> >> 3.) Merge information:
> >> Oct 25 12:41:58 bastard pgld: Merging ranges:
> >> 222.255.74.0-222.255.223.255 222.255.74.16-222.255.74.23
> >> 222.255.74.48-222.255.74.55 222.255.74.64-222.255.74.71
> >> 222.255.76.0-222.255.76.63 222.255.76.96-222.255.76.127
> >> 222.255.76.160-222.255.76.191 222.255.77.16-222.255.77.31
> >> 222.255.77.32-222.255.77.47 222.255.77.64-222.255.77.79
> >> 222.255.78.56-222.255.78.63 222.255.78.120-222.255.78.127
> >> 222.255.78.144-222.255.78.151 222.255.79.28-222.255.79.31
> >> 222.255.79.36-222.255.79.39 222.255.79.40-222.255.79.43
> >> 222.255.79.76-222.255.79.79 222.255.79.96-222.255.79.99
> >> 222.255.79.100-222.255.79.103 222.255.79.128-222.255.79.131
> >> 222.255.83.160-222.255.83.175 222.255.96.0-222.255.98.255
> >> 222.255.160.0-222.255.160.255 into 222.255.74.0-222.255.223.255
> >> Oct 25 12:41:58 bastard pgld: Merging ranges:
> >> 222.255.228.0-222.255.255.255 223.0.0.0-255.255.255.254 into
> >> 222.255.228.0-255.255.255.254
> >>
> >> Merge information is nearly zero for single lists like ipfilter.dat,
> >> but with the current multiple blocklist setup 6 MB!
> >>
> >>
> >> ==========================================================
> >>
> >>
> >> Now the proposal:
> >>
> >>
> >> Log targets:
> >> ============
> >>
> >> Per default do not log.
> >> Enable logging to syslog with option "--syslog|-s"
> >> Enable logging to logfile with option "--logfile|-l LOGFILE""
> >> Enable logging to dbus with option "--dbus|-b"
> >>
> >> So I'd like to add the logfile option, and change from the "disable
> >> hit logging" approach to an "enable log target" approach.
> >>
> >> Note: Personally I see no need for output to STDOUT. But I always run
> >> pgld in daemon mode.
> >>
> >>
> >> Verbosity level:
> >> ================
> >>
> >> If logging is disabled (see above), do not log at all.
> >> Except errors, log them to syslog.
> >>
> >> If logging to one or several targets is enabled:
> >> - default: log general information
> >> - -v: also log hits
> >> - -vv: also log merge information
> >>
> >> Note: I see no need for different verbosity levels per log target.
> >>
> >>
> >> Hit logging:
> >> ============
> >>
> >> Change this to format
> >> CHAIN SRC:PORT DEST:PORT PROTO | DESC
> >>
> >> IN: 65.55.207.135:34874 192.168.178.20:8803 TCP | Microsoft Corp
> >> OUT: 192.168.178.20:35884 79.176.17.1:18142 TCP | Slot Blocker
> >>
> >> Note: In a low memory mode (for routers and NAS) at least the
> >> description would be omitted. Perhaps this should be reduced even
> >> more.
> >>
> >>
> >> Greets
> >> jre
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> >> is the only developer event you need to attend this year. Jumpstart your
> >> developing skills, take BlackBerry mobile applications to market and
> stay
> >> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> >> http://p.sf.net/sfu/devconference
> >> _______________________________________________
> >> Peerguardian-devel mailing list
> >> Peerguardian-devel@...
> >> https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
> >
> >
> >
> > --
> > "The man who trades freedom for security does not deserve nor will he
> ever
> > receive either."
> > -Benjamin Franklin
> >
> >
> ------------------------------------------------------------------------------
> > Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> > is the only developer event you need to attend this year. Jumpstart your
> > developing skills, take BlackBerry mobile applications to market and stay
> > ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> > http://p.sf.net/sfu/devconference
> > _______________________________________________
> > Peerguardian-devel mailing list
> > Peerguardian-devel@...
> > https://lists.sourceforge.net/lists/listinfo/peerguardian-devel
> >
> >
>
>
>
> --
> Cade
>
--
"The man who trades freedom for security does not deserve nor will he ever
receive either."
-Benjamin Franklin
|