wallfire-users — WallFire users mailing-list

Re: [wallfire-users] Help about how the wflogs reads the logs

[Herve E. <rv...@wa...>]

On Fri, Nov 07, 2008 at 04:42:56PM -0200, Vinícius Batistela wrote:

> Are you sure that nepenthes don't do any kind of filter? For example, i
> have many packets that is part of a DNS connection (my host did a DNS
> request) and, with wflogs, the port that my host used to do that appears
> only one time. It don't consider all the packets.

Well, if packets get logged by netfilter, wflogs should parse (by
default) these log entries and reflect what's in it, no more no less.
If you can provide a log sample that is not correctly parsed by
wflogs, I'll be happy to fix that.

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: [wallfire-users] Help about how the wflogs reads the logs

[V. B. <vin...@ba...>]

Are you sure that nepenthes don't do any kind of filter? For example, i have
many packets that is part of a DNS connection (my host did a DNS request)
and, with wflogs, the port that my host used to do that appears only one
time. It don't consider all the packets.

On Fri, Nov 7, 2008 at 7:49 AM, Herve Eychenne <rv...@wa...> wrote:

> On Thu, Nov 06, 2008 at 06:39:18PM -0200, Vinícius Batistela wrote:
>
>  Hi Vinícius,
>
> > i am using wflogs with the logs generated by iptables and i have a doubt.
> > I used the wflogs filtrating by protocols (TCP and UDP) and i generated a
> > XML file. Then, i wrote a script to read this XML. But, looking for the
> > results i got with this script and looking for the results i got using a
> > script that reads directly the iptables' log, i saw they are different.
>
> > So, i think that for TCP, the wflogs just consider the packages with the
> > SYN flag activated, that represent a try of start a connection. Am i
> > wrong?
>
> Well, it's supposed to consider all packets (SYN or not) by default,
> even if you can filter only SYN ones if you wish to do so.
>
> > So, and for UDP, i have different results too, using the xml and
> > reading the iptables' log directly. But the UDP protocol do not have
> > control mechanisms. What wflogs do about UDP?
>
> It's the same (except that there is no connection/SYN in UDP), it considers
> all packets by default.
>
> > Thank you for the answers.
>
> Regards,
>
>  Hervé
>



-- 
Vinícius Batistela

Re: [wallfire-users] Help about how the wflogs reads the logs

[Herve E. <rv...@wa...>]

On Thu, Nov 06, 2008 at 06:39:18PM -0200, Vinícius Batistela wrote:

 Hi Vinícius,

> i am using wflogs with the logs generated by iptables and i have a doubt.
> I used the wflogs filtrating by protocols (TCP and UDP) and i generated a
> XML file. Then, i wrote a script to read this XML. But, looking for the
> results i got with this script and looking for the results i got using a
> script that reads directly the iptables' log, i saw they are different.

> So, i think that for TCP, the wflogs just consider the packages with the
> SYN flag activated, that represent a try of start a connection. Am i
> wrong?

Well, it's supposed to consider all packets (SYN or not) by default,
even if you can filter only SYN ones if you wish to do so.

> So, and for UDP, i have different results too, using the xml and
> reading the iptables' log directly. But the UDP protocol do not have
> control mechanisms. What wflogs do about UDP?

It's the same (except that there is no connection/SYN in UDP), it considers
all packets by default.

> Thank you for the answers.

Regards,

 Hervé

[wallfire-users] Help about how the wflogs reads the logs

[V. B. <vin...@ba...>]

Hi,

i am using wflogs with the logs generated by iptables and i have a doubt. I
used the wflogs filtrating by protocols (TCP and UDP) and i generated a XML
file. Then, i wrote a script to read this XML. But, looking for the results
i got with this script and looking for the results i got using a script that
reads directly the iptables' log, i saw they are different. So, i think that
for TCP, the wflogs just consider the packages with the SYN flag activated,
that represent a try of start a connection. Am i wrong? So, and for UDP, i
have different results too, using the xml and reading the iptables' log
directly. But the UDP protocol do not have control mechanisms. What wflogs
do about UDP?

Thank you for the answers.

-- 
Vinícius Batistela

Re: [wallfire-users] Pix Question

[Anabella C. <ana...@gm...>]

Thank you very much!
 If I can help with something just let me know!
Regards
Anabella


 On 9/21/05, Herve Eychenne <rv...@wa...> wrote:
>
> On Tue, Sep 20, 2005 at 02:52:07PM -0300, Anabella Cristaldi wrote:
>
> > Hi Herve,
> > I have done a copy and paste. A longer piece of logs...
>
> > Aug 11 16:04:40 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:15 :
> %PIX-2-106006: Deny
> > inbound UDP from 199.203.54.218/27219 <http://199.203.54.218/27219> to
> 64.116.226.187/1434 <http://64.116.226.187/1434> on interface
> > outside
> > Aug 11 16:04:40 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:16 :
> %PIX-3-710003: UDP
> > access denied by ACL from 10.101.8.102/33622 <http://10.101.8.102/33622=
>to inside:
> 10.101.31.255/10105 <http://10.101.31.255/10105>
> > Aug 11 16:04:41 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:17 :
> %PIX-3-710003: UDP
> > access denied by ACL from 10.101.8.102/33622 <http://10.101.8.102/33622=
>to inside:
> 10.101.31.255/10105 <http://10.101.31.255/10105>
> > Aug 11 16:04:41 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:17 :
> %PIX-3-710003: UDP
> > access denied by ACL from 10.101.3.233/10150 <http://10.101.3.233/10150=
>to inside:
> 10.101.31.255/10150 <http://10.101.31.255/10150>
> > Aug 11 16:04:41 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:17 :
> %PIX-3-710003: UDP
> > access denied by ACL from 10.101.9.10/138 <http://10.101.9.10/138> to
> inside:10.101.31.255/138 <http://10.101.31.255/138>
>
> Ok, I've added what is necessary to parse these logs.
>
> But there is a very big bunch of different CISCO PIX log
> messages. I fear supporting any of them would be a full-time job.
> So wflogs will probably never support every possible CISCO log
> message, even if patches are always welcome.
>
> I've committed the changes into CVS already. Feel free to test.
>
> I also plan to do a release of wflogs in the next few days, stay tuned.
>
> Cheers,
>
> Herve
>
> --
> _
> (=B0=3D Herv=E9 Eychenne
> //)
> v_/_ WallFire project: http://www.wallfire.org/
>
>

Re: [wallfire-users] Pix Question

[Herve E. <rv...@wa...>]

On Tue, Sep 20, 2005 at 02:52:07PM -0300, Anabella Cristaldi wrote:

> Hi Herve,
> I have done a copy and paste. A longer piece of logs...

> Aug 11 16:04:40 10.101.5.50 Aug 11 2005 15:54:15 : %PIX-2-106006: Deny
> inbound UDP from 199.203.54.218/27219 to 64.116.226.187/1434 on interface
> outside
> Aug 11 16:04:40 10.101.5.50 Aug 11 2005 15:54:16 : %PIX-3-710003: UDP
> access denied by ACL from 10.101.8.102/33622 to inside:10.101.31.255/10105
> Aug 11 16:04:41 10.101.5.50 Aug 11 2005 15:54:17 : %PIX-3-710003: UDP
> access denied by ACL from 10.101.8.102/33622 to inside:10.101.31.255/10105
> Aug 11 16:04:41 10.101.5.50 Aug 11 2005 15:54:17 : %PIX-3-710003: UDP
> access denied by ACL from 10.101.3.233/10150 to inside:10.101.31.255/10150
> Aug 11 16:04:41 10.101.5.50 Aug 11 2005 15:54:17 : %PIX-3-710003: UDP
> access denied by ACL from 10.101.9.10/138 to inside:10.101.31.255/138

Ok, I've added what is necessary to parse these logs.

But there is a very big bunch of different CISCO PIX log
messages. I fear supporting any of them would be a full-time job.
So wflogs will probably never support every possible CISCO log
message, even if patches are always welcome.

I've committed the changes into CVS already. Feel free to test.

I also plan to do a release of wflogs in the next few days, stay tuned.

Cheers,

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: [wallfire-users] Needing help with $count filter option

[Herve E. <rv...@wa...>]

On Fri, Sep 16, 2005 at 12:02:45AM +0200, Herve Eychenne wrote:

> On Thu, Sep 15, 2005 at 02:36:45PM -0600, James Lay wrote:

> Or maybe --filter-before-processing and --filter-after-processing
> would be more explicit...

That is what I eventually implemented.
I've commited the changes into CVS. Feel free to test.

> Anyway, I'll release a version with these two options soon, like I
> said. I guess you'll be interested by the second. Thanks for having
> inspired this future improvment.

Like I said, I plan to do a release within the next few days, stay tuned.

Cheers,

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: [wallfire-users] Pix Question

[Anabella C. <ana...@gm...>]

Hi Herve,
I have done a copy and paste. A longer piece of logs...
 Aug 11 16:04:40 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:15 :=20
%PIX-2-106006: Deny inbound UDP from
199.203.54.218/27219<http://199.203.54.218/27219>to
64.116.226.187/1434 <http://64.116.226.187/1434> on interface outside
Aug 11 16:04:40 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:16 :=20
%PIX-3-710003: UDP access denied by ACL from
10.101.8.102/33622<http://10.101.8.102/33622>to inside:
10.101.31.255/10105 <http://10.101.31.255/10105>
Aug 11 16:04:41 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:17 :=20
%PIX-3-710003: UDP access denied by ACL from
10.101.8.102/33622<http://10.101.8.102/33622>to inside:
10.101.31.255/10105 <http://10.101.31.255/10105>
Aug 11 16:04:41 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:17 :=20
%PIX-3-710003: UDP access denied by ACL from
10.101.3.233/10150<http://10.101.3.233/10150>to inside:
10.101.31.255/10150 <http://10.101.31.255/10150>
Aug 11 16:04:41 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:17 :=20
%PIX-3-710003: UDP access denied by ACL from
10.101.9.10/138<http://10.101.9.10/138>to inside:
10.101.31.255/138 <http://10.101.31.255/138>

Regards
 On 9/20/05, Herve Eychenne <rv...@wa...> wrote:=20
>=20
> On Mon, Sep 19, 2005 at 02:22:11PM -0300, Anabella Cristaldi wrote:
>=20
> > yes, you are right
> > The lines like this one...
>=20
> > Aug 11 16:04:26 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:02 :=
=20
> %PIX-3-710003: UDP
> > access denied by ACL from 10.101.0.167/138 <http://10.101.0.167/138> to=
=20
> inside: 10.101.31.255/138 <http://10.101.31.255/138>
>=20
> Are you sure there is a space between "15:54:02" and ":"?
> And are you sure there is a space between "inside:" and
> "10.101.31.255/138 <http://10.101.31.255/138>"?
> I looked at CISCO documentation, and it seems there should be no
> space... I don't understand. Did you really copy paste, or copy by
> hand? Spacing is important...
>=20
> > arises the error:
> > warning: wrong log entry.
>=20
> > Thanks a lot!
>=20
>=20
> > On 9/19/05, Herve Eychenne <rv...@wa...> wrote:
>=20
> > On Mon, Sep 19, 2005 at 01:00:02PM -0300, Anabella Cristaldi wrote:
>=20
> > > Hi,
> > > I have a problem when parsing lines like this one
>=20
> > > Aug 11 16:04:25 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:01=
=20
> : %PIX-3-710003:
>=20
> > > I get the error
>=20
> > > pix2:581: warning: wrong log entry.
>=20
> > > Is this message type not supported?
>=20
> > Are you sure the line you're presenting here is complete?
> > I think "%PIX-3-710003:" is supposed to be followed by something... am
> > I wrong?
>=20
> > Herve
>=20
> > --
> > _
> > (DEG=3D Herve Eychenne
> > //)
> > v_/_ WallFire project: http://www.wallfire.org/
>=20
> Herve
>=20
> --
> _
> (=B0=3D Herv=E9 Eychenne
> //)
> v_/_ WallFire project: http://www.wallfire.org/
>=20
>

Re: [wallfire-users] Pix Question

[Herve E. <rv...@wa...>]

On Mon, Sep 19, 2005 at 02:22:11PM -0300, Anabella Cristaldi wrote:

> yes, you are right
> The lines like this one...

> Aug 11 16:04:26 10.101.5.50 Aug 11 2005 15:54:02 : %PIX-3-710003: UDP
> access denied by ACL from 10.101.0.167/138 to inside: 10.101.31.255/138

Are you sure there is a space between "15:54:02" and ":"?
And are you sure there is a space between "inside:" and
"10.101.31.255/138"?
I looked at CISCO documentation, and it seems there should be no
space... I don't understand.  Did you really copy paste, or copy by
hand?  Spacing is important...

> arises the error:
> warning: wrong log entry.

> Thanks a lot!


> On 9/19/05, Herve Eychenne <rv...@wa...> wrote:

>   On Mon, Sep 19, 2005 at 01:00:02PM -0300, Anabella Cristaldi wrote:

>   > Hi,
>   > I have a problem when parsing lines like this one

>   > Aug 11 16:04:25 10.101.5.50 Aug 11 2005 15:54:01 : %PIX-3-710003:

>   > I get the error

>   > pix2:581: warning: wrong log entry.

>   > Is this message type not supported?

>   Are you sure the line you're presenting here is complete?
>   I think "%PIX-3-710003:" is supposed to be followed by something... am
>   I wrong?

>   Herve

>   --
>   _
>   (DEG=  Herve Eychenne
>   //)
>   v_/_ WallFire project:  http://www.wallfire.org/

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: [wallfire-users] Pix Question

[Anabella C. <ana...@gm...>]

yes, you are right
The lines like this one...
 Aug 11 16:04:26 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:02 :=20
%PIX-3-710003: UDP access denied by ACL from
10.101.0.167/138<http://10.101.0.167/138>to inside:
10.101.31.255/138 <http://10.101.31.255/138>
 arises the error:=20
warning: wrong log entry.
 Thanks a lot!
=20

 On 9/19/05, Herve Eychenne <rv...@wa...> wrote:=20
>=20
> On Mon, Sep 19, 2005 at 01:00:02PM -0300, Anabella Cristaldi wrote:
>=20
> > Hi,
> > I have a problem when parsing lines like this one
>=20
> > Aug 11 16:04:25 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:01 :=
=20
> %PIX-3-710003:
>=20
> > I get the error
>=20
> > pix2:581: warning: wrong log entry.
>=20
> > Is this message type not supported?
>=20
> Are you sure the line you're presenting here is complete?
> I think "%PIX-3-710003:" is supposed to be followed by something... am
> I wrong?
>=20
> Herve
>=20
> --
> _
> (=B0=3D Herv=E9 Eychenne
> //)
> v_/_ WallFire project: http://www.wallfire.org/
>=20
>

Re: [wallfire-users] Pix Question

[Herve E. <rv...@wa...>]

On Mon, Sep 19, 2005 at 01:00:02PM -0300, Anabella Cristaldi wrote:

> Hi,
> I have a problem when parsing lines like this one

> Aug 11 16:04:25 10.101.5.50 Aug 11 2005 15:54:01 : %PIX-3-710003:

> I get the error

> pix2:581: warning: wrong log entry.

> Is this message type not supported?

Are you sure the line you're presenting here is complete?
I think "%PIX-3-710003:" is supposed to be followed by something... am
I wrong?

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

[wallfire-users] Pix Question

[Anabella C. <ana...@gm...>]

Hi,
I have a problem when parsing lines like this one
 Aug 11 16:04:25 10.101.5.50 <http://10.101.5.50> Aug 11 2005 15:54:01 :=20
%PIX-3-710003:
 I get the error
 pix2:581: warning: wrong log entry.
 Is this message type not supported?=20
 Thank you very much in advanced
Regards
Anabella

Re: [wallfire-users] Needing help with $count filter option

[Herve E. <rv...@wa...>]

On Thu, Sep 15, 2005 at 02:36:45PM -0600, James Lay wrote:

 Hi,

> If it were me, I would have it position dependent,i.e. :

> wflogs -f ''$start_time >=3D [00:00:00] && $count > 6'

> filters before count, and

> wflogs -f '$count > 6 && $start_time >=3D [00:00:00]'

> will count before filter.

Wow, seems like there's been some misunderstanding somewhere.

Well... the filtering expression is a mathematical expression, so it
has to obey to fundamental logic (and its operators, like "and",
"or", etc.) which among others implies that a && b is equivalent to
b && a, for example.

Besides, here is the current wflogs process:
- parse the logs (for netfilter logs lines, there are only one packet
  per line, so count =3D=3D 1 for each log entry/packet)
- logs entries are filtered (-f)
- logs entries are processed/changed (summarized, obfuscated, sorted,
  etc.). So it entries are summarized, after that process, similar entrie=
s
  are grouped into one entry, with a count reflecting the number of
  original entries (packets) that formed the group
- logs entries are "printed"

So there is not really a counting process by itself. Rather a process
that may update counts accordingly, among other things.
Take ipfilter logs, for example. Ipfilter log lines can have
a count > 1, which means that ipfilter can aggregate similar packets
in one log line already.
What you call counting in wflogs is in fact aggregating similar entries
(and increment the count accordingly, of course).
Filtering is definitely a clearly separated operation, which can take
place before or after (both make sense, and are different) this aggregati=
on.

Not only I'm unable to think of any algorithm that would enable to mix
the two operations (aggregating and filtering), but even if it was
technically possible, I would consider it very confusing to introduce
position dependent concepts in things that are position independant
(a && b <=3D> b && a) by essence.

The names you propose (--count-before-filter and
--filter-before-count) reflect your (biased) view, which seems to be
very "count-centric", whereas the packet counter is only one of
the many parameters one would want to base one's filter on.

> Just some thoughts.

I was proposing --filter-after-parsing and --filter-before-output
but it doesn't really reflect that it occurs _just after_ parsing
(before any additional processing) and _just before_ output (after
processing, if there actually is one).
That can be documented in the man page, though.

Or maybe --filter-before-processing and --filter-after-processing
would be more explicit... Or --filter-before-mangling and
--filter-after-mangling, I don't know.

Anyway, I'll release a version with these two options soon, like I
said. I guess you'll be interested by the second. Thanks for having
inspired this future improvment.

Cheers,

 Herv=E9

> On Thu, 15 Sep 2005 20:38:15 +0200
> Herve Eychenne <rv...@wa...> wrote:

> > On Thu, Sep 15, 2005 at 02:03:26AM -0600, James Lay wrote:
> >=20
> >  Hi!
> >=20
> > > Here's what I'm trying to do:
> >=20
> > > wflogs -i netfilter -f '$start_time >=3D [00:00:00] && $count > 1' =
-o
> > > html --sort=3Ddport,-count --resolve=3D0 --whois=3D0  /var/log/kern=
el >
> > > test.html
> >=20
> > > The above yields nothing at all :(  If I remove the $count > 1 then
> > > I get all sorts of info...including a lot of things that have count=
s
> > > above one.  Am I missing something?  Help!
> >=20
> > Oh, yes.
> > Filtering currently takes place before summary.
> > So, as netfilter logs lines concern only one packet at a time, $count
> > is always equal to 1 (for netfilter).
> >=20
> > I guess filtering _after_ summary would make sense too...
> > so we should probably enable both.
> >=20
> > Now, the question is : how would we name the long options so that it
> > is clear that
> > - the first filter is done before summary (or any other operation suc=
h
> >   as sort, obfusctation, etc...). In fact, it is done just after
> >   parsing, so maybe a name like --filter-after-parsing would be good
> > - the second filter is done after summary (and all), so a name like
> >   --filter-before-output would be good.
> >=20
> > Now, we must keep backward compatibility, by keeping the old -f
> > letter.
> >=20
> > So once --filter-before-output is implemented (which I intend to do i=
n
> > the very next days as it is only a few lines of code), I'll have to
> > choose if it would be better to assign -f to --filter-after-parsing
> > (which it was until now) or to --filter-before-output (-F been create=
d
> > and assigned to the other).
> > Assigning it to --filter-after-parsing would ensure strict
> > compatibility, but I don't know if it would be that clever, as that
> > would not make so much difference for most users to assign -f to
> > --filter-before-output, and it's probably what people would
> > intuitively expect (as most of them want to use the summary option).
> >=20
> > What do you think?
> >=20
> >  Herve
> >=20
> > --=20
> >  _
> > (=B0=3D  Herv=E9 Eychenne
> > //)
> > v_/_ WallFire project:  http://www.wallfire.org/

 Herve

--=20
 _
(=B0=3D  Herv=E9 Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: [wallfire-users] Needing help with $count filter option

[James L. <jl...@sl...>]

Herve,

Didn't address your question in my first email...but I would have these
long options:

--count-before-filter
--filter-before-count

James

On Thu, 15 Sep 2005 20:38:15 +0200
Herve Eychenne <rv...@wa...> wrote:

> On Thu, Sep 15, 2005 at 02:03:26AM -0600, James Lay wrote:
>=20
>  Hi!
>=20
> > Here's what I'm trying to do:
>=20
> > wflogs -i netfilter -f '$start_time >=3D [00:00:00] && $count > 1' -o
> > html --sort=3Ddport,-count --resolve=3D0 --whois=3D0  /var/log/kernel >=

> > test.html
>=20
> > The above yields nothing at all :(  If I remove the $count > 1 then
> > I get all sorts of info...including a lot of things that have counts
> > above one.  Am I missing something?  Help!
>=20
> Oh, yes.
> Filtering currently takes place before summary.
> So, as netfilter logs lines concern only one packet at a time, $count
> is always equal to 1 (for netfilter).
>=20
> I guess filtering _after_ summary would make sense too...
> so we should probably enable both.
>=20
> Now, the question is : how would we name the long options so that it
> is clear that
> - the first filter is done before summary (or any other operation such
>   as sort, obfusctation, etc...). In fact, it is done just after
>   parsing, so maybe a name like --filter-after-parsing would be good
> - the second filter is done after summary (and all), so a name like
>   --filter-before-output would be good.
>=20
> Now, we must keep backward compatibility, by keeping the old -f
> letter.
>=20
> So once --filter-before-output is implemented (which I intend to do in
> the very next days as it is only a few lines of code), I'll have to
> choose if it would be better to assign -f to --filter-after-parsing
> (which it was until now) or to --filter-before-output (-F been created
> and assigned to the other).
> Assigning it to --filter-after-parsing would ensure strict
> compatibility, but I don't know if it would be that clever, as that
> would not make so much difference for most users to assign -f to
> --filter-before-output, and it's probably what people would
> intuitively expect (as most of them want to use the summary option).
>=20
> What do you think?
>=20
>  Herve
>=20
> --=20
>  _
> (=B0=3D  Herv=E9 Eychenne
> //)
> v_/_ WallFire project:  http://www.wallfire.org/
>=20
>=20
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server.
> Download it for free - -and be entered to win a 42" plasma tv or your
> very own Sony(tm)PSP.  Click here to play:
> http://sourceforge.net/geronimo.php
> _______________________________________________ wallfire-users
> mailing list wal...@li...
> https://lists.sourceforge.net/lists/listinfo/wallfire-users

Re: [wallfire-users] Needing help with $count filter option

[James L. <jl...@sl...>]

Herve,

If it were me, I would have it position dependent,i.e. :

wflogs -f ''$start_time >=3D [00:00:00] && $count > 6'

filters before count, and

wflogs -f '$count > 6 && $start_time >=3D [00:00:00]'

will count before filter.

Personally I know I would want to count before filter...I think it
would chew up less cpu cycles by dropping my one and two hit items and
then filtering.  Also, this way you wouldn't have to have any new
commands....and, it makes sense logically...just by looking at the line
you could be able to see where the counting occurs.

Just some thoughts.

James


On Thu, 15 Sep 2005 20:38:15 +0200
Herve Eychenne <rv...@wa...> wrote:

> On Thu, Sep 15, 2005 at 02:03:26AM -0600, James Lay wrote:
>=20
>  Hi!
>=20
> > Here's what I'm trying to do:
>=20
> > wflogs -i netfilter -f '$start_time >=3D [00:00:00] && $count > 1' -o
> > html --sort=3Ddport,-count --resolve=3D0 --whois=3D0  /var/log/kernel >=

> > test.html
>=20
> > The above yields nothing at all :(  If I remove the $count > 1 then
> > I get all sorts of info...including a lot of things that have counts
> > above one.  Am I missing something?  Help!
>=20
> Oh, yes.
> Filtering currently takes place before summary.
> So, as netfilter logs lines concern only one packet at a time, $count
> is always equal to 1 (for netfilter).
>=20
> I guess filtering _after_ summary would make sense too...
> so we should probably enable both.
>=20
> Now, the question is : how would we name the long options so that it
> is clear that
> - the first filter is done before summary (or any other operation such
>   as sort, obfusctation, etc...). In fact, it is done just after
>   parsing, so maybe a name like --filter-after-parsing would be good
> - the second filter is done after summary (and all), so a name like
>   --filter-before-output would be good.
>=20
> Now, we must keep backward compatibility, by keeping the old -f
> letter.
>=20
> So once --filter-before-output is implemented (which I intend to do in
> the very next days as it is only a few lines of code), I'll have to
> choose if it would be better to assign -f to --filter-after-parsing
> (which it was until now) or to --filter-before-output (-F been created
> and assigned to the other).
> Assigning it to --filter-after-parsing would ensure strict
> compatibility, but I don't know if it would be that clever, as that
> would not make so much difference for most users to assign -f to
> --filter-before-output, and it's probably what people would
> intuitively expect (as most of them want to use the summary option).
>=20
> What do you think?
>=20
>  Herve
>=20
> --=20
>  _
> (=B0=3D  Herv=E9 Eychenne
> //)
> v_/_ WallFire project:  http://www.wallfire.org/
>=20
>=20
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server.
> Download it for free - -and be entered to win a 42" plasma tv or your
> very own Sony(tm)PSP.  Click here to play:
> http://sourceforge.net/geronimo.php
> _______________________________________________ wallfire-users
> mailing list wal...@li...
> https://lists.sourceforge.net/lists/listinfo/wallfire-users

Re: [wallfire-users] Needing help with $count filter option

[Herve E. <rv...@wa...>]

On Thu, Sep 15, 2005 at 02:03:26AM -0600, James Lay wrote:

 Hi!

> Here's what I'm trying to do:

> wflogs -i netfilter -f '$start_time >=3D [00:00:00] && $count > 1' -o
> html --sort=3Ddport,-count --resolve=3D0 --whois=3D0  /var/log/kernel >
> test.html

> The above yields nothing at all :(  If I remove the $count > 1 then I
> get all sorts of info...including a lot of things that have counts
> above one.  Am I missing something?  Help!

Oh, yes.
Filtering currently takes place before summary.
So, as netfilter logs lines concern only one packet at a time, $count
is always equal to 1 (for netfilter).

I guess filtering _after_ summary would make sense too...
so we should probably enable both.

Now, the question is : how would we name the long options so that it is
clear that
- the first filter is done before summary (or any other operation such
  as sort, obfusctation, etc...). In fact, it is done just after
  parsing, so maybe a name like --filter-after-parsing would be good
- the second filter is done after summary (and all), so a name like
  --filter-before-output would be good.

Now, we must keep backward compatibility, by keeping the old -f letter.

So once --filter-before-output is implemented (which I intend to do in
the very next days as it is only a few lines of code), I'll have to
choose if it would be better to assign -f to --filter-after-parsing
(which it was until now) or to --filter-before-output (-F been created
and assigned to the other).
Assigning it to --filter-after-parsing would ensure strict
compatibility, but I don't know if it would be that clever, as that
would not make so much difference for most users to assign -f to
--filter-before-output, and it's probably what people would
intuitively expect (as most of them want to use the summary option).

What do you think?

 Herve

--=20
 _
(=B0=3D  Herv=E9 Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

[wallfire-users] Needing help with $count filter option

[James L. <jl...@sl...>]

Hey all!

Here's what I'm trying to do:

wflogs -i netfilter -f '$start_time >= [00:00:00] && $count > 1' -o
html --sort=dport,-count --resolve=0 --whois=0  /var/log/kernel >
test.html

The above yields nothing at all :(  If I remove the $count > 1 then I
get all sorts of info...including a lot of things that have counts
above one.  Am I missing something?  Help!

James

Re: [wallfire-users] wfnetobjs install problem

[Herve E. <rv...@wa...>]

On Tue, Jun 28, 2005 at 02:33:38PM -0400, jaskaran singh wrote:

> i am unable to make the wfnetobjs-0.2.4 properly. this is the error i am 
> receiving. strsep is a regular c/c++ function so why can't it be compiled.

> ipaddr.cc: In static member function `static bool wf_ipaddr::check(const 
> char*)
> ':
> ipaddr.cc:219: error: `strsep' undeclared (first use this function)
> ipaddr.cc:219: error: (Each undeclared identifier is reported only once for
> each function it appears in.)
> make[2]: *** [ipaddr.lo] Error 1
> make[2]: Leaving directory `/usr/local/adm/src/wfnetobjs-0.2.4/netobjs'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory `/usr/local/adm/src/wfnetobjs-0.2.4'
> make: *** [all-recursive-am] Error 2

It compiles fine for me like it was, but please add
#include <string.h>
to the other system includes in ipaddr.cc.
That should do the trick. It's now "fixed" in my tree.
Thanks for the report.

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

[wallfire-users] wfnetobjs install problem

[jaskaran s. <js...@fd...>]

i am unable to make the wfnetobjs-0.2.4 properly. this is the error i am 
receiving. strsep is a regular c/c++ function so why can't it be compiled.

ipaddr.cc: In static member function `static bool wf_ipaddr::check(const 
char*)
    ':
ipaddr.cc:219: error: `strsep' undeclared (first use this function)
ipaddr.cc:219: error: (Each undeclared identifier is reported only once for
    each function it appears in.)
make[2]: *** [ipaddr.lo] Error 1
make[2]: Leaving directory `/usr/local/adm/src/wfnetobjs-0.2.4/netobjs'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/local/adm/src/wfnetobjs-0.2.4'
make: *** [all-recursive-am] Error 2
-- 
Jaskaran Singh
Systems Analyst
******************************
Fairleigh Dickinson University
1000 River Road
Teaneck,NJ 07666 TBH1-01
Tel 201-692-2414
Fax 201-692-2494
******************************

Re: [wallfire-users] how to use the files generated by wfconvert netfilter output filter ?

[Herve E. <rv...@wa...>]

On Mon, Mar 21, 2005 at 04:20:44PM +0100, Eric Lacroix wrote:

 Hi Eric,

> I'm trying to use wfconvert 0.4.1 to configure a netfilter simple
> firewall. I don't know what to do with the scripts generated. They seem
> to be 'sh' scripts, but there's still some stuff missing.

Yes.

> I know iptables a very little, what may explains what I don't
> understand.

Did you read the README?
Here's a copy-paste of the interesting parts:
--------------------------------------------------------------------

For the moment, you can play with the example given under
doc/example.wf,
which is quite self-explainable, I think.
$ wfconvert -i wallfire -o netfilter example.wf
translates the rules into a netfilter/iptables script, by producing a
wallfire profile directory (probably in an insecure way, for the
moment :-().
See netfilter output below.

[...]

Netfilter output
----------------

WallFire provides a powerful infrastructure for netfilter rules.  You
can store several configurations on your machine, each one being for a
given firewalling ruleset in a given place, which is useful for
laptops, for example. Such a configuration is called a "profile".

$ wfconvert -i wallfire -o netfilter example.wf
produces a temporary directory containing scripts using iptables or
iptables-save commands. This directory is a "profile".
You can then apply the rules of this profile with the command named
wallfire.
If the above wfconvert execution produced files in the directory
/tmp/dir,
you can (hopefully) activate the firewalling with:
# WF_CONFDIR=3D/tmp/dir wallfire start

--------------------------------------------------------------------

Is should be sufficient for a start, isn't?  If not, what can I do to mak=
e
it more clear?
Anyway, I know that documentation is far from complete at that stage...
sorry about that.

 Herve

--=20
 _
(=B0=3D  Herv=E9 Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

[wallfire-users] how to use the files generated by wfconvert netfilter output filter ?

[Eric L. <eri...@fr...>]

Hello rv,

I'm trying to use wfconvert 0.4.1 to configure a netfilter simple
firewall. I don't know what to do with the scripts generated. They seem
to be 'sh' scripts, but there's still some stuff missing.
I know iptables a very little, what may explains what I don't
understand.

Regards,

Eric

[wallfire-users] (no subject)

[Herve E. <rv...@wa...>]

kelbert
Bcc:=20
Subject: Re: [wallfire-users] wflogs_report error when run from cron.dail=
y
Reply-To:=20
In-Reply-To: <111...@lo...>

On Thu, Mar 10, 2005 at 10:07:15AM +0100, wolvie.news wrote:

 Hi,

> __MARK__ Dumping FILTER: $start_time >=3D [] && $start_time < [ 1 day]
> error before `[': syntax error
> wflogs: Error: wrong filter expression.
> run-parts: /etc/cron.daily/wflogs_report exited with return code 1

> KERN.LOG.0 ends with:
> Mar 10 06:26:42 localhost kernel: blah...

> (after logrotate)
> KERN.LOG starts with:
> Mar 10 06:28:06 localhost kernel: blah...

> So at 06:27:37 kern.log is empty (or I'm wrong). Can this be considered
> an bug?

No, I don't think so... ;-) Log files can perfectly be empty when nothing
was logged (yet). ;-)

> Howto fix this not complicating the whole script too much?
> Concatenating the INPUT_FILE* files into some temp file? Maybe... The
> problem is that even if wflogs doesn't hit the exact time of log
> rotation it still will not give the full daily report.

> Hope this was somehow helpful and my deduction was right.

You're right, this occurs when the file is empty, which breaks the
script. I asked Jean-Michel Kelbert (Debian package maintainer) if he
wanted to take care of that, as he wrote the script initially.
I'm waiting for his feedback or patch, even if he seems quite busy
these days.

Thanks for your report, stay tuned.

 Herve

--=20
 _
(=B0=3D  Herv=E9 Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: [wallfire-users] wflogs_report error when run from cron.daily

[wolvie.news <wol...@wp...>]

W liście z pon, 28-02-2005, godz. 17:00, Herve Eychenne pisze: 
> On Mon, Feb 28, 2005 at 05:51:54PM +0100, wolvie.news wrote:
> 
>  Hi,
> 
> > I've encountered a problem running the J.M. Kelbert's wflogs_report
> > script from the cron.daily crontask. I get the following error e-mail
> > message:
> > 
> >         Subject: Cron <root@core> test -x /usr/sbin/anacron || run-parts
> >         --report
> >         +/etc/cron.daily
> 
> >         /etc/cron.daily/wflogs_report:
> >         error before `[': syntax error
> >         wflogs: Error: wrong filter expression.
> >         run-parts: /etc/cron.daily/wflogs_report exited with return code
> >         1
> > 
> > and empty file in /var/www/yyyy/mm/ directory. 
> > I'm running wflogs on my sarge machine installed via the apt-get
> > command. Since I'm a rookie I cannot find the solution to the problem.
> > Running the same script from bash shell and from the root's crontab file
> > is fine.
> > 
> > Thanks in advance for any hints/solutions.
> 
> Yes, we are aware of that.  This error seems to show up in a unpredictable
> way, so it happens from time to time.  I guess it's a quoting problem in
> the script, but I couldn't figure out what causes this exactly, nor trigger
> it in a reproductible way.
> 
> Well... I guess the "best" solution is to add "set -x" at the begining of the
> script, and wait for the error to occur again.
> Feel free to do that and report the detailed error dump when you'lle face
> this bug again.
> I'll probably do the same on my machine.
> Cheers,
> 
>  Herve
> 
> --
>  _
> ( =  Herve Eychenne
> //)
> v_/_ WallFire project:  http://www.wallfire.org/
> 

I've done so research and here are the results:
Firstly I wrote a script generating a report on hourly basis (via
cron.hourly) which was supposed to overwrite a file with a -24h report .
Similar errors occured from time to time. I thought it was a cron/bash
env/locales error (parsing a date) but after doing some dumps I knew I
was wrong. 
Secondly I did some dumps (echo $whatever_might_be_wrong)  in an
oryginal /etc/cron.daily/wflogs_report script (still thinking it was a
problem related to date localisation). What I had to do next was to wait
and read the mail. 
And it hit me today: The date looks ok. The file is generated although
it's empty. And the filter expression is somehow wrong. I took a look at
the logs and compared the timestamps of lines in logs with the time of
running wflogs_report. And... there is no error.
DATE_BEGIN_LOG=`head -n 1 $INPUT_FILE | awk '{print $1,$2}'`
does it's job.. or doesn't it? :) Example:

MAIL FROM CRON:

/etc/cron.daily/wflogs_report:
Running /etc/cron.daily/wflogs_report
(...)
export LC_ALL="C" set.
Date looks like this:
Thu Mar 10 06:27:37 CET 2005
(...)
__MARK__ Dumping FILTER: $start_time >= [] && $start_time < [ 1 day]
error before `[': syntax error
wflogs: Error: wrong filter expression.
run-parts: /etc/cron.daily/wflogs_report exited with return code 1

KERN.LOG.0 ends with:
Mar 10 06:26:42 localhost kernel: blah...

(after logrotate)
KERN.LOG starts with:
Mar 10 06:28:06 localhost kernel: blah...

So at 06:27:37 kern.log is empty (or I'm wrong). Can this be considered
an bug? Howto fix this not complicating the whole script too much?
Concatenating the INPUT_FILE* files into some temp file? Maybe... The
problem is that even if wflogs doesn't hit the exact time of log
rotation it still will not give the full daily report.

Hope this was somehow helpful and my deduction was right.

-- 
Lukasz Kusmirek wolvie<dot>news<at>wp<dot>pl

[wallfire-users] [ANNOUNCE] Release of wfconvert-0.4.1

[Herve E. <rv...@wa...>]

 Hi!

here's new version wfconvert-0.4.1

wfconvert is the firewall rule compiler of the WallFire project.

Bugfixes from 0.4.0:

- wfwizard:
  * fixed a crash occuring when there was no Internet access
  * no NAT rules needed when there is no Internet access
  * added correct NAT (and corresponding filter) rules
  * fixed a bug with the (stupid) inversion of the result of the
    question "Do you have an Internet access?"

User-visible changes from 0.4.0:

- wfwizard:
  * warn when removing an interface pointing to an unknown network
  * allow again the modification of the name of a network
- wfconvert:
  * added usage examples to "wfconvert --help" output
- wallfire language:
  * interfaces can now be commented (comments introduced by #, as
    usual)
- wallfire_xml language:
  * interfaces can now be commented
  * the whole XML dump is now enclosed into a ruleset tag
- a language revision number has been added to files parsed and
  produced by wallfire and wallfire_xml input/output modules. This
  enables to know exactly in which format version they are. Of course,
  the version number is different for both formats and will be
  incremented each time the syntax changes.
- added ignore_version config option to wallfire input module. This
  option enables to ignore the error if trying to parse an unsupported
  language version.


Full ChangeLog from the previous version:
  http://www.wallfire.org/wfconvert/ChangeLog-0.4.1

Download:
  http://www.wallfire.org/download/wfconvert-0.4.1.tar.gz
  http://www.wallfire.org/download/wfconvert-0.4.1.tar.bz2



In order to compile wfconvert and wflogs, remember that you need the last
version of wfnetobjs. Read INSTALL instructions in both source trees.
wfnetobjs is available at:
http://www.wallfire.org/wfnetobjs/
Both source trees must be untarred from the same directory, and you'll ha=
ve
to rename wfnetobjs-version to wfnetobjs, or create a symbolic link.


wfconvert homepage:
  http://www.wallfire.org/wfconvert/

General information about the WallFire project can be found at:
  http://www.wallfire.org/

Happy firewalling,

 Herve

--=20
 _
(=B0=3D  Herv=E9 Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

[wallfire-users] [ANNOUNCE] Release of wfnetobjs-0.2.4

[Herve E. <rv...@wa...>]

 Hi!

here's new version wfnetobjs-0.2.4.
wfconvert-0.4.1 is coming very soon.

wfnetobjs is the network objects library of the WallFire project.


Changes from 0.2.3:

* netobjs/: host.cc, wfhost.h: added wf_host::rename_network(...)
* rvlog/Makefile.am: install rvlog.h header
* moved config/ directory in wfnetobjs, containing module config manageme=
nt.
  This used to be duplicated code included directly into wfconvert and wf=
logs.
* netobjs/host.cc: improved wf_host::debugprint(...)
* netobjs/firewall.cc, netobjs/host.cc, netobjs/iface.cc,
  netobjs/ipaddr.cc, netobjs/ipaddr_range.cc, netobjs/macaddr.cc,
  netobjs/metahost.cc, netobjs/metahost_elem.cc,
  netobjs/metaport.cc, netobjs/metaport_elem.cc,
  netobjs/network.cc, netobjs/wffirewall.h, netobjs/wfhost.h,
  netobjs/wfiface.h, netobjs/wfipaddr.h, netobjs/wfipaddr_range.h,
  netobjs/wfmacaddr.h, netobjs/wfmetahost.h, netobjs/wfmetaport.h,
  netobjs/wfnetwork.h, netobjs/wfservice.h, tools/Makefile.am,
  tools/debiface2netobj.cc: add #ifdef DEBUG around debugprint()
  methods


Download:
  http://www.wallfire.org/download/wfnetobjs-0.2.4.tar.gz
  http://www.wallfire.org/download/wfnetobjs-0.2.4.tar.bz2




wfnetobjs homepage:
  http://www.wallfire.org/wfnetobjs/

General information about the WallFire project can be found at:
  http://www.wallfire.org/

Happy firewalling,

 Herve

--=20
 _
(=B0=3D  Herv=E9 Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/