hank-devel — Hank development discussion

[hank-devel] Snort in need of overhaul, time for Hank to reappear?

[Josh M. <jos...@tr...>]

In light of the article and /. discussion on Snort's creator saying 
Snort needs an overhaul, is there an interest in drawing more attention 
to Hank? I'm willing to help out where I can if there's an interest.

http://slashdot.org/article.pl?sid=04/05/24/1113219&mode=nested&tid=126&tid=172
http://www.zdnet.com.au/news/security/0,2000061744,39148508,00.htm


Josh

[hank-devel] Schema

[Aaron B. <aar...@in...>]

Is there a schema for the XML files? Either XSD or DTD?

=20

[hank-devel] signs of life?

[Josh M. <jos...@tr...>]

I was at the NYLUG presentation on Hank and thought it sounded great. 
Anyone on this list? Anyone using Hank, or interested in helping to 
revive it?

Josh

Re: [hank-devel] problems compiling hank

[<tl...@mi...>]

On Sun, 16 Jun 2002, Patrick McHardy wrote:

> I want to use it to pass packets from netfilter to hank to match them
> against known service signatures (
> like ssh, edonkey, http, ...) and then change their nfmark value. The
> nfmark is then used for QoS classification.
> Do you think it is suited for this task ?

Yep, no prob.  If you want detailed upper-level protocol analysis, like
only matching ssh that uses DSA keys or http 1.1 or something, then
you'll have to write a protocol engine for ssh or http, but other than
that it should be fine.

> BTW, i managed to compile hank now and it seems to work, however with
> libpcap i get segmentation faults (libipq works fine):
>
> (gdb) bt
> #0  0x40023c50 in pcap_dispatch () from /usr/lib/libpcap.so.0
> #1  0x080562c4 in get_packet (id=33554432) at paengine_pcap.c:141
> #2  0x08056f55 in main (argc=3, argv=0xbffffae4) at hank.c:361
>
> (gdb) disassemble
> 0x40023c50 <pcap_dispatch+32>:    cmpl   $0x0,0x14(%eax)
>
> (gdb) info registers eax
> eax            0x0    0

The problem line is this:

141    if((i=pcap_dispatch(config.pcap, 1, &callback, NULL))==0){

Try this patch:

**********************************************************************
--- paengine_pcap.c.orig	Sun Jun 16 16:49:45 2002
+++ paengine_pcap.c	Sun Jun 16 16:50:22 2002
@@ -138,6 +138,8 @@
 	}

 	/* do callback */
+	printf("DOING CALLBACK with config %p, callback %p.\n",
+		config.pcap, &callback);
 	if((i=pcap_dispatch(config.pcap, 1, &callback, NULL))==0){
 		/* FIXME: log */
 		/* fprintf(stderr, "AIEE!  pcap_dispatch returned %i!\n", i); */
**********************************************************************

that will at least tell if they're not getting set somehow.

> I used the files coming with hank  except for one change: i removed the
> "firewall" part from rules.xml because hank complained
> about unrecognized keyword "firewall" ...

That's fine.  That whole area is on the verge of a cleanup; it's
got rather a lot of cruft in it.  Unfortunately, right now I am
in the middle of another project; if someone wanted to undertake
the work, I'd be happy to send guidance.

--
Todd Lewis
tl...@mi...

"Bonsoir, Monet.  Work, work.  It is the most beautiful thing there is
       in the world."  -- Clemenceau

Re: [hank-devel] problems compiling hank

[Patrick M. <ka...@tr...>]

I want to use it to pass packets from netfilter to hank to match them 
against known service signatures (
like ssh, edonkey, http, ...) and then change their nfmark value. The 
nfmark is then used for QoS classification.
Do you think it is suited for this task ?
BTW, i managed to compile hank now and it seems to work, however with 
libpcap i get segmentation faults (libipq works fine):

(gdb) bt
#0  0x40023c50 in pcap_dispatch () from /usr/lib/libpcap.so.0
#1  0x080562c4 in get_packet (id=33554432) at paengine_pcap.c:141
#2  0x08056f55 in main (argc=3, argv=0xbffffae4) at hank.c:361

(gdb) disassemble
0x40023c50 <pcap_dispatch+32>:    cmpl   $0x0,0x14(%eax)

(gdb) info registers eax
eax            0x0    0

I used the files coming with hank  except for one change: i removed the 
"firewall" part from rules.xml because hank complained
about unrecognized keyword "firewall" ...

bye
patrick


tl...@mi... wrote:

>If I may ask, since Hank is on something of a hiatus at the moment,
>how did you find out about Hank?  What is your interest in the project?
>
>--
>Todd Lewis
>tl...@mi...
>
>"Bonsoir, Monet.  Work, work.  It is the most beautiful thing there is
>       in the world."  -- Clemenceau
>

Re: [hank-devel] problems compiling hank

[<tl...@mi...>]

If I may ask, since Hank is on something of a hiatus at the moment,
how did you find out about Hank?  What is your interest in the project?

--
Todd Lewis
tl...@mi...

"Bonsoir, Monet.  Work, work.  It is the most beautiful thing there is
       in the world."  -- Clemenceau

Re: [hank-devel] problems compiling hank

[<tl...@mi...>]

On Sat, 15 Jun 2002, Patrick McHardy wrote:

> tl...@mi... wrote:
>
> >Dude, that's wierd.  My german is a little rusty; what does
> >
> >	> grep: arp.hX: Datei oder Verzeichnis nicht gefunden
> >
> >mean?  Data or Verzeichnis not found?
> >
> it means "No such file or directory" ..

Did I mention that I'm rusty?  8^)

> I also don't know much about it, but i guess it should be easy to locate
> the problem .. i'll send you a patch as soon as i've fixed it.

Groovy.

--
Todd Lewis
tl...@mi...

"Bonsoir, Monet.  Work, work.  It is the most beautiful thing there is
       in the world."  -- Clemenceau

Re: [hank-devel] problems compiling hank

[Patrick M. <ka...@tr...>]

tl...@mi... wrote:

>On Sat, 15 Jun 2002, Patrick McHardy wrote:
>
>>Hi.
>>I had some troubles compiling hang,
>>when i wanted to produce english make output i tried LANG=en make and
>>suddenly it worked ?!
>>
>>$ make
>>rm -f *.o pe_* pes hank pe/*.h[a-zX] pe/pe_*.[ch] gmon.out hdt_blob_test
>>rcsids.c
>>make peos
>>make[1]: Entering directory `/home/kaber/test/hank-0.5.1'
>>cd pe && rm -f *X && ./Xdoer *.xml && cp pe* .. && cd ..
>>Host byte order: endian_little
>>Using arp.hX, arp, ARP.
>>grep: arp.hX: Datei oder Verzeichnis nicht gefunden
>>ERROR: Protocol endianness () not specified in file arp.hX; exiting.
>>make[1]: *** [peos] Fehler 255
>>make[1]: Leaving directory `/home/kaber/test/hank-0.5.1'
>>make: *** [hank] Fehler 2
>>
>>---------------------------
>>
>>$ LANG=en make
>>rm -f *.o pe_* pes hank pe/*.h[a-zX] pe/pe_*.[ch] gmon.out hdt_blob_test
>>rcsids.c
>>make peos
>>make[1]: Entering directory `/home/kaber/test/hank-0.5.1'
>>cd pe && rm -f *X && ./Xdoer *.xml && cp pe* .. && cd ..
>>Host byte order: endian_little
>>Using arp.hX, arp, ARP.
>>Skipping disabled pe: ddp.hX
>>Using dns.hX, dns, DNS.
>>Using ethernet.hX, ethernet, ETHERNET.
>>Using http.hX, http, HTTP.
>>Using icmp.hX, icmp, ICMP.
>>Using ipv4.hX, ipv4, IPV4.
>>Using ntp.hX, ntp, NTP.
>>Using oncrpc.hX, oncrpc, ONCRPC.
>>Skipping disabled pe: ppp.hX
>>Using tcp.hX, tcp, TCP.
>>Skipping disabled pe: tftp.hX
>>Using udp.hX, udp, UDP.
>>Skipping disabled pe: udp2.hX
>>make `gcc -E targets.c |grep '^pe'`
>>....
>>
>>Has someone got a explanation for this ?
>>
>
>Dude, that's wierd.  My german is a little rusty; what does
>
>	> grep: arp.hX: Datei oder Verzeichnis nicht gefunden
>
>mean?  Data or Verzeichnis not found?
>
it means "No such file or directory" ..

>
>The error message comes from pe/Xdoer:
>
>   pendian=`grep '^E' $i | awk '{print $2}'`
>   if [ -z "$pendian" ]
>   then
>      echo "ERROR: Protocol endianness ($pendian) not specified in file $i; exiting." >&2
>      exit -1
>   fi
>
>So, maybe there are i18n issues with grep or with xsltproc that result
>in the endian macro ("E endian_big") not getting set.
>
>Being an English speaker, I don't understand a whole lot about i18n,
>but if you wanted to track down the problem and submit a fix, I'd
>be happy to accept it.
>

I also don't know much about it, but i guess it should be easy to locate 
the problem .. i'll send you a patch as soon as
i've fixed it.

bye
patrick

Re: [hank-devel] problems compiling hank

[<tl...@mi...>]

On Sat, 15 Jun 2002, Patrick McHardy wrote:

> Hi.
> I had some troubles compiling hang,
> when i wanted to produce english make output i tried LANG=en make and
> suddenly it worked ?!
>
> $ make
> rm -f *.o pe_* pes hank pe/*.h[a-zX] pe/pe_*.[ch] gmon.out hdt_blob_test
> rcsids.c
> make peos
> make[1]: Entering directory `/home/kaber/test/hank-0.5.1'
> cd pe && rm -f *X && ./Xdoer *.xml && cp pe* .. && cd ..
> Host byte order: endian_little
> Using arp.hX, arp, ARP.
> grep: arp.hX: Datei oder Verzeichnis nicht gefunden
> ERROR: Protocol endianness () not specified in file arp.hX; exiting.
> make[1]: *** [peos] Fehler 255
> make[1]: Leaving directory `/home/kaber/test/hank-0.5.1'
> make: *** [hank] Fehler 2
>
> ---------------------------
>
> $ LANG=en make
> rm -f *.o pe_* pes hank pe/*.h[a-zX] pe/pe_*.[ch] gmon.out hdt_blob_test
> rcsids.c
> make peos
> make[1]: Entering directory `/home/kaber/test/hank-0.5.1'
> cd pe && rm -f *X && ./Xdoer *.xml && cp pe* .. && cd ..
> Host byte order: endian_little
> Using arp.hX, arp, ARP.
> Skipping disabled pe: ddp.hX
> Using dns.hX, dns, DNS.
> Using ethernet.hX, ethernet, ETHERNET.
> Using http.hX, http, HTTP.
> Using icmp.hX, icmp, ICMP.
> Using ipv4.hX, ipv4, IPV4.
> Using ntp.hX, ntp, NTP.
> Using oncrpc.hX, oncrpc, ONCRPC.
> Skipping disabled pe: ppp.hX
> Using tcp.hX, tcp, TCP.
> Skipping disabled pe: tftp.hX
> Using udp.hX, udp, UDP.
> Skipping disabled pe: udp2.hX
> make `gcc -E targets.c |grep '^pe'`
> ....
>
> Has someone got a explanation for this ?

Dude, that's wierd.  My german is a little rusty; what does

	> grep: arp.hX: Datei oder Verzeichnis nicht gefunden

mean?  Data or Verzeichnis not found?

The error message comes from pe/Xdoer:

   pendian=`grep '^E' $i | awk '{print $2}'`
   if [ -z "$pendian" ]
   then
      echo "ERROR: Protocol endianness ($pendian) not specified in file $i; exiting." >&2
      exit -1
   fi

So, maybe there are i18n issues with grep or with xsltproc that result
in the endian macro ("E endian_big") not getting set.

Being an English speaker, I don't understand a whole lot about i18n,
but if you wanted to track down the problem and submit a fix, I'd
be happy to accept it.

--
Todd Lewis
tl...@mi...

"Bonsoir, Monet.  Work, work.  It is the most beautiful thing there is
       in the world."  -- Clemenceau

[hank-devel] problems compiling hank

[Patrick M. <ka...@tr...>]

Hi.
I had some troubles compiling hang,
when i wanted to produce english make output i tried LANG=en make and 
suddenly it worked ?!

$ make
rm -f *.o pe_* pes hank pe/*.h[a-zX] pe/pe_*.[ch] gmon.out hdt_blob_test 
rcsids.c
make peos
make[1]: Entering directory `/home/kaber/test/hank-0.5.1'
cd pe && rm -f *X && ./Xdoer *.xml && cp pe* .. && cd ..
Host byte order: endian_little
Using arp.hX, arp, ARP.
grep: arp.hX: Datei oder Verzeichnis nicht gefunden
ERROR: Protocol endianness () not specified in file arp.hX; exiting.
make[1]: *** [peos] Fehler 255
make[1]: Leaving directory `/home/kaber/test/hank-0.5.1'
make: *** [hank] Fehler 2

---------------------------

$ LANG=en make
rm -f *.o pe_* pes hank pe/*.h[a-zX] pe/pe_*.[ch] gmon.out hdt_blob_test 
rcsids.c
make peos
make[1]: Entering directory `/home/kaber/test/hank-0.5.1'
cd pe && rm -f *X && ./Xdoer *.xml && cp pe* .. && cd ..
Host byte order: endian_little
Using arp.hX, arp, ARP.
Skipping disabled pe: ddp.hX
Using dns.hX, dns, DNS.
Using ethernet.hX, ethernet, ETHERNET.
Using http.hX, http, HTTP.
Using icmp.hX, icmp, ICMP.
Using ipv4.hX, ipv4, IPV4.
Using ntp.hX, ntp, NTP.
Using oncrpc.hX, oncrpc, ONCRPC.
Skipping disabled pe: ppp.hX
Using tcp.hX, tcp, TCP.
Skipping disabled pe: tftp.hX
Using udp.hX, udp, UDP.
Skipping disabled pe: udp2.hX
make `gcc -E targets.c |grep '^pe'`
....

Has someone got a explanation for this ?
Bye
Patrick

[hank-devel] new release

[<tl...@mi...>]

http://hank.sourceforge.net/hank-0.5.1-raw.tar.gz
http://hank.sourceforge.net/hank-0.5.1-cooked.tar.gz

This fixes a lot of the bugs in 0.5.0, most of all the hdt_blob.c mistakes
steming from my inability to understand CVS branches.  I'll be uploading
binaries soon.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

[hank-devel] Re: hank questions (fwd)

[<tl...@mi...>]

Forwarded FYI.  Please, if you're sending me a question on Hank, please
consider sending it instead to the mailing list if it's a question on
which others might have interest, like this one.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

---------- Forwarded message ----------
Date: Fri, 18 Jan 2002 13:22:39 -0500 (EST)
From: tl...@mi...
To: K.William Young <wy...@wy...>
Subject: Re: hank questions

I don't think that libipq allows rewrites.  I need to update the
capabilities flags of the paenine to indicate that.  The way to modify
a packet is to drop the original and inject the new with libpacket or
whatever; ditto for creating new ones.  However, if you want to pursue
rewriting within the main flow of things, I suggest taking a look at some
other libipq applications that rewrite packets, seeing what they do, and
then let's figure out how to make it work.  That's the long and correct
way; dropping and injecting anew is the easy, portable and gross way to do it.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

On 18 Jan 2002, K.William Young wrote:

> I read firewall.c and matcher_tlewis.c.
> 
> 1) To alter a packet, can I make the changes to the packet->payload and then let libipq take care of the rest?
> 
> 2) Can I generate a new packet* and then insert it with ipq_set_verdict? Is this the best way to add ICMP rejects?
> 
> Will
> 
> K. William Young
> wy...@wy...
> 

[hank-devel] Approval?

[Ben R. <are...@ho...>]

When I set the list up, I told it to require moderator approval to join the 
list. I've had no reason to deny anyone, though, and I don't forsee the need 
to do so anytime soon, so unless anyone has any specific complaints I intend 
to switch the list to open (ie, no approval required). If anyone does 
disagree with this decision, please take it up with me via email 
(are...@ho...) and I'll reconsider. If I don't have any complaints 
by Friday I'm going to open up the list.

Your friendly list moderator
Ben


_________________________________________________________________
Chat with friends online, try MSN Messenger: http://messenger.msn.com

[hank-devel] Re: announcing new applicatio: Hank

[<tl...@mi...>]

The problem that you describe strikes me as a netfilter issue, not a
Hank issue.  Right now, Hank's netfilter paengine just accepts packets
and handles them.  I would answer your question with another one: what
tools does netfilter give userspace programs to deal with this class of
behavior?  I don't see a way to deal with this problem from the user-space
side of things, but if there is one, then I would love to hear about it.
If/when netfilter/ipqueue has something like BSD divert sockets' port
numbering system where you can have different user-space targets with
potentially different behaviors, Hank will support it.  Until then,
unless there's another solution, complain to the netfilter-volken, not me.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

On Mon, 7 Jan 2002, Mauricio Oliveto wrote:

> Very interesting !
> But I have a question: How does Hank deals with SNATed
> (POSTROUTING) packets received from netfilter ip queue
> on FORWARD hook if you want to do some filtering based
> on the ip src that netfilter will put at SNAT
> POSTROUTING ?
> 
> What you think?
> moliveto
> 
> --- tl...@mi... wrote:
> > Allen,
> > 
> > I've copied this response to the netfilter and hank
> > lists,
> > since it's probably a common question.
> > 
> > The packet acquisition mechanism is modular.  You
> > can chose
> > pcap, netfilter or divert at runtime.  E.g.,
> > 
> > hank -p pcap -P int=eth0
> > hank -p pcap -P rf=./pcap_capture_file
> > hank -p netfilter
> > 
> > all work with hank.  Take a look at paengine.h and
> > paengine_*.c in the
> > source code to see what's going on.
> > 
> > --
> > Todd Lewis
> > tl...@mi...
> > 
> > Q: How many marxists does it take to screw in a
> > lightbulb?
> > A: None - the bulb contains the seeds of its own
> > revolution. 
> > 
> > On Mon, 7 Jan 2002, Allen Francom wrote:
> > 
> > > 
> > > Interesting...
> > > 
> > > Why does it work with Netfilter if you're using
> > > pcap ?
> > > 
> > > -AEF
> > > 
> > > --- tl...@mi... wrote:
> > > > I have recently released Hank, a combination
> > network
> > > > IDS and firewall
> > > > engine.  Hank works with netfilter, BSD divert
> > > > sockets and pcap.  (The
> > > > divert socket code is in patch form.)  You can
> > write
> > > > IDS-style rules,
> > > > like this:
> > > > 
> > > > <!-- converted snort rule: -->
> > > > <!-- alert tcp $EXTERNAL_NET any -> $HOME_NET
> > 110
> > > > (msg:"EXPLOIT pop3 x86 bsd overflow";flags: A+;
> > > > content:"|5e0 e31c 0b03 b8d7 e0e8 9fa 89f9|";
> > > > classtype:attempted-admin; sid:286; rev:1;) -->
> > > > 
> > > > <rule>
> > > >    <collection>
> > > >       <report>EXPLOIT pop3 x86 bsd
> > overflow</report>
> > > >       <classtype>attempted-admin</classtype>
> > > >       <sid>286</sid>
> > > >    </collection>
> > > >    <match>
> > > >       <AND>
> > > >          <PRIM proto="ipv4" field="source"
> > > > op="&lt;">&EXTERNAL_NET;</PRIM>
> > > >          <PRIM proto="ipv4" field="destination"
> > > > op="&lt;">&HOME_NET;</PRIM>
> > > >          <PRIM proto="tcp"
> > field="destination_port"
> > > > op="=">110</PRIM>
> > > >          <!-- flags: A+ -->
> > > >          <PRIM proto="tcp" field="flags"
> > > > op="mask_all">16:16</PRIM>
> > > >          <!--  content:"|5e0 e31c 0b03 b8d7 e0e8
> > 9fa
> > > > 89f9|" -->
> > > >          <PRIM proto="tcp" field="payload"
> > > > op="acbm_case_match">,,|5e0 e31c 0b03 b8d7 e0e8
> > 9fa
> > > > 89f9|</PRIM>
> > > >       </AND>
> > > >    </match>
> > > > </rule>
> > > > 
> > > > An XML style sheet performs a policy
> > transformation
> > > > that determines the
> > > > firewall verdict, if any; you can write your own
> > > > policy transforms or
> > > > specify the verdict yourself in the rule if you
> > > > like.
> > > > 
> > > > Hank is completely protocol independent.  Adding
> > new
> > > > protocols is very
> > > > easy.  Protocols can exist at any layer in the
> > > > stack; neither tunneling
> > > > (IP-over-IP) or unusual combinations
> > > > (AppletalkDDP-over-UDP) are problems.
> > > > Protocols are defined in XML files which also
> > become
> > > > the documentation;
> > > > complicated protocols might require a bit of C
> > glue
> > > > code for the sticky
> > > > parts, but that's on the order of 10-50 lines
> > and is
> > > > rare.
> > > > 
> > > > All fields are of a given datatype, and datatype
> > > > operations are
> > > > separate code, so you get the full combination
> > of
> > > > (fieldsxoperations),
> > > > yielding a much richer range of possible matches
> > > > than is possible,
> > > > to my knowledge, under any other system.  Add an
> > > > integer operations,
> > > > and it is supported for all integer fields. 
> > Hank's
> > > > got a single-pass
> > > > content matching mechanism based on an
> > Aho-Corasick
> > > > Boyer-Moore tree,
> > > > so adding more patterns does not noticeably slow
> > > > down content matching.
> > > > (The speed constraint is the size of the
> > smallest
> > > > pattern, in keeping
> > > > with the Boyer Moore spirit.)
> > > > 
> > > > I want for Hank to be your one-stop-shop for
> > > > user-space packet handling
> > > > needs.  I want to write NAT modules, rate
> > limiting
> > > > modules, packet
> > > > normalizing modules, etc.  I do not wish to
> > supplant
> > > > the very excellent
> > > > netfilter architecture, but rather give people
> > the
> > > > option of where to
> > > > put functionality: user or kernel space.  I hope
> > > > that Hank can serve to
> > > > prototype netfilter modules, and that existing
> > > > netfilter modules can be
> > > > ported up to Hank.
> > > > 
> > > > It's a new projet and still has some rough
> > spots,
> > > > but it's good shit,
> > > > and is something that I think the people on this
> > > > list would be interested
> > > > in.  Please check it out and let me know what
> > you
> > > > think.
> > > > 
> > > > http://hank.sourceforge.net/  -- Hank's home
> > page
> > > > http://hank.sourceforge.net/docs/lwn.html  -- an
> > > > article I wrote for
> > > >                                              
> > Linux
> > > > Weekly News,
> > > >                                              
> > > > hopefully to be pub'd soon
> > > > 
> > > > --
> > > > Todd Lewis
> > > > tl...@mi...
> > > > 
> > > > Q: How many marxists does it take to screw in a
> > > > lightbulb?
> > > > A: None - the bulb contains the seeds of its own
> > > > revolution. 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Send FREE video emails in Yahoo! Mail!
> > > http://promo.yahoo.com/videomail/
> > > 
> > 
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Send FREE video emails in Yahoo! Mail!
> http://promo.yahoo.com/videomail/
> 

[hank-devel] Re: announcing new applicatio: Hank

[Mauricio O. <mau...@ya...>]

Very interesting !
But I have a question: How does Hank deals with SNATed
(POSTROUTING) packets received from netfilter ip queue
on FORWARD hook if you want to do some filtering based
on the ip src that netfilter will put at SNAT
POSTROUTING ?

What you think?
moliveto

--- tl...@mi... wrote:
> Allen,
> 
> I've copied this response to the netfilter and hank
> lists,
> since it's probably a common question.
> 
> The packet acquisition mechanism is modular.  You
> can chose
> pcap, netfilter or divert at runtime.  E.g.,
> 
> hank -p pcap -P int=eth0
> hank -p pcap -P rf=./pcap_capture_file
> hank -p netfilter
> 
> all work with hank.  Take a look at paengine.h and
> paengine_*.c in the
> source code to see what's going on.
> 
> --
> Todd Lewis
> tl...@mi...
> 
> Q: How many marxists does it take to screw in a
> lightbulb?
> A: None - the bulb contains the seeds of its own
> revolution. 
> 
> On Mon, 7 Jan 2002, Allen Francom wrote:
> 
> > 
> > Interesting...
> > 
> > Why does it work with Netfilter if you're using
> > pcap ?
> > 
> > -AEF
> > 
> > --- tl...@mi... wrote:
> > > I have recently released Hank, a combination
> network
> > > IDS and firewall
> > > engine.  Hank works with netfilter, BSD divert
> > > sockets and pcap.  (The
> > > divert socket code is in patch form.)  You can
> write
> > > IDS-style rules,
> > > like this:
> > > 
> > > <!-- converted snort rule: -->
> > > <!-- alert tcp $EXTERNAL_NET any -> $HOME_NET
> 110
> > > (msg:"EXPLOIT pop3 x86 bsd overflow";flags: A+;
> > > content:"|5e0 e31c 0b03 b8d7 e0e8 9fa 89f9|";
> > > classtype:attempted-admin; sid:286; rev:1;) -->
> > > 
> > > <rule>
> > >    <collection>
> > >       <report>EXPLOIT pop3 x86 bsd
> overflow</report>
> > >       <classtype>attempted-admin</classtype>
> > >       <sid>286</sid>
> > >    </collection>
> > >    <match>
> > >       <AND>
> > >          <PRIM proto="ipv4" field="source"
> > > op="&lt;">&EXTERNAL_NET;</PRIM>
> > >          <PRIM proto="ipv4" field="destination"
> > > op="&lt;">&HOME_NET;</PRIM>
> > >          <PRIM proto="tcp"
> field="destination_port"
> > > op="=">110</PRIM>
> > >          <!-- flags: A+ -->
> > >          <PRIM proto="tcp" field="flags"
> > > op="mask_all">16:16</PRIM>
> > >          <!--  content:"|5e0 e31c 0b03 b8d7 e0e8
> 9fa
> > > 89f9|" -->
> > >          <PRIM proto="tcp" field="payload"
> > > op="acbm_case_match">,,|5e0 e31c 0b03 b8d7 e0e8
> 9fa
> > > 89f9|</PRIM>
> > >       </AND>
> > >    </match>
> > > </rule>
> > > 
> > > An XML style sheet performs a policy
> transformation
> > > that determines the
> > > firewall verdict, if any; you can write your own
> > > policy transforms or
> > > specify the verdict yourself in the rule if you
> > > like.
> > > 
> > > Hank is completely protocol independent.  Adding
> new
> > > protocols is very
> > > easy.  Protocols can exist at any layer in the
> > > stack; neither tunneling
> > > (IP-over-IP) or unusual combinations
> > > (AppletalkDDP-over-UDP) are problems.
> > > Protocols are defined in XML files which also
> become
> > > the documentation;
> > > complicated protocols might require a bit of C
> glue
> > > code for the sticky
> > > parts, but that's on the order of 10-50 lines
> and is
> > > rare.
> > > 
> > > All fields are of a given datatype, and datatype
> > > operations are
> > > separate code, so you get the full combination
> of
> > > (fieldsxoperations),
> > > yielding a much richer range of possible matches
> > > than is possible,
> > > to my knowledge, under any other system.  Add an
> > > integer operations,
> > > and it is supported for all integer fields. 
> Hank's
> > > got a single-pass
> > > content matching mechanism based on an
> Aho-Corasick
> > > Boyer-Moore tree,
> > > so adding more patterns does not noticeably slow
> > > down content matching.
> > > (The speed constraint is the size of the
> smallest
> > > pattern, in keeping
> > > with the Boyer Moore spirit.)
> > > 
> > > I want for Hank to be your one-stop-shop for
> > > user-space packet handling
> > > needs.  I want to write NAT modules, rate
> limiting
> > > modules, packet
> > > normalizing modules, etc.  I do not wish to
> supplant
> > > the very excellent
> > > netfilter architecture, but rather give people
> the
> > > option of where to
> > > put functionality: user or kernel space.  I hope
> > > that Hank can serve to
> > > prototype netfilter modules, and that existing
> > > netfilter modules can be
> > > ported up to Hank.
> > > 
> > > It's a new projet and still has some rough
> spots,
> > > but it's good shit,
> > > and is something that I think the people on this
> > > list would be interested
> > > in.  Please check it out and let me know what
> you
> > > think.
> > > 
> > > http://hank.sourceforge.net/  -- Hank's home
> page
> > > http://hank.sourceforge.net/docs/lwn.html  -- an
> > > article I wrote for
> > >                                              
> Linux
> > > Weekly News,
> > >                                              
> > > hopefully to be pub'd soon
> > > 
> > > --
> > > Todd Lewis
> > > tl...@mi...
> > > 
> > > Q: How many marxists does it take to screw in a
> > > lightbulb?
> > > A: None - the bulb contains the seeds of its own
> > > revolution. 
> > > 
> > > 
> > > 
> > > 
> > 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Send FREE video emails in Yahoo! Mail!
> > http://promo.yahoo.com/videomail/
> > 
> 
> 


__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

[hank-devel] Re: announcing new applicatio: Hank

[<tl...@mi...>]

Allen,

I've copied this response to the netfilter and hank lists,
since it's probably a common question.

The packet acquisition mechanism is modular.  You can chose
pcap, netfilter or divert at runtime.  E.g.,

hank -p pcap -P int=eth0
hank -p pcap -P rf=./pcap_capture_file
hank -p netfilter

all work with hank.  Take a look at paengine.h and paengine_*.c in the
source code to see what's going on.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

On Mon, 7 Jan 2002, Allen Francom wrote:

> 
> Interesting...
> 
> Why does it work with Netfilter if you're using
> pcap ?
> 
> -AEF
> 
> --- tl...@mi... wrote:
> > I have recently released Hank, a combination network
> > IDS and firewall
> > engine.  Hank works with netfilter, BSD divert
> > sockets and pcap.  (The
> > divert socket code is in patch form.)  You can write
> > IDS-style rules,
> > like this:
> > 
> > <!-- converted snort rule: -->
> > <!-- alert tcp $EXTERNAL_NET any -> $HOME_NET 110
> > (msg:"EXPLOIT pop3 x86 bsd overflow";flags: A+;
> > content:"|5e0 e31c 0b03 b8d7 e0e8 9fa 89f9|";
> > classtype:attempted-admin; sid:286; rev:1;) -->
> > 
> > <rule>
> >    <collection>
> >       <report>EXPLOIT pop3 x86 bsd overflow</report>
> >       <classtype>attempted-admin</classtype>
> >       <sid>286</sid>
> >    </collection>
> >    <match>
> >       <AND>
> >          <PRIM proto="ipv4" field="source"
> > op="&lt;">&EXTERNAL_NET;</PRIM>
> >          <PRIM proto="ipv4" field="destination"
> > op="&lt;">&HOME_NET;</PRIM>
> >          <PRIM proto="tcp" field="destination_port"
> > op="=">110</PRIM>
> >          <!-- flags: A+ -->
> >          <PRIM proto="tcp" field="flags"
> > op="mask_all">16:16</PRIM>
> >          <!--  content:"|5e0 e31c 0b03 b8d7 e0e8 9fa
> > 89f9|" -->
> >          <PRIM proto="tcp" field="payload"
> > op="acbm_case_match">,,|5e0 e31c 0b03 b8d7 e0e8 9fa
> > 89f9|</PRIM>
> >       </AND>
> >    </match>
> > </rule>
> > 
> > An XML style sheet performs a policy transformation
> > that determines the
> > firewall verdict, if any; you can write your own
> > policy transforms or
> > specify the verdict yourself in the rule if you
> > like.
> > 
> > Hank is completely protocol independent.  Adding new
> > protocols is very
> > easy.  Protocols can exist at any layer in the
> > stack; neither tunneling
> > (IP-over-IP) or unusual combinations
> > (AppletalkDDP-over-UDP) are problems.
> > Protocols are defined in XML files which also become
> > the documentation;
> > complicated protocols might require a bit of C glue
> > code for the sticky
> > parts, but that's on the order of 10-50 lines and is
> > rare.
> > 
> > All fields are of a given datatype, and datatype
> > operations are
> > separate code, so you get the full combination of
> > (fieldsxoperations),
> > yielding a much richer range of possible matches
> > than is possible,
> > to my knowledge, under any other system.  Add an
> > integer operations,
> > and it is supported for all integer fields.  Hank's
> > got a single-pass
> > content matching mechanism based on an Aho-Corasick
> > Boyer-Moore tree,
> > so adding more patterns does not noticeably slow
> > down content matching.
> > (The speed constraint is the size of the smallest
> > pattern, in keeping
> > with the Boyer Moore spirit.)
> > 
> > I want for Hank to be your one-stop-shop for
> > user-space packet handling
> > needs.  I want to write NAT modules, rate limiting
> > modules, packet
> > normalizing modules, etc.  I do not wish to supplant
> > the very excellent
> > netfilter architecture, but rather give people the
> > option of where to
> > put functionality: user or kernel space.  I hope
> > that Hank can serve to
> > prototype netfilter modules, and that existing
> > netfilter modules can be
> > ported up to Hank.
> > 
> > It's a new projet and still has some rough spots,
> > but it's good shit,
> > and is something that I think the people on this
> > list would be interested
> > in.  Please check it out and let me know what you
> > think.
> > 
> > http://hank.sourceforge.net/  -- Hank's home page
> > http://hank.sourceforge.net/docs/lwn.html  -- an
> > article I wrote for
> >                                               Linux
> > Weekly News,
> >                                              
> > hopefully to be pub'd soon
> > 
> > --
> > Todd Lewis
> > tl...@mi...
> > 
> > Q: How many marxists does it take to screw in a
> > lightbulb?
> > A: None - the bulb contains the seeds of its own
> > revolution. 
> > 
> > 
> > 
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Send FREE video emails in Yahoo! Mail!
> http://promo.yahoo.com/videomail/
> 

[hank-devel] Web design (was: Re: let's release!)

[Casey A. S. <cs...@so...>]

On Sunday 06 January 2002 01:54 pm, Ben Ranker wrote:
> For those who haven't heard, the site has been set up over at
> http://hank.sourceforge.net. As you'll see if you go there, it's pretty
> barebones at the moment. We would *love* it if someone could donate some
> time and effort to making up some nicer-looking content for the Hank site.
> Contact me (are...@ho...) or Todd (tl...@mi...) or mail
> the list if you're interested.

I just contacted Todd regarding this last night offering design services.  
But as far as content, well, I'm not as familiar with hank as most of the 
rest of you, so I'll be relying on the list for that.

-- 
Casey Allen Shobe
cs...@so...

'Why do people with closed minds always open their mouths?'

Re: [hank-devel] let's release!

[Ben R. <are...@ho...>]

>From: <tl...@mi...>
>To: Hank Developers <han...@li...>
>Subject: [hank-devel] let's release!
>
>Someone mirror this to sourceforge, renaming main.html to index.html,
>and we can release.  Let's roll, mofos!

For those who haven't heard, the site has been set up over at 
http://hank.sourceforge.net. As you'll see if you go there, it's pretty 
barebones at the moment. We would *love* it if someone could donate some 
time and effort to making up some nicer-looking content for the Hank site. 
Contact me (are...@ho...) or Todd (tl...@mi...) or mail 
the list if you're interested.

And of course, all of you download it and start playing with it. CVS 
instructions are available on the website, and you can ask here if you've 
got any questions.

Todd, any chance of getting you to write up some basic architecture docs to 
help ease people into how it all fits together? To the best of my knowledge 
you're still the only one who really groks how it's all put together. If you 
can get something basic for the rest of us, that can serve as a springboard 
for someone to write a full developer's guide.

Ben

_________________________________________________________________
Join the world’s largest e-mail service with MSN Hotmail. 
http://www.hotmail.com

[hank-devel] let's release!

[<tl...@mi...>]

http://www.mindspring.com/~tlewis/webtree

Someone mirror this to sourceforge, renaming main.html to index.html,
and we can release.  Let's roll, mofos!

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

Re: [hank-devel] slight tweak

[<tl...@mi...>]

On Fri, 7 Dec 2001 tl...@mi... wrote:

>        struct rbuf_c {
>                struct rbuf_c *next;
>                struct encapsulation *c;
>                unsigned char *rbuf;
>        };
(...)
> Then, evaluating the value
> of primitive (UDP, 2) doesn't involve walking the encapsulation list;
> instead it's:
> 
> 	rbuf[1] ? rbuf[1][2] : 0;
> 
> The find_matches interface would become:
> 
> 	struct rbuf_c* (*find_matches)(struct capsule *e, struct prim_can *c);

Additionally, for rules that specify positional dependence, like "For
the topmost of all UDP encapsulations present in the packet", or "Any
UDP encapsulation following an IPv4 encapsulation", you could do:

	if(rbuf[1]){
		if(rbuf[1]->next){
			/* multiple encapsulations are present */
			r = find_appripriate_encap(packet, rule);
		} else {
			r = rbuf[1][2];
		}
	}

Cheap in the common case, respectable in the rare case, powerful in
all cases.  This is, I think, a definite improvement over what is in
there now.  I'm still waiting until after the release to do it, though.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

[hank-devel] slight tweak

[<tl...@mi...>]

This is an idea I have that I wanted to throw out there.  I don't plan
to touch it until this version is out the door.

After Hank reached 100% snort-compatibility, I inserted some
instrumentation and started counting cycles.  It turns out that evaluating
the primitives in a rule, in other words, just finding and returning
the already-computed value of the primitive evaluation, is taking a lot
(~1000) of cycles.  Multiply that by 1000 rules, and you're talking some
serious CPU heat.  Now, this isn't as big a problem as it may first see,
because we also need to make the ruleset more narrow and deep, i.e.,
to make rule evaluation more conditional than it is today; that will
reduce the number of rules evaluated to well below 1000.  Nonetheless,
I was expecting finding a primitive's retval to take ~10 cycles.

A big part of the cost here is going to traversing the linked list of
encapsulations on every single evaluation.  There's a related problem,
in that the rbuf[] into which primitive values are inserted is associated
with the encapsulation itself; it needs to be parameterized if we are
going to be able to have multiple match sets.

So, I was thinking that we could have a small integer associated with
each registered protocol.  We could have a struct like this:

	struct rbuf_c {
		struct rbuf_c *next;
		struct encapsulation *c;
		unsigned char *rbuf;
	};

	struct rbuf_c rbuf[N];

where "N" is the number of registered protocols.  Thus, if UDP were
the second protocol registered at startup time, then rbuf[1] would be
a pointer to the rbufs for all of the UDP encapsulations.  We would
tell the UDP pe what protocol number it is.  Then, evaluating the value
of primitive (UDP, 2) doesn't involve walking the encapsulation list;
instead it's:

	rbuf[1] ? rbuf[1][2] : 0;

The find_matches interface would become:

	struct rbuf_c* (*find_matches)(struct capsule *e, struct prim_can *c);

This seems like a clear win to me, and so I plan to do it.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

[hank-devel] we are snort compatible; release imminent

[<tl...@mi...>]

All snort incompatibilities and their related bugs have been squished.
There are three snort rules in my collection that snort detects and hank
does not; all three have syntax errors that happen to get through snort's
decoder.  This bug has been reported to the snort development community,
and Fyodor has told me that he will look into fixing it in snort.

According to my tests, hank is 100% snort-compatible using rules
convertered by snort_converter.pl.  We are in pre-release mode.

Anyone who can do so should perform their own tests and report
any outstanding bugs in hank.

This is by far the best program I've ever written.  I hope that
people like it.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

[hank-devel] bug update; snort compatible very soon

[<tl...@mi...>]

It turns out that I was not interpreting depth/offset according
to their proper definitions.  Fixed and on its way to CVS.

With that fix, there are 17 buggy rules (out of 1226) left:

52 case
72 case
121 ipv4:flags_offset mask_all int32
178 ipv4:length = int32
276 ipv4:flags_offset mask_all int32
282 ipv4:length = int32
283 case
368 ipv4:length = int32
369 ipv4:length = int32
370 ipv4:length = int32
375 ipv4:length = int32
379 ipv4:length = int32
531 ipv4:flags_offset mask_all int32
534 ipv4:flags_offset mask_all int32
1097 case
1221 ipv4:length = int32
1222 ipv4:length < int32

The length cases require int32-style access to the length of an
hdt_blob entity, which should be as simple as a little C++-style
multiple-inheritance, only I get to manage the vtable myself.  8^)
(Apologies if I mangled the C++ side.)  Fortunately, the in32 support and
length data are already there.  I'm going to think for a while (read: have
another beer) before deciding how precisely the syntax for that will work.

By the way, if you're an 80s fan like me, then I recommend Cyndi Lauper's
greatest hits album, "12 Deadly Cyns".  Windows open on this warm and
sunny december day, Cyndi playing on the stereo, Pabst in hand, squashing
hank bugs, life is very good today for the Toddster.

Finally, I checked for rules that snort misses that we catch.  There are
35 of them.  We may be more snort-compatible than snort!

If anyone wants to share in the snort-compatibility glory, then they
had better hurry up!

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

[hank-devel] home stretch

[<tl...@mi...>]

I have evening fixed the greatest remaining bug in hank.  There was a
place in hdt_blob where I used strlen instead of the proper length value,
and so null characters were screwing me up.

With this change, of the 1226 snort rules I have, hank is failing
on 39 of them.  They are in three classes:

     26	case
      9	ipv4:length
      4	ipv4:flags_offset

It turns out that the snort definition of dsize is actually very subtle
and bullshitty.  That one is a simple change to snort_converter.pl.
The flags_offset bug is, I suspect, an even simpler fix.  Of the remaining
acbm_case_match failures, here are the actual patterns:

<PRIM proto="tcp" field="payload" op="acbm_case_match">,,c|3A|\</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">,,..\</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">,,70" reference</PRIM>
<PRIM proto="icmp" field="payload" op="acbm_case_match">,16,|61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F 70|</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">1,3,|00 0200|</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">1,3,|00 0600|</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">1,3,|00 cb00|</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">1,3,|00 5f02|</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">16,32,|00 01 86 F3 00 00 00 01 00 00 00 0F 00 00 00 01|</PRIM>
<PRIM proto="tcp" field="payload" op="acbm_case_match">16,32,|00 01 86 A5 00 00 00 01 00 00 00 05 00 00 00 01|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 F7 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 87 03 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 BA 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 E4 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 A5 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 87 cc 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|02 49 f1 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 B1 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 A2 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 87 88 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 AF 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 B8 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 F3 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 A9 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 A4 00 00|</PRIM>
<PRIM proto="udp" field="payload" op="acbm_case_match">40,8,|01 86 BC 00 00|</PRIM>

The first two have to do with backslash encoding, the second one is a
mistranslation, and the rest of depth/offset.  I've tracked down snort's
behaviour in decode.c and sp_pattern_match.c, and it's all related to
snort's definition of dsize.  It should be simple to fix.

So, that is five bugs to go before we have 100% snort compatibility.
If anyone were to want to tackle some of these bugs, then I would be
happy to help and appreciative of any progress they made.

--
Todd Lewis
tl...@mi...

Q: How many marxists does it take to screw in a lightbulb?
A: None - the bulb contains the seeds of its own revolution. 

Re: [hank-devel] cvs question

[<ah...@se...>]

Everytime I have seen that message it means that the version of the file
you are working on is older then the on in cvs.  HTH.

Drew

On Wed, 14 Nov 2001 tl...@mi... wrote:

> So, I branched off hdt_blob.c and hdt_blob_impl.c in order to
> convert them from a simple linked-list to a dynamically-sized
> hash table.  The change appears to be working fine, and I want
> to merge the changes back into the main branch and abandon the
> hash branch.  I've tried "cvs commit -r HEAD", but I get the
> complaint:
>
> cvs server: Up-to-date check failed for `hdt_blob.c'
> cvs server: Up-to-date check failed for `hdt_blob_impl.c'
> cvs [server aborted]: correct above errors first!
>
> Is this the right way to merge the files?  If so, what do I
> need to do to remove the errors, which I don't understand?
>
> Help appreciated.
>
> --
> Todd Lewis
> tl...@mi...
>
> Q: How many marxists does it take to screw in a lightbulb?
> A: None - the bulb contains the seeds of its own revolution.
>
>
> _______________________________________________
> hank-devel mailing list
> han...@li...
> https://lists.sourceforge.net/lists/listinfo/hank-devel
>