Re: [Openpacket-devel] Use cases and meta data

[Aaron T. <syn...@gm...>]

On 1/29/07, Eric Hacker <li...@er...> wrote:
> On 1/29/07, Aaron Turner <syn...@gm...> wrote:
> > Comments inline...
> >
> > On 1/29/07, Eric Hacker <li...@er...> wrote:
> > > Dear Packet Fanatics,
> >

[snip]

> > > IP in data, boolean, must indicate if the IP address of either the
> > > client or server is represented in the data. If so, it would be nice
> > > to have a way to define where (for this particular flow) the IPs are
> > > in the data so that they can be changed. Something like packet 4, byte
> > > offset 54, client IP.
> >
> > I assume the reason you want the above is so that you can properly
> > rewrite IP addresses to match your test bed network.   I would argue
> > that this really should be automated (who wants to look through pcap's
> > in Wirkshark and find IP addresses??) and hence should be a separate
> > tool.  Perhaps the OpenPacket team develops such a tool, but that
> > seems outside of the scope of this project at this time.  On a side
> > note, I'd suggest looking at tshark's pdml's output.  At least that
> > way you won't be spending all your time writing protocol decoders.
>
> Rewriting supports more than just matching the testbed. It allows the
> simulation of multiple flows. Sure the IDS catches one attack, but
> does it handle 500 all at different targets?

Fair enough, but it reduces to the same problem once you automate it.

> I agree that rewriting IP addresses needs to be automated, and perhaps
> a tool can do it automatically most of the time so this isn't
> necessary. It just seems to me that often the protocols are new or
> complex enough that the tools can't support this. Whereas if the meta
> data can explicitly provide the location, the tool can do it without
> knowing the protocol.
>
> PDML sucks as far as XML goes and using other tools to parse it. It is
> a Packet DISPLAY ML. A good Packet ML would look much like RFC 3252,
> fixed up a bit.

No argument here.  I'm just not aware of any actual implementation of
anything better.  The hard part isn't changing the IP address, it's
doing all the packet decoding to find the IP address/checksums/etc to
edit.  I really have no interest in writing decoders for every lame
POS protocol which decides to encode L3 data in L4+.

Anyways, that said, if people can agree on an reasonable format to
encode packet editing commands, something like (or not, I'm just
making this up as I type):

Packet 57:               # packet to modify
   Offset: 89              # byte offset starting from start of packet
   Direction: C2S     # direction of packet
   Type: IPv4            # type of field
   Encoding: big_endian # encoding of new value
   Value: 192.168.2.34    # optional new value

I'll see about making sure libtcpedit/tcprewrite supports rewriting
the data appropriately.  The goal being that you'd take the .pcap,
this edit command and a new value and you'd get a new pcap containing
the appropriate changes (including L3/4 checksum re-calcuation).  That
would hopefully allow anyone to easily modify the packets based on
their need.

> > > Flow-pointer, normalized, a URL for the actual packet dump file.
> >
> > I'm not sure what a "flow-pointer" is.  More info?
>
> I was assuming that the meta-data and the actual pcap are not in a
> single file, so the meta-data ought to point to the pcap. Flow pointer
> is probably a poor choice of words.

Ah, that makes sense.  Yeah, that seems the way to go, at least until
the pcap-ng format takes off, but that's at least a few years off.

> > > Not all elements need to be a part of the system as stored in the
> > > OpenPacket repository, however the method of creating the meta-data
> > > should support having all these elements.
> >
> > Not sure I 100% follow, but I will say that data is only useful if
> > people have access to it.  And people are less likely to contribute
> > data if they don't see the immediate value.
>
> What I meant here is that the format should be extensible and systems
> shouldn't choke on meta-data that they don't understand. There is a
> core set of required meta-data, a set of optional meta-data, but if it
> is there, this is what it has to look like, and then there's the
> meta-data that someone needed and decided to give back because someone
> else might find it useful too. There are limits of course.

Reasonable.

> > I think a lot of the vuln target information should come from the CVE.
> >  No need to duplicate info IMHO.
>
> I agree, when it is CVE related. What if just some nice Microsoft
> Exchange traffic that happens to trigger some dumb ISS signature all
> the time? I think this is the kind of info that is optional, but needs
> to have rules established on how to put it in.

In those cases, my feeling you'll have better luck getting people to
use 'tags' or labels rather then a formalized taxonomy.   Basically my
experience is that contributors of this sort can't be bothered to
figure things out and just want something simple (think upload a
tarball of pcaps).  Even getting per-pcap metadata may be an uphill
climb.  IMHO you're better off making it super simple for them and
writing tools to make it easy for you to clean up.   I'd hope for the
best, plan for the worst.

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix