openpacket-devel — OpenPacket development list

Re: [Openpacket-devel] New Beta Site Live

[James P. <jp...@gm...>]

Are there plans to implement an SSL cert for logins?

On 10/13/07, Richard Bejtlich <tao...@gm...> wrote:
> Thanks to more excellent, all-volunteer work of our developer Sharri
> Parsell, I am happy to notify you that a beta demo of OpenPacket.org
> is now available at
>
> http://beta.openpacket.org:8080/
>
> Thank you also to JJC of www.redsphereglobal.com for providing free
> temporary hosting.  We are working on a more permanent solution, but
> if you have ideas now please let me know.
>
> Please take a look and report feedback to the
> ope...@li... mailing list.
>
> This site is considered "beta," and we are getting closer to public
> announcements of the site. If you are interested in assessing the
> security of the site, please contact me directly. We can coordinate
> with Sharri and JJC to ensure your discoveries do not catch us by
> surprise.
>
> Thank you,
>
> Richard
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Openpacket-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openpacket-devel
>


-- 
James Pleger
p: 623.298.7966
e: jp...@gm...

[Openpacket-devel] New Beta Site Live

[Richard B. <tao...@gm...>]

Thanks to more excellent, all-volunteer work of our developer Sharri
Parsell, I am happy to notify you that a beta demo of OpenPacket.org
is now available at

http://beta.openpacket.org:8080/

Thank you also to JJC of www.redsphereglobal.com for providing free
temporary hosting.  We are working on a more permanent solution, but
if you have ideas now please let me know.

Please take a look and report feedback to the
ope...@li... mailing list.

This site is considered "beta," and we are getting closer to public
announcements of the site. If you are interested in assessing the
security of the site, please contact me directly. We can coordinate
with Sharri and JJC to ensure your discoveries do not catch us by
surprise.

Thank you,

Richard

Re: [Openpacket-devel] Openpacket site

[CS L. <ge...@gm...>]

Hi Jeff,

Yeah, I have this problem too.

On 10/10/07, Jeff Lake <jl...@kc...> wrote:
>
> Is anyone else getting this error?:
>
> 502 Proxy Error
> The proxy server received an invalid response from an upstream server.
>
> The proxy server could not handle the request GET /.
>
> Reason: Could not connect to remote machine: Connection refused
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> Openpacket-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openpacket-devel
>
>
>


-- 
Best Regards,

CS Lee<geekooL[at]gmail.com>

http://geek00l.blogspot.com

[Openpacket-devel] Openpacket site

[Jeff L. <jl...@kc...>]

Is anyone else getting this error?:

502 Proxy Error
The proxy server received an invalid response from an upstream server.

The proxy server could not handle the request GET /.

Reason: Could not connect to remote machine: Connection refused

[Openpacket-devel] Unable to add new thread in forums (tested General)

[M. S. <shi...@ho...>]

  ActiveRecord::RecordNotFound in
  Post#new



  Showing app/views/post/new.rhtml where line #12 raised:
 =20

Shirkdog=20
' or 1=3D1--=20
http://www.shirkdog.us
_________________________________________________________________
Climb to the top of the charts!=A0 Play Star Shuffle:=A0 the word scramble =
challenge with star power.
http://club.live.com/star_shuffle.aspx?icid=3Dstarshuffle_wlmailtextlink_oc=
t=

Re: [Openpacket-devel] Alpha demo site online

[David J. B. <da...@vo...>]

Great work, Sharri!  I had a look, and everything was pretty smooth
except that when I clicked on the submitter name to view their profiles,
it gave me the same error someone else mentioned here earlier.

I suggest that registered users should be able to modify the
description of the packet captures directly, rather than add notes
in the associated forum thread.  That way, as more information surfaces
about a capture, anyone in the know can update the text, modify the tags
and generally increase the accuracy of that information.  This will also
make it easier to search for and find items that may not have been
properly recognized when they were originally posted.

Also, from the alpha site, I wasn't sure if there were still going to be
such things as moderators to approve uploaded pcaps?  Just wondering.

Great job on the site, though.  I'm looking forward to it going live!

	Thanks,
	  David




Richard Bejtlich wrote:
> Hello all,
> 
> Thanks to the excellent, all-volunteer work of our developer Sharri
> Parsell, I am happy to notify you that an alpha demo of OpenPacket.org
> is now available at Sharri's temporary site:
> 
> http://openpacket.spaceegg.net/
> 
> Please take a look and report feedback to the
> ope...@li... mailing list.
> 
> This site is considered an "alpha" because we are still working on
> functionality, administration, appearance, and so on.  If you are
> interested in assessing the security of the site, please contact me
> directly.  We can coordinate with Sharri to ensure your discoveries do
> not catch us by surprise.
> 
> Sincerely,
> 
> Richard
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Openpacket-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openpacket-devel

Re: [Openpacket-devel] Alpha demo site online

[Tom L. <do...@gm...>]

Very nice.  I like the look & feel a lot.  I couldn't post on the forums
though, error:

ActiveRecord::RecordNotFound in Post#new

Showing *app/views/post/new.rhtml* where line *#12* raised:

Couldn't find Capture without an ID

Extracted source (around line *#12*):

9: <% form_for 'post', :url => {:action => 'create'} do |form| %>
10:
11: <p><label for="post_subject">Subject</label><br/>
12: <%= form.text_field 'subject', :size => 60, :value => "Re:
#{capture_filename}" %></p>
13:
14: <p><label for="post_body">Body</label><br/>
15: <%= form.text_area 'body', :rows => 20, :cols => 80 %></p>

RAILS_ROOT: /home/html/spaceegg/openpacket/config/..
Application Trace
<http://openpacket.spaceegg.net/post/new?forum_id=2#> | Framework
Trace <http://openpacket.spaceegg.net/post/new?forum_id=2#> | Full
Trace<http://openpacket.spaceegg.net/post/new?forum_id=2#>

/usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/base.rb:1012:in
`find_from_ids'
/usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/base.rb:419:in
`find'
#{RAILS_ROOT}/app/helpers/post_helper.rb:4:in `capture_filename'
#{RAILS_ROOT}/app/views/post/new.rhtml:12:in
`_run_rhtml_47app47views47post47new46rhtml'
#{RAILS_ROOT}/app/views/post/new.rhtml:9:in
`_run_rhtml_47app47views47post47new46rhtml'
/usr/bin/mongrel_rails:16:in `load'
/usr/bin/mongrel_rails:16

Re: [Openpacket-devel] Alpha demo site online

[James P. <jp...@gm...>]

Looks very good!

A feature request:

It would be kind of cool(and maybe a cya) to have all the malicious
and suspicious pcaps zipped and password protected(that way people
using av won't have problems, and will require interaction).

Another thing would be cool to have the most popular tags on the sidebar.

Other than that.. Looks good, When is the site gonna go live :)

On 9/28/07, Richard Bejtlich <tao...@gm...> wrote:
> Hello all,
>
> Thanks to the excellent, all-volunteer work of our developer Sharri
> Parsell, I am happy to notify you that an alpha demo of OpenPacket.org
> is now available at Sharri's temporary site:
>
> http://openpacket.spaceegg.net/
>
> Please take a look and report feedback to the
> ope...@li... mailing list.
>
> This site is considered an "alpha" because we are still working on
> functionality, administration, appearance, and so on.  If you are
> interested in assessing the security of the site, please contact me
> directly.  We can coordinate with Sharri to ensure your discoveries do
> not catch us by surprise.
>
> Sincerely,
>
> Richard
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Openpacket-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openpacket-devel
>

[Openpacket-devel] Alpha demo site online

[Richard B. <tao...@gm...>]

Hello all,

Thanks to the excellent, all-volunteer work of our developer Sharri
Parsell, I am happy to notify you that an alpha demo of OpenPacket.org
is now available at Sharri's temporary site:

http://openpacket.spaceegg.net/

Please take a look and report feedback to the
ope...@li... mailing list.

This site is considered an "alpha" because we are still working on
functionality, administration, appearance, and so on.  If you are
interested in assessing the security of the site, please contact me
directly.  We can coordinate with Sharri to ensure your discoveries do
not catch us by surprise.

Sincerely,

Richard

Re: [Openpacket-devel] Interested in working on OpenPacket.org

[Richard B. <tao...@gm...>]

On 8/14/07, l0...@l0... <l0...@l0...> wrote:
> Hi there,
>
> I found out about OpenPacket.org reading Richard's blog. I read through
> the requirements draft, and it seems like a really cool project. If you're
> still looking for additional developers, I'd love to help out. I'm not in
> a position to contribute interesting packet captures, unfortunately, but I
> can assist with the development work.
>
> I have a solid background in networking, security, and web design
> (primarily PHP), so a project like this is right up my alley. For more
> about me, check out http://l0gic.net.
>
> Thanks,
>
> Jeremy
>

Hi Jeremy,

Thanks for writing.  We should have an alpha site ready soon.  Please
take a look once it's ready.

Sincerely,

Richard

[Openpacket-devel] Interested in working on OpenPacket.org

[<l0...@l0...>]

Hi there,

I found out about OpenPacket.org reading Richard's blog. I read through
the requirements draft, and it seems like a really cool project. If you're
still looking for additional developers, I'd love to help out. I'm not in
a position to contribute interesting packet captures, unfortunately, but I
can assist with the development work.

I have a solid background in networking, security, and web design
(primarily PHP), so a project like this is right up my alley. For more
about me, check out http://l0gic.net.

Thanks,

Jeremy

[Openpacket-devel] OpenPacket.org developments

[Richard B. <tao...@gm...>]

I am happy to report that work on OpenPacket.org is back on track,
thanks to a new volunteer Web application developer.

Several months ago Richard Fifarek (http://synfulpacket.blogspot.com/)
put me in touch with his friend Sharri Parsell.  She read the
requirements document
(http://openpacket.sourceforge.net/openpacket_req_doc_draft_21jul06.pdf)
I wrote last year and discussed the project with me.

Today she showed me a pre-alpha version based on our discussions over
the last few months.  I was very impressed and we plan to have an
alpha ready for review by openpacket-devel list members within the
next two weeks.  I've also contacted a few guys I trust who perform
Web app assessments to review the site prior to general public
availability.

If development stays on track we plan to have a beta available for the
public prior to Black Hat at the end of this month.

Stay tuned to this mailing list and/or the OpenPacket.org Blog
(openpacket.blogspot.com) for more developments.

Thank you especially to Sharri for volunteering to do this work and
getting us close to public availability!

Sincerely,

Richard

Re: [Openpacket-devel] Bleeding threats doc wiki

[David J. B. <da...@vo...>]

Richard Bejtlich wrote:

> Somehow this whole thread went straight to my Gmail trash.  I blame
> David Bianco.  :)

A reasonable filter rule, I assure you...

	David

Re: [Openpacket-devel] OpenPacket ideas

[Aaron T. <syn...@gm...>]

On 3/12/07, Richard Bejtlich <tao...@gm...> wrote:
> OpenPacket.org fans,
>
> I just had a conversation with a volunteer Web architect for
> OpenPacket.org.  She's BCC'd on this email because I'm not sure if she
> wants any attention yet.  I decided to send this message to the
> openpacket-devel list so others could potentially reply to this post
> with their thoughts.
>
> One of the problems we just discussed was pcap trace classification.
> How do people do searches on traces in an efficient manner?  One
> option I considered would be to run a trace through Tshark to produce
> statistics, then use the output to create tags.  For example, the
> Wireshark sample captures Wiki

[snip]

Do whatever is easiest and quickest.  Right now OpenPacket doesn't
exist except in our dreams.  Just get *something* out there that is
usable and worry about adding features later.  Until you get enough
momentum and pcap's where people have to search for them, this feature
isn't critical.

Once people start using the site, I'm sure you'll get all sorts of
requests and suggestions which will help set priorities and
requirements.

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix

Re: [Openpacket-devel] Bleeding threats doc wiki

[Matt J. <jo...@bl...>]

Glad it's moving again!

I'm open to whatever we can do. The more the merrier!

I have to say though, so far the wiki format for our documentation is
spectacularly easy to use. No experience with pcaps or files yet
though,. We'll be learning as we go.

Matt

Richard Bejtlich wrote:
> On 3/7/07, Matt Jonkman <jo...@bl...> wrote:
>> I hope folks start using it as well. :)
>>
>> I don't want to take from any momentum that might develop here. I think
>> openpacket is a very valuable thing we'll all need.
>>
>> Would you all be interested in some form of cooperation with the
>> bleeding wiki? I don't know how many pcaps we'll have attached, but I'll
>> be trying to set the example by attaching them myself when available.
>>
>> Matt
> 
> Somehow this whole thread went straight to my Gmail trash.  I blame
> David Bianco.  :)
> 
> Momentum is certainly a relative term, because until a few hours ago
> OpenPacket.org was dead in the water.  However, with the possible
> discovery of a new Web architect I am hopeful again.  I would like to
> cooperate with Bleeding, so let's see what happens.
> 
> Thank you,
> 
> Richard

-- 
--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
765-429-0398
765-807-3060 fax
http://www.bleedingthreats.net
--------------------------------------------

PGP: http://www.bleedingthreats.com/mattjonkman.asc

Re: [Openpacket-devel] Bleeding threats doc wiki

[Richard B. <tao...@gm...>]

On 3/7/07, Matt Jonkman <jo...@bl...> wrote:
> I hope folks start using it as well. :)
>
> I don't want to take from any momentum that might develop here. I think
> openpacket is a very valuable thing we'll all need.
>
> Would you all be interested in some form of cooperation with the
> bleeding wiki? I don't know how many pcaps we'll have attached, but I'll
> be trying to set the example by attaching them myself when available.
>
> Matt

Somehow this whole thread went straight to my Gmail trash.  I blame
David Bianco.  :)

Momentum is certainly a relative term, because until a few hours ago
OpenPacket.org was dead in the water.  However, with the possible
discovery of a new Web architect I am hopeful again.  I would like to
cooperate with Bleeding, so let's see what happens.

Thank you,

Richard

[Openpacket-devel] OpenPacket ideas

[Richard B. <tao...@gm...>]

OpenPacket.org fans,

I just had a conversation with a volunteer Web architect for
OpenPacket.org.  She's BCC'd on this email because I'm not sure if she
wants any attention yet.  I decided to send this message to the
openpacket-devel list so others could potentially reply to this post
with their thoughts.

One of the problems we just discussed was pcap trace classification.
How do people do searches on traces in an efficient manner?  One
option I considered would be to run a trace through Tshark to produce
statistics, then use the output to create tags.  For example, the
Wireshark sample captures Wiki

http://wiki.wireshark.org/SampleCaptures

includes a trace

http://wiki.wireshark.org/SampleCaptures?action=AttachFile&do=get&target=dssetup_DsRoleUpgradeDownlevelServer_MS04-011_exploit.cap

that looks like this when run through Tshark statistics:

$ tshark -n -q -r
dssetup_DsRoleUpgradeDownlevelServer_MS04-011_exploit.cap -z io,phs

===================================================================
Protocol Hierarchy Statistics
Filter: frame

frame                                    frames:16 bytes:4962
  eth                                    frames:16 bytes:4962
    ip                                   frames:16 bytes:4962
      tcp                                frames:16 bytes:4962
        nbss                             frames:8 bytes:4418
          smb                            frames:6 bytes:2414
            pipe                         frames:3 bytes:1934
              dcerpc                     frames:3 bytes:1934
                dssetup                  frames:1 bytes:1514
                  dcerpc.cn_deseg_req    frames:1 bytes:1514
===================================================================

The protocol name (eth, ip, tcp, nbss, etc.) could be used to create
tags.  Someone could query OpenPacket for all traffic involving
"dcerpc" and find this trace.

We might be able to do something similar with Argus if we wanted to
create classifications for IP protocols and/or TCP/UDP ports.

I appreciate any thoughts.

Sincerely,

Richard

Re: [Openpacket-devel] Bleeding threats doc wiki

[Matt J. <jo...@bl...>]

I hope folks start using it as well. :)

I don't want to take from any momentum that might develop here. I think
openpacket is a very valuable thing we'll all need.

Would you all be interested in some form of cooperation with the
bleeding wiki? I don't know how many pcaps we'll have attached, but I'll
be trying to set the example by attaching them myself when available.

Matt

Eric Hacker wrote:
> David,
> 
> I hadn't noticed. Thanks for the tip.
> 
> Let's hope some people start using it, so there is more interest in
> doing it Right.
> 
> On 3/6/07, David J. Bianco <da...@vo...> wrote:
>> Did anyone else notice that the new beta for the Bleeding Threats
>> documentation wiki includes pcap file uploads?  Thought that might
>> be interesting to the folks on this list...
> 
> 
> Regards,

-- 
--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
765-429-0398
765-807-3060 fax
http://www.bleedingthreats.net
--------------------------------------------

PGP: http://www.bleedingthreats.com/mattjonkman.asc

Re: [Openpacket-devel] Bleeding threats doc wiki

[Eric H. <li...@er...>]

David,

I hadn't noticed. Thanks for the tip.

Let's hope some people start using it, so there is more interest in
doing it Right.

On 3/6/07, David J. Bianco <da...@vo...> wrote:
> Did anyone else notice that the new beta for the Bleeding Threats
> documentation wiki includes pcap file uploads?  Thought that might
> be interesting to the folks on this list...


Regards,
-- 
Eric Hacker, CISSP

aptronym (AP-troh-NIM) noun
A name that is especially suited to the profession of its owner

I _can_ leave well enough alone, but my criteria for well enough is
pretty darn high.

[Openpacket-devel] Bleeding threats doc wiki

[David J. B. <da...@vo...>]

Did anyone else notice that the new beta for the Bleeding Threats
documentation wiki includes pcap file uploads?  Thought that might
be interesting to the folks on this list...

See http://doc.bleedingthreats.net/2003434 for an example of a rule
doc page that allows pcap uploads.

	David

Re: [Openpacket-devel] Use cases and meta data

[Aaron T. <syn...@gm...>]

On 1/29/07, Eric Hacker <li...@er...> wrote:
> On 1/29/07, Aaron Turner <syn...@gm...> wrote:
> > Comments inline...
> >
> > On 1/29/07, Eric Hacker <li...@er...> wrote:
> > > Dear Packet Fanatics,
> >

[snip]

> > > IP in data, boolean, must indicate if the IP address of either the
> > > client or server is represented in the data. If so, it would be nice
> > > to have a way to define where (for this particular flow) the IPs are
> > > in the data so that they can be changed. Something like packet 4, byte
> > > offset 54, client IP.
> >
> > I assume the reason you want the above is so that you can properly
> > rewrite IP addresses to match your test bed network.   I would argue
> > that this really should be automated (who wants to look through pcap's
> > in Wirkshark and find IP addresses??) and hence should be a separate
> > tool.  Perhaps the OpenPacket team develops such a tool, but that
> > seems outside of the scope of this project at this time.  On a side
> > note, I'd suggest looking at tshark's pdml's output.  At least that
> > way you won't be spending all your time writing protocol decoders.
>
> Rewriting supports more than just matching the testbed. It allows the
> simulation of multiple flows. Sure the IDS catches one attack, but
> does it handle 500 all at different targets?

Fair enough, but it reduces to the same problem once you automate it.

> I agree that rewriting IP addresses needs to be automated, and perhaps
> a tool can do it automatically most of the time so this isn't
> necessary. It just seems to me that often the protocols are new or
> complex enough that the tools can't support this. Whereas if the meta
> data can explicitly provide the location, the tool can do it without
> knowing the protocol.
>
> PDML sucks as far as XML goes and using other tools to parse it. It is
> a Packet DISPLAY ML. A good Packet ML would look much like RFC 3252,
> fixed up a bit.

No argument here.  I'm just not aware of any actual implementation of
anything better.  The hard part isn't changing the IP address, it's
doing all the packet decoding to find the IP address/checksums/etc to
edit.  I really have no interest in writing decoders for every lame
POS protocol which decides to encode L3 data in L4+.

Anyways, that said, if people can agree on an reasonable format to
encode packet editing commands, something like (or not, I'm just
making this up as I type):

Packet 57:               # packet to modify
   Offset: 89              # byte offset starting from start of packet
   Direction: C2S     # direction of packet
   Type: IPv4            # type of field
   Encoding: big_endian # encoding of new value
   Value: 192.168.2.34    # optional new value

I'll see about making sure libtcpedit/tcprewrite supports rewriting
the data appropriately.  The goal being that you'd take the .pcap,
this edit command and a new value and you'd get a new pcap containing
the appropriate changes (including L3/4 checksum re-calcuation).  That
would hopefully allow anyone to easily modify the packets based on
their need.

> > > Flow-pointer, normalized, a URL for the actual packet dump file.
> >
> > I'm not sure what a "flow-pointer" is.  More info?
>
> I was assuming that the meta-data and the actual pcap are not in a
> single file, so the meta-data ought to point to the pcap. Flow pointer
> is probably a poor choice of words.

Ah, that makes sense.  Yeah, that seems the way to go, at least until
the pcap-ng format takes off, but that's at least a few years off.

> > > Not all elements need to be a part of the system as stored in the
> > > OpenPacket repository, however the method of creating the meta-data
> > > should support having all these elements.
> >
> > Not sure I 100% follow, but I will say that data is only useful if
> > people have access to it.  And people are less likely to contribute
> > data if they don't see the immediate value.
>
> What I meant here is that the format should be extensible and systems
> shouldn't choke on meta-data that they don't understand. There is a
> core set of required meta-data, a set of optional meta-data, but if it
> is there, this is what it has to look like, and then there's the
> meta-data that someone needed and decided to give back because someone
> else might find it useful too. There are limits of course.

Reasonable.

> > I think a lot of the vuln target information should come from the CVE.
> >  No need to duplicate info IMHO.
>
> I agree, when it is CVE related. What if just some nice Microsoft
> Exchange traffic that happens to trigger some dumb ISS signature all
> the time? I think this is the kind of info that is optional, but needs
> to have rules established on how to put it in.

In those cases, my feeling you'll have better luck getting people to
use 'tags' or labels rather then a formalized taxonomy.   Basically my
experience is that contributors of this sort can't be bothered to
figure things out and just want something simple (think upload a
tarball of pcaps).  Even getting per-pcap metadata may be an uphill
climb.  IMHO you're better off making it super simple for them and
writing tools to make it easy for you to clean up.   I'd hope for the
best, plan for the worst.

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix

Re: [Openpacket-devel] Use cases and meta data

[Eric H. <li...@er...>]

On 1/29/07, Aaron Turner <syn...@gm...> wrote:
> Comments inline...
>
> On 1/29/07, Eric Hacker <li...@er...> wrote:
> > Dear Packet Fanatics,
>
> [snip]
>

> Having some experience creating a taxonomy in this area, I can say
> it's quite doable.  However, it does take time to manage the taxonomy
> and time to connect the dots.  In general I suspect that most people
> with pcap's they'd contribute no longer have this sort of information,
> and if they did would be unwilling to spend the time fill out the data
> for each pcap.  I'm not saying some people won't, but a majority won't
> bother.

Sadly, I agree. That's what karma points are for though, right. :)

> > IP in data, boolean, must indicate if the IP address of either the
> > client or server is represented in the data. If so, it would be nice
> > to have a way to define where (for this particular flow) the IPs are
> > in the data so that they can be changed. Something like packet 4, byte
> > offset 54, client IP.
>
> I assume the reason you want the above is so that you can properly
> rewrite IP addresses to match your test bed network.   I would argue
> that this really should be automated (who wants to look through pcap's
> in Wirkshark and find IP addresses??) and hence should be a separate
> tool.  Perhaps the OpenPacket team develops such a tool, but that
> seems outside of the scope of this project at this time.  On a side
> note, I'd suggest looking at tshark's pdml's output.  At least that
> way you won't be spending all your time writing protocol decoders.

Rewriting supports more than just matching the testbed. It allows the
simulation of multiple flows. Sure the IDS catches one attack, but
does it handle 500 all at different targets?

I agree that rewriting IP addresses needs to be automated, and perhaps
a tool can do it automatically most of the time so this isn't
necessary. It just seems to me that often the protocols are new or
complex enough that the tools can't support this. Whereas if the meta
data can explicitly provide the location, the tool can do it without
knowing the protocol.

PDML sucks as far as XML goes and using other tools to parse it. It is
a Packet DISPLAY ML. A good Packet ML would look much like RFC 3252,
fixed up a bit.

> > Flow-pointer, normalized, a URL for the actual packet dump file.
>
> I'm not sure what a "flow-pointer" is.  More info?

I was assuming that the meta-data and the actual pcap are not in a
single file, so the meta-data ought to point to the pcap. Flow pointer
is probably a poor choice of words.

> > Not all elements need to be a part of the system as stored in the
> > OpenPacket repository, however the method of creating the meta-data
> > should support having all these elements.
>
> Not sure I 100% follow, but I will say that data is only useful if
> people have access to it.  And people are less likely to contribute
> data if they don't see the immediate value.

What I meant here is that the format should be extensible and systems
shouldn't choke on meta-data that they don't understand. There is a
core set of required meta-data, a set of optional meta-data, but if it
is there, this is what it has to look like, and then there's the
meta-data that someone needed and decided to give back because someone
else might find it useful too. There are limits of course.

> I think a lot of the vuln target information should come from the CVE.
>  No need to duplicate info IMHO.

I agree, when it is CVE related. What if just some nice Microsoft
Exchange traffic that happens to trigger some dumb ISS signature all
the time? I think this is the kind of info that is optional, but needs
to have rules established on how to put it in.

> Yep.  Actually, I would love to see OpenPacket ship tcpreplay
> (tcpprep) cache files along with the pcap's so people can use them in
> inline mode right away.   I'm sure a lot of people would find that
> useful.

That's a perfect reason to keep the meta-data format extensible. It
enables you to create a way to ship extra information in the
meta-data.

Thanks for tcpreplay, BTW.

Peace,
Hacker

Re: [Openpacket-devel] Use cases and meta data

[Aaron T. <syn...@gm...>]

Comments inline...

On 1/29/07, Eric Hacker <li...@er...> wrote:
> Dear Packet Fanatics,

[snip]

> The following elements are needed to support this. Each element may be
> either standardized if a value space is defined or normalized if it
> needs to be. For these purposes, I'll break out the flows into the
> client and the server, understanding that sometimes that is not
> appropriate, but mostly it is. Shoulds and musts are apply to  this
> use case only.

[snip list of description oriented metadata]

Having some experience creating a taxonomy in this area, I can say
it's quite doable.  However, it does take time to manage the taxonomy
and time to connect the dots.  In general I suspect that most people
with pcap's they'd contribute no longer have this sort of information,
and if they did would be unwilling to spend the time fill out the data
for each pcap.  I'm not saying some people won't, but a majority won't
bother.

> IP in data, boolean, must indicate if the IP address of either the
> client or server is represented in the data. If so, it would be nice
> to have a way to define where (for this particular flow) the IPs are
> in the data so that they can be changed. Something like packet 4, byte
> offset 54, client IP.

I assume the reason you want the above is so that you can properly
rewrite IP addresses to match your test bed network.   I would argue
that this really should be automated (who wants to look through pcap's
in Wirkshark and find IP addresses??) and hence should be a separate
tool.  Perhaps the OpenPacket team develops such a tool, but that
seems outside of the scope of this project at this time.  On a side
note, I'd suggest looking at tshark's pdml's output.  At least that
way you won't be spending all your time writing protocol decoders.

> Port in data, boolean, same as IP in data.
> Flow-pointer, normalized, a URL for the actual packet dump file.

I'm not sure what a "flow-pointer" is.  More info?

> OpenPacket meta-data, such as reliability of the source or if verified
> by others.
> Meta-data URL, standardized, for getting updates to the meta-data,
> especially the open-packet portion of the meta-data.

Good idea.

> Not all elements need to be a part of the system as stored in the
> OpenPacket repository, however the method of creating the meta-data
> should support having all these elements.

Not sure I 100% follow, but I will say that data is only useful if
people have access to it.  And people are less likely to contribute
data if they don't see the immediate value.

> Sometimes the OS is what is vulnerable and sometimes it is just the
> application. Having the OS separate is nice. It may allow a
> sophisticated testing system to manipulate OS specific fingerprints in
> a flow, such as Source ports, sequence numbers, options etc. to
> simulate different OSs from one capture. Is that dreaming, or what. :)

I think a lot of the vuln target information should come from the CVE.
 No need to duplicate info IMHO.

> I don't recall if there has been a final decision regarding packaging
> of captures. In order to support meta-data, they probably need to be
> gzipped-tarballs that include both the meta-data and the packet
> capture. Alternatively, if the capture file names could be completely
> unique and normalized, then a testing system could look locally for a
> cached copy, by filename, and then use the full URL if not found. This
> is probably a feature for a later version of OpenPacket.
>
> A wonderful feature for tcpreplay that would support this use case is
> client-server mode. Here the tcpreplay client and server would 'play'
> their respective side of the conversation after receiving the packet
> from the other side. This would be very useful for inline NSD testing.

Yep.  Actually, I would love to see OpenPacket ship tcpreplay
(tcpprep) cache files along with the pcap's so people can use them in
inline mode right away.   I'm sure a lot of people would find that
useful.

[snip]

-- 
Aaron Turner
http://synfin.net/
http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix

[Openpacket-devel] Use cases and meta data

[Eric H. <li...@er...>]

Dear Packet Fanatics,

I'm going to talk out loud here in hopes that by not editing myself
too much I will be freer to innovate. That means a lot of this could
have been said before, but maybe I'll present a new twist.

One of the things we'll need with packet captures is meta data to
describe the packet flow. One of the way to determine what meta-data
is necessary is to go through use cases and see what becomes necessary
and desirable. My use case here will be testing Network Security
Devices (NSD) as that is where the need for OpenPacket is extremely
high and where my interest is.

For testing NSDs, one will want to send a stream of packets past the
NSD. For inline perhaps a stream separated into sides going through
the NSD. This process should be automated, so that a program should be
able to associate the flow with the desired outcome from the NSD.
There should be normal flows as well as attack flows available. There
may be 'false positive' flows that should not trip the NSD, but might,
and these should be labeled normal and associated with the attack.

The following elements are needed to support this. Each element may be
either standardized if a value space is defined or normalized if it
needs to be. For these purposes, I'll break out the flows into the
client and the server, understanding that sometimes that is not
appropriate, but mostly it is. Shoulds and musts are apply to  this
use case only.

CVE, standardized, if may be that more than one CVE applies, but for
now, if any apply, one must be filled in.
Server Application vendor, normalized, must.
Server Application version, normalized, must.
Client Application vendor, normalized, should.
Client Application version, normalized, should.
Server OS vendor, normalized, should.
Server OS version, normalized, should.
Client OS vendor, normalized, should.
Client OS version, normalized, should.
Vulnerability exploited?, boolean,
Vulnerable, boolean, for passive sensors.
IP in data, boolean, must indicate if the IP address of either the
client or server is represented in the data. If so, it would be nice
to have a way to define where (for this particular flow) the IPs are
in the data so that they can be changed. Something like packet 4, byte
offset 54, client IP.
Port in data, boolean, same as IP in data.
Flow-pointer, normalized, a URL for the actual packet dump file.
OpenPacket meta-data, such as reliability of the source or if verified
by others.
Meta-data URL, standardized, for getting updates to the meta-data,
especially the open-packet portion of the meta-data.

Not all elements need to be a part of the system as stored in the
OpenPacket repository, however the method of creating the meta-data
should support having all these elements.

Sometimes the OS is what is vulnerable and sometimes it is just the
application. Having the OS separate is nice. It may allow a
sophisticated testing system to manipulate OS specific fingerprints in
a flow, such as Source ports, sequence numbers, options etc. to
simulate different OSs from one capture. Is that dreaming, or what. :)

I don't recall if there has been a final decision regarding packaging
of captures. In order to support meta-data, they probably need to be
gzipped-tarballs that include both the meta-data and the packet
capture. Alternatively, if the capture file names could be completely
unique and normalized, then a testing system could look locally for a
cached copy, by filename, and then use the full URL if not found. This
is probably a feature for a later version of OpenPacket.

A wonderful feature for tcpreplay that would support this use case is
client-server mode. Here the tcpreplay client and server would 'play'
their respective side of the conversation after receiving the packet
from the other side. This would be very useful for inline NSD testing.

The reason I have been thinking about this use case is that I am
putting together a system that will be able to manage the automated
testing. It's current functionality is limited. It can watch a log
file, request a remote attack agent to attack, receive confirmation
that the attack was made, and check to see if the attack was observed
in the logs. The attack agent can only launch HTTP attacks or
prewritten HPING attacks right now. It is all in Perl and extensible
through plug-ins. I begun to think about a tcpreplay plug-in and that
brought about this email.

I hope to appease my employer's intellectual property demigods and
clean up the functionality, code and docs enough to release the system
to CPAN in the not too distant future.

Regards,
Eric Hacker, CISSP

aptronym (AP-troh-NIM) noun
A name that is especially suited to the profession of its owner

I _can_ leave well enough alone, but my criteria for well enough is
pretty darn high.

[Openpacket-devel] Latest OpenPacket.org developments

[Richard B. <tao...@gm...>]

Hello everyone,

If you've seen my blog you'll know a new addition to my family has
occupied my time recently.  :)

However, I am talking with a potential hosting site and partner
affliation for OpenPacket.org.

I also plan to deploy my own Django server and familiarize myself with
the security options for its deployment.

I welcome help from anyone with experience in this area.

Thank you,

Richard