You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
|
Feb
|
Mar
|
Apr
(4) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Tim F. <tim...@gm...> - 2006-04-26 18:29:41
|
Hey guys (pretty sure we're still girl-less), I've committed some changes to the SVN repository - include/ contains links to a couple header files and lib/ has links to the libraries so that you ca= n build external things using libqcap using e.g. -I$(QCAP)/include -L$(QCAP)/lib -lqcap. Right now it's just qcap.h and segment.h in the include directory, because everything my code needs is in qcap.h and qcap.hincludes segment.h. If someone else needs something else, I'd suggest creating a ne= w symlink in the include dir for it. I've also changed the examples to link against the libraries (the src/libqcap/examples/Makefile builds as if it were external to qcap, so that in itself is an example :-) ). Thanks, -Tim On 4/26/06, Evan Hughes <eva...@gm...> wrote: > > Howdy gents (and any ladies that may be lurking), > > We're now using Subversion for revision tracking. Please check-out > from that repository and make all future commits to it. > > e > > > ------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job > easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronim= o > http://sel.as-us.falkag.net/sel?cmdlnk&kid=120709&bid&3057&dat=121642 > _______________________________________________ > Qcap-lib-dev mailing list > Qca...@li... > https://lists.sourceforge.net/lists/listinfo/qcap-lib-dev > -- Tim Furlong tim...@gm... |
|
From: Evan H. <eva...@gm...> - 2006-04-26 14:44:35
|
Howdy gents (and any ladies that may be lurking), We're now using Subversion for revision tracking. Please check-out from that repository and make all future commits to it. e |
|
From: Evan H. <eva...@gm...> - 2006-04-25 21:58:53
|
Remember, ladies and germs, the _correct_ CVS directory to be working from is: qcap/qcap/src/libqcap/ (clearly in the qcap module) The other version are left as a tool to detect which developers read the docs, and which developers get a lump of coal for Xmas. e |
|
From: <hu...@us...> - 2006-04-24 18:30:59
|
Revision: 1 Author: hughes Date: 2006-04-24 11:30:45 -0700 (Mon, 24 Apr 2006) ViewCVS: http://svn.sourceforge.net/qcap/?rev=1&view=rev Log Message: ----------- - The skeletal sourceforge website for qcap. Added Paths: ----------- www/ www/contact.php www/docs.php www/download.php www/footer.inc www/header.inc www/images/ www/images/rss.gif www/images/sf.png www/images/stock_book-16.png www/images/stock_preferences-16.png www/images/stock_save-16.png www/images/user_comment.png www/index.php www/status.php www/style.css Added: www/contact.php =================================================================== --- www/contact.php (rev 0) +++ www/contact.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,36 @@ +<?php + $Title = 'Contact'; + include_once("header.inc"); +?> + +<h1>Contacting the Maintainers</h1> + +<p> +<a href="http://www.ccsl.carleton.ca/~ehughes">Evan Hughes</a> is the +maintainer of +qcap. If you wish to be added to the SourceForge project, please contact him +directly, otherwise use the +<a href="http://sourceforge.net/mail/?group_id=149547">project mailing +list</a>. +</p> + +<p> +This project is actively seeking developers. If you are interested in working +on this project, join the +<a href="http://sourceforge.net/mail/?group_id=149547">project mailing +list</a> and get in touch with the current team. The project also produces an +<a href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1">RSS feed</a>. +</p> + + +<h1>Website Credits</h1> + +<p> +All page content was written by Evan Hughes. Icons were provided by +Mark James of <a href="http://www.famfamfam.com/">FamFamFam</a>, and Matt +Brett of <a href="http://www.feedicons.com/">FeedIcons.com</a>. +</p> + +<?php + include_once("footer.inc"); +?> Added: www/docs.php =================================================================== --- www/docs.php (rev 0) +++ www/docs.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,403 @@ +<?php + $Title = 'Documentation'; + include_once("header.inc"); + +function fn($name) { + return '<tt>' . $name . '</tt>'; +} + +function td($name) { + return fn($name); +} +?> + +<h1>qcap Documentation</h1> + +<p> +qcap is designed to read application data out of network traffic. Development +has focused on TCP protocols (specifically HTTP, FTP, and SMTP), but +mechanisms exist to read other protocols as well. +</p> + +<p> +For any topics that are not discussed here, please post to the <a +href="http://sourceforge.net/mail/?group_id=149547">project mailing +list</a>. +</p> + +<h2>Getting Started</h2> + +<h3>Design and Layout</h3> + +<p> +libqcap is designed to be similar to libpcap. Like libpcap, qcap uses a +session object, that is associated with each open trace. Also like libpcap, +qcap uses callbacks to inform the application of events. To improve +performance, qcap allows the application to request notifications for specific +events; enabling qcap to limit processing to data the applications cares +about. +</p> + +<p> +Unlike libpcap, qcap provides a number of types specific to stream-based +protocols for handling data. Those are: + +<dl class="types"> + <dt>qcap_tcpstr_t</dt> + <dd>A TCP stream. Every stream being tracked has one of these + structures. As new segments arrive in the stream, they are added + to the stream. A <tt>qcap_tcpstr_t</tt> can be queried for its + contents with <tt>qcap_tcpstr_pos_t</tt>. + </dd> + + <dt>qcap_tcpstr_pos_t</dt> + <dd>A position within a TCP stream. Positions are used to track the + start and end of fields within a stream. + </dd> + + <dt>qcap_grammar_t</dt> + <dd>Represents a protocol grammar. Streams can be parsed with + grammars. Normally, applications should not need to deal directly + with grammars, unless they are dealing with a new protocol. + </dd> +</dl> + + + +<p> +qcap also provides a family of callbacks that may be installed into the +protocol parsing engine, to detect events and modify how parsing occurs: +</p> + +<dl class="types"> + <dt>qcap_packet_handler_t</tt> + <dd>Called when a new packet arrives. No reordering is performed on + the packet. + </dd> + + <dt>qcap_stream_handler_t</tt> + <dd>Called when a new TCP stream is detected. The handler controls + how segments are treated: whether they should automatically be + stored, which direction(s) of the stream should be monitored, etc. + </dd> + + <dt>qcap_segment_handler_t</dt> + <dd>Called for every packet a TCP stream receives. Unless specified + otherwise when the callback is registered, packets are only passed + to this handler in order. + </dd> + + <dt>qcap_grammar_parser_callback_t</dt> + <dd>Called during protocol parsing to indicate that new data has + arrived, also indicating which protocol field the data falls into. + </dd> +</dl> + +<p class="note"> +As a rule, any typedef that ends with "handler_t" is a callback. +</p> + + + +<h3>Idioms in qcap</h3> + +<p> +qcap has a fairly large API. To try and minimize the amount of thinking the +programmer must do, there are a number of idioms that are followed fairly +consistently throughout the project. +</p> + +<h4>API Return Values</h4> +<p> +Function calls are one of two forms: those that cannot fail are declared as +<code>void</code>, while those that can fail are declared as +<code>int</code>. Any function declared as <code>int</code> will return +<code>QCAP_ERR_OKAY</code> when it succeeds, or some other value if it fails. +</p> + + +<h4>Function Naming</h4> +<p> +Functions are of the form <code>qcap_<i>type</i>_<i>op</i></code>. <i>type</i> +is the +short form of one of the types described above, and should be considered the +type that the function operates upon. <i>op</i> is the action being performed +on the type. +</p> + + +<h4 id="handler_control">Handler Control</h4> +<p> +Callbacks can disassociate themselves from repeating events with their return +value. They do this by returning <?= td('QCAP_CONTROL_DETACH');?>. +Alternatively, they can signal an error with <?= td('QCAP_CONTROL_ERROR')?> +or indicate that they wish to receive future notifications with +<?= td('QCAP_CONTROL_CONTINUE')?>. +</p> + + +<h3>Downloading qcap</h3> + +<p> +qcap is currently only available via CVS. See <a +href="http://sourceforge.net/cvs/?group_id=149547">the SourceForge CVS +page for qcap</a> for instructions on how to download the source. +</p> + + +<h3>Generating Documentation</h3> + +<p> +qcap uses <a href="http://www.doxygen.org">Doxygen</a> for code documentation. +Assuming that you have Doxygen installed, you can generate this documentation +by running <kbd>make doc</kbd> in +<span class="location">lib/src/libqcap/</span>. +</p> + + + +<h2>Opening Traces</h2> + +<p> +qcap currently only operates in an offline mode. A trace can be opened with +</p> + +<div class="code">char *ERRBUF[QCAP_ERRBUF_SIZE]; + +int main(int argc, char *argv[]) { + qcap_t *qp = NULL; + + int r = <?= fn('qcap_open_offline')?>(&qp, "trace.pcap", ERRBUF); + + if (r != QCAP_ERR_OKAY) { + printf("Error: %s\n", ERRBUF); + return -1; + } + + <span class="boring">// Do stuff</span> + + qcap_close(qp); +}</div> + +<p> +If all goes according to plan, <?= fn('qcap_open_offline()')?> returns +<tt>QCAP_ERR_OKAY</tt>, indicating that the file was properly opened, and the +<tt>qcap_t</tt> returned through the first argument has been properly +initialized. If any other value is returned, then the last argument will have +a string written into it, describing the nature of the error. +</p> + +<p> +After the trace has been read, the resources allocated by the stream must be +banished back to the heap. This is done with a call to +<?= fn('qcap_close()')?>. +</p> + + + + +<h2>Reading Streams</h2> + +<h3>Following Individual Streams</h3> + +<p> +qcap assumes that applications will not with to treat all streams equally. The +application may wish to track certain streams, or may wish to handle +different classes of stream in differing ways. qcap supports this by dividing +stream handlers into two. When a new stream is detected, instances of +<?= td('qcap_stream_handler_t')?> are called, and are given the ability to +attach one or more <?= td('qcap_segment_handler_t')?>s to the stream. The +segment handlers are called for each subsequent packet received in the stream. +</p> + +<p> +The application must take three steps: +</p> + +<p> +First, a stream handler must be registered. It is +responsible for telling qcap which streams are interesting to the application, +and how qcap should treat them. Any number of stream handlers may be assigned +to a qcap instance. +</p> + +<p> +Second, <?= fn('qcap_loop()')?> is called. <?= fn('qcap_loop()')?> reads +packets and processes them. If one or more stream handlers have been +registered, then qcap performs IP defragmentation and TCP reconstruction. For +each new stream discovered, each stream handler is called. +</p> + +<p> +Third, each stream handler may decide to track the newly found stream. They do +so by registering a segment handler with the new stream. The segment handler +will be called, in order, for every new segment ACK'd into the stream. +</p> + +<p> +The process is somewhat involved. We shall illustrate this behaviour through +an example. +</p> + + +<div class="code">int main(int argc, char *argv[]) { + qcap_t *qp = NULL; + + <span class="boring">int r = <?= fn('qcap_open_offline')?>(&qp, "trace.pcap", NULL); + + if (r != QCAP_ERR_OKAY) { + return -1; + }</span> + + r = qcap_stream_handler_add(qp, cb_stream, NULL, ERRBUF); + <span class="boring">if (r != QCAP_ERR_OKAY) { + return -1; + }</span> + + qcap_loop(qp, 0, NULL); + + <span class="boring">qcap_close(qp);</span> +}</div> + + +<p> +The code above opens a trace file, assigns the callback <tt>cb_stream</tt> to +the qcap handle, and then tells qcap to read in all packets with +<?= fn('qcap_loop()');?>. At this point, <tt>cb_stream</tt> remains undefined. +Let's define it: +</p> + +<div class="code">qcap_stream_control_t +cb_stream(struct qcap_packet *p, struct tcp_stream *a_tcp, int isOriginalSyn, void *data) { + <span class="boring">struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + u_short dstPort = ntohs(hdr->th_dport); + + if (dstPort == 80) {</span> + qcap_tcphdr_add_handler(p->qcap, a_tcp, cb_seg, + QCAP_SEGMENT_OPTS_ALL, NULL, NULL); + <span class="boring">} + + return QCAP_CONTROL_CONTINUE; +</span>}</div> + +<p> +<tt>cb_stream()</tt> decides if a stream is interesting by checking the server +port. If the server port is 80, it associates the callback <tt>cb_seg()</tt> +with the stream. For every packet (including the one <tt>cb_stream()</tt> was +called on), <tt>cb_seg()</tt> will be triggered. It does this by calling +<?= fn('qcap_tcphdr_add_handler()')?>. +</p> + + +<p> +<tt>cb_seg()</tt> can do anything at this point. For now, let's tell it to +print the sequence numbers of the segments it receives. +<p> + +<div class="code">qcap_control_t +seg_cb( + struct qcap_packet *p, + struct tcp_stream *stream, + struct half_stream *sender, + qcap_direction_t dir, + u_char *segment, + u_int segmentLen, + qcap_stream_state_t + *state, + void *data +) { + struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + int seq = ntohl(hdr->th_seq); + + printf("Got sequence number %x\n", seq); + + return QCAP_CONTROL_CONTINUE; +}</div> + + +<h3>Getting Text Out of Streams</h3> + +<p> +If an application wants to see the TCP payload of a stream, the have two +options. They can reconstruct it by hand, or they can let qcap do the work. +The first option is left as an exercise to the reader. The second option is +described here. +</p> + +<p> +In order to query a stream, <?= td('qcap_tcpstr_pos_t');?> structures must be +used. Each structure contains a tuple of the packet identity, and the position +within the segment. To query a stream for a substring, you must create two +positions: one at the start, and one at the end of the substring. Then you can +call <?= fn('qcap_tcpstr_pos_get_between_str()')?> to pull the text out. +</p> + +<p> +The following sample code will read the first 60 bytes out of a stream. It +builds upon the examples shown above, by replacing the stream and segment +handlers. The stream handler asks qcap to maintain a +<?= td('qcap_tcpstr_t')?> for this stream, and creates two +<?= td('qcap_tcpstr_pos_t')?> structures to point to the start and end of the +portion of the stream that we're interested in. Subsequently, the segment +handler checks to see if the <?= td('qcap_tcpstr_t')?> has accumulated enough +data to provide us with our answer. If it has, it prints the first 60 bytes, +and disassociates itself from the stream. +</p> + +<div class="code">qcap_control_t +seg_cb( + struct qcap_packet *p, + struct tcp_stream *stream, + struct half_stream *sender, + qcap_direction_t dir, + u_char *segment, + u_int segmentLen, + qcap_stream_state_t + *state, + void *data +) { + struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + int seq = ntohl(hdr->th_seq); + + printf("Got sequence number %x\n", seq); + + return QCAP_CONTROL_CONTINUE; +} + + +qcap_control_t +seg_cb( + struct qcap_packet *p, + struct tcp_stream *stream, + struct half_stream *sender, + qcap_direction_t dir, + u_char *segment, + u_int segmentLen, + qcap_stream_state_t *state, + void *user +) { + struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + int seq = ntohl(hdr->th_seq); + + struct buffer *buf = user; + + int r = qcap_tcpstr_pos_get_between(buf->start, buf->end, buf->data); + + + printf("Got sequence number %x\n", seq); + + return QCAP_CONTROL_CONTINUE; +}</div> + + +<h2>Reading Specific Fields from Specific Protocols</h2> + +<h2>Errata</h2> + +<p> +Yes, we know it's very easy to confuse "qcap" with "pcap". The HCI issues of +the name were not considered at design time. We're sorry. + +<?php + include_once("footer.inc"); +?> Added: www/download.php =================================================================== --- www/download.php (rev 0) +++ www/download.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,16 @@ +<?php + $Title = 'Download'; + include_once("header.inc"); +?> + +<h1>Downloading qcap</h1> + +<p> +qcap is currently only available via CVS. See <a +href="http://sourceforge.net/cvs/?group_id=149547">the SourceForge CVS +page for qcap</a> for instructions on how to download the source. +</p> + +<?php + include_once("footer.inc"); +?> Added: www/footer.inc =================================================================== --- www/footer.inc (rev 0) +++ www/footer.inc 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,6 @@ + + </td><!-- td:content --> + </tr> +</table> +</body> +</html> Added: www/header.inc =================================================================== --- www/header.inc (rev 0) +++ www/header.inc 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,23 @@ +<html> +<head> + <title>qcap - <?= $GLOBALS['Title'] ?></title> + <link href="style.css" rel="StyleSheet"/> + <link rel="alternate" type="application/rss+xml" + href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1" title="qcap news"/> +</head> +<body> +<table class="page"> + <tr> + <td class="nav" valign="top"> + <div><a href="docs.php">Documentation</a></div> + <div><a href="download.php">Download</a></div> + <div><a href="status.php">Project Plan</a></div> + + <div class="special"><a href="contact.php">Contact</a></div> + + <div class="special"><a href="http://sourceforge.net/projects/qcap/">Sourceforge project</a></div> + + <div class="special"><a href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1">Project News Feed</a></div> + </td> + + <td class="content" valign="top"> Added: www/images/rss.gif =================================================================== (Binary files differ) Property changes on: www/images/rss.gif ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/sf.png =================================================================== (Binary files differ) Property changes on: www/images/sf.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/stock_book-16.png =================================================================== (Binary files differ) Property changes on: www/images/stock_book-16.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/stock_preferences-16.png =================================================================== (Binary files differ) Property changes on: www/images/stock_preferences-16.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/stock_save-16.png =================================================================== (Binary files differ) Property changes on: www/images/stock_save-16.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/user_comment.png =================================================================== (Binary files differ) Property changes on: www/images/user_comment.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/index.php =================================================================== --- www/index.php (rev 0) +++ www/index.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,37 @@ +<?php + $GLOBALS['Title'] = "Home page"; + include_once("header.inc"); +?> + +<h1>qcap</h1> + +<p> +qcap is a library for network traffic analysis. It provides mechanisms for an +application to: +</p> + +<ul> + + <li>access <a href="http://www.tcpdump.org">libpcap</a> data sources + (files and network devices), + + <li>defragment IP packets, + + <li>reconstruct TCP streams, + + <li>parse TCP streams, + + <li>decode encrypted/obfuscated portions of TCP streams. + +</ul> + +<p> +It is designed to provide applications with easy access to the content of TCP +streams. The parsing facilities identify portions of layer-7 streams, and hand +them off to application-level callbacks. If the stream is encoded with +credentials available to qcap, qcap will decode that portion of the stream. +</p> + +<?php + include_once("footer.inc"); +?> Added: www/status.php =================================================================== --- www/status.php (rev 0) +++ www/status.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,41 @@ +<?php + $Title = 'Status'; + include_once("header.inc"); +?> + +<h1>Project Status</h1> + +<p> +qcap is currently around version 0.1.1. We are currently finding a stable +API, and bug hunting. +</p> + +<h2>Upcoming Changes</h2> + +<p> +The project plan can be found in the CVS repository. It provides a detailed +plan of upcoming changes, and an explanation of the version number scheme. +<a +href="http://cvs.sourceforge.net/viewcvs.py/*checkout*/qcap/lib/doc/plan.txt">The +current version is available via ViewCVS</a>. +</p> + +<h2>Known Bugs</h2> + +<ul> + <li>The libnids modifications prevent duplicate packets from being handled + properly. Thus far, we have only been able to test traces with + multiple ACK packets, that are without payload. We're not too sure + what duplicate packets with payloads will cause.</li> + + <li>Grammars are currently one-sided, the parser of the server side of a + conversation is independent of the client side.</li> + + <li>Grammar acceptance is not handled gracefully. We have a greedy parser + that notifies its callbacks whenever a match is made, even if the + match is part of a larger production that later fails.</li> +</ul> + +<?php + include_once("footer.inc"); +?> Added: www/style.css =================================================================== --- www/style.css (rev 0) +++ www/style.css 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,98 @@ +/** + * Page layout + */ + +body { + height: 100%; + padding: 5px; + margin: 0px; +} + +table.page { + height: 100%; + width: 90%; + border: black solid 1px; +} + +table.page tr td.nav { + height: 100%; + border-right: #ccc solid 1px; + + font-size: smaller; +} + +table.page tr td.nav div.special { + margin-top: 2ex; +} + +table.page tr td.nav div { + margin-top: .25ex; +} + +table.page tr td.nav div a { + background-repeat: no-repeat; + padding-left: 18px; + margin-right: 18px; + line-height: 18px; +} + +table.page tr td.nav div a[href="docs.php"] { + background-image: url(images/stock_book-16.png); +} + +table.page tr td.nav div a[href="download.php"] { + background-image: url(images/stock_save-16.png); +} + +table.page tr td.nav div a[href="status.php"] { + background-image: url(images/stock_preferences-16.png); +} + +table.page tr td.nav div a[href="contact.php"] { + background-image: url(images/user_comment.png); +} + +table.page tr td.nav div a[href="http://sourceforge.net/projects/qcap/"] { + background-image: url(images/sf.png); +} + +table.page tr td.nav div a[href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1"] { + background-image: url(images/rss.gif); +} + +table tr td.content { + height: 100%; +} + + +/** + * Text for code documentation. + */ +dl.types { + margin-left: 2ex; +} + +dl.types dt { + margin-top: 1ex; +} + + +dl.types dt, +tt { + font-family: monospace; +} + + +div.code { + padding: 1ex; + + white-space: pre; + font-family: monospace; + overflow: auto; + + background-color: #ccc; +} + +div.code span.boring { + color: #444; +} This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
|
From: Evan H. <eva...@gm...> - 2005-12-20 23:28:37
|
Howdy folks, As you can see, the qcap webpage has been updated, as has the version in CVS. The CVS code is slightly buggy at the moment, and I'm looking for folks to take on some of the items listed in the project plan. The plan is available at http://cvs.sourceforge.net/viewcvs.py/*checkout*/qcap/lib/doc/plan.txt. Any volunteers? e |