Menu
▾
▴
[Qcap-lib-dev] SF.net SVN: qcap: [1]
|
From: <hu...@us...> - 2006-04-24 18:30:59
|
Revision: 1 Author: hughes Date: 2006-04-24 11:30:45 -0700 (Mon, 24 Apr 2006) ViewCVS: http://svn.sourceforge.net/qcap/?rev=1&view=rev Log Message: ----------- - The skeletal sourceforge website for qcap. Added Paths: ----------- www/ www/contact.php www/docs.php www/download.php www/footer.inc www/header.inc www/images/ www/images/rss.gif www/images/sf.png www/images/stock_book-16.png www/images/stock_preferences-16.png www/images/stock_save-16.png www/images/user_comment.png www/index.php www/status.php www/style.css Added: www/contact.php =================================================================== --- www/contact.php (rev 0) +++ www/contact.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,36 @@ +<?php + $Title = 'Contact'; + include_once("header.inc"); +?> + +<h1>Contacting the Maintainers</h1> + +<p> +<a href="http://www.ccsl.carleton.ca/~ehughes">Evan Hughes</a> is the +maintainer of +qcap. If you wish to be added to the SourceForge project, please contact him +directly, otherwise use the +<a href="http://sourceforge.net/mail/?group_id=149547">project mailing +list</a>. +</p> + +<p> +This project is actively seeking developers. If you are interested in working +on this project, join the +<a href="http://sourceforge.net/mail/?group_id=149547">project mailing +list</a> and get in touch with the current team. The project also produces an +<a href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1">RSS feed</a>. +</p> + + +<h1>Website Credits</h1> + +<p> +All page content was written by Evan Hughes. Icons were provided by +Mark James of <a href="http://www.famfamfam.com/">FamFamFam</a>, and Matt +Brett of <a href="http://www.feedicons.com/">FeedIcons.com</a>. +</p> + +<?php + include_once("footer.inc"); +?> Added: www/docs.php =================================================================== --- www/docs.php (rev 0) +++ www/docs.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,403 @@ +<?php + $Title = 'Documentation'; + include_once("header.inc"); + +function fn($name) { + return '<tt>' . $name . '</tt>'; +} + +function td($name) { + return fn($name); +} +?> + +<h1>qcap Documentation</h1> + +<p> +qcap is designed to read application data out of network traffic. Development +has focused on TCP protocols (specifically HTTP, FTP, and SMTP), but +mechanisms exist to read other protocols as well. +</p> + +<p> +For any topics that are not discussed here, please post to the <a +href="http://sourceforge.net/mail/?group_id=149547">project mailing +list</a>. +</p> + +<h2>Getting Started</h2> + +<h3>Design and Layout</h3> + +<p> +libqcap is designed to be similar to libpcap. Like libpcap, qcap uses a +session object, that is associated with each open trace. Also like libpcap, +qcap uses callbacks to inform the application of events. To improve +performance, qcap allows the application to request notifications for specific +events; enabling qcap to limit processing to data the applications cares +about. +</p> + +<p> +Unlike libpcap, qcap provides a number of types specific to stream-based +protocols for handling data. Those are: + +<dl class="types"> + <dt>qcap_tcpstr_t</dt> + <dd>A TCP stream. Every stream being tracked has one of these + structures. As new segments arrive in the stream, they are added + to the stream. A <tt>qcap_tcpstr_t</tt> can be queried for its + contents with <tt>qcap_tcpstr_pos_t</tt>. + </dd> + + <dt>qcap_tcpstr_pos_t</dt> + <dd>A position within a TCP stream. Positions are used to track the + start and end of fields within a stream. + </dd> + + <dt>qcap_grammar_t</dt> + <dd>Represents a protocol grammar. Streams can be parsed with + grammars. Normally, applications should not need to deal directly + with grammars, unless they are dealing with a new protocol. + </dd> +</dl> + + + +<p> +qcap also provides a family of callbacks that may be installed into the +protocol parsing engine, to detect events and modify how parsing occurs: +</p> + +<dl class="types"> + <dt>qcap_packet_handler_t</tt> + <dd>Called when a new packet arrives. No reordering is performed on + the packet. + </dd> + + <dt>qcap_stream_handler_t</tt> + <dd>Called when a new TCP stream is detected. The handler controls + how segments are treated: whether they should automatically be + stored, which direction(s) of the stream should be monitored, etc. + </dd> + + <dt>qcap_segment_handler_t</dt> + <dd>Called for every packet a TCP stream receives. Unless specified + otherwise when the callback is registered, packets are only passed + to this handler in order. + </dd> + + <dt>qcap_grammar_parser_callback_t</dt> + <dd>Called during protocol parsing to indicate that new data has + arrived, also indicating which protocol field the data falls into. + </dd> +</dl> + +<p class="note"> +As a rule, any typedef that ends with "handler_t" is a callback. +</p> + + + +<h3>Idioms in qcap</h3> + +<p> +qcap has a fairly large API. To try and minimize the amount of thinking the +programmer must do, there are a number of idioms that are followed fairly +consistently throughout the project. +</p> + +<h4>API Return Values</h4> +<p> +Function calls are one of two forms: those that cannot fail are declared as +<code>void</code>, while those that can fail are declared as +<code>int</code>. Any function declared as <code>int</code> will return +<code>QCAP_ERR_OKAY</code> when it succeeds, or some other value if it fails. +</p> + + +<h4>Function Naming</h4> +<p> +Functions are of the form <code>qcap_<i>type</i>_<i>op</i></code>. <i>type</i> +is the +short form of one of the types described above, and should be considered the +type that the function operates upon. <i>op</i> is the action being performed +on the type. +</p> + + +<h4 id="handler_control">Handler Control</h4> +<p> +Callbacks can disassociate themselves from repeating events with their return +value. They do this by returning <?= td('QCAP_CONTROL_DETACH');?>. +Alternatively, they can signal an error with <?= td('QCAP_CONTROL_ERROR')?> +or indicate that they wish to receive future notifications with +<?= td('QCAP_CONTROL_CONTINUE')?>. +</p> + + +<h3>Downloading qcap</h3> + +<p> +qcap is currently only available via CVS. See <a +href="http://sourceforge.net/cvs/?group_id=149547">the SourceForge CVS +page for qcap</a> for instructions on how to download the source. +</p> + + +<h3>Generating Documentation</h3> + +<p> +qcap uses <a href="http://www.doxygen.org">Doxygen</a> for code documentation. +Assuming that you have Doxygen installed, you can generate this documentation +by running <kbd>make doc</kbd> in +<span class="location">lib/src/libqcap/</span>. +</p> + + + +<h2>Opening Traces</h2> + +<p> +qcap currently only operates in an offline mode. A trace can be opened with +</p> + +<div class="code">char *ERRBUF[QCAP_ERRBUF_SIZE]; + +int main(int argc, char *argv[]) { + qcap_t *qp = NULL; + + int r = <?= fn('qcap_open_offline')?>(&qp, "trace.pcap", ERRBUF); + + if (r != QCAP_ERR_OKAY) { + printf("Error: %s\n", ERRBUF); + return -1; + } + + <span class="boring">// Do stuff</span> + + qcap_close(qp); +}</div> + +<p> +If all goes according to plan, <?= fn('qcap_open_offline()')?> returns +<tt>QCAP_ERR_OKAY</tt>, indicating that the file was properly opened, and the +<tt>qcap_t</tt> returned through the first argument has been properly +initialized. If any other value is returned, then the last argument will have +a string written into it, describing the nature of the error. +</p> + +<p> +After the trace has been read, the resources allocated by the stream must be +banished back to the heap. This is done with a call to +<?= fn('qcap_close()')?>. +</p> + + + + +<h2>Reading Streams</h2> + +<h3>Following Individual Streams</h3> + +<p> +qcap assumes that applications will not with to treat all streams equally. The +application may wish to track certain streams, or may wish to handle +different classes of stream in differing ways. qcap supports this by dividing +stream handlers into two. When a new stream is detected, instances of +<?= td('qcap_stream_handler_t')?> are called, and are given the ability to +attach one or more <?= td('qcap_segment_handler_t')?>s to the stream. The +segment handlers are called for each subsequent packet received in the stream. +</p> + +<p> +The application must take three steps: +</p> + +<p> +First, a stream handler must be registered. It is +responsible for telling qcap which streams are interesting to the application, +and how qcap should treat them. Any number of stream handlers may be assigned +to a qcap instance. +</p> + +<p> +Second, <?= fn('qcap_loop()')?> is called. <?= fn('qcap_loop()')?> reads +packets and processes them. If one or more stream handlers have been +registered, then qcap performs IP defragmentation and TCP reconstruction. For +each new stream discovered, each stream handler is called. +</p> + +<p> +Third, each stream handler may decide to track the newly found stream. They do +so by registering a segment handler with the new stream. The segment handler +will be called, in order, for every new segment ACK'd into the stream. +</p> + +<p> +The process is somewhat involved. We shall illustrate this behaviour through +an example. +</p> + + +<div class="code">int main(int argc, char *argv[]) { + qcap_t *qp = NULL; + + <span class="boring">int r = <?= fn('qcap_open_offline')?>(&qp, "trace.pcap", NULL); + + if (r != QCAP_ERR_OKAY) { + return -1; + }</span> + + r = qcap_stream_handler_add(qp, cb_stream, NULL, ERRBUF); + <span class="boring">if (r != QCAP_ERR_OKAY) { + return -1; + }</span> + + qcap_loop(qp, 0, NULL); + + <span class="boring">qcap_close(qp);</span> +}</div> + + +<p> +The code above opens a trace file, assigns the callback <tt>cb_stream</tt> to +the qcap handle, and then tells qcap to read in all packets with +<?= fn('qcap_loop()');?>. At this point, <tt>cb_stream</tt> remains undefined. +Let's define it: +</p> + +<div class="code">qcap_stream_control_t +cb_stream(struct qcap_packet *p, struct tcp_stream *a_tcp, int isOriginalSyn, void *data) { + <span class="boring">struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + u_short dstPort = ntohs(hdr->th_dport); + + if (dstPort == 80) {</span> + qcap_tcphdr_add_handler(p->qcap, a_tcp, cb_seg, + QCAP_SEGMENT_OPTS_ALL, NULL, NULL); + <span class="boring">} + + return QCAP_CONTROL_CONTINUE; +</span>}</div> + +<p> +<tt>cb_stream()</tt> decides if a stream is interesting by checking the server +port. If the server port is 80, it associates the callback <tt>cb_seg()</tt> +with the stream. For every packet (including the one <tt>cb_stream()</tt> was +called on), <tt>cb_seg()</tt> will be triggered. It does this by calling +<?= fn('qcap_tcphdr_add_handler()')?>. +</p> + + +<p> +<tt>cb_seg()</tt> can do anything at this point. For now, let's tell it to +print the sequence numbers of the segments it receives. +<p> + +<div class="code">qcap_control_t +seg_cb( + struct qcap_packet *p, + struct tcp_stream *stream, + struct half_stream *sender, + qcap_direction_t dir, + u_char *segment, + u_int segmentLen, + qcap_stream_state_t + *state, + void *data +) { + struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + int seq = ntohl(hdr->th_seq); + + printf("Got sequence number %x\n", seq); + + return QCAP_CONTROL_CONTINUE; +}</div> + + +<h3>Getting Text Out of Streams</h3> + +<p> +If an application wants to see the TCP payload of a stream, the have two +options. They can reconstruct it by hand, or they can let qcap do the work. +The first option is left as an exercise to the reader. The second option is +described here. +</p> + +<p> +In order to query a stream, <?= td('qcap_tcpstr_pos_t');?> structures must be +used. Each structure contains a tuple of the packet identity, and the position +within the segment. To query a stream for a substring, you must create two +positions: one at the start, and one at the end of the substring. Then you can +call <?= fn('qcap_tcpstr_pos_get_between_str()')?> to pull the text out. +</p> + +<p> +The following sample code will read the first 60 bytes out of a stream. It +builds upon the examples shown above, by replacing the stream and segment +handlers. The stream handler asks qcap to maintain a +<?= td('qcap_tcpstr_t')?> for this stream, and creates two +<?= td('qcap_tcpstr_pos_t')?> structures to point to the start and end of the +portion of the stream that we're interested in. Subsequently, the segment +handler checks to see if the <?= td('qcap_tcpstr_t')?> has accumulated enough +data to provide us with our answer. If it has, it prints the first 60 bytes, +and disassociates itself from the stream. +</p> + +<div class="code">qcap_control_t +seg_cb( + struct qcap_packet *p, + struct tcp_stream *stream, + struct half_stream *sender, + qcap_direction_t dir, + u_char *segment, + u_int segmentLen, + qcap_stream_state_t + *state, + void *data +) { + struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + int seq = ntohl(hdr->th_seq); + + printf("Got sequence number %x\n", seq); + + return QCAP_CONTROL_CONTINUE; +} + + +qcap_control_t +seg_cb( + struct qcap_packet *p, + struct tcp_stream *stream, + struct half_stream *sender, + qcap_direction_t dir, + u_char *segment, + u_int segmentLen, + qcap_stream_state_t *state, + void *user +) { + struct tcphdr *hdr = (struct tcphdr *)(p->ip_packet + 4 * ((struct ip*)p->ip_packet)->ip_hl); + int seq = ntohl(hdr->th_seq); + + struct buffer *buf = user; + + int r = qcap_tcpstr_pos_get_between(buf->start, buf->end, buf->data); + + + printf("Got sequence number %x\n", seq); + + return QCAP_CONTROL_CONTINUE; +}</div> + + +<h2>Reading Specific Fields from Specific Protocols</h2> + +<h2>Errata</h2> + +<p> +Yes, we know it's very easy to confuse "qcap" with "pcap". The HCI issues of +the name were not considered at design time. We're sorry. + +<?php + include_once("footer.inc"); +?> Added: www/download.php =================================================================== --- www/download.php (rev 0) +++ www/download.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,16 @@ +<?php + $Title = 'Download'; + include_once("header.inc"); +?> + +<h1>Downloading qcap</h1> + +<p> +qcap is currently only available via CVS. See <a +href="http://sourceforge.net/cvs/?group_id=149547">the SourceForge CVS +page for qcap</a> for instructions on how to download the source. +</p> + +<?php + include_once("footer.inc"); +?> Added: www/footer.inc =================================================================== --- www/footer.inc (rev 0) +++ www/footer.inc 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,6 @@ + + </td><!-- td:content --> + </tr> +</table> +</body> +</html> Added: www/header.inc =================================================================== --- www/header.inc (rev 0) +++ www/header.inc 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,23 @@ +<html> +<head> + <title>qcap - <?= $GLOBALS['Title'] ?></title> + <link href="style.css" rel="StyleSheet"/> + <link rel="alternate" type="application/rss+xml" + href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1" title="qcap news"/> +</head> +<body> +<table class="page"> + <tr> + <td class="nav" valign="top"> + <div><a href="docs.php">Documentation</a></div> + <div><a href="download.php">Download</a></div> + <div><a href="status.php">Project Plan</a></div> + + <div class="special"><a href="contact.php">Contact</a></div> + + <div class="special"><a href="http://sourceforge.net/projects/qcap/">Sourceforge project</a></div> + + <div class="special"><a href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1">Project News Feed</a></div> + </td> + + <td class="content" valign="top"> Added: www/images/rss.gif =================================================================== (Binary files differ) Property changes on: www/images/rss.gif ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/sf.png =================================================================== (Binary files differ) Property changes on: www/images/sf.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/stock_book-16.png =================================================================== (Binary files differ) Property changes on: www/images/stock_book-16.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/stock_preferences-16.png =================================================================== (Binary files differ) Property changes on: www/images/stock_preferences-16.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/stock_save-16.png =================================================================== (Binary files differ) Property changes on: www/images/stock_save-16.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/images/user_comment.png =================================================================== (Binary files differ) Property changes on: www/images/user_comment.png ___________________________________________________________________ Name: svn:mime-type + application/octet-stream Added: www/index.php =================================================================== --- www/index.php (rev 0) +++ www/index.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,37 @@ +<?php + $GLOBALS['Title'] = "Home page"; + include_once("header.inc"); +?> + +<h1>qcap</h1> + +<p> +qcap is a library for network traffic analysis. It provides mechanisms for an +application to: +</p> + +<ul> + + <li>access <a href="http://www.tcpdump.org">libpcap</a> data sources + (files and network devices), + + <li>defragment IP packets, + + <li>reconstruct TCP streams, + + <li>parse TCP streams, + + <li>decode encrypted/obfuscated portions of TCP streams. + +</ul> + +<p> +It is designed to provide applications with easy access to the content of TCP +streams. The parsing facilities identify portions of layer-7 streams, and hand +them off to application-level callbacks. If the stream is encoded with +credentials available to qcap, qcap will decode that portion of the stream. +</p> + +<?php + include_once("footer.inc"); +?> Added: www/status.php =================================================================== --- www/status.php (rev 0) +++ www/status.php 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,41 @@ +<?php + $Title = 'Status'; + include_once("header.inc"); +?> + +<h1>Project Status</h1> + +<p> +qcap is currently around version 0.1.1. We are currently finding a stable +API, and bug hunting. +</p> + +<h2>Upcoming Changes</h2> + +<p> +The project plan can be found in the CVS repository. It provides a detailed +plan of upcoming changes, and an explanation of the version number scheme. +<a +href="http://cvs.sourceforge.net/viewcvs.py/*checkout*/qcap/lib/doc/plan.txt">The +current version is available via ViewCVS</a>. +</p> + +<h2>Known Bugs</h2> + +<ul> + <li>The libnids modifications prevent duplicate packets from being handled + properly. Thus far, we have only been able to test traces with + multiple ACK packets, that are without payload. We're not too sure + what duplicate packets with payloads will cause.</li> + + <li>Grammars are currently one-sided, the parser of the server side of a + conversation is independent of the client side.</li> + + <li>Grammar acceptance is not handled gracefully. We have a greedy parser + that notifies its callbacks whenever a match is made, even if the + match is part of a larger production that later fails.</li> +</ul> + +<?php + include_once("footer.inc"); +?> Added: www/style.css =================================================================== --- www/style.css (rev 0) +++ www/style.css 2006-04-24 18:30:45 UTC (rev 1) @@ -0,0 +1,98 @@ +/** + * Page layout + */ + +body { + height: 100%; + padding: 5px; + margin: 0px; +} + +table.page { + height: 100%; + width: 90%; + border: black solid 1px; +} + +table.page tr td.nav { + height: 100%; + border-right: #ccc solid 1px; + + font-size: smaller; +} + +table.page tr td.nav div.special { + margin-top: 2ex; +} + +table.page tr td.nav div { + margin-top: .25ex; +} + +table.page tr td.nav div a { + background-repeat: no-repeat; + padding-left: 18px; + margin-right: 18px; + line-height: 18px; +} + +table.page tr td.nav div a[href="docs.php"] { + background-image: url(images/stock_book-16.png); +} + +table.page tr td.nav div a[href="download.php"] { + background-image: url(images/stock_save-16.png); +} + +table.page tr td.nav div a[href="status.php"] { + background-image: url(images/stock_preferences-16.png); +} + +table.page tr td.nav div a[href="contact.php"] { + background-image: url(images/user_comment.png); +} + +table.page tr td.nav div a[href="http://sourceforge.net/projects/qcap/"] { + background-image: url(images/sf.png); +} + +table.page tr td.nav div a[href="http://sourceforge.net/export/rss2_projnews.php?group_id=149547&rss_fulltext=1"] { + background-image: url(images/rss.gif); +} + +table tr td.content { + height: 100%; +} + + +/** + * Text for code documentation. + */ +dl.types { + margin-left: 2ex; +} + +dl.types dt { + margin-top: 1ex; +} + + +dl.types dt, +tt { + font-family: monospace; +} + + +div.code { + padding: 1ex; + + white-space: pre; + font-family: monospace; + overflow: auto; + + background-color: #ccc; +} + +div.code span.boring { + color: #444; +} This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
