[Redoubt-checkins] CVS: docs protocol.html,NONE,1.1 index.html,1.2,1.3

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Update of /cvsroot/redoubt/docs
In directory usw-pr-cvs1:/tmp/cvs-serv10128

Modified Files:
	index.html 
Added Files:
	protocol.html 
Log Message:
Added protocol spec and toc to index file

--- NEW FILE: protocol.html ---
<html><head><title>Redoubt: network protocol version 1</title></head>
<body bgcolor=ffffff><h1>Redoubt: network protocol version 1</h1>

Redoubt uses a subset of the the HTTP/1.1 protocol (see 
<a href='http://www.rfc-editor.org/rfc/rfc2068.txt'>rfc 2068</a>). It does
allow the implementation of a server using CGI scripts and a normal HTTP server
like Apache. For security reasons all connections should be done over SSL.
Some servers may accept normal http connections though.

<h2>Authentication</h2>

Authentication is nor mandated nor enforced by the protocol. However,
not using authentication is not advisable for obvious reasons.
One of the following authentication methods should be used
(not all of them have already been implemented):

<ul>
<li><b>Basic Authentication:</b><br>
     This scheme (see <a href=http://www.rfc-editor.org/rfc/rfc2617.txt>RFC 2617</a>) 
     uses the standard http authentication mechanism. As it
     sends passwords in clear text over the wire it is not advisable to
     use it unless anonymous SSL (i.e. SSL without client side authentication)
     is used.
<li><b>Digest Authentication:</b><br>
     This authentication system (see <a href=http://www.rfc-editor.org/rfc/rfc2617.txt>RFC 2617</a>)
     does not send passwords in clear over the wire, and is 
     more secure than basic authentication. It still allows
     man-in-the-middle attacks.
<li><b>SSL with public keys:</b><br>
     Most web servers allow public key authentication with SSL. This
     is one of the most secure ways to authenticate and encrypt
     traffic.
<li><b>SSH and http tunneling:</b><br>
     If none of the above authentication systems can be used, it
     is always possible to tunnel the traffic through an ssh tunnel.
     In this scenario ssh can either use passwords or public key
     authentication.
</ul>

<h2>Base URL</h2>

All URLs a backup client uses may have to be prefixed by a fixed string (as in
/backup/username/) called the base URL. This is server specific and usually
only necessary for servers that are not exclusive backup servers. This URL
is shown as <tt>/&lt;base-url&gt;</tt>.

<h2>Fields</h2>

The following fields should be supported by the client and server and always
used if appropriate.

<ul>
<li>Range
<li>Content-MD5
<li>Content-Length
<li>Server / Client
</ul>

All connections should specify 'no-cache' in order to not confuse caches.
Multiple requests per connection should be supported by the server and the
client for bigger throughput, but are not really necessary.

<h2>Filenames</h2>

Filenames include important information and must conform to the
Perl regular expression shown below. They consist of five fields,
separated by '.':
<ol>
<li><b>Random Filename:</b><br>
     Every filename on the client is mapped to a random filename. This avoids
     the leakage of information in filenames. Only the client keeps the
     mapping in a file, if that information is lost all files need to be
     downloaded by the client and decrypted in order to recreated the mapping.
     The field must be between 5 and 50 characters long. It is recommended
     that a random string is converted to a suitable filename with base64
     (all '/' need to be replaced with '-').
<li><b>Filetype:</b><br>
     The archive stores four kind of file types: full files, meta data only,
     deletions, and hardlink information (the format of those files are
     described elsewhere). In order to speed up recovery the type of file 
     is encoded in the filename, using one character: 'F', 'M', 'D' or 'L'.
<li><b>Generation:</b><br>
     Every version of a file gets a new generation number. This field contains
     the time when the file was backed up in seconds since the epoch.
<li><b>Block:</b><br>
     Large files must be broken up into pieces. This fields contains the
     block number for this part of the file, and the total number of blocks
     used. The two numbers are separated by a '-' sign.
<li><b>Redundancy and Mirror information:</b><br>
     In order to recreate a file the client needs to know how it was stored
     on different servers. This field stores this information. Its format
     depends on the redundancy algorithm used (mirror, raid, ...) and
     generally contains between 2 and 5 characters.
</ol>

The filename must match the following Perl regular expression:
<pre>
     ([A-Za-z0-9+-]{5,50}).([FMDL]).([0-9]+).([0-9+]-[0-9]+).([A-Za-z0-9+-]+) 
</pre>

Note that the server will make use of some of the data in the filename (in
particular the generation information). The rest of the fields are only
for clients, and standardizing those makes it easier to develop clients that
can inter operate with each other.

<h2>Options</h2>

The Client sends a <tt>GET</tt> request for the URL <tt>/&lt;base-url&gt;/options</tt>

<p>
The Server returns an ASCII file (content-type text/plain)
that lists options and their values. The option name is separated by a tab
from the value, which goes to the end of the line.
<p>
The following options are sent out (a serer may include additional ones,
clients should just ignore options they don't recognize).

<p><table border=1 align=center>
<tr><td><tt>VERSION</tt></td><td>protocol version the server supports, currently always 1.</td></tr>
<tr><td><tt>SPACE</tt></td><td>Total space in bytes for this archive user.</td></tr>
<tr><td><tt>USED</tt></td><td>Total bytes already used in this backup.</td></tr>
<tr><td><tt>FILES</tt></td><td>Total number of files currently stored.</td></tr>
<tr><td><tt>RESTORE_BANDWIDTH</tt></td><td>Current restore bandwidth in kbits/second for this archive user.</td></tr>
<tr><td><tt>BACKUP_BANDWIDTH</tt></td><td>Current bqckup bandwidth in kbits/second for this archive user.</td></tr>
<tr><td><tt>MAX_FILE_SIZE</tt></td><td>Maximum size a file can have.</td></tr>
</table>

<p>Note that the <tt>OPTIONS</tt> request that is part of the http protocol cannot
be used as it does not allow extensions. The client should not be surprised
if other fields and informations are sent by the server.

<h2>Listing</h2>

The client can request a full listing of all files on the server. This should
only be done to either check the archive to find dead or missing files, or
to start a full restore.
<p>
The listing is requested by sending a <tt>GET</tt> request for
<tt>/&lt;base-url&gt;/</tt>

<p>
The server responds with an ASCII file (content type text/plain) that lists
each filename in the index on a line by itself.

<h2>Requesting a file</h2>

The client sends a <tt>GET</tt> request with a URL of the form
<tt>/&lt;base-url&gt;/filename</tt>, and the
server returns the file (if it exists). The server should support
<tt>Range:</tt> requests, and always add the size and md5sum in the response
header. The content-type should always be application/octet-stream.

<h2>Storing a file</h2>

The client sends a <tt>PUT</tt> request with the URL
<tt>/&lt;base-url&gt;/filename</tt> and the file as part of the request with a
content-type of application/octet-stream. It may only send parts of the file
using <tt>Range:</tt>, and the server is expected to support this.

<h2>Deleting a file</h2>

The client sends the <tt>DELETE</tt> request with the URL
<tt>/&lt;base-url&gt;/filename</tt>. The server acts
accordingly.

<h2>Checking for a file</h2>

If the client wants to check if a file is still in the archive, it can issue a
<tt>HEAD</tt> request with the same URL used to <tt>GET</tt> or <tt>PUT</tt>
the file.  The server is supposed to return the
correct information and include the md5sum of the file (or of the range of
bytes requested).

Index: index.html
===================================================================
RCS file: /cvsroot/redoubt/docs/index.html,v
retrieving revision 1.2
retrieving revision 1.3
diff -C2 -d -r1.2 -r1.3
*** index.html	2002/01/17 20:09:48	1.2
--- index.html	2002/01/20 20:23:36	1.3
***************
*** 4,8 ****

! <H2>What it does</h2>

  The redoubt backup system lets you backup your files to one or more
--- 4,17 ----

! <h2>Table of Contents</h2>
! <ul>
! <li><a href=#what>What it does</a>
! <li><a href=#stat>Current Status</a>
! <li><a href=#docs>Documentation</a>
! <li><a href=#download>Download</a>
! <li><a href=#links>Links</a>
! </ul>
! 
! <H2><a name=what>What it does</h2>

  The redoubt backup system lets you backup your files to one or more
***************
*** 100,104 ****

! <h2>Current Status</h2>

  We are currently creating the specifications for backup clients,
--- 109,113 ----

! <h2><a name=stat>Current Status</h2>

  We are currently creating the specifications for backup clients,
***************
*** 106,112 ****
  prototype.

! <h2>Sourceforge</h2>

! All important files are accessible via the <a
! href=http://sourceforge.net/projects/redoubt/>redoubt SourceForge
! page</a>
--- 115,141 ----
  prototype.

! <h2><a name=docs>Documentation</h2>

! <ul>
! <li><a href=http://redoubt.sourceforge.net/protocol.html>Client/Server Protocol</a>
! </ul>
! 
! 
! <h2><a name=download>Download</h2>
! 
! All files are accessible via the 
! <a href=http://sourceforge.net/projects/redoubt/>redoubt sourceforge page</a>.
! 
! <h2><a name=links>Links</h2>
! 
! <ul>
! <li><b>Mirrors:</b><br>
!      The main page is hosted at 
!      <a href=http://redoubt.sourceforge.net>http://redoubt.sourceforge.net</a>. A
!      mirror of this web site is at <a href=http://www.ars.net/redoubt/>http://www.ars.net/redoubt/</a>.
! <li><b>Software:</b><br>
!      All the code is available via our
!      <a href=http://sourceforge.net/projects/redoubt/>redoubt sourceforge page</a>
! <li><b>Source forge:</b><br>
!      For our development we make extensive use of <A href="http://sourceforge.net"><IMG src="http://sourceforge.net/sflogo.php?group_id=44154&type=1"width="88"height="31"border="0"alt="SourceForge Logo"></A>
! </ul>