Re: [mbackup-devel] xml as a tape header: the cons

[John H. <Jo...@mw...>]

{{ Nick you know all about XML, is this reasonable??}}


----- Original Message -----
From: James O'Kane <jo...@mi...>
To: <mba...@li...>
Sent: Wednesday, 23 August 2000 09:49
Subject: Re: [mbackup-devel] xml as a tape header: the cons


> On Wed, 23 Aug 2000, John Huttley wrote:
> > Implementation wise, the client would create the XML header with the data and
> > pass it,
> > locally or over a network , to the server process.
>
> I'm not sure I understand this. Do you plan to chance the file_tag struct
> into a pointer to an xml formatted string? If that's they plan, I'm
> curious about the overhead of making each module understand how to read
> and add to the xml format. Right now, to change something in the file_tag
> struct you can just do a file_tag->current_size = 100. If we change that
> to xml, won't we have to file_tag->xml_set_current_size(100)? I'm not sure
> I see how that is better.
> On the other hand, if we're just talking about having the tape-writing
> module create the xml header just as we write to tape then we're on the
> same page.

Not quite. the existing file_tag only makes sense in the context of a unix
system.
If the intent is to backup _data_ as against _files_ then a general purpose
interface is required.
However this does not mean that we need to pass around an XML formatted string
and have every filter
parse it and re write it.  Your example shows it as worse case, because you have
picked an attribute that is
already supported in the file_tag. A simple struct dereference is hard to beat!

There is certainly going to be overhead in having each filter understand DOM, but
it is,
I think, survivable. It does not increase with the size of the file.. unlike
compression, for example.


Suppose we were backing up a netware 3 file system.
Our header needs to look like this..

<stream_code>NETWARE 3</stream_code>                 <<+++ Actually I suppose we
specify the DTD

<stream_id>3</stream_id>
<object_id>489894</objectid>
## Quick ref to this file, makes subsequent headers much shorter

<length>1234567</length>
<owner>someone</owner>
<cdate>2000-03-01 23:45:01</cdate>                    ## ISO standard dates.
<mdate>2000-08-21 20:45:01</mdate>
<trustee type=user>someuser1<rights>RWCM</rights></trustee>            ##
trustees are unlimited in number
<trustee type=user>someuser2<rights>SRWCEMFA</rights></trustee>
<trustee type=group>somegroup1<rights>RF</rights></trustee>
<trustee type=group>admingroup1<rights>SRWCEMFA</rights></trustee>
<type>directory<IRmask>SRF</IRmask>
<attribute>DPR</attribute>


<server>MYSERVER</server>
<volume>VOL1</volume>
<path>home\john\documents</path>

<namespace>long
    <filename>mydatadirectory</filename>
    <OS2EA encoding=BASE64>                    ## if resource fork size is a
problem, we can write them out as data in their own block
            kilobytes of base 64 data
    </OS2EA>
</namespace>

<namespace>mac
    <filename>mydatadirectory</filename>
    <resource_fork encoding=BASE64>
            even more kilobytes of base 64 data
    </resource_fork>
</namespace>

<namespace>DOS
    <filename>MYDAT~1</filename>
</namespace>

<block>
    <sequence>1</sequence>
    <length>524288</length>                        # we are writing out in 512kb
blocks..
     <offset>0</offset>
</block>

============

Then the next block can just have


<stream_code>NETWARE 3</stream_code
<stream_id>3</stream_id>
<object_id>489894</objectid>
<block>
    <sequence>2</sequence>
    <length>524288</length>
     <offset>524288</offset>
</block>

=============

When this hits the filters, they may act on the data to change the size of the
blocks (compression etc)
and add additional entities into the header.


As you can see, there isn't much in there that matches the existing file_tag
structure.
Netware 4 and higher is even worse.


XML/DOM headers are better in that we can do more with them. Not as simple or as
fast though.



> I was also planning on using XML to talk between modules and the GUI such
> as libglade and possibly using XML for talking between client and server,
> but I'm not sure if this would be the same XML format as the data on tape.

Good old glade! I never did work it out. I'm sure though, the the XML will be
quite different.

Regards

John