dump-users

Re: [Dump-users] That old chesnut - Dump, QFA and DLT IV drives.

[Stelian P. <st...@po...>]

Le jeudi 31 janvier 2008 à 11:49 +0000, John Lewis a écrit :
> Hi,
> 
> I cannot get the subject to work, 90% of the time I get "error getting
> tapepos". If I disable QFA it works reliably.
[...]

I'm afraid you won't get much help here. Your problem looks like it is
driver related, so I think you should ask the developers of the st
driver for suggestions.

Stelian.
-- 
Stelian Pop <st...@po...>

[Dump-users] That old chesnut - Dump, QFA and DLT IV drives.

[John L. <jl...@jo...>]

Hi,

I cannot get the subject to work, 90% of the time I get "error getting
tapepos". If I disable QFA it works reliably.

Having read a post on here from a couple of years ago Stelian suggested
enabling scsi2logical i.e. "mt -f /dev/nst0 stsetoptions scsi2logical".
However when I run this command I don't see any output in either dmesg,
syslog or messages, so I am not confident it has done anything.

To make sure I have scsi2logical enabled I have changed the relevant
option in st_option.h and recompiled the kernel. But QFA still doesn't
work.

I have 3 identical Quantum DLT-IV drives, have tried 3 different models
of SATA controller (Intel ICH 7, Sil3112 and Sil3124) and have sent one
drive back, so I am confident hardware is not a problem. 

If somebody could tell me how to work around the problem whilst still
using QFA (I want to be able to restore single files quickly) I would
appreciate it.

Thanks,

John.

Re: [Dump-users] error in EA block 1, magic = 0

[Stelian P. <st...@po...>]

Le dimanche 20 janvier 2008 à 16:18 -0800, Kenneth Porter a écrit :
> What does this message mean? I'm seeing it a few times during "restore -C".

This message means that restore found an 'extended attributes block' on
the tape, but the block doesn't contain valid EA information.

I'm not sure if this is a problem in dump or in restore, it would be
nice if you could try and see if the problem can be reproduced with
0.4b41 (both dump and restore), or even give a try to the CVS version.

Stelian. 
-- 
Stelian Pop <st...@po...>

Re: [Dump-users] mode file too large?

[Stelian P. <st...@po...>]

Le dimanche 20 janvier 2008 à 16:20 -0800, Kenneth Porter a écrit :
> --On Friday, January 04, 2008 2:15 PM +0100 Stelian Pop 
> <st...@po...> wrote:
> 
> > If you modify restore/dirs.c and replace the two occurences of:
> > 	mf = fopen(modefile, "r");
> > with:
> > 	mf = FOPEN(modefile, "r");
> > the problem should be hopefully fixed.
> 
> I just want to confirm that this fixed the problem.

Thanks, I'll commit the fix into CVS.

-- 
Stelian Pop <st...@po...>

Re: [Dump-users] mode file too large?

[Kenneth P. <sh...@se...>]

--On Friday, January 04, 2008 2:15 PM +0100 Stelian Pop 
<st...@po...> wrote:

> If you modify restore/dirs.c and replace the two occurences of:
> 	mf = fopen(modefile, "r");
> with:
> 	mf = FOPEN(modefile, "r");
> the problem should be hopefully fixed.

I just want to confirm that this fixed the problem.

[Dump-users] error in EA block 1, magic = 0

[Kenneth P. <sh...@se...>]

What does this message mean? I'm seeing it a few times during "restore -C".

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Stelian P. <st...@po...>]

Le samedi 05 janvier 2008 à 10:42 -0800, Kenneth Porter a écrit :
> On Saturday, January 05, 2008 3:33 PM +0100 Stelian Pop 
> <st...@po...> wrote:
> 
> > It's composed of a map of inodes on the tape, then a sequential stream
> > of all the directories on the tape, then a sequential stream of all the
> > other type of files on the tape.
> 
> So given a dump file on disk, the inode map, and a fast seek mechanism (ie. 
> a QFA file), one should be able to quickly navigate the metadata.

Yes.

-- 
Stelian Pop <st...@po...>

Re: [Dump-users] Help with -A option

[Stelian P. <st...@po...>]

Le samedi 05 janvier 2008 à 11:56 -0800, dvg a écrit :
> 
> I decided to compile dump 0.4b41 from source. The restore
> command would not work on an archive_file created with the
> old dump. However, when I used 0.4b41 dump to perform the
> backup then all worked well! There must be a bug in the
> dump command shipped by Red Hat. Is this a known bug in the
> dump source code? If so, I'd like to get Red Hat to ship the
> fix.

I suspect you're encountering this one:

Changes between versions 0.4b40 and 0.4b41 (released January 2, 2006)
=====================================================================
[...]
5.      Fixed dump to not include extended attributes information
        in the toc (archive) file which confused restore -t.

This bug only affects restore -t, other modes should work file.

You should also not have problems if you dump a filesystem without
extended attributes (selinux labels probably and/or ACLs).

Can you confirm this ?

-- 
Stelian Pop <st...@po...>

Re: [Dump-users] Help with -A option

[Kenneth P. <sh...@se...>]

On Saturday, January 05, 2008 11:56 AM -0800 dvg 
<dim...@jp...> wrote:

> Is this a known bug in the
> dump source code? If so, I'd like to get Red Hat to ship the
> fix.

You could grab the source RPM from RHEL5.1 and rebuild it ("rpmbuild 
--rebuild foo.src.rpm") and install the new binary RPM.

(I'm using CentOS 5.1 and except for an issue I'm working from yesterday, 
it's been working fine. My issue doesn't appear to be fatal, and as soon as 
I finish recovering a corrupted Windows server from yesterday's wind storm 
and power outage, I'll go back to confirming that Stelian's fix worked.)

I don't see the bug your describing listed here:

<https://bugzilla.redhat.com/buglist.cgi?component=dump>

Wait for Stelian to comment, and if it's indeed a bug with that version, 
open a new RH bug and report what you learn here, so they'll have a way to 
assign someone to cut a new RPM.

Re: [Dump-users] Help with -A option

[dvg <dim...@jp...>]

I decided to compile dump 0.4b41 from source. The restore
command would not work on an archive_file created with the
old dump. However, when I used 0.4b41 dump to perform the
backup then all worked well! There must be a bug in the
dump command shipped by Red Hat. Is this a known bug in the
dump source code? If so, I'd like to get Red Hat to ship the
fix.


Dimitri

--
Dimitrios Gerasimatos       dim...@jp...
Section 343                          Jet Propulsion Laboratory
4800 Oak Grove Dr.    Mail Stop 264-820,   Pasadena, CA  91109
Voice: 818.354.4910    FAX: 818.393.7413    Cell: 818.726.8617

[Dump-users] Help with -A option

[dvg <dim...@jp...>]

I am playing with the -A option for the first time and not
having much luck. This is on RHEL4U6 with dump-0.4b39-3.EL4.2
provided by Red Hat.


I created a test dump file with:

dump -0 -A contents -f dump.file /share

(/share is the /dev/sdc2 partition on this system and
README.txt is one of the existing files)


Now I want to use the "contents" file to restore:


restore -t -A contents -f dump.file  README.txt
Dump   date: Sat Jan  5 11:09:46 2008
Dumped from: the epoch
Level 0 dump of /share on host.jpl.nasa.gov:/dev/sdc2
Label: none
resync restore, skipped 1 blocks
resync restore, skipped 1 blocks
Segmentation fault


I thought maybe I don't need -f so I tried it that way, too:

restore -t -A contents  README.txt
Dump   date: Sat Jan  5 11:09:46 2008
Dumped from: the epoch
Level 0 dump of /share on host.jpl.nasa.gov:/dev/sdc2
Label: none
resync restore, skipped 1 blocks
resync restore, skipped 1 blocks
Segmentation fault


What is wrong with my syntax here?


Dimitri

--
Dimitrios Gerasimatos       dim...@jp...
Section 343                          Jet Propulsion Laboratory
4800 Oak Grove Dr.    Mail Stop 264-820,   Pasadena, CA  91109
Voice: 818.354.4910    FAX: 818.393.7413    Cell: 818.726.8617

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Kenneth P. <sh...@se...>]

On Saturday, January 05, 2008 3:33 PM +0100 Stelian Pop 
<st...@po...> wrote:

> It's composed of a map of inodes on the tape, then a sequential stream
> of all the directories on the tape, then a sequential stream of all the
> other type of files on the tape.

So given a dump file on disk, the inode map, and a fast seek mechanism (ie. 
a QFA file), one should be able to quickly navigate the metadata.

Re: [Dump-users] Definition of dump levels?

[Scott E. <sc...@MI...>]

On Sat, 5 Jan 2008, Stelian Pop wrote:

>
> Le samedi 05 janvier 2008 =E0 08:23 -0500, Scott Ehrlich a =E9crit :
>> I've seen so many references to dump levels, but none of them, including
>> the man page, actually says what, specifically, each level covers.
>
> Well, the man page says what each level covers pretty clearly:
>
>   A level 0, full backup,  specified
>   by  -0 guarantees the entire file system is copied
>   ...
>   A  level  number  above  0,  incremental
>   backup,  tells  dump to copy all files new or modified since the
>   last dump of a lower level.
>
>> For example, a level 0 would presumably back up everything.
>
> Yes.
>
>>  But will it
>> still do so if I perform a level 0 today, update /etc/dumpdates, then
>> perform another level 0 just after, and no files have changed?
>
> Yes.
>
>>
>> What about the other levels?  What do they do?
>
> level 0 =3D always full dump
> level 1 =3D changed files since the last level 0
> level 2 =3D changed files since the last level 0 _or_ 1
> etc...
>
> /etc/dumpdates is here just to record when the last level 0, last level
> 1 etc were performed. If this file is missing (or not updated), dump
> acts as if the lowel-level dump was never performed. This means that a
> 'dump -5' for example will do a full dump is you haven't done a level 0,
> 1, 2, 3 or 4 at some time before.

I guess what is/was confusing me was the tower example in the man page.=20
It made me think there was something special to the levels beyond just=20
making differentials.  From another email I received, I learned that a=20
level 3 cannot occur until a level 2 exists, and a level 2 cannot occur=20
until a level 1 exists, etc.  The tower example in the man page, at least=
=20
to me, made me think a level 3 could occur now, just after a level 0, then=
=20
a level 2, then...

Scott

>
> Stelian.
> --=20
> Stelian Pop <st...@po...>
>
>

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Stelian P. <st...@po...>]

Le vendredi 04 janvier 2008 à 18:26 +0000, Keith G. Robertson-Turner a
écrit :
> Verily I say unto thee, that Stelian Pop spake thusly:
> > On Fri, Jan 04, 2008 at 06:49:06AM -0800, Kenneth Porter wrote:
> 
> >> This is one of those use cases where being able to mount the dump 
> >> like an ISO9660 filesystem would be handy. ;)
> > 
> > I'm not sure the sequential nature of a dump is compatible with some
> > sort of loop mount... Especially if the dump is located on a real 
> > tape :)
> 
> That'd make an interesting side project.
> 
> What does the layout of a dump file look like, semantically?

It's composed of a map of inodes on the tape, then a sequential stream
of all the directories on the tape, then a sequential stream of all the
other type of files on the tape.

This means that if you want the directory structure you only have to
extract the directory part. But unfortunately there isn't much more
information than the inode number and the pathname here, if you want the
size, the permission bits etc you'll have to seek forward until you
encounter the real inode (meta)data.

> Wouldn't it be possible to create a virtual filesystem based on the dump
> header, which subsequently calls "restore" when actually working with
> files? 

Not a full filesystem, see above why. For a simple (names only) view,
sure, this is what restore -i do.

> These would then be temporarily extracted to /tmp ... somewhat
> like file-roller or similar?
> 
> That way, the actual data would not need to be extracted first, it would
> be a much faster operation, use less memory and disk space, and
> (assuming the inode info can be retrieved without fully restoring the
> entire dump) this would work just as well with tape as a dump file.

No, you'll have to either generate 'false' information about the inodes
or analyse the whole dump to extract the real information.

> I'm thinking along the lines of integration with gnome-vfs or similar.

> Something to think about in the long term, anyway.

Stelian.

-- 
Stelian Pop <st...@po...>

Re: [Dump-users] Definition of dump levels?

[Stelian P. <st...@po...>]

Le samedi 05 janvier 2008 à 08:23 -0500, Scott Ehrlich a écrit :
> I've seen so many references to dump levels, but none of them, including 
> the man page, actually says what, specifically, each level covers.

Well, the man page says what each level covers pretty clearly:

   A level 0, full backup,  specified
   by  -0 guarantees the entire file system is copied 
   ...
   A  level  number  above  0,  incremental
   backup,  tells  dump to copy all files new or modified since the
   last dump of a lower level.

> For example, a level 0 would presumably back up everything. 

Yes.

>  But will it 
> still do so if I perform a level 0 today, update /etc/dumpdates, then 
> perform another level 0 just after, and no files have changed?

Yes.

> 
> What about the other levels?  What do they do?

level 0 = always full dump
level 1 = changed files since the last level 0
level 2 = changed files since the last level 0 _or_ 1
etc...

/etc/dumpdates is here just to record when the last level 0, last level
1 etc were performed. If this file is missing (or not updated), dump
acts as if the lowel-level dump was never performed. This means that a
'dump -5' for example will do a full dump is you haven't done a level 0,
1, 2, 3 or 4 at some time before.

Stelian.
-- 
Stelian Pop <st...@po...>

[Dump-users] Definition of dump levels?

[Scott E. <sc...@MI...>]

I've seen so many references to dump levels, but none of them, including 
the man page, actually says what, specifically, each level covers.

For example, a level 0 would presumably back up everything.  But will it 
still do so if I perform a level 0 today, update /etc/dumpdates, then 
perform another level 0 just after, and no files have changed?

What about the other levels?  What do they do?

Thanks.

Scott

[Dump-users] Dump levels?

[Scott E. <sc...@MI...>]

I created a cron job to invoke a dump script according to Tower Of Hanoi. I am 
dumping a subdirectory and a filesystem.

The backup script, with some other lines removed:
/path/to/dump -0 -fv /dev/nst0 /var/log
/path/to-dump -3 -fv /dev/nst0 /home

When viewing the dump logs, it looks like it is taking the whole job as a level 
0.   Am missing something?

Thanks.

Scott

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Kenneth P. <sh...@se...>]

--On Friday, January 04, 2008 6:26 PM +0000 "Keith G. Robertson-Turner" 
<dum...@ge...> wrote:

> I'm thinking along the lines of integration with gnome-vfs or similar.

FUSE is probably the best thing to layer it on. Check out all the 
filesystems implemented on top of FUSE:

<http://fuse.sourceforge.net/wiki/index.php/FileSystems>

<http://fuse.sourceforge.net/>

Re: [Dump-users] limiting io-rate

[Peter <pm...@fr...>]

On Sat, Dec 22, 2007 at 10:41:41AM -0500, Tony Nelson wrote:
> >
> >Is it possible to limit the io-rate of dump?
> 
> Perhaps ionice, using idle priority?
> 
>     # ionice -c3 dump -dumpoptions

Hello Tony, and happy new year!
I've tested this today and it works great!
Thank you very much, Peter

-- 
http://pmrb.free.fr/contact/

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Keith G. Robertson-T. <dum...@ge...>]

Verily I say unto thee, that Stelian Pop spake thusly:
> On Fri, Jan 04, 2008 at 06:49:06AM -0800, Kenneth Porter wrote:

>> This is one of those use cases where being able to mount the dump 
>> like an ISO9660 filesystem would be handy. ;)
> 
> I'm not sure the sequential nature of a dump is compatible with some
> sort of loop mount... Especially if the dump is located on a real 
> tape :)

That'd make an interesting side project.

What does the layout of a dump file look like, semantically?

Wouldn't it be possible to create a virtual filesystem based on the dump
header, which subsequently calls "restore" when actually working with
files? These would then be temporarily extracted to /tmp ... somewhat
like file-roller or similar?

That way, the actual data would not need to be extracted first, it would
be a much faster operation, use less memory and disk space, and
(assuming the inode info can be retrieved without fully restoring the
entire dump) this would work just as well with tape as a dump file.

I'm thinking along the lines of integration with gnome-vfs or similar.

Something to think about in the long term, anyway.

-- 
Regards,
Keith G. Robertson-Turner

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Kenneth P. <sh...@se...>]

--On Friday, January 04, 2008 3:58 PM +0100 Stelian Pop 
<st...@po...> wrote:

> I'm not sure the sequential nature of a dump is compatible with some
> sort of loop mount... Especially if the dump is located on a real tape :)

It would probably require either a disk-based dump image or a QFA file. I'd 
probably just require the QFA file.

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Stelian P. <st...@po...>]

On Fri, Jan 04, 2008 at 06:49:06AM -0800, Kenneth Porter wrote:

> --On Friday, January 04, 2008 3:12 PM +0100 Stelian Pop 
> <st...@po...> wrote:
> 
> > In order to print such information the restore code should scan the entire
> > dump just like when it does a 'restore -C' for example, and instead of
> > comparing the file just print out the information. It shouldn't be very
> > hard to implement if you feel like doing it :)
> 
> Ideally it would list everything in the inode.

Yup.

> 
> This is one of those use cases where being able to mount the dump like an 
> ISO9660 filesystem would be handy. ;)

I'm not sure the sequential nature of a dump is compatible with some
sort of loop mount... Especially if the dump is located on a real tape :)

Stelian.
-- 
Stelian Pop <st...@po...>

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Kenneth P. <sh...@se...>]

--On Friday, January 04, 2008 3:12 PM +0100 Stelian Pop 
<st...@po...> wrote:

> In order to print such information the restore code should scan the entire
> dump just like when it does a 'restore -C' for example, and instead of
> comparing the file just print out the information. It shouldn't be very
> hard to implement if you feel like doing it :)

Ideally it would list everything in the inode.

This is one of those use cases where being able to mount the dump like an 
ISO9660 filesystem would be handy. ;) Might make a good Google Summer of 
Code project.

Re: [Dump-users] Listing the content of a dump file without writing any data?

[Stelian P. <st...@po...>]

On Fri, Jan 04, 2008 at 11:50:44AM +0100, Ernest Beinrohr wrote:

> Hi, I would like to index all my dumps and would therefore like to read 
> the contents of a dump file somehow. I can extract it, I know. But my 
> dumps are quite big 500gb so writimg whem somewhere might be a problem.
> 
> I have found that "restore tvf dumpfile.dmp" gives me a file listing, 
> but it is not sufficent, as there ano NO file sizes and no dates also. 
> How could I get this information from the dump?

Unfortunately 'restore -t' reads a table of contents which is located at
the beginning of the dump, and this table does not contain size and 
date information.

In order to print such information the restore code should scan the entire
dump just like when it does a 'restore -C' for example, and instead of
comparing the file just print out the information. It shouldn't be very
hard to implement if you feel like doing it :)

Stelian.
-- 
Stelian Pop <st...@po...>

Re: [Dump-users] mode file too large?

[Kenneth P. <sh...@se...>]

--On Friday, January 04, 2008 2:15 PM +0100 Stelian Pop 
<st...@po...> wrote:

> Yes. This error is probably caused by the fact that your
> dump is huge and the mode file has exceeded 2 GB in size.
>
> As the error says, since restore cannot open the modefile,
> it will not be able to set the mode, owner and time information on
> any of the restored file.
>
> If you modify restore/dirs.c and replace the two occurences of:
> 	mf = fopen(modefile, "r");
> with:
> 	mf = FOPEN(modefile, "r");
> the problem should be hopefully fixed.

Thankfully the issue is in restore, not dump, so at least my existing 
backups are likely ok. I'll make the change....