DIBS: Distributed Internet Backup System / Bugs / #10 unable to recover file

Tomas Pospisek - 2004-09-14

Logged In: YES
user_id=15179

Emin wrote:

> Look in ~/.dibs/recovery.

$ ls -a /home/tpo/.dibs/recovery/tmp/
. ..
$

unfortunately that's empty as well

> You would probably be annoyed if Dibs overwrote something
else in
> doing a backup.

I guess different backup systems take different approaches.
The ones I know will:

if (allready exists target file) {
echo "the user that there is allready a file and that he
needs to".
"move it away in order to restore the stored
version"
} else {
restore_file():
}

But anyway, whatever aproach you choose, I suggest to
discribe the behaveour where the restore_file and the
restore_all commands are documented.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Emin Martinian - 2004-09-15

Logged In: YES
user_id=665772

Make sure DIBS has processed all incoming messages. For
example, issue the poll_passives command (with no arguments)
to get messages from peers who are communicate with you in
passie mode, then issue the process_message command (with no
arguments).

If you are running the daemon on your local machine (the
place you are issuing the recover_file command) and the
remote machine can connect to you in active mode, this
should happen automatically. Even if the remote machine
communictes with your machine in passive mode, things should
happen automatically once the daemon does its periodic
poll_passives, process_message routine.

Let me know if this works.

In any case, you are correct that there is a bug in the
documentation regarding the recover_file command and where
the recovered data should go. A better description of
recovery would also be useful. Therefore, I'll accept this
as a documentation bug for now pending your response.

Thank you very much for the useful feedback,
-Emin

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Emin Martinian - 2004-09-15

assigned_to: nobody --> emin63

status: open --> open-accepted
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Tomas Pospisek - 2004-09-15

Logged In: YES
user_id=15179

> Make sure DIBS has processed all incoming messages. For
> example, issue the poll_passives command (with no arguments)
> to get messages from peers who are communicate with you in
> passie mode, then issue the process_message command (with no
> arguments).

I issue localy:

$ HOST=localhost dibs.py poll_passives
opening log /home/tpo/.dibs/logfile at level 0
Connecting to localhost:9999
Doing TransmitGet:GET tpo.dibs@localhost
GET finished after receiving 0 files.
closing log

The remote peer daemon says meanwhile:

$ dibs.py start_daemon
opening log /var/lib/dibs/.dibs/daemonLog at level 0

Connected by ('127.0.0.1', 49375)
Handling incoming files:
opening log /var/lib/dibs/.dibs/logfile at level 0
No messages to process.
closing log
Connection from ('127.0.0.1', 49375) closed.

> If you are running the daemon on your local machine (the
> place you are issuing the recover_file command) and the
> remote machine can connect to you in active mode, this
> should happen automatically.

No the remote machine is not allowed to connect (I'm behind
a firewall).

Still, allthough the directory structure is there under
~/.dibs/recovery, the file itself has not been restored.
*t

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Emin Martinian - 2004-09-15

Logged In: YES
user_id=665772

Did you issue the "process_message" command? It looks like
you only did poll_passives and my hunch is one or both of
your machines have unprocessed messages hanging around.

You could also try checking in the .dibs/incoming and
.dibs/outgoing messages of the two machines.

-Emin

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Tomas Pospisek - 2004-09-15

Logged In: YES
user_id=15179

> Did you issue the "process_message" command?

No I forgot, but here it is again:

$ HOST=localhost dibs.py poll_passives
opening log /home/tpo/.dibs/logfile at level 0
Connecting to localhost:9999
Doing TransmitGet:GET tpo.dibs@localhost
GET finished after receiving 0 files.
closing log

$ HOST=localhost dibs.py process_message
opening log /home/tpo/.dibs/logfile at level 0
No messages to process.
closing log

And remote:

$ HOST=example.org dibs.py start_daemon
opening log /var/lib/dibs/.dibs/daemonLog at level 0
Connected by ('127.0.0.1', 49645)
Handling incoming files:
opening log /var/lib/dibs/.dibs/logfile at level 0
No messages to process.
closing log
Connection from ('127.0.0.1', 49645) closed.

Still nothing in ~/.dibs/recovery/

> It looks like
> you only did poll_passives and my hunch is one or both of
> your machines have unprocessed messages hanging around.
>
> You could also try checking in the .dibs/incoming and
> .dibs/outgoing messages of the two machines.

Local:

$ ls -aR /home/tpo/.dibs/incoming
/home/tpo/.dibs/incoming:
. ..
$ ls -aR /home/tpo/.dibs/outgoing/
/home/tpo/.dibs/outgoing/:
. .. 749404A9

/home/tpo/.dibs/outgoing/749404A9:
. ..

Remote:

# ls -aR .dibs/incoming/
.dibs/incoming/:
. ..
# ls -aR .dibs/outgoing/
.dibs/outgoing/:
. .. tpo.dibs@wal.dyn.sourcepole.ch

.dibs/outgoing/tpo.dibs@wal.dyn.sourcepole.ch:
.
1095168057593.0BijJ+uzNhlxFRqEtPSq8Ug
1095168317484.0SK6kpOKTIPROSi+tYkSQ8Q
..
1095168203921.0tIQATlSLAxUttCDv+IZr4w
1095168318200.0GhMWmq+noMLaKJPsRLQUEw
1095167843881.0EifZn5hl02lHE12dx8aVAg
1095168204698.0vuO1F8djjFE6K+n6OZfHEw
1095168456834.0tMTwOjRf1tZQrNsKnO9NSA
1095167846174.0QSzdgnkNFAeX4ZEO2T3JBw
1095168205503.0CqWQr+xqLcuMjjJMMrQLvA
1095168457636.0SPtd7UuNSypRm0xKT+XMeg
1095168055280.0UI32TLdNcyZbwsnnDEDGng
1095168316704.0K8A1BJUmV7RC6r7zS+zlJw
1095168458351.0ZfDaRztHhLotEIWh92KDrQ

So yes, there seems to be a whole lot of stuff there. If I
issue again:

$ HOST=localhost dibs.py recover_file --file /tmp/test
opening log /home/tpo/.dibs/logfile at level 0
Connecting to localhost:9999
Connecting to localhost:9999
Connecting to localhost:9999

WARNING: Re-attempting to recover file /tmp/test.

Starting recovery of file /tmp/test.
closing log

And remotely I see:

Connected by ('127.0.0.1', 49657)
GET finished after receiving 1 files.
Handling incoming files:
opening log /var/lib/dibs/.dibs/logfile at level 0
Processed msg in file
/var/lib/dibs/.dibs/incoming/1095280488220.0u7BG9NMf21gwjCKmp2JMLQ.
closing log
Connection from ('127.0.0.1', 49657) closed.
Connected by ('127.0.0.1', 49658)
GET finished after receiving 1 files.
Handling incoming files:
opening log /var/lib/dibs/.dibs/logfile at level 0
Processed msg in file
/var/lib/dibs/.dibs/incoming/1095280488801.0vVhJyNNLG0ueuaJGHimj+g.
closing log
Connection from ('127.0.0.1', 49658) closed.
Connected by ('127.0.0.1', 49659)
GET finished after receiving 1 files.
Handling incoming files:
opening log /var/lib/dibs/.dibs/logfile at level 0
Processed msg in file
/var/lib/dibs/.dibs/incoming/1095280489461.0ijdGUnS+GjswwCPrjG+jAg.
closing log
Connection from ('127.0.0.1', 49659) closed.

And after that 3 new files appear on the remote peer,
allthough only in the outgoing folder and not with the
previous name:

# ls -l .dibs/outgoing/tpo.dibs\@host.example.org/
total 828
-rw-r--r-- 1 dibs nogroup 1531 15. Sep 22:34
1095280488670.0kI1cGYUicm4ChISZd5KAmQ
-rw-r--r-- 1 dibs nogroup 1538 15. Sep 22:34
1095280489258.0eXWtC6AJpiILmsN8RTjRwg
-rw-r--r-- 1 dibs nogroup 1531 15. Sep 22:34
1095280489917.06KoZvmv7kz+7prbbGMbuyQ

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Emin Martinian - 2004-09-16

Logged In: YES
user_id=665772

The new messages that appeared on the remote machine are the
pieces of the file you are trying to recover. You need to
get these to the local machine and process them.

In general, to recover a file with the communication setup
you have, you would issue the following commands

recover_file (asks peer for recovery; peer creates new
messages)
poll_passives (gets new messages from peer)
process_message (processes the new messages from the peer)

Since you have already issued the recover_file command you
should be able to just do poll_passives and process_message.
I'm not sure what went wrong when you issued these commands
before. I did notice that you seem to be using "749404A9"
as the peer name for one of your peers. The peer name used
in the add_peer command *must* be the name of the GPG key
for that peer (e.g., something like foo.dibs@example.org).
You might consider setting things up all over again.

-Emin

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Tomas Pospisek - 2004-09-20

Logged In: YES
user_id=15179

OK, as you suggested, I've reinstalled everything. And it
does work now:-) !!!

I guess the problem was - again - the "gpg-ids" that DIBS is
using. I'll expand on that in another feature request.

Thanks a lot so far,
*t

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

unable to recover file

Group

Searches

Help

#10 unable to recover file

Discussion