I have some questions about redundancy. In thi FAQ it says that to det N/2 redundandancy you need N chunks of size S/N-P where P is (N+1)/2. So the number of chunks depends on N and size on N-P.
In this program however the number of chunks seems to depend on 2 constants: kbPerFile and redundantPieces that dont seem to be connected to N or P and size of peices is a function of S and kbperFile. And how is redundancy to be computed if K (=Size/kbperFile) chages with file size and
redundantPieces doesnt. how do we do m-fold redundancy in this case? Can it be done with this algorithm without changes?
What am I missing?
Thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You are correct that the number of chunks depends on two constants: kbPerFile and redundantPieces.
The first of these, kbPerFile, is not intended to have anything to do with redundancy. It is just there to prevent DIBS from working with really long chunks.
The second paramter, redundantPieces, is the interesting parameter from the point of view of redundancy. Regardless of how many chunks a file is chopped into, DIBS adds this many redundant chunks. For example, if a file is chopped into K chunks based on the kbPerFile parameter, then the total number of chunks is N=K+redundantPieces. The idea is that you should set redundantPieces to be greater than the number of peers which you think might fail so that you will always be able to recieve at least K pieces and recover the file.
You asked "how do we do m-fold redudancy?". If by m-fold redudnancy you mean you want to be able to recover the file even if any m peers crash, then simply set redudnantPieces = m and don't worry about kbPerFile. If you mean something else by m-fold redudnancy and think it would be more useful than the current method of choosing redudnancy, please let me know.
-Emin
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have some questions about redundancy. In thi FAQ it says that to det N/2 redundandancy you need N chunks of size S/N-P where P is (N+1)/2. So the number of chunks depends on N and size on N-P.
In this program however the number of chunks seems to depend on 2 constants: kbPerFile and redundantPieces that dont seem to be connected to N or P and size of peices is a function of S and kbperFile. And how is redundancy to be computed if K (=Size/kbperFile) chages with file size and
redundantPieces doesnt. how do we do m-fold redundancy in this case? Can it be done with this algorithm without changes?
What am I missing?
Thanks
You are correct that the number of chunks depends on two constants: kbPerFile and redundantPieces.
The first of these, kbPerFile, is not intended to have anything to do with redundancy. It is just there to prevent DIBS from working with really long chunks.
The second paramter, redundantPieces, is the interesting parameter from the point of view of redundancy. Regardless of how many chunks a file is chopped into, DIBS adds this many redundant chunks. For example, if a file is chopped into K chunks based on the kbPerFile parameter, then the total number of chunks is N=K+redundantPieces. The idea is that you should set redundantPieces to be greater than the number of peers which you think might fail so that you will always be able to recieve at least K pieces and recover the file.
You asked "how do we do m-fold redudancy?". If by m-fold redudnancy you mean you want to be able to recover the file even if any m peers crash, then simply set redudnantPieces = m and don't worry about kbPerFile. If you mean something else by m-fold redudnancy and think it would be more useful than the current method of choosing redudnancy, please let me know.
-Emin