Menu
▾
▴
#7 Interpreting .aria2 file
Hi,
I would like to interpret the .aria2 file from php. Using fopen with rb & fread command, i was able to read the header details as mentioned on the home page. https://sourceforge.net/apps/trac/aria2/wiki/ControlFileFormat . But is there is any way to interpret the completed size from bitfield?
Each bit of bitfield corresponds to 1 piece and if nth bit is set, nth piece has been acquired.
To calculate the completed length, calculate the number of set bit in bitfield and multiply it with piece length.
Since total length is not multiples of piece length, the length of last piece is not always equal to piece length.
The last piece length is calculated by total length - piece length * (number of pieces -1).
Here is the pseudo code to calculate completed length(assume that isBitSet(b,i) function returns true if
ith bit of b is set otherwise returns false):
completedLength = 0;
for(int i = 0; i < numberOfPieces-1; ++i)
if(isBitSet(bitfield, i)) completedLength += pieceLength;
if(isBitSet(bitfield, numberOfPieces-1))
completedLength += totalLength - pieceLength * (numberOfPieces -1)
Hi, Thanks for the response.
I have tried to implement in php and I can't get the completed download. as you have mentioned bitfield & bit, the values for the above we need to take it from bitfield_length & bitfield or after NUM_IN_FLIGHT_PIECE (no. of piece downloads) n times? Here is my php code.
$filename = "test.aria2";
$handle = fopen($filename, "rb");
$version = fread($handle, 2);
$extension = fread($handle, 4);
$infoHashLength = hexdec(bin2hex(fread($handle, 4)));
if ($infoHashLength > 0)
$infoHash = hexdec(bin2hex(fread($handle, $infoHashLength)));
$pieceLength=hexdec(bin2hex(fread($handle, 4)));
echo "pieceLength=".$pieceLength;
$totalLength=hexdec(bin2hex(fread($handle, 8)));
echo "totalLength=".$totalLength;
$uploadLength = hexdec(bin2hex(fread($handle, 8)));
$bitfieldLength=hexdec(bin2hex(fread($handle, 4)));
echo "bitFieldLength=".$bitfieldLength;
for($i=0;$i<$bitfieldLength;$i++)
$bitfieldData[$i] = hexdec(bin2hex(fread($handle, 1)));
$numberOfPieces=hexdec(bin2hex(fread($handle, 4)));
echo "numberOfPieces=".$numberOfPieces;
$completedLength = 0;
for($i=0; $i < $numberOfPieces-1; ++$i)
if(isBitSet($bitfieldData, $i)) $completedLength += $pieceLength;
if(isBitSet($bitfieldData, $numberOfPieces-1))
$completedLength += $totalLength - $pieceLength * ($numberOfPieces -1);
echo "completedLength=".$completedLength;
function isBitSet($b,$i) {
if ($b[$i])
return true;
else
return false;
}
numberOfPiece is not NUM_IN_FLIGHT_PIECE.
numberOfPIece can be calculated using totalLength and pieceLength:
numberOfPiece = (totalLength+pieceLength-1)/pieceLength
.aria2 file uses bitfield in 2 levels: entire file level and
piece level.
The pseudo code I wrote before is for first one. This bitfield
records completely acquired pieces. Now that you have
numberOfPiece above, you can calculate completed length for fully
acquired pieces.
I didn't mention piece level bitfield in the previous post. This
bitfield records the download progress within a piece. The chunk
size(this corresponds to pieceLength to entire file level
btifield) is 16KiB. Last bit of last piece has probably less
chunk size than 16KiB. You can calculate bytes downloaded in
each in-flight piece in the same way. The real completed length
is the sum of completed length of in-flight pieces plus completed
length calcuated using entire file level bitfield.
I updated wiki page:
http://sourceforge.net/apps/trac/aria2/wiki/ControlFileFormat
because INDEX of in-flight piece is missing.
I don't know php much, but the code looks like it doesn't test
whether bit is set properly.
function isBitSet($b,$i) {
if ($b[$i])
return true;
else
return false;
}
I think $b[$i] returns $i-th bytes not $i-th bit.
To calculate i-th bit in bitfield,
mask = 128 >> (i%8)
b = (bitfield[i/8]&mask) != 0
If i-th bit is set, b is true.
Thanks,
I can able to get the downloaded size, but only from the main bitfield and not from num_in_flight_piece, here is my code.
$completedLength = 0;
for($i=0;$i<$numInFlightPiece;$i++){
$index_p=hexdec(bin2hex(fread($handle, 4)));
echo "Index=".$index_p;
$Length_p=hexdec(bin2hex(fread($handle, 4)));
echo "Length=".$Length_p;
$bitfieldLength_p=hexdec(bin2hex(fread($handle, 4)));
echo "bitFieldLength=".$bitfieldLength_p;
for($j=0;$j<$bitfieldLength_p;$j++)
$bitfieldData_p[$j] = hexdec(bin2hex(fread($handle, 1)));
for($k=0; $k < $numInFlightPiece; ++$k)
if(isBitSet($bitfieldData_p, $k)) $completedLength += $Length_p;
}
for($i=0; $i < $numberOfPieces-1; ++$i)
if(isBitSet($bitfieldData, $i)) $completedLength += $pieceLength;
if(isBitSet($bitfieldData, $numberOfPieces-1))
$completedLength += $totalLength - $pieceLength * ($numberOfPieces -1);
echo "completedLength=".$completedLength;
You need to calculate the number of 16KiB chunk in a piece.
$chunkLength = 16*1024;
$completedLength = 0;
for($i=0;$i<$numInFlightPiece;$i++){
$index_p=hexdec(bin2hex(fread($handle, 4)));
echo "Index=".$index_p;
$Length_p=hexdec(bin2hex(fread($handle, 4)));
echo "Length=".$Length_p;
$bitfieldLength_p=hexdec(bin2hex(fread($handle, 4)));
echo "bitFieldLength=".$bitfieldLength_p;
for($j=0;$j<$bitfieldLength_p;$j++)
$bitfieldData_p[$j] = hexdec(bin2hex(fread($handle, 1)));
$numChunk=($Length_p - $chunkLength - 1)/$chunkLength;
for($k=0; $k < $numChunk-1; ++$k)
if(isBitSet($bitfieldData_p, $k)) $completedLength += $chunkLength;
if(isBitSet($bitfieldData_p, $numChunk-1))
$completedLength += $Length_p - $chunkLength * ($numChunk-1);
}
for($i=0; $i < $numberOfPieces-1; ++$i)
if(isBitSet($bitfieldData, $i)) $completedLength += $pieceLength;
if(isBitSet($bitfieldData, $numberOfPieces-1))
$completedLength += $totalLength - $pieceLength * ($numberOfPieces-1);
echo "completedLength=".$completedLength;
That was really helpful, thanks for the info. By the way, I was trying to interpret the .aria2 from commandline using a C program, do you have any idea how to read the header with C?. I was trying with fread, but I was not able to read 8bytes header.
DefaultBtProgressInfoFile.cc reads/writes .aria2 file. It is written in C++ not pure C, but I think it tells how to read header in C style.
You mean 8byte integer field?
You can use "long long int" type. Since all integers in .aria2 file are written in network byte order, multi byte integers should be converted using ntoh function. There are no ntoh family for 8byte integers, so we use following functions in aria2.
ifdef WORDS_BIGENDIAN
inline uint64_t ntoh64(uint64_t x) { return x; }
inline uint64_t hton64(uint64_t x) { return x; }
else // !WORDS_BIGENDIAN
inline uint64_t byteswap64(uint64_t x) {
uint64_t v1 = ntohl(x & 0x00000000ffffffff);
uint64_t v2 = ntohl(x >> 32);
return (v1 << 32)|v2;
}
inline uint64_t ntoh64(uint64_t x) { return byteswap64(x); }
inline uint64_t hton64(uint64_t x) { return byteswap64(x); }
endif // !WORDS_BIGENDIAN