Menu
▾
▴
#23 Serious data loss problem while downloading.
First of all: when crashing, 1.1.5 used to leave my system in a kernel-panic-like status, and corrupted my .met files; 1.1.8, however, peacefully segfaults and doesn't corrupt them.
The bug I'm submitting, though, is about data loss in downloads.
I'll start with an example of a download I have before my eyes:
- Size: 28.15 MB
- Transferred: 32.40 MB
- Completed: 17.90 MB
- Lost to corruption: 3.59 MB
- ICH: 0
- Gained by compression: 238 KB
As you can see, even escluding detected corruption (which is high anyway), there's still a good 12 MB that just vanished. This happens (although not in such scary % proportions) with all my lmule downloads. In the last 2 years, I've used edonkey and emule under Windows, mldonkey and emule with Wine under Linux, and I never had a similar problem - so I guess I can't blame my OS or my net connection.
The bug is easily reproducable (at least on my computer) by simply downloading anything. I'll be glad to provide any additional info if needed.
Logged In: YES
user_id=5462
I've encountered into this same problem once myself too, but
I couldn't figure out what caused it
Logged In: YES
user_id=5462
I've encountered into this same problem once myself too, but
I couldn't figure out what caused it
Logged In: YES
user_id=729045
Well, I still experience it (on almost all my downloads) in 1.1.8. Not a big issue, since files complete correctly anyway, but just curious. By the way, is ICH implemented?
Logged In: YES
user_id=5462
Yup, ICH should be there.
Logged In: NO
doesn't only happen to lmule but mldonkey and emule under windows, too. seems like spammer clients uploading zero data to gain credits or something...
Logged In: YES
user_id=627910
This can be caused by some organitations (in the USA I
think) that are introducing "intentionally" corrupted parts in
the P2P networks. The solution for this is to use the
ipfilter.dat (at least in Windows version of eMule it works, I
don't know if it's implemented in lMule too). You can
download an updated ipfilter.dat from
http://cvs.suche.org/horde/chora/cvs.php/ip.prefix?
sbt=1&login=2&Horde=1cbfdc482329c48f3d8cd420aff8dc02
Logged In: YES
user_id=729045
I know about the intentional corruption, but I don't think this is the case. I was experiencing the problem on a wide variety of files, many of which are definitely not likely to be in those organisations' interests. Things got better lately, I see much less data loss and when it's there, it's almost always reported in the "Lost to corruption" details entry.
By the way, ipfilter.dat is not implemented in lmule, AFAIK.
Logged In: YES
user_id=729045
Again on the topic, using the latest lmule version. It seems that the data loss occurs when a file is downloaded from various sources simultaneously. Although this is told by observation only, I'm getting pretty sure, and I have evident examples before my eyes. I'm really sorry that my coding skills don't allow me to try and submit a patch or something, but I think the problem could be the following:
when you download data of a certain chunk from multiple sources, sometimes you get the same data block more than once, so one "copy", so to say, is redundant and dumped. Maybe this could be solved by defining internal offsets in chunks, a bit like download managers do with http. Or maybe I'm completely wrong, but I think it would be worth a check.
Logged In: YES
user_id=729045
Again on the topic, using the latest lmule version. It seems that the data loss occurs when a file is downloaded from various sources simultaneously. Although this is told by observation only, I'm getting pretty sure, and I have evident examples before my eyes. I'm really sorry that my coding skills don't allow me to try and submit a patch or something, but I think the problem could be the following:
when you download data of a certain chunk from multiple sources, sometimes you get the same data block more than once, so one "copy", so to say, is redundant and dumped. Maybe this could be solved by defining internal offsets in chunks, a bit like download managers do with http. Or maybe I'm completely wrong, but I think it would be worth a check.
Logged In: YES
user_id=23127
let me know if this still occurs in 1.4.0
Logged In: YES
user_id=729045
I'm installing 1.4.0 right now, I'll let you know what happens (and what doesn't) as soon as possible.
BTW, sorry for the double post here below... I noticed it just now.
Logged In: YES
user_id=729045
Sorry to say that the problem is still there (in 1.4.0). I just started a new download, I am getting the same chunk from two users and the "Completed" value is 2.08 Mb while "Transferred" is 3.06 Mb. The missing data are not stated under "Lost to corruption"; on the contrary, details say I gained 220 Kb by compression.
Another download just started now(two sources) and the exact same thing is happening.
By the way, am I the only one experiencing this?
Logged In: YES
user_id=23127
no just the only person griping about it :-)
are you positive this doesn't happen in emule? cuz it
happens to me there too.
Logged In: YES
user_id=23127
no just the only person griping about it :-)
are you positive this doesn't happen in emule? cuz it
happens to me there too.
Logged In: YES
user_id=23127
no just the only person griping about it :-)
are you positive this doesn't happen in emule? cuz it
happens to me there too.
Logged In: YES
user_id=729045
Oh, ok! And sorry for not replying earlier...
About emule... well, corruption happens - although less frequently than in xmule - (mind that I'm not using emule anymore since lmule was born), but I have never (in more than one year of usage) noticed the behavior I'm complaining about here :)
If my humbly suggested solution (in-chunk offsets) is too messy/boring/hard to code, a decent workaround could be to tell different sources to upload different chunks whenever it's possible. What do you think? Or is this already happening?