Hi,
I tried both the latest x64 and latest and nightly latest x86 builds.
Using windows 7 on NTFS drives... adding new torrents with big files even with preallocate 1 or preallocate 0 causes a huge slowdown once the torrent starts on a file.
I can see my drive loaded up, writing the whole file, and the client can't connect at times... but transmissiondaemon.exe is not frozen or nowhere near 10% cpu use on a quad core (25% being a full core).
I'm not sure if this is also a problem in the linux version, but it can be quite annoying. I'd think preallocate 1 or 0 would prevent this, but not so on big files that get allocated at once!
I also have an incomplete directory. I may try to use .part instead..
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ah thank you... and I was thinking to run transmission in a VM as a temp fix! lol
We just had a 6+ month old bug with a specific router ethernet switch get fixed by the openwrt developers... but it was one of the reporters that started the ball rolling by finding a temporary hacked that worked...
I wonder if transmission were multithreaded, would it be able to get around this issue?
Too bad there isn't a good client for deluge as this awesome transmission one!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I wonder if transmission were multithreaded, would it be able to get around this issue?
There have been many suggestions at the 1753 post and in the forums, but none have addressed the issue at its core since it is really in the lowest level of design for libtransmission. Part of the reason I created my forks of the project was to maintain a patch that allowed for temporary pieces https://trac.transmissionbt.com/ticket/532 This stops the allocation problem for any file that is deselected. In the official version even deselected files are created when a piece is downloaded that crosses a file boundary and this is especially troublesome with big files.
Last edit: cfp p2p 2014-09-15
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
With the streaming mode set to 4 (forced) this has resolved the issue for me. I've tested with both flash drives and hard drives and it works well. You can also set the streaming mode for individual torrents' files if you want to use default streaming mode 0 and then set particular file(s) to streaming mode 4 (forced). https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/57f64ca4#1466
When adding new torrents with big files there is no longer a huge slowdown once the torrent starts on a file when the streaming mode for that file is set to 4 (forced).
Please verify that this works for your system. Thanks.
Last edit: cfp p2p 2015-05-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
...make the file sparse. Mark the file as sparse with the FSCTL_SET_SPARSE control code, and immediately after setting the end of file, use the FSCTL_SET_ZERO_DATA control code to make the entire file sparse. This logically fills the file with zeroes without committing physical disk space. Anywhere you actually write gets converted from "sparse" to "real". This does open the possibility that a later write into the middle of the file will encounter a disk-full error, so it's not a "just do this and you won't have to worry about anything" solution, and depending on how randomly you convert the file from "sparse" to "real", the file may end up more fragmented than it would have been if you had "kept it real" the whole time.
The fragmentation would be dependent on what other applications also might be writing to the same disk at the same time. If you have a lot of simultaneous downloads that could affect fragmentation too. You could in these cases try a larger transmission cache.
But Windows OS has its own cache and so that takes precedence. I've used cache sizes of 4MB to 16MB but haven't been able to get any effective change over that. Don't go big or huge on cache, there's lots of examples on the regular transmission forums of problems when doing that.
references on cache:
Other news
I also tested Transmission-Qt with
"cache-size-mb": 48,
in settings.json (in fact the daemon was also using that), and it didn't seem
to make any difference.
Interesting on the cache. In deluge they have libtorrent settings that allow you to change the way it interacts with windows cache, example: to not use windows file caching.
I just started using the x64 version and no issues so far.
I just added a 12GB torrent and it loaded fine!
Only weirdness is testing my forwarded port fails, despite even disabling the firewall and using the old deluge port (with deluge shut down).
In testing my ports via browser, it shows the port as open (as compared to open|filtered for the unused ports that are forwarded).
If anyone has issues with the gui paths mappings, here is my setup:
X:\ is my mapped drive to the server D:\ where torrents are
D:/torrents=X:\torrents
d:/torrents=X:\torrents
Last edit: robnitro 2015-05-25
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Only issue happens when the drive is busy doing other things, like accessing a big file on a share on that drive. The transmission client freezes up similarly to how it did in the past, when creating new torrents. I haven't changed the I/O priority, so that's not the issue. It's how the transmission daemon handles file delays etc. I think I can live with it though, as the transmission remote gui is 2x better than deluge headless client.
(I like being able to open the folders directly from the gui and GTK of deluge is kind of buggy at times).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The libraries of cygwin 64bit might be better suited for
the 64bit environment, but I can't say myself of
any noticeable difference. Try both. If you've got
64bit machine I don't know of any reason not to use
transmission 64.
As far as the port problem, I've sometimes found it can be
quite simple. Try this:
I am double NATted (fiber but cable company needs their router to do cable box stuff and caller-id- but its crap when it comes to qos/management), where most ports are forwarded to my main openwrt/gargoyle router which does proper qos.
I have the ports correct, in any other application tests fine, like utorrent or hosting games/ftp and so on. I think that transmission fails because it's only seeing the first hop, which is still behind the main router.
I'm using the cygrunsrv created service, so when I shutdown the server (and services stop) will it be improper? So far, no issues with resuming torrents even when I stop and start the service. The settings.json changes stick through these stop and starts. I suppose because I am logged into the rpc?
Last edit: robnitro 2015-05-26
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If the settings stick then it means your getting a clean shutdown, otherwise settings can revert themselves. To be safe use the shift web client or the transmission-remote.exe methods I posted above. In order to edit settings.json manually or read what the settings actually saved will be, the daemon must be shut down. i.e. settings.json elements "peer-port": 51413, (substitute your port for the 51413) "port-forwarding-enabled": false,
The port check of transmission remote GUI is using like (substitute your port for the 51413) http://portcheck.transmissionbt.com/51413 but a failed result doesn't necessarily mean the port is not open correctly, it is just how portcheck.transmissionbt.com is responding.
try can-you-see-me, grc.com or other tests and see what they say.
Last edit: cfp p2p 2015-05-26
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This discussion post has become somewhat disorganized.
I would like to correct that now.
1.) can't connect to daemon service while it is creating new files
Set the the streaming mode for file(s) to 4 (forced).
This can be set as default for all files.
When adding torrents, of any file size, there should
be no huge slowdown when the torrent starts files
with streaming mode for those files set to 4 (forced).
With the streaming mode set to 4 (forced) this should
prevent daemon and GUI temporary freeze that could
occur otherwise. Additionally, the streaming mode for
individual torrents' files can be set if you want to use,
for example, default streaming mode 0 and then set
particular file(s) to streaming mode 4 (forced). https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/57f64ca4#1466
2.) port forwarding issues
This is unrelated to the original discussion topic,
nevertheless I will elaborate.
https://www.grc.com/x/ne.dll?bh0bkyd2
Enter the port and select "User Specified Custom Port Probe"
You must scroll down the page and
look for Open or Closed in the status column.
http://portcheck.transmissionbt.com/XXXXX
substitute your port for XXXXX
0 means port is closed, 1 the port is open
Transmission's GUI internally uses portcheck.transmissionbt.com
Mode 4 for forced in the settings.json doesn't work. Just now I had it get stuck on allocating a 47GB file and my preallocation is set to 0. It doesn't actually allocate the whole file but parts, but during this process it locks up the ability to interact with the daemon.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ok it seems like this only happens when I changed the file from high/medium/or low to do not download... thats when it moved from the "incomplete" folder and started the whole write, slowww to do 47GB! heheh
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The preallocation issue isn't happening now in the latest Dec update. I don't think that was changed but maybe some windows update on win7 x64 was making the allocation system act up?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I tried both the latest x64 and latest and nightly latest x86 builds.
Using windows 7 on NTFS drives... adding new torrents with big files even with preallocate 1 or preallocate 0 causes a huge slowdown once the torrent starts on a file.
I can see my drive loaded up, writing the whole file, and the client can't connect at times... but transmissiondaemon.exe is not frozen or nowhere near 10% cpu use on a quad core (25% being a full core).
I'm not sure if this is also a problem in the linux version, but it can be quite annoying. I'd think preallocate 1 or 0 would prevent this, but not so on big files that get allocated at once!
I also have an incomplete directory. I may try to use .part instead..
.part didn't help. So it seems like with the windows file system handling, it is still writing out the whole file even with preallocation 0.
https://trac.transmissionbt.com/ticket/1753
The ticket has been open for six years on the official site. The developers concede:
"but it's likely a huge undertaking."
https://trac.transmissionbt.com/ticket/1753#comment:41
It would have to be a collaborative effort and the main developers don't seem interested. :-(
Last edit: cfp p2p 2014-09-15
Ah thank you... and I was thinking to run transmission in a VM as a temp fix! lol
We just had a 6+ month old bug with a specific router ethernet switch get fixed by the openwrt developers... but it was one of the reporters that started the ball rolling by finding a temporary hacked that worked...
I wonder if transmission were multithreaded, would it be able to get around this issue?
Too bad there isn't a good client for deluge as this awesome transmission one!
There have been many suggestions at the 1753 post and in the forums, but none have addressed the issue at its core since it is really in the lowest level of design for libtransmission. Part of the reason I created my forks of the project was to maintain a patch that allowed for temporary pieces https://trac.transmissionbt.com/ticket/532 This stops the allocation problem for any file that is deselected. In the official version even deselected files are created when a piece is downloaded that crosses a file boundary and this is especially troublesome with big files.
Last edit: cfp p2p 2014-09-15
Hi,
I haven't been using transmission and was wondering whether this got fixed or is still an issue.
There's a good workaround for this issue. See my post dated 2015-05-06 here: http://sourceforge.net/p/transmissiondaemon/discussion/general/thread/57f64ca4/#2f5a
settings.json setting
"stream-mode-default": 4,
With the streaming mode set to 4 (forced) this has resolved the issue for me. I've tested with both flash drives and hard drives and it works well. You can also set the streaming mode for individual torrents' files if you want to use default streaming mode 0 and then set particular file(s) to streaming mode 4 (forced).
https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/57f64ca4#1466
When adding new torrents with big files there is no longer a huge slowdown once the torrent starts on a file when the streaming mode for that file is set to 4 (forced).
Please verify that this works for your system. Thanks.
Last edit: cfp p2p 2015-05-17
I have to wait until finishing current torrents in deluge. Is there huge fragmentation with this kind of allocation?
Thanks for coming back on this :)
There shouldn't be any huge amount of fragmentation.
With streaming mode forced and when blocks begin download at the front of a file, cygwin uses sparse instead of writing all zeros first.
http://blogs.msdn.com/b/oldnewthing/archive/2011/09/22/10215053.aspx
The fragmentation would be dependent on what other applications also might be writing to the same disk at the same time. If you have a lot of simultaneous downloads that could affect fragmentation too. You could in these cases try a larger transmission cache.
But Windows OS has its own cache and so that takes precedence. I've used cache sizes of 4MB to 16MB but haven't been able to get any effective change over that. Don't go big or huge on cache, there's lots of examples on the regular transmission forums of problems when doing that.
references on cache:
http://sourceforge.net/p/trqtw/discussion/1315797/thread/73243864/?limit=25&page=1#caa4
http://sourceforge.net/p/trqtw/discussion/1315797/thread/48bb4e0b/#1704
Let me know how this goes for you and how it compares to deluge. I'd be interested to know.
Last edit: cfp p2p 2015-05-25
Interesting on the cache. In deluge they have libtorrent settings that allow you to change the way it interacts with windows cache, example: to not use windows file caching.
I just started using the x64 version and no issues so far.
I just added a 12GB torrent and it loaded fine!
Only weirdness is testing my forwarded port fails, despite even disabling the firewall and using the old deluge port (with deluge shut down).
In testing my ports via browser, it shows the port as open (as compared to open|filtered for the unused ports that are forwarded).
Any benefit to x64 vs x86?
By the way, I have been using a great blocklist here:
http://john.bitsurge.net/public/biglist.p2p.gz
If anyone has issues with the gui paths mappings, here is my setup:
X:\ is my mapped drive to the server D:\ where torrents are
D:/torrents=X:\torrents
d:/torrents=X:\torrents
Last edit: robnitro 2015-05-25
Only issue happens when the drive is busy doing other things, like accessing a big file on a share on that drive. The transmission client freezes up similarly to how it did in the past, when creating new torrents. I haven't changed the I/O priority, so that's not the issue. It's how the transmission daemon handles file delays etc. I think I can live with it though, as the transmission remote gui is 2x better than deluge headless client.
(I like being able to open the folders directly from the gui and GTK of deluge is kind of buggy at times).
The libraries of cygwin 64bit might be better suited for
the 64bit environment, but I can't say myself of
any noticeable difference. Try both. If you've got
64bit machine I don't know of any reason not to use
transmission 64.
As far as the port problem, I've sometimes found it can be
quite simple. Try this:
transmission remote GUI
Tools-->Transmission options...--> |Network|
UNcheck [ ]Enable port forwarding
Shut down transmission daemon properly like this
http://sourceforge.net/p/transmissiondaemon/discussion/general/thread/b4eea1a5#22f6
or use this to shutdown:
in order
localhost:9091 or whatever your daemon's host:port is
-n username:password user-name and password if you set them
http:// followed by nothing.
Login to your router and manually set up port forwarding,
Restart transmission daemon and verify that port forwarding is functioning.
Reboot the whole system after this if it still doesn't work.
There's the wiki and tons of posts on the official transmission forums
pertaining to ports.
https://forum.transmissionbt.com/search.php?keywords=port+closed
https://trac.transmissionbt.com/wiki/PortClosed
Last edit: cfp p2p 2015-05-26
Thanks for the help.
I am double NATted (fiber but cable company needs their router to do cable box stuff and caller-id- but its crap when it comes to qos/management), where most ports are forwarded to my main openwrt/gargoyle router which does proper qos.
I have the ports correct, in any other application tests fine, like utorrent or hosting games/ftp and so on. I think that transmission fails because it's only seeing the first hop, which is still behind the main router.
I'm using the cygrunsrv created service, so when I shutdown the server (and services stop) will it be improper? So far, no issues with resuming torrents even when I stop and start the service. The settings.json changes stick through these stop and starts. I suppose because I am logged into the rpc?
Last edit: robnitro 2015-05-26
If the settings stick then it means your getting a clean shutdown, otherwise settings can revert themselves. To be safe use the shift web client or the transmission-remote.exe methods I posted above. In order to edit settings.json manually or read what the settings actually saved will be, the daemon must be shut down. i.e. settings.json elements
"peer-port": 51413, (substitute your port for the 51413)
"port-forwarding-enabled": false,
example for a manual set port forwarding
read this post, it will give you some idea of what's going on inside libtransmission:
https://forum.transmissionbt.com/viewtopic.php?f=1&t=14274
The port check of transmission remote GUI is using like (substitute your port for the 51413)
http://portcheck.transmissionbt.com/51413 but a failed result doesn't necessarily mean the port is not open correctly, it is just how portcheck.transmissionbt.com is responding.
try can-you-see-me, grc.com or other tests and see what they say.
Last edit: cfp p2p 2015-05-26
this made a fixed for me to asynchronous I/O the daemon is no longer frezzing
OK
This discussion post has become somewhat disorganized.
I would like to correct that now.
1.) can't connect to daemon service while it is creating new files
Set the the streaming mode for file(s) to 4 (forced).
This can be set as default for all files.
When adding torrents, of any file size, there should
be no huge slowdown when the torrent starts files
with streaming mode for those files set to 4 (forced).
To enable -
See post dated 2015-05-06 here:
http://sourceforge.net/p/transmissiondaemon/discussion/general/thread/57f64ca4/#2f5a
settings.json setting
"stream-mode-default": 4,
With the streaming mode set to 4 (forced) this should
prevent daemon and GUI temporary freeze that could
occur otherwise. Additionally, the streaming mode for
individual torrents' files can be set if you want to use,
for example, default streaming mode 0 and then set
particular file(s) to streaming mode 4 (forced).
https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/57f64ca4#1466
2.) port forwarding issues
This is unrelated to the original discussion topic,
nevertheless I will elaborate.
With your browser try these:
http://canyouseeme.org/
https://www.grc.com/x/ne.dll?bh0bkyd2
Enter the port and select "User Specified Custom Port Probe"
You must scroll down the page and
look for Open or Closed in the status column.
http://portcheck.transmissionbt.com/XXXXX
substitute your port for XXXXX
0 means port is closed, 1 the port is open
Transmission's GUI internally uses portcheck.transmissionbt.com
If you have a closed port try setting it manually
using information from this:
https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/2d0fd14b/#23c3
and this:
https://sourceforge.net/p/transmissiondaemon/discussion/general/thread/2d0fd14b/#da37
Last edit: cfp p2p 2015-05-27
Mode 4 for forced in the settings.json doesn't work. Just now I had it get stuck on allocating a 47GB file and my preallocation is set to 0. It doesn't actually allocate the whole file but parts, but during this process it locks up the ability to interact with the daemon.
Ok it seems like this only happens when I changed the file from high/medium/or low to do not download... thats when it moved from the "incomplete" folder and started the whole write, slowww to do 47GB! heheh
The preallocation issue isn't happening now in the latest Dec update. I don't think that was changed but maybe some windows update on win7 x64 was making the allocation system act up?