Thread: [Fileshack-general] unicode charters in file names
fileshack allows you to create your own hosted web file storage, easy
Status: Beta
Brought to you by:
peterkuma
Menu
▾
▴
fileshack-general
|
From: Kyrylo S. <ksh...@gm...> - 2012-06-11 10:35:34
|
This is probably a question to the developer of Fileshack, but maybe someone else could provide insight. I have managed to setup Fileshack on lighttpd server via fcgi, and it seems to work more or less ok (although it seems like with Opera uploading big files causes excessive memory usage). My issue is that I cannot upload files containing unicode characters in names. On different browsers the process fails in different ways, but none succeed. I have checked that the test server at fileshackweb.appspot.com does not exhibit this problem, so I assume this is a problem with my setup. Since the unicode aspect does not seem to be covered in setup manual for fileshack, I don't really know where to start at other then diving into the code and trying to debug it but I hope there is an easier way to that, and also, if I encountered this problem, someone else might too. django process is running with LC_ALL=en_US.UTF-8. OS: gentoo x64 django: v 1.4 python: 2.72 sqlite: 3.7.11 Thanks, -- * * * -=- Kyrylo Shpytsya -=- |
|
From: Peter K. <pet...@us...> - 2012-06-11 17:34:46
|
Hi Kyrylo, What you describe is indeed a bug. The file name is sent unencoded in an HTTP header, and the UTF-8 byte sequence is expected to be ASCII by python. The master branch already contains a fix to this bug. It should be enough if you just run: git pull ./manage.py collectstatic --noinput and reload the fcgi server (or perhaps try `./manage.py runserver` to test first). The fix seems to be OK on linux Firefox 13, Opera 11.64, Chromium 6.0 and win IE9 and Safari 5.1.4. The memory usage in browsers is a bit of a problem, because (as far as I know) HTML5 provides no way of reading a fraction of a file, it has to be loaded fully into memory if it is to be sent in chunks to the server. When uploading by a simple upload form, browsers deal with the IO and it should be much more efficient, but one loses the ability to resume the upload, which is particularly interesting for very large files. The best I can think of is to have a cut-off of some 256MB, and upload larger files by FormData or by a simple upload form. Thank you for reporting the bug! Regards, Peter On 06/11/2012 12:35 PM, Kyrylo Shpytsya wrote: > This is probably a question to the developer of Fileshack, but maybe > someone else could provide insight. > > I have managed to setup Fileshack on lighttpd server via fcgi, and it > seems to work more or less ok (although it seems like with Opera > uploading big files causes excessive memory usage). > > My issue is that I cannot upload files containing unicode characters > in names. On different browsers the process fails in different ways, > but none succeed. I have checked that the test server at > fileshackweb.appspot.com does not exhibit this problem, so I assume > this is a problem with my setup. Since the unicode aspect does not > seem to be covered in setup manual for fileshack, I don't really know > where to start at other then diving into the code and trying to debug > it but I hope there is an easier way to that, and also, if I > encountered this problem, someone else might too. > > django process is running with LC_ALL=en_US.UTF-8. > > OS: gentoo x64 > django: v 1.4 > python: 2.72 > sqlite: 3.7.11 > > Thanks, > |
|
From: Peter K. <pet...@us...> - 2012-06-11 21:18:53
|
Dear Kyrylo, Thank you for the suggestion, slice looks perfect for the task. I'll try to incorporate it into fileshack soon. Regards, Peter On 06/11/2012 10:41 PM, Kyrylo Shpytsya wrote: > Hello Peter, > > thank you for FileShack itself and a prompt reply. I can confirm that > it is fixed by your last change. > > As to partial reads, maybe you have seen this? > http://www.html5rocks.com/en/tutorials/file/dndfiles/ /slice > > I am not an active web/javascript developer myself, so I am not sure > whether would work or not. > > Thanks. > > On Mon, Jun 11, 2012 at 8:18 PM, Peter Kuma > <pet...@us...> wrote: >> Hi Kyrylo, >> >> What you describe is indeed a bug. >> >> The file name is sent unencoded in an HTTP header, and the UTF-8 byte >> sequence is expected to be ASCII by python. >> >> The master branch already contains a fix to this bug. It should be >> enough if you just run: >> >> git pull >> ./manage.py collectstatic --noinput >> >> and reload the fcgi server (or perhaps try `./manage.py runserver` to >> test first). >> >> The fix seems to be OK on linux Firefox 13, Opera 11.64, Chromium 6.0 >> and win IE9 and Safari 5.1.4. >> >> The memory usage in browsers is a bit of a problem, because (as far as I >> know) HTML5 provides no way of reading a fraction of a file, it has to >> be loaded fully into memory if it is to be sent in chunks to the server. >> When uploading by a simple upload form, browsers deal with the IO and it >> should be much more efficient, but one loses the ability to resume the >> upload, which is particularly interesting for very large files. The best >> I can think of is to have a cut-off of some 256MB, and upload larger >> files by FormData or by a simple upload form. >> >> Thank you for reporting the bug! >> >> Regards, >> >> Peter >> >> On 06/11/2012 12:35 PM, Kyrylo Shpytsya wrote: >>> This is probably a question to the developer of Fileshack, but maybe >>> someone else could provide insight. >>> >>> I have managed to setup Fileshack on lighttpd server via fcgi, and it >>> seems to work more or less ok (although it seems like with Opera >>> uploading big files causes excessive memory usage). >>> >>> My issue is that I cannot upload files containing unicode characters >>> in names. On different browsers the process fails in different ways, >>> but none succeed. I have checked that the test server at >>> fileshackweb.appspot.com does not exhibit this problem, so I assume >>> this is a problem with my setup. Since the unicode aspect does not >>> seem to be covered in setup manual for fileshack, I don't really know >>> where to start at other then diving into the code and trying to debug >>> it but I hope there is an easier way to that, and also, if I >>> encountered this problem, someone else might too. >>> >>> django process is running with LC_ALL=en_US.UTF-8. >>> >>> OS: gentoo x64 >>> django: v 1.4 >>> python: 2.72 >>> sqlite: 3.7.11 >>> >>> Thanks, >>> >> > > > |
|
From: Peter K. <pet...@us...> - 2012-06-12 12:47:07
|
Hi Kyrylo, It is somewhat better to use the github issues, since others can easily check if a bug they are experiencing has already been reported. Multiple file uploading was meant to be supported in the future, but never really tested. It turned out to be prevented by a small bug, and now it should work at least in recent Firefox, Chromium, Opera and Safari on win. Certain browsers do not support multiple file uploading, slightly older Safari, and of course IE9. When the session has expired the user should get the error message: Your session expired Please log in again where "log in again" is a link to the login form. Sure, this is not as good as it could be - the page could switch to the login form automatically right when the session expires. In any case, the session expiry of 1h is rather strict for this kind of application. I extended it to last until the browser is closed. Therefore, the "Your session expired" error should happen only when the browser cache is deleted. There is actually an update mechanism in place which synchronises the list of items with the server, but it is disabled at the moment, as there are still some unresolved race conditions (though they are unlikely to occur when update is say once per 60s). This enables one to see files as they are being uploaded by other clients, and start downloading them before they are fully uploaded. The fixes are in the master branch. Regards, Peter On 06/12/2012 11:38 AM, Kyrylo Shpytsya wrote: > Hello again. > > what is your preferred way for bug reports? A formal one via > https://github.com/peterkuma/fileshackproject/issues, or just via > email? > > Meanwhile two more possible bugs/issues: > > - Is multiple file upload supported? I.e. when user selects more than > one file in a file selector dialog of browser. I have checked on Opera > and Firefox selecting three files, and behavior appears to be quite > inconsistent and erratic. > - Even if you choose not to do any timer-based periodical updates from > UI (or whatever other mechanism available in AJAX for keeping client > representation in sync with server), it would be nice to somehow check > session expiration before letting user to choose files for upload > instead of having a mysterious upload failure. > > Thanks. > > On Tue, Jun 12, 2012 at 12:18 AM, Peter Kuma > <pet...@us...> wrote: >> Dear Kyrylo, >> >> Thank you for the suggestion, slice looks perfect for the task. I'll try >> to incorporate it into fileshack soon. >> >> Regards, >> >> Peter >> >> On 06/11/2012 10:41 PM, Kyrylo Shpytsya wrote: >>> Hello Peter, >>> >>> thank you for FileShack itself and a prompt reply. I can confirm that >>> it is fixed by your last change. >>> >>> As to partial reads, maybe you have seen this? >>> http://www.html5rocks.com/en/tutorials/file/dndfiles/ /slice >>> >>> I am not an active web/javascript developer myself, so I am not sure >>> whether would work or not. >>> >>> Thanks. >>> >>> On Mon, Jun 11, 2012 at 8:18 PM, Peter Kuma >>> <pet...@us...> wrote: >>>> Hi Kyrylo, >>>> >>>> What you describe is indeed a bug. >>>> >>>> The file name is sent unencoded in an HTTP header, and the UTF-8 byte >>>> sequence is expected to be ASCII by python. >>>> >>>> The master branch already contains a fix to this bug. It should be >>>> enough if you just run: >>>> >>>> git pull >>>> ./manage.py collectstatic --noinput >>>> >>>> and reload the fcgi server (or perhaps try `./manage.py runserver` to >>>> test first). >>>> >>>> The fix seems to be OK on linux Firefox 13, Opera 11.64, Chromium 6.0 >>>> and win IE9 and Safari 5.1.4. >>>> >>>> The memory usage in browsers is a bit of a problem, because (as far as I >>>> know) HTML5 provides no way of reading a fraction of a file, it has to >>>> be loaded fully into memory if it is to be sent in chunks to the server. >>>> When uploading by a simple upload form, browsers deal with the IO and it >>>> should be much more efficient, but one loses the ability to resume the >>>> upload, which is particularly interesting for very large files. The best >>>> I can think of is to have a cut-off of some 256MB, and upload larger >>>> files by FormData or by a simple upload form. >>>> >>>> Thank you for reporting the bug! >>>> >>>> Regards, >>>> >>>> Peter >>>> >>>> On 06/11/2012 12:35 PM, Kyrylo Shpytsya wrote: >>>>> This is probably a question to the developer of Fileshack, but maybe >>>>> someone else could provide insight. >>>>> >>>>> I have managed to setup Fileshack on lighttpd server via fcgi, and it >>>>> seems to work more or less ok (although it seems like with Opera >>>>> uploading big files causes excessive memory usage). >>>>> >>>>> My issue is that I cannot upload files containing unicode characters >>>>> in names. On different browsers the process fails in different ways, >>>>> but none succeed. I have checked that the test server at >>>>> fileshackweb.appspot.com does not exhibit this problem, so I assume >>>>> this is a problem with my setup. Since the unicode aspect does not >>>>> seem to be covered in setup manual for fileshack, I don't really know >>>>> where to start at other then diving into the code and trying to debug >>>>> it but I hope there is an easier way to that, and also, if I >>>>> encountered this problem, someone else might too. >>>>> >>>>> django process is running with LC_ALL=en_US.UTF-8. >>>>> >>>>> OS: gentoo x64 >>>>> django: v 1.4 >>>>> python: 2.72 >>>>> sqlite: 3.7.11 >>>>> >>>>> Thanks, >>>>> >>>> >>> >>> >>> >> > > > |
|
From: Peter K. <pet...@us...> - 2012-06-13 11:47:40
|
Hi Kyrylo, File slicing is now supported, although for the time being only in recent Firefox and Google Chrome/Chromium. Possibly, it could be extended to Opera and some old versions of Firefox, but I'm not sure if it is worth the effort. Opera will likely support FormData soon, which is the missing bit. Performance is greatly improved as a result. Regards, Peter On 06/11/2012 11:18 PM, Peter Kuma wrote: > Dear Kyrylo, > > Thank you for the suggestion, slice looks perfect for the task. I'll try > to incorporate it into fileshack soon. > > Regards, > > Peter > > On 06/11/2012 10:41 PM, Kyrylo Shpytsya wrote: >> Hello Peter, >> >> thank you for FileShack itself and a prompt reply. I can confirm that >> it is fixed by your last change. >> >> As to partial reads, maybe you have seen this? >> http://www.html5rocks.com/en/tutorials/file/dndfiles/ /slice >> >> I am not an active web/javascript developer myself, so I am not sure >> whether would work or not. >> >> Thanks. >> >> On Mon, Jun 11, 2012 at 8:18 PM, Peter Kuma >> <pet...@us...> wrote: >>> Hi Kyrylo, >>> >>> What you describe is indeed a bug. >>> >>> The file name is sent unencoded in an HTTP header, and the UTF-8 byte >>> sequence is expected to be ASCII by python. >>> >>> The master branch already contains a fix to this bug. It should be >>> enough if you just run: >>> >>> git pull >>> ./manage.py collectstatic --noinput >>> >>> and reload the fcgi server (or perhaps try `./manage.py runserver` to >>> test first). >>> >>> The fix seems to be OK on linux Firefox 13, Opera 11.64, Chromium 6.0 >>> and win IE9 and Safari 5.1.4. >>> >>> The memory usage in browsers is a bit of a problem, because (as far as I >>> know) HTML5 provides no way of reading a fraction of a file, it has to >>> be loaded fully into memory if it is to be sent in chunks to the server. >>> When uploading by a simple upload form, browsers deal with the IO and it >>> should be much more efficient, but one loses the ability to resume the >>> upload, which is particularly interesting for very large files. The best >>> I can think of is to have a cut-off of some 256MB, and upload larger >>> files by FormData or by a simple upload form. >>> >>> Thank you for reporting the bug! >>> >>> Regards, >>> >>> Peter >>> >>> On 06/11/2012 12:35 PM, Kyrylo Shpytsya wrote: >>>> This is probably a question to the developer of Fileshack, but maybe >>>> someone else could provide insight. >>>> >>>> I have managed to setup Fileshack on lighttpd server via fcgi, and it >>>> seems to work more or less ok (although it seems like with Opera >>>> uploading big files causes excessive memory usage). >>>> >>>> My issue is that I cannot upload files containing unicode characters >>>> in names. On different browsers the process fails in different ways, >>>> but none succeed. I have checked that the test server at >>>> fileshackweb.appspot.com does not exhibit this problem, so I assume >>>> this is a problem with my setup. Since the unicode aspect does not >>>> seem to be covered in setup manual for fileshack, I don't really know >>>> where to start at other then diving into the code and trying to debug >>>> it but I hope there is an easier way to that, and also, if I >>>> encountered this problem, someone else might too. >>>> >>>> django process is running with LC_ALL=en_US.UTF-8. >>>> >>>> OS: gentoo x64 >>>> django: v 1.4 >>>> python: 2.72 >>>> sqlite: 3.7.11 >>>> >>>> Thanks, |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X