Re: [chiPHPug-discuss] Very Large Uploads Problem

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

You know, part of me wants to say if you're trying to move gigs of  
data around via web form, youre doing something wrong.

Think:
  * tmp or /var space could run out, because its getting written to / 
var/tmp then rebuilt to /tmp
  * PHP could run out of memory rebuilding the 1MB /var/tmp chunks (or  
so I read)
  * You're tying up an httpd instance for who knows how long
  * No Fault Tolerance... process manager might kill the process or  
the client might time out.

Plus, HTTP wasn't designed for large file transfers.

Then again, if its hosted locally, some of this may not apply.

Try a Java or Flash applet instead, perhaps? Yes, i know... *groan*.

On Nov 19, 2008, at 11:52 , Anacreo wrote:

> Jason,
>  That's a great review of what the issues could be...  I didn't know  
> about
> the IIS config setting and the Firewall info is intriguing.  I'm most
> inclined to think it's the max time for the script to run.
>
>  Is there anything recorded in the http_error log?
>
> Alec
>
> On Wed, Nov 19, 2008 at 8:12 AM, Jason Rexilius  
> <ja...@ho...>wrote:
>
>> A couple of feedback items from the google (off the top of my head I
>> cant see anything obvious):
>>
>> 1) are you running newer PHP5? from manual:
>> "G (for Gigabytes; available since PHP 5.1.0)"
>>
>>
>> 2) IIS has som limits if you are on Windows (why, oh why):
>> IIS7 has a upload limit of 30000000 (about 30mb)
>> You can change this with the command (for 250mb):
>> c:\windows\system32\inetsrv\appcmd set config - 
>> section:requestFiltering
>> -requestLimits.maxAllowedContentLength:262144000
>> Or manual define it in:
>> %windir%\system32\inetsrv\config\applicationhost.config
>> Add this rule before the </requestFiltering> tag:
>> <requestLimits maxAllowedContentLength ="262144000" />
>>
>>
>> 3) There has been reference to session.gc_maxlifetime default value  
>> 1440
>> (24min) causing GC to wack session duriung upload and cause problems.
>> Not sure on that one..
>>
>>
>> 4) Also reported issues of MAX_FILE_SIZE hidden form field, and
>> zlib.output_compression = On in Apache..
>>
>>
>> 5) Check Apache LimitRequestBody (max size in bytes)
>>
>>
>> 6) Obscure but possible:
>> "Because some systems send the packets with the headers last (or the
>> header packet may be received after some of the data packets),  
>> firewalls
>> can't filter this traffic based on destination port and address. Many
>> firewalls (including iptables) have to be configured to allow  
>> fragments
>> separately from standard traffic. Unfortunately, it's an all-or- 
>> nothing
>> thing in these cases, and exploits based on packet fragmentation have
>> been a problem in the past (teardrop, boink, etc.). Note that ICMP  
>> may
>> be used to notify the host (your server) of oncoming fragmentation,  
>> so
>> you may need to allow ICMP traffic as well."
>>
>> The iptables rules for this are as follows:
>> # allow all fragments
>> -A INPUT -f -j ACCEPT
>> # allow icmp traffic
>> -A INPUT -p icmp -j ACCEPT
>>
>>
>> Most of the above thanks to manual page:
>>
>> http://us3.php.net/features.file-upload
>>
>>
>>
>> Peter Snyder wrote:
>>> Hello Everyone,
>>>
>>> I'm having a problem creating an app for a client who will need to
>>> upload very large files, around 1GB.  I'm running into problems  
>>> though
>>> with the upload.  In an effort to rule out problems, I've boosted  
>>> all
>>> the normal suspects to absurd levels (see below), but am still  
>>> getting
>>> problems.  If I just watch the uploads in the temp dir I can see  
>>> that
>>> the server is receiving the uploads correctly, but for some reason  
>>> the
>>> upload stops at around 40 - 50 MB each time.  The browser will  
>>> putter
>>> on for a while and then say that the server has timed out.
>>>
>>> Does anyone have any other ideas for possible problems?  I've tried
>>> this on two different servers (clients and my staging one), with no
>>> luck.  Any hints anyone could provide would be greatly appreciated.
>>>
>>> Thanks!
>>>
>>> Sincerely,
>>> Pete Snyder
>>>
>>>
>>>
>>>
>>> -----
>>> .htaccess file
>>> php_value memory_limit 2G
>>> php_value post_max_size 2G
>>> php_value upload_max_filesize 2G
>>> php_value max_input_time 144000
>>> php_value max_execution_time 144000
>>>
>>>
>>>
>>> -------------------------------------------------------------------------
>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>>> Build the coolest Linux based applications with Moblin SDK & win  
>>> great
>> prizes
>>> Grand prize is a trip for two to an Open Source event anywhere in  
>>> the
>> world
>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>> _______________________________________________
>>> chiPHPug-discuss mailing list
>>> chi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win  
>> great
>> prizes
>> Grand prize is a trip for two to an Open Source event anywhere in  
>> the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> chiPHPug-discuss mailing list
>> chi...@li...
>> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss
>>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> chiPHPug-discuss mailing list
> chi...@li...
> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss