chiphpug-discuss

Re: [chiPHPug-discuss] cURL problem

[Neil R. <Nei...@rc...>]

Bingo!

max_execution_time did it.

Thank you Pete, thank you Wilfried.


At 05:32 PM 8/25/2008, Peter Snyder <sn...@gm...>  wrote:
>Have you tried setting max_execution_time to a larger value value with ini_set?
>
>Just a first guess.
>


At 05:34 PM 8/25/2008, Wilfried Schobeiri <ws...@de...> wrote:
>the time limit is in php.ini, and set_time_limit() (per request) can 
>be found at http://us2.php.net/set_time_limit
>memory_limit is found in php.ini
>curl timeouts are set with curl_setopt and can be found at 
>http://us2.php.net/curl_setopt
>

Neil
--
Nei...@rc...

Liberals have always loved conspiracy theories because raising the specter of foul play and dirty tricks is an easy and convenient justification for ignoring their own political and policy failures.
 -- Oliver North

Re: [chiPHPug-discuss] cURL problem

[Neil R. <Nei...@rc...>]

Yes.

But this is my own script running on my own machine, while I watch it.  Or at least watch it until I'm sure it's running right.
I'm scraping some directories, so there's nested lookups &c., and some of them really are taking 5 or 10 minutes or more on my cheesy little machine and cheesy little "fast" internet connection.

Nervous Nellie that I am, my script now starts

echo 'original max_execution_time = ' . ini_get('max_execution_time') . "<br />\n";
ini_set('max_execution_time', 1200);
echo 'reset max_execution_time = ' . ini_get('max_execution_time') . "<br />\n<br />\n";


At 07:43 PM 8/25/2008, "Jough Dempsey" <jou...@gm...> wrote:
>Not that the previous answers won't work, but I'm wondering if just
>taking off the safeties that are in place to stop things like scripts
>that take a minute or two is really the Right Answer.
>
>A minute in PHP execution is a LONG time.  Is what the script is doing
>really that intensive, or are you just executing the same operations
>many times (e.g. if you were writing a spider that would request
>thousands of web pages)?  Even then, you should be careful that your
>script isn't doing Bad Things or capitalizing things Inappropriately.
>
>Regardless, I'd suggest setting max_execution_time to a higher value,
>but not "0" which means your script just runs and runs and runs.
>

Neil
--
Nei...@rc...

Never lift what you can drag, never drag what you can roll, never roll what you can leave.

Re: [chiPHPug-discuss] cURL problem

[Jason R. <ja...@ho...>]

You really should fork-exec that long running process and then poll a 
results page or something, rather than upping max_execution_time.

If there is a good reason for this process to run longer than a couple 
seconds, then I would get it out of the way of browser interaction.



Neil Rest wrote:
> Yes.
> 
> But this is my own script running on my own machine, while I watch it.  Or at least watch it until I'm sure it's running right.
> I'm scraping some directories, so there's nested lookups &c., and some of them really are taking 5 or 10 minutes or more on my cheesy little machine and cheesy little "fast" internet connection.
> 
> Nervous Nellie that I am, my script now starts
> 
> echo 'original max_execution_time = ' . ini_get('max_execution_time') . "<br />\n";
> ini_set('max_execution_time', 1200);
> echo 'reset max_execution_time = ' . ini_get('max_execution_time') . "<br />\n<br />\n";
> 
> 
> At 07:43 PM 8/25/2008, "Jough Dempsey" <jou...@gm...> wrote:
>> Not that the previous answers won't work, but I'm wondering if just
>> taking off the safeties that are in place to stop things like scripts
>> that take a minute or two is really the Right Answer.
>>
>> A minute in PHP execution is a LONG time.  Is what the script is doing
>> really that intensive, or are you just executing the same operations
>> many times (e.g. if you were writing a spider that would request
>> thousands of web pages)?  Even then, you should be careful that your
>> script isn't doing Bad Things or capitalizing things Inappropriately.
>>
>> Regardless, I'd suggest setting max_execution_time to a higher value,
>> but not "0" which means your script just runs and runs and runs.
>>
> 
> Neil
> --
> Nei...@rc...
> 
> Never lift what you can drag, never drag what you can roll, never roll what you can leave.
> 
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> chiPHPug-discuss mailing list
> chi...@li...
> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss

Re: [chiPHPug-discuss] cURL problem

[Wilfried S. <ws...@de...>]

I'm sure if the script is still in dev, he's going to want to see the  
results/debug output of it.. be it browser or CLI (though I do agree..  
getting it out of the browser would serve you much better, Neil)

On Aug 26, 2008, at 08:53 , Jason Rexilius wrote:

> You really should fork-exec that long running process and then poll a
> results page or something, rather than upping max_execution_time.
>
> If there is a good reason for this process to run longer than a couple
> seconds, then I would get it out of the way of browser interaction.
>
>
>
> Neil Rest wrote:
>> Yes.
>>
>> But this is my own script running on my own machine, while I watch  
>> it.  Or at least watch it until I'm sure it's running right.
>> I'm scraping some directories, so there's nested lookups &c., and  
>> some of them really are taking 5 or 10 minutes or more on my cheesy  
>> little machine and cheesy little "fast" internet connection.
>>
>> Nervous Nellie that I am, my script now starts
>>
>> echo 'original max_execution_time = ' .  
>> ini_get('max_execution_time') . "<br />\n";
>> ini_set('max_execution_time', 1200);
>> echo 'reset max_execution_time = ' .  
>> ini_get('max_execution_time') . "<br />\n<br />\n";
>>
>>
>> At 07:43 PM 8/25/2008, "Jough Dempsey" <jou...@gm...>  
>> wrote:
>>> Not that the previous answers won't work, but I'm wondering if just
>>> taking off the safeties that are in place to stop things like  
>>> scripts
>>> that take a minute or two is really the Right Answer.
>>>
>>> A minute in PHP execution is a LONG time.  Is what the script is  
>>> doing
>>> really that intensive, or are you just executing the same operations
>>> many times (e.g. if you were writing a spider that would request
>>> thousands of web pages)?  Even then, you should be careful that your
>>> script isn't doing Bad Things or capitalizing things  
>>> Inappropriately.
>>>
>>> Regardless, I'd suggest setting max_execution_time to a higher  
>>> value,
>>> but not "0" which means your script just runs and runs and runs.
>>>
>>
>> Neil
>> --
>> Nei...@rc...
>>
>> Never lift what you can drag, never drag what you can roll, never  
>> roll what you can leave.
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's  
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win  
>> great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in  
>> the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> chiPHPug-discuss mailing list
>> chi...@li...
>> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> chiPHPug-discuss mailing list
> chi...@li...
> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss

Re: [chiPHPug-discuss] cURL problem

[Jough D. <jou...@gm...>]

Not that the previous answers won't work, but I'm wondering if just
taking off the safeties that are in place to stop things like scripts
that take a minute or two is really the Right Answer.

A minute in PHP execution is a LONG time.  Is what the script is doing
really that intensive, or are you just executing the same operations
many times (e.g. if you were writing a spider that would request
thousands of web pages)?  Even then, you should be careful that your
script isn't doing Bad Things or capitalizing things Inappropriately.

Regardless, I'd suggest setting max_execution_time to a higher value,
but not "0" which means your script just runs and runs and runs.

Also, setting the php.ini directive means that all PHP scripts will
run up until that max execution time, which also may not be what the
doctor ordered.

You can use the set_time_limit(int) function instead:

http://us.php.net/manual/en/function.set-time-limit.php

And you get the added benefit of having it reset every time it's
called, which is really helpful if you use it in a loop where it
resets the counter per iteration, but doesn't stop the script from
timing out when it is doing something Bad.

My $.02 (USD, now worth much less than it used to be).

-- Jough


On Mon, Aug 25, 2008 at 6:39 PM, Neil Rest <Nei...@rc...> wrote:
> Bingo!
>
> max_execution_time did it.
>
> Thank you Pete, thank you Wilfried.

Re: [chiPHPug-discuss] cURL problem

[Adam L. <apl...@gm...>]

I had these same thoughts.  If you are simply scraping one page, 60
seconds is extremely long.  Like Jough says, some spider activity
might take that long, but processing one scrape seems kind of crazy
for that long of execution time.  Maybe describe what you're up to and
see if someone can help you optimize the process.  I'm imagining a
bunch of nested FOR loops in my head, or something else that could be
done more efficiently.

My $.04 (Jough, you have to adjust for inflation and weak currency.
Don't sell yourself short.)

Adam P. Larsen
apl...@gm...



On Mon, Aug 25, 2008 at 7:43 PM, Jough Dempsey <jou...@gm...> wrote:
> Not that the previous answers won't work, but I'm wondering if just
> taking off the safeties that are in place to stop things like scripts
> that take a minute or two is really the Right Answer.
>
> A minute in PHP execution is a LONG time.  Is what the script is doing
> really that intensive, or are you just executing the same operations
> many times (e.g. if you were writing a spider that would request
> thousands of web pages)?  Even then, you should be careful that your
> script isn't doing Bad Things or capitalizing things Inappropriately.
>
> Regardless, I'd suggest setting max_execution_time to a higher value,
> but not "0" which means your script just runs and runs and runs.
>
> Also, setting the php.ini directive means that all PHP scripts will
> run up until that max execution time, which also may not be what the
> doctor ordered.
>
> You can use the set_time_limit(int) function instead:
>
> http://us.php.net/manual/en/function.set-time-limit.php
>
> And you get the added benefit of having it reset every time it's
> called, which is really helpful if you use it in a loop where it
> resets the counter per iteration, but doesn't stop the script from
> timing out when it is doing something Bad.
>
> My $.02 (USD, now worth much less than it used to be).
>
> -- Jough
>
>
> On Mon, Aug 25, 2008 at 6:39 PM, Neil Rest <Nei...@rc...> wrote:
>> Bingo!
>>
>> max_execution_time did it.
>>
>> Thank you Pete, thank you Wilfried.
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> chiPHPug-discuss mailing list
> chi...@li...
> https://lists.sourceforge.net/lists/listinfo/chiphpug-discuss
>