Thread: Re: [Persistentperl-users] Indefinite wait on first pass through?

Brought to you by: samh

persistentperl-users

Re: [Persistentperl-users] Indefinite wait on first pass through?

From: Sam H. <sa...@da...> - 2003-02-11 22:32:13

There is a known issue with DBI::Oracle causing hangs like that if you're
using IPC for your oracle connection.  A workaround is provided in the
last question under "Frequently Asked Questions".

As for #2, does the compile fail initially when the script is first
loaded, or is it later on in an eval, do or require?  Can you provide a small
code example that demonstrates the problem?

 > Hello all
 > 
 > (I see the bandwidth on this list is pretty low ... 4 messages in a 
 > year, so i hope there's still people out there).
 > 
 > We've installed perperl on a Solaris box using Iplanet webserver
 > and are running into two critical issues.
 > 
 > #!/usr/bin/perperl -X -- -M1 -t604800 
 > is the options we're using.
 > 
 > 1. The first time a web browser connects to the website running perperl,
 > the browser never pulls the site up.  You have to physically hit the
 > stop and reload buttons on the browser (doesn't matter what browser)
 > before the site comes up.  After that things are fine.  We can repeat
 > this process at will by touching the main perl program that runs.
 > 
 > 2. If there is a compilation error in the code ... the perperl process
 > can't ever load a page again ... we either have to kill the backend
 > processes or touch the main perl program to force the backends to 
 > reload... which of course causes us the issue in point #1.
 > 
 > Does anyone have any insight into why this is happening?  We've
 > tried setting timeout to Zero and then to a large number to no avail.
 > We're doing database connections via DBI:Oracle in the code as 
 > well, if that is known to be problematic.
 > 
 > Thanks, todd
 > 
 > 
 > -------------------------------------------------------
 > This SF.NET email is sponsored by:
 > SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
 > http://www.vasoftware.com
 > _______________________________________________
 > Persistentperl-users mailing list
 > Per...@li...
 > https://lists.sourceforge.net/lists/listinfo/persistentperl-users
 >

Re: [Persistentperl-users] Indefinite wait on first pass through?

From: Sam H. <sa...@da...> - 2003-02-12 22:41:52

 I think oracle usually uses ipc to connect to the database, when the
 database is on the same system as the client.  But if your database is on
 a different box, you must be using TCP/IP.

 From your description of the problem, it sounds like the oracle problem
 or something similar.  Do you fork any programs that continue running
 even after your perl code is done?  That's what oracle does, and that's
 why it can hang up a backend.  The background oracle process keeps stdout
 open, and since that file is a pipe into the web-server, the web-server
 never sees an EOF on the script's stdout, and just waits, and eventually
 times out.

 You could try applying the close-on-exec fcntl on STDERR in addition to
 STDOUT.  That might fix it if there is a forked program holding onto
 that file.

 Or, if you dupe STDERR/STDOUT in your perl code and don't close the duped
 file before exiting, that might be causing the hang.

 I think the five minute thing is due to your web server's cgi timeout.
 After five minutes, the web server gives up on the cgi altogether, and
 displays the results, even if the cgi hasn't shut down stdout and stderr.

 > Right; but i'm not specifically using an IPC connection to the database.
 > Or am I?  Why would i use anything except a TCP connection if i'm on
 > a TCP/IP network?  I'm not ever compiling in an IPC module and i'm on a 
 > standard tcp/ip network.

 > Nonetheless, i put in the fcntl call into the code for testing and am now
 > seeing very odd results.  I have two browser windows open; i touch the
 > main perl program w/ the perperl call and hit reload in browser window one:
 > it starts spinning, saying its "Opening page http://..."  Meanwhile,
 > as Soon as you hit reload in browser 1, you can open the page in browser
 > two and it works like a charm.  Fully FIVE minutes later the first browser
 > comes up.  Five minutes??  AFter this, both browsers work jsut fine.
 > This behavior is repeatable ... its about 5:15 each time.
 > 
 > Any ideas?  The perl code loaded up totals 271790 lines (pretty big) and the
 > image size of each perperl_backend process is 70Mb.  Could it just be that
 > the code is so big and complex that it just takes 5 minutes for perperl to
 > store the first time through (and if this is the case...why does the 
 > SECOND load attempt work so well, WHILE the first load attempt is still
 > going on?).  I killed all backends inbetween tests to prevent a "cached"
 > backend from servicing the program and got the same results.
 > 
 > 
 > As for #2, i fixed the bug that was causing it (there were ^M characters
 > in a module we were using ...) and now i can't recreate it necessarily.
 > What was happening was, our code ran fine until it tried to read in
 > the module w/ the bug.  I'll have to dig to re-create it.
 > 
 > Any insight anyone can provide is greatly appreciated.  
 > Thanks, todd
 > 
 > 
 > > 
 > > There is a known issue with DBI::Oracle causing hangs like that if you're
 > > using IPC for your oracle connection.  A workaround is provided in the
 > > last question under "Frequently Asked Questions".
 > > 
 > > As for #2, does the compile fail initially when the script is first
 > > loaded, or is it later on in an eval, do or require?  Can you provide a small
 > > code example that demonstrates the problem?
 > > 
 > >  > Hello all
 > >  > 
 > >  > (I see the bandwidth on this list is pretty low ... 4 messages in a 
 > >  > year, so i hope there's still people out there).
 > >  > 
 > >  > We've installed perperl on a Solaris box using Iplanet webserver
 > >  > and are running into two critical issues.
 > >  > 
 > >  > #!/usr/bin/perperl -X -- -M1 -t604800 
 > >  > is the options we're using.
 > >  > 
 > >  > 1. The first time a web browser connects to the website running perperl,
 > >  > the browser never pulls the site up.  You have to physically hit the
 > >  > stop and reload buttons on the browser (doesn't matter what browser)
 > >  > before the site comes up.  After that things are fine.  We can repeat
 > >  > this process at will by touching the main perl program that runs.
 > >  > 
 > >  > 2. If there is a compilation error in the code ... the perperl process
 > >  > can't ever load a page again ... we either have to kill the backend
 > >  > processes or touch the main perl program to force the backends to 
 > >  > reload... which of course causes us the issue in point #1.
 > >  > 
 > >  > Does anyone have any insight into why this is happening?  We've
 > >  > tried setting timeout to Zero and then to a large number to no avail.
 > >  > We're doing database connections via DBI:Oracle in the code as 
 > >  > well, if that is known to be problematic.
 > >  > 
 > >  > Thanks, todd
 > >  > 
 > >  > 
 > >  > -------------------------------------------------------
 > >  > This SF.NET email is sponsored by:
 > >  > SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
 > >  > http://www.vasoftware.com
 > >  > _______________________________________________
 > >  > Persistentperl-users mailing list
 > >  > Per...@li...
 > >  > https://lists.sourceforge.net/lists/listinfo/persistentperl-users
 > >  > 
 > > 
 > 
 > 
 > 
 > -------------------------------------------------------
 > This SF.NET email is sponsored by:
 > SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
 > http://www.vasoftware.com
 > _______________________________________________
 > Persistentperl-users mailing list
 > Per...@li...
 > https://lists.sourceforge.net/lists/listinfo/persistentperl-users
 >

Re: [Persistentperl-users] Indefinite wait on first pass through?

From: Joel P. <pl...@ej...> - 2003-02-14 17:52:41

Hi Sam,

Todd & I have narrowed down the issue a bit, but we could still use your advice.  We added
timing/debug/log statements to our core software and it revealed a few things.  It is not really an
Oracle issue.  I think it is more related to how we get and process the CGI request and if our main
program should ever exit(0)?  Here's what we've found:

1) background info:  a small version of our code is:
#!/usr/bin/perperl -X -- -M1 -t604800
MAIN:
{
    &log("step1");
      $query = new CGI;

    &log("step2");
     &process_request();

    &log("step3");
     #    exit(0);
}

sub process_request
{
.... our main application here...
}

where process_request(); is the bulk of our program.

It flies through steps 1,2,3 within 1-3 seconds.... then after step3, it hangs for 5 minutes... if we
add the exit(0), it still hangs...

If we recraft the code to be:
MAIN:
{
    &log("step1");

    while(1) {
     $query = new CGI;
     &log("step2");

     &process_request();
     &log("step3");
   }
}

Then it processes the first request over & over again, but really quickly....

If we recraft the code to be:
MAIN:
{
    &log("step1");

    while($query = new CGI) {
     &log("step2");

     &process_request();
     &log("step3");
   }
}

Then it takes the long time to process even the first request...

My guess is that we should be putting this into a loop or not using "new CGI" but using something like
CGI::Request() or something.  Can you e-mail us a 5 line sample script on what you had in mind?  I'm
sure it's some confusion about how the main loop should either poll forever, or exit and what CGI::
method you (the perperl code) is expecting us to use.

Many thanks,

Joel


______________________________________
Joel Plotkin
eJournalPress
http://www.ejournalpress.com
301-254-5900
301-564-4006 (fax)

Re: [Persistentperl-users] Indefinite wait on first pass through?

From: Sam H. <sa...@da...> - 2003-02-14 18:17:21

The five minute hang is probably a file that is being held onto either
by your perl script, or by a process you spawn (like oracle).  The five
minutes isn't being consumed by one part of your code or the other,
it's the web server thinking that your code is still running even after
it's done, because you still have an open file.  At least that's my best
guess from what I've seen so far.

If you could start tearing parts of your code out in testing, like remove
the oracle database open altogether, or remove other parts where you may
dup files or spawn other processes, and see if the hang goes away, you
could probably narrow it down that way, to see which part is causing it.

To answer your question, here's what your main program should look like:

 #!/usr/bin/perperl
 &log("step1");
 $query = new CGI;

 &log("step2");
 &process_request();

 &log("step3");

No while loop is required.  There's no need to call exit.  If your
script is written correctly you should be able to switch from "perperl"
to "perl" with no changes to the code.



 > Hi Sam,
 > 
 > Todd & I have narrowed down the issue a bit, but we could still use your advice.  We added
 > timing/debug/log statements to our core software and it revealed a few things.  It is not really an
 > Oracle issue.  I think it is more related to how we get and process the CGI request and if our main
 > program should ever exit(0)?  Here's what we've found:
 > 
 > 1) background info:  a small version of our code is:
 > #!/usr/bin/perperl -X -- -M1 -t604800
 > MAIN:
 > {
 >     &log("step1");
 >       $query = new CGI;
 > 
 >     &log("step2");
 >      &process_request();
 > 
 >     &log("step3");
 >      #    exit(0);
 > }
 > 
 > sub process_request
 > {
 > .... our main application here...
 > }
 > 
 > where process_request(); is the bulk of our program.
 > 
 > It flies through steps 1,2,3 within 1-3 seconds.... then after step3, it hangs for 5 minutes... if we
 > add the exit(0), it still hangs...
 > 
 > If we recraft the code to be:
 > MAIN:
 > {
 >     &log("step1");
 > 
 >     while(1) {
 >      $query = new CGI;
 >      &log("step2");
 > 
 >      &process_request();
 >      &log("step3");
 >    }
 > }
 > 
 > Then it processes the first request over & over again, but really quickly....
 > 
 > If we recraft the code to be:
 > MAIN:
 > {
 >     &log("step1");
 > 
 >     while($query = new CGI) {
 >      &log("step2");
 > 
 >      &process_request();
 >      &log("step3");
 >    }
 > }
 > 
 > Then it takes the long time to process even the first request...
 > 
 > My guess is that we should be putting this into a loop or not using "new CGI" but using something like
 > CGI::Request() or something.  Can you e-mail us a 5 line sample script on what you had in mind?  I'm
 > sure it's some confusion about how the main loop should either poll forever, or exit and what CGI::
 > method you (the perperl code) is expecting us to use.
 > 
 > Many thanks,
 > 
 > Joel
 > 
 > 
 > ______________________________________
 > Joel Plotkin
 > eJournalPress
 > http://www.ejournalpress.com
 > 301-254-5900
 > 301-564-4006 (fax)
 > 
 > 
 > 
 > 
 > 
 > --------------87E8EBFE6E648645532F8BFB
 > Content-Type: text/html; charset=us-ascii
 > Content-Transfer-Encoding: 7bit
 > 
 > <!doctype html public "-//w3c//dtd html 4.0 transitional//en">
 > <html>
 > Hi Sam,
 > <p>Todd &amp; I have narrowed down the issue a bit, but we could still
 > use your advice.&nbsp; We added timing/debug/log statements to our core
 > software and it revealed a few things.&nbsp; It is not really an Oracle
 > issue.&nbsp; I think it is more related to how we get and process the CGI
 > request and if our main program should ever exit(0)?&nbsp; Here's what
 > we've found:
 > <p>1) background info:&nbsp; a small version of our code is:
 > <br>#!/usr/bin/perperl -X -- -M1 -t604800
 > <br><tt>MAIN:</tt>
 > <br><tt>{</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp; &amp;log("step1");</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; $query = new CGI;</tt><tt></tt>
 > <p><tt>&nbsp;&nbsp;&nbsp; &amp;log("step2");</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;process_request();</tt><tt></tt>
 > <p><tt>&nbsp;&nbsp;&nbsp; &amp;log("step3");</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; #&nbsp;&nbsp;&nbsp; exit(0);</tt>
 > <br><tt>}</tt><tt></tt>
 > <p><tt>sub process_request</tt>
 > <br><tt>{</tt>
 > <br><tt>.... our main application here...</tt>
 > <br><tt>}</tt>
 > <p>where process_request(); is the bulk of our program.
 > <p>It flies through steps 1,2,3 within 1-3 seconds.... then after step3,
 > it hangs for 5 minutes... if we add the exit(0), it still hangs...
 > <p>If we recraft the code to be:
 > <br><tt>MAIN:</tt>
 > <br><tt>{</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp; &amp;log("step1");</tt><tt></tt>
 > <p><tt>&nbsp;&nbsp;&nbsp; while(1) {</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; $query = new CGI;</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;log("step2");</tt><tt></tt>
 > <p><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;process_request();</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;log("step3");</tt>
 > <br><tt>&nbsp;&nbsp; }</tt>
 > <br><tt>}</tt>
 > <p>Then it processes the first request over &amp; over again, but really
 > quickly....
 > <p>If we recraft the code to be:
 > <br><tt>MAIN:</tt>
 > <br><tt>{</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp; &amp;log("step1");</tt><tt></tt>
 > <p><tt>&nbsp;&nbsp;&nbsp; while($query = new CGI) {</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;log("step2");</tt><tt></tt>
 > <p><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;process_request();</tt>
 > <br><tt>&nbsp;&nbsp;&nbsp;&nbsp; &amp;log("step3");</tt>
 > <br><tt>&nbsp;&nbsp; }</tt>
 > <br><tt>}</tt>
 > <p>Then it takes the long time to process even the first request...
 > <p>My guess is that we should be putting this into a loop or not using
 > "new CGI" but using something like CGI::Request() or something.&nbsp; Can
 > you e-mail us a 5 line sample script on what you had in mind?&nbsp; I'm
 > sure it's some confusion about how the main loop should either poll forever,
 > or exit and what CGI:: method you (the perperl code) is expecting us to
 > use.
 > <p>Many thanks,
 > <p>Joel
 > <br>&nbsp;
 > <p>______________________________________
 > <br>Joel Plotkin
 > <br>eJournalPress
 > <br><A HREF="http://www.ejournalpress.com">http://www.ejournalpress.com</A>
 > <br>301-254-5900
 > <br>301-564-4006 (fax)
 > <br>&nbsp;
 > <br>&nbsp;
 > <br>&nbsp;
 > <br>&nbsp;</html>
 > 
 > --------------87E8EBFE6E648645532F8BFB--
 > 
 > 
 > 
 > -------------------------------------------------------
 > This SF.NET email is sponsored by: FREE  SSL Guide from Thawte
 > are you planning your Web Server Security? Click here to get a FREE
 > Thawte SSL guide and find the answers to all your  SSL security issues.
 > http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
 > _______________________________________________
 > Persistentperl-users mailing list
 > Per...@li...
 > https://lists.sourceforge.net/lists/listinfo/persistentperl-users
 >

Re: [Persistentperl-users] Indefinite wait on first pass through?

From: boss <bo...@sh...> - 2003-02-12 16:32:28

Right; but i'm not specifically using an IPC connection to the database.
Or am I?  Why would i use anything except a TCP connection if i'm on
a TCP/IP network?  I'm not ever compiling in an IPC module and i'm on a 
standard tcp/ip network.

Nonetheless, i put in the fcntl call into the code for testing and am now
seeing very odd results.  I have two browser windows open; i touch the
main perl program w/ the perperl call and hit reload in browser window one:
it starts spinning, saying its "Opening page http://..."  Meanwhile,
as Soon as you hit reload in browser 1, you can open the page in browser
two and it works like a charm.  Fully FIVE minutes later the first browser
comes up.  Five minutes??  AFter this, both browsers work jsut fine.
This behavior is repeatable ... its about 5:15 each time.

Any ideas?  The perl code loaded up totals 271790 lines (pretty big) and the
image size of each perperl_backend process is 70Mb.  Could it just be that
the code is so big and complex that it just takes 5 minutes for perperl to
store the first time through (and if this is the case...why does the 
SECOND load attempt work so well, WHILE the first load attempt is still
going on?).  I killed all backends inbetween tests to prevent a "cached"
backend from servicing the program and got the same results.


As for #2, i fixed the bug that was causing it (there were ^M characters
in a module we were using ...) and now i can't recreate it necessarily.
What was happening was, our code ran fine until it tried to read in
the module w/ the bug.  I'll have to dig to re-create it.

Any insight anyone can provide is greatly appreciated.  
Thanks, todd


> 
> There is a known issue with DBI::Oracle causing hangs like that if you're
> using IPC for your oracle connection.  A workaround is provided in the
> last question under "Frequently Asked Questions".
> 
> As for #2, does the compile fail initially when the script is first
> loaded, or is it later on in an eval, do or require?  Can you provide a small
> code example that demonstrates the problem?
> 
>  > Hello all
>  > 
>  > (I see the bandwidth on this list is pretty low ... 4 messages in a 
>  > year, so i hope there's still people out there).
>  > 
>  > We've installed perperl on a Solaris box using Iplanet webserver
>  > and are running into two critical issues.
>  > 
>  > #!/usr/bin/perperl -X -- -M1 -t604800 
>  > is the options we're using.
>  > 
>  > 1. The first time a web browser connects to the website running perperl,
>  > the browser never pulls the site up.  You have to physically hit the
>  > stop and reload buttons on the browser (doesn't matter what browser)
>  > before the site comes up.  After that things are fine.  We can repeat
>  > this process at will by touching the main perl program that runs.
>  > 
>  > 2. If there is a compilation error in the code ... the perperl process
>  > can't ever load a page again ... we either have to kill the backend
>  > processes or touch the main perl program to force the backends to 
>  > reload... which of course causes us the issue in point #1.
>  > 
>  > Does anyone have any insight into why this is happening?  We've
>  > tried setting timeout to Zero and then to a large number to no avail.
>  > We're doing database connections via DBI:Oracle in the code as 
>  > well, if that is known to be problematic.
>  > 
>  > Thanks, todd
>  > 
>  > 
>  > -------------------------------------------------------
>  > This SF.NET email is sponsored by:
>  > SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
>  > http://www.vasoftware.com
>  > _______________________________________________
>  > Persistentperl-users mailing list
>  > Per...@li...
>  > https://lists.sourceforge.net/lists/listinfo/persistentperl-users
>  > 
>