Webware for Python

We broke something in our latest round of merges.  It wasn't leaking at
all on sequential requests.  It should only leak on high concurrent
loads.

Jay

Chuck Esterbrook wrote:
> 
> Using WebKit/Testing/stress.py, I ran 10,000 requests on the 2 socket based
> app servers. Performance and memory leaks are practically identical.
> 
> Note that if we had the "limited lifetime feature" of app servers going,
> this probably wouldn't be a big deal. At 6MB of leakage per 10,000 requests
> I think for my current sites I could set the lifetime to 10,000 or even
> 25,000 requests and feel fine about it. It's not like I get even get 10,000
> hits a day.
> 
> Yeah, I'd still like to nab the leak, but it's not critical for me.
> 
> Regardless of leak or no leak, I don't want to use the app server until:
>         * We have better exception catching for WebKit itself (not just the servlets)
>         * A back up for when our somewhat-complex exception handler throws
> exceptions itself.  :-)
>         * More stress testing.
>         * Auto-launching by the adapters.
> 
> So in the mean time I continue to use OneShot.cgi, which I need for
> development purposes anyway.
> 
> My next mini-project is an auto-filesystem-pickle option for sessions in
> order to enable OneShot.cgi to be "complete".
> 
> -Chuck
> 
> Here are the results:
> 
> *** AsyncAppServer ***
> 
> C:\All\echuck\Projects\Webware\WebKit\Testing>python stress.py 10000 1 1 0
> STRESS TEST for Webware.WebKit.AppServer
> 
> time                = Thu Jul 13 14:02:24 2000
> requestFilenames    = ['CountVisits.rr', 'ListBox.rr', 'Time.rr', 'Welcome.rr']
> maxRequests         = 10000
> minParallelRequests = 1
> maxParallelRequests = 1
> delay               = 0.00
> sequential          = 1
> Running...
> count                = 10000
> duration             = 1405.70
> secs/page            = 0.14
> pages/sec            = 7.11
> Done.
> 
> Memory: 4,868 --> 11,084
> 
> *** ThreadedAppServer ***
> 
> C:\All\echuck\Projects\Webware\WebKit\Testing>python stress.py 10000 1 1 0
> STRESS TEST for Webware.WebKit.AppServer
> 
> time                = Thu Jul 13 14:28:58 2000
> requestFilenames    = ['CountVisits.rr', 'ListBox.rr', 'Time.rr', 'Welcome.rr']
> maxRequests         = 10000
> minParallelRequests = 1
> maxParallelRequests = 1
> delay               = 0.00
> sequential          = 1
> Running...
> count                = 10000
> duration             = 1365.50
> secs/page            = 0.14
> pages/sec            = 7.32
> Done.
> 
> Memory: 4,804 -> 11,068
> 
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> http://lists.sourceforge.net/mailman/listinfo/webware-discuss

That's interesting. I guess we'll dig on this deeper.

-Chuck


At 10:24 PM 7/13/00 -0400, Jay Love wrote:

>We broke something in our latest round of merges.  It wasn't leaking at
>all on sequential requests.  It should only leak on high concurrent
>loads.
>
>Jay
>
>Chuck Esterbrook wrote:
> >
> > Using WebKit/Testing/stress.py, I ran 10,000 requests on the 2 socket based
> > app servers. Performance and memory leaks are practically identical.
> >
> > Note that if we had the "limited lifetime feature" of app servers going,
> > this probably wouldn't be a big deal. At 6MB of leakage per 10,000 requests
> > I think for my current sites I could set the lifetime to 10,000 or even
> > 25,000 requests and feel fine about it. It's not like I get even get 10,000
> > hits a day.
> >
> > Yeah, I'd still like to nab the leak, but it's not critical for me.
> >
> > Regardless of leak or no leak, I don't want to use the app server until:
> >         * We have better exception catching for WebKit itself (not just 
> the servlets)
> >         * A back up for when our somewhat-complex exception handler throws
> > exceptions itself.  :-)
> >         * More stress testing.
> >         * Auto-launching by the adapters.
> >
> > So in the mean time I continue to use OneShot.cgi, which I need for
> > development purposes anyway.
> >
> > My next mini-project is an auto-filesystem-pickle option for sessions in
> > order to enable OneShot.cgi to be "complete".
> >
> > -Chuck
> >
> > Here are the results:
> >
> > *** AsyncAppServer ***
> >
> > C:\All\echuck\Projects\Webware\WebKit\Testing>python stress.py 10000 1 1 0
> > STRESS TEST for Webware.WebKit.AppServer
> >
> > time                = Thu Jul 13 14:02:24 2000
> > requestFilenames    = ['CountVisits.rr', 'ListBox.rr', 'Time.rr', 
> 'Welcome.rr']
> > maxRequests         = 10000
> > minParallelRequests = 1
> > maxParallelRequests = 1
> > delay               = 0.00
> > sequential          = 1
> > Running...
> > count                = 10000
> > duration             = 1405.70
> > secs/page            = 0.14
> > pages/sec            = 7.11
> > Done.
> >
> > Memory: 4,868 --> 11,084
> >
> > *** ThreadedAppServer ***
> >
> > C:\All\echuck\Projects\Webware\WebKit\Testing>python stress.py 10000 1 1 0
> > STRESS TEST for Webware.WebKit.AppServer
> >
> > time                = Thu Jul 13 14:28:58 2000
> > requestFilenames    = ['CountVisits.rr', 'ListBox.rr', 'Time.rr', 
> 'Welcome.rr']
> > maxRequests         = 10000
> > minParallelRequests = 1
> > maxParallelRequests = 1
> > delay               = 0.00
> > sequential          = 1
> > Running...
> > count                = 10000
> > duration             = 1365.50
> > secs/page            = 0.14
> > pages/sec            = 7.32
> > Done.
> >
> > Memory: 4,804 -> 11,068
> >
> > _______________________________________________
> > Webware-discuss mailing list
> > Webware-discuss@...
> > http://lists.sourceforge.net/mailman/listinfo/webware-discuss
>
>_______________________________________________
>Webware-discuss mailing list
>Webware-discuss@...
>http://lists.sourceforge.net/mailman/listinfo/webware-discuss

Try dropping your session timeout down to 10 seconds.  THat's at least
part of it.

J

Chuck Esterbrook wrote:
> 
> That's interesting. I guess we'll dig on this deeper.
> 
> -Chuck
> 
> At 10:24 PM 7/13/00 -0400, Jay Love wrote:
> 
> >We broke something in our latest round of merges.  It wasn't leaking at
> >all on sequential requests.  It should only leak on high concurrent
> >loads.
> >
> >Jay
> >
> >Chuck Esterbrook wrote:
> > >
> > > Using WebKit/Testing/stress.py, I ran 10,000 requests on the 2 socket based
> > > app servers. Performance and memory leaks are practically identical.
> > >
> > > Note that if we had the "limited lifetime feature" of app servers going,
> > > this probably wouldn't be a big deal. At 6MB of leakage per 10,000 requests
> > > I think for my current sites I could set the lifetime to 10,000 or even
> > > 25,000 requests and feel fine about it. It's not like I get even get 10,000
> > > hits a day.
> > >
> > > Yeah, I'd still like to nab the leak, but it's not critical for me.
> > >
> > > Regardless of leak or no leak, I don't want to use the app server until:
> > >         * We have better exception catching for WebKit itself (not just
> > the servlets)
> > >         * A back up for when our somewhat-complex exception handler throws
> > > exceptions itself.  :-)
> > >         * More stress testing.
> > >         * Auto-launching by the adapters.
> > >
> > > So in the mean time I continue to use OneShot.cgi, which I need for
> > > development purposes anyway.
> > >
> > > My next mini-project is an auto-filesystem-pickle option for sessions in
> > > order to enable OneShot.cgi to be "complete".
> > >
> > > -Chuck
> > >
> > > Here are the results:
> > >
> > > *** AsyncAppServer ***
> > >
> > > C:\All\echuck\Projects\Webware\WebKit\Testing>python stress.py 10000 1 1 0
> > > STRESS TEST for Webware.WebKit.AppServer
> > >
> > > time                = Thu Jul 13 14:02:24 2000
> > > requestFilenames    = ['CountVisits.rr', 'ListBox.rr', 'Time.rr',
> > 'Welcome.rr']
> > > maxRequests         = 10000
> > > minParallelRequests = 1
> > > maxParallelRequests = 1
> > > delay               = 0.00
> > > sequential          = 1
> > > Running...
> > > count                = 10000
> > > duration             = 1405.70
> > > secs/page            = 0.14
> > > pages/sec            = 7.11
> > > Done.
> > >
> > > Memory: 4,868 --> 11,084
> > >
> > > *** ThreadedAppServer ***
> > >
> > > C:\All\echuck\Projects\Webware\WebKit\Testing>python stress.py 10000 1 1 0
> > > STRESS TEST for Webware.WebKit.AppServer
> > >
> > > time                = Thu Jul 13 14:28:58 2000
> > > requestFilenames    = ['CountVisits.rr', 'ListBox.rr', 'Time.rr',
> > 'Welcome.rr']
> > > maxRequests         = 10000
> > > minParallelRequests = 1
> > > maxParallelRequests = 1
> > > delay               = 0.00
> > > sequential          = 1
> > > Running...
> > > count                = 10000
> > > duration             = 1365.50
> > > secs/page            = 0.14
> > > pages/sec            = 7.32
> > > Done.
> > >
> > > Memory: 4,804 -> 11,068
> > >
> > > _______________________________________________
> > > Webware-discuss mailing list
> > > Webware-discuss@...
> > > http://lists.sourceforge.net/mailman/listinfo/webware-discuss
> >
> >_______________________________________________
> >Webware-discuss mailing list
> >Webware-discuss@...
> >http://lists.sourceforge.net/mailman/listinfo/webware-discuss
> 
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> http://lists.sourceforge.net/mailman/listinfo/webware-discuss

Yeah, looks like that's most  or all of it here.  Don't freak me out
like that, man.  I've torn this thing apart searching for leaks!

THis is one good reason for only generating a session object/dict when
it is requested.  Dicts are by far the largest structures in python, and
all objects are dicts.

Jay

Jay Love wrote:
> 
> Try dropping your session timeout down to 10 seconds.  THat's at least
> part of it.
> 
> J