Project WONDER

you're going to have to do some profiling, either with a real  
profiler or by just adding in some timing statements throughout the  
call that is slow to see where the time is actually going ... there's  
just not enough information here to answer your question.  That ANY  
operation takes even 100 seconds is HIGHLY suspect to me, though.  I  
would say you have to serious performance issues at a lower level  
that need to be resolved.  You should really turn on sql debugging, etc.

On Apr 17, 2007, at 10:43 AM, Frank Stock wrote:

> Hi,
>
> I use a lot of SOAP-calls and some of them are using several select
> statements.
> In my random-testing I have done some testing with the
> StoreCoordinatorPool. (maxCoordinators=10)
> Strange is that the total time is beter (from 140 sec to 100 seconds)
> but some SOAP-calls needs a lot more time.
> Some SOAP-calls take from less then 1 second to 60 seconds!! (I then
> get a out of time error) How can I speed up things?
>
> Thanks,
> Frank Stock
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Wonder-disc mailing list
> Wonder-disc@...
> https://lists.sourceforge.net/lists/listinfo/wonder-disc

Hi, Frank,

Although I agree with Mike here, you might take a shortcut first and  
simply set EOAdaptorDebugEnabled to check the times for your SQL  
execution. I'm suspecting an indexing problem. If so, it should be  
readily apparent from the times on the select statements. I've seen  
indexing problems cause just such orders of magnitude variations in  
execution times.

If that turns out to not be the problem, go with Mike's advice.

Regards,
Jerry

On Apr 17, 2007, at 11:35 AM, Mike Schrag wrote:

> you're going to have to do some profiling, either with a real
> profiler or by just adding in some timing statements throughout the
> call that is slow to see where the time is actually going ... there's
> just not enough information here to answer your question.  That ANY
> operation takes even 100 seconds is HIGHLY suspect to me, though.  I
> would say you have to serious performance issues at a lower level
> that need to be resolved.  You should really turn on sql debugging,  
> etc.
>
> On Apr 17, 2007, at 10:43 AM, Frank Stock wrote:
>
>> Hi,
>>
>> I use a lot of SOAP-calls and some of them are using several select
>> statements.
>> In my random-testing I have done some testing with the
>> StoreCoordinatorPool. (maxCoordinators=10)
>> Strange is that the total time is beter (from 140 sec to 100 seconds)
>> but some SOAP-calls needs a lot more time.
>> Some SOAP-calls take from less then 1 second to 60 seconds!! (I then
>> get a out of time error) How can I speed up things?
>>
>> Thanks,
>> Frank Stock
>>
>> --------------------------------------------------------------------- 
>> -
>> ---
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> Wonder-disc mailing list
>> Wonder-disc@...
>> https://lists.sourceforge.net/lists/listinfo/wonder-disc
>
>
>
> ---------------------------------------------------------------------- 
> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> Wonder-disc mailing list
> Wonder-disc@...
> https://lists.sourceforge.net/lists/listinfo/wonder-disc


--
__ Jerry W. Walker,
    WebObjects Developer/Instructor for High Performance Industrial  
Strength Internet Enabled Systems

     jerrywwalker@...
     203 278-4085        office

Thanks,

I will explain my testing:


I have > 500 different SOAP-Calls in my application.
I have a test application where I test only 10 different SOAP-calls,  
some have to do a lot off select before they return a result.
In my test-app start 40 times the same thread wich randomly pick a  
SOAP-call (50 calls) so togheter 2000 calls; which is a normal workload.
So, the time from 1 of the SOAP-calls goes from less that a second to  
60 seconds.

I will do the sql-logging for testing

Thanks
Frank

Op 17-apr-07, om 18:20 heeft Jerry W. Walker het volgende geschreven:

> Hi, Frank,
>
> Although I agree with Mike here, you might take a shortcut first  
> and simply set EOAdaptorDebugEnabled to check the times for your  
> SQL execution. I'm suspecting an indexing problem. If so, it should  
> be readily apparent from the times on the select statements. I've  
> seen indexing problems cause just such orders of magnitude  
> variations in execution times.
>
> If that turns out to not be the problem, go with Mike's advice.
>
> Regards,
> Jerry
>
> On Apr 17, 2007, at 11:35 AM, Mike Schrag wrote:
>
>> you're going to have to do some profiling, either with a real
>> profiler or by just adding in some timing statements throughout the
>> call that is slow to see where the time is actually going ... there's
>> just not enough information here to answer your question.  That ANY
>> operation takes even 100 seconds is HIGHLY suspect to me, though.  I
>> would say you have to serious performance issues at a lower level
>> that need to be resolved.  You should really turn on sql  
>> debugging, etc.
>>
>> On Apr 17, 2007, at 10:43 AM, Frank Stock wrote:
>>
>>> Hi,
>>>
>>> I use a lot of SOAP-calls and some of them are using several select
>>> statements.
>>> In my random-testing I have done some testing with the
>>> StoreCoordinatorPool. (maxCoordinators=10)
>>> Strange is that the total time is beter (from 140 sec to 100  
>>> seconds)
>>> but some SOAP-calls needs a lot more time.
>>> Some SOAP-calls take from less then 1 second to 60 seconds!! (I then
>>> get a out of time error) How can I speed up things?
>>>
>>> Thanks,
>>> Frank Stock
>>>
>>> -------------------------------------------------------------------- 
>>> --
>>> ---
>>> This SF.net email is sponsored by DB2 Express
>>> Download DB2 Express C - the FREE version of DB2 express and take
>>> control of your XML. No limits. Just data. Click to get it now.
>>> http://sourceforge.net/powerbar/db2/
>>> _______________________________________________
>>> Wonder-disc mailing list
>>> Wonder-disc@...
>>> https://lists.sourceforge.net/lists/listinfo/wonder-disc
>>
>>
>>
>> --------------------------------------------------------------------- 
>> ----
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> Wonder-disc mailing list
>> Wonder-disc@...
>> https://lists.sourceforge.net/lists/listinfo/wonder-disc
>
>
> --
> __ Jerry W. Walker,
>    WebObjects Developer/Instructor for High Performance Industrial  
> Strength Internet Enabled Systems
>
>     jerrywwalker@...
>     203 278-4085        office
>
>
>
>

On 17.04.2007, at 12:33, Frank Stock wrote:

> I have > 500 different SOAP-Calls in my application.
> I have a test application where I test only 10 different SOAP-calls,
> some have to do a lot off select before they return a result.
> In my test-app start 40 times the same thread wich randomly pick a
> SOAP-call (50 calls) so togheter 2000 calls; which is a normal  
> workload.
> So, the time from 1 of the SOAP-calls goes from less that a second to
> 60 seconds.

Just an idea: do you have concurrent request handling turned on?

cug

yes,

WOAllowsConcurrentRequestHandling=true


Frank
Op 17-apr-07, om 21:18 heeft Guido Neitzer het volgende geschreven:

> On 17.04.2007, at 12:33, Frank Stock wrote:
>
>> I have > 500 different SOAP-Calls in my application.
>> I have a test application where I test only 10 different SOAP-calls,
>> some have to do a lot off select before they return a result.
>> In my test-app start 40 times the same thread wich randomly pick a
>> SOAP-call (50 calls) so togheter 2000 calls; which is a normal  
>> workload.
>> So, the time from 1 of the SOAP-calls goes from less that a second to
>> 60 seconds.
>
> Just an idea: do you have concurrent request handling turned on?
>
> cug
>

On 17.04.2007, at 13:27, Frank Stock wrote:

> WOAllowsConcurrentRequestHandling=true

And this is executed? I have:

-WOAllowsConcurrentRequestHandling=true

With the "-" in the beginning. I'm never sure what actually works  
with all the different kinds of Property styles.

Apple uses that as an example for commandline start:

./myApp -DWOAllowsConcurrentRequestHandling=true

You can ask your Application class whether it's turned on. So, if  
concurrent requests are not the problem, perhaps concurrent database  
access might still be a problem. You'll have to figure out query  
runtimes and WHAT actions take long and in what circumstance. Is it  
only we you have concurrent requests (lots of threads in the test  
app) or is it with many requests hammering the app (fast test app on  
fast computer with fast connection).

It might be slow at the database level when concurrent requests come  
in, so check the query runtimes - compare them in single threaded  
test to concurrent incoming requests. Might they just "add up" when  
you have high concurrency? Is you DB server perhaps the bottleneck  
(Indexes? Queries optmized? IO?). Is your server perhaps swapping in  
such a case when a lot of connections are opened?

cug

Op 17-apr-07, om 21:52 heeft Guido Neitzer het volgende geschreven:

> On 17.04.2007, at 13:27, Frank Stock wrote:
>
>> WOAllowsConcurrentRequestHandling=true
>
> And this is executed? I have:
>
> -WOAllowsConcurrentRequestHandling=true
>
> With the "-" in the beginning. I'm never sure what actually works  
> with all the different kinds of Property styles.
>
> Apple uses that as an example for commandline start:
>
> ./myApp -DWOAllowsConcurrentRequestHandling=true
I will have to test that later becaus I am working on another project  
now and have to reboot to be in MacOS Server.
>
> You can ask your Application class whether it's turned on. So, if  
> concurrent requests are not the problem, perhaps concurrent  
> database access might still be a problem. You'll have to figure out  
> query runtimes and WHAT actions take long and in what circumstance.  
> Is it only we you have concurrent requests (lots of threads in the  
> test app) or is it with many requests hammering the app (fast test  
> app on fast computer with fast connection).
My testmachine is a iMac Intel dual 2GHz running MacOSX server,  
application in develop-mode (XCode), the other machine is a Powerbook  
Intel, so yes both are realtive fast machines
>
> It might be slow at the database level when concurrent requests  
> come in, so check the query runtimes - compare them in single  
> threaded test to concurrent incoming requests. Might they just "add  
> up" when you have high concurrency? Is you DB server perhaps the  
> bottleneck (Indexes? Queries optmized? IO?). Is your server perhaps  
> swapping in such a case when a lot of connections are opened?
Could it be MySQL that the bottleneck (in top -u MySQL is going to 90  
- 110%)


Frank
>
> cug
>

> Could it be MySQL that the bottleneck (in top -u MySQL is going to 90
> - 110%)
I would definitely check how many/what kind of query you are  
attacking mysql with.  If I had to guess just offhand, I would guess  
maybe traversing a keypath on the other size of a to-many  
relationship and firing tons of faults by accident.  You should see  
this really quickly by turning on sql debug (it would be hundreds of  
insane one-off "where id = x" queries).  If that's the case, check  
out my wotips post about ERXRecursiveBatchFetching from a couple  
weeks ago for some tips on making that less horrible.

ms

On 17.04.2007, at 14:02, Frank Stock wrote:

> Could it be MySQL that the bottleneck (in top -u MySQL is going to  
> 90 - 110%)

Maybe. As others suggested: turn on SQL logging in the application  
and look at the queries. Then: MySQL is not famous for concurrent  
request handling but it should be able to handle 10 concurrent  
connections as long as it can execute the statements itself in a  
reasonable time.

cug

Le 07-04-17 =E0 16:02, Frank Stock a =E9crit :

> Could it be MySQL that the bottleneck (in top -u MySQL is going to 90
> - 110%)

You should use the slow queries feature of MySQL to check for... slow =20=

queries.  It will log long queries in a text log so that you can find =20=

the culprits.=

On Apr 17, 2007, at 12:52 PM, Guido Neitzer wrote:

> On 17.04.2007, at 13:27, Frank Stock wrote:
>
>> WOAllowsConcurrentRequestHandling=true
>
> And this is executed? I have:
>
> -WOAllowsConcurrentRequestHandling=true
> With the "-" in the beginning. I'm never sure what actually works
> with all the different kinds of Property styles.


I don't think that works.  The two formats that I know work are the  
WO Way:

-WOAllowsConcurrentRequestHandling YES

and the Java Way

-DWOAllowsConcurrentRequestHandling=true


> You can ask your Application class whether it's turned on. So, if
> concurrent requests are not the problem, perhaps concurrent database
> access might still be a problem.

It could also be the default isolation/locking levels of the JDBC  
connection.  FrontBase works a lot better if you don't use the defaults.

Chuck

-- 

Practical WebObjects - for developers who want to increase their  
overall knowledge of WebObjects or who are trying to solve specific  
problems.
http://www.global-village.net/products/practical_webobjects

Am 17.04.2007 um 22:57 schrieb Chuck Hill:

>
> I don't think that works.  The two formats that I know work are the
> WO Way:
>
> -WOAllowsConcurrentRequestHandling YES
>
> and the Java Way
>
> -DWOAllowsConcurrentRequestHandling=true

And the Java way when in a Properties file:

>> WOAllowsConcurrentRequestHandling=true