From: Christopher P. L. <cl...@ma...> - 2004-01-28 15:52:33
|
Fair enough, I'm running: eXist 0.9.2 under Tomcat 5.0.16. The clients are Coldfusion apps talking via SOAP. When you run SOAP you have to request a session ID (this may be true for embedded as well, I'm not sure). I generally request a single ID and run multiple queries under that ID. What I've found is that doing all my work under a single ID results in processor utilization that never is greater than 50%. I've also found that if I split the work up under two ID's, I can max the processor out at 100% (which is what I want). The same is true if I run a large XPath via the query interface and I'm running the app at the same time, I can peg the processor at 100%. I realize this is a big question due to the fact that I'm using Tomcat to run eXist. As I understand it there are many subcomponents of eXist that could be the issue. Or it may be Tomcat...or a combination of the two. Thanks for your time. Best, Chris -----Original Message----- From: Christofer Dutz [mailto:du...@c-...] Sent: Tuesday, January 27, 2004 6:18 PM To: Christopher P. Lyman Subject: Re: [Exist-open] eXist Performance As far as I understand eXist there is no fine granular locking mechanism. Normaly this should only effect writes (stroing and updates). So if one process is updating all others may have to wait and don't do any CPU-time consuming stuff. By the way, what are you talking about when you are talking about "Sessions"? If you were a little more specific with your scenario, mabe you could get more detailed help. Chris Christopher P. Lyman wrote: > Hello All, > > I've got an issue that I could use some help on. I have several > processes that make intensive uses of eXist, either by storing files, > XUpdates, or XPath queries and reads. What I've noticed is that > processor utilization never goes above 50% for a single session. I > can start a process under a different session and then I can max out > the processor, but this introduces the overhead of establishing > multiple sessions. (By the way, I'm using SOAP) > > My goal would be to use a single session ID and to max out the > processors. Does anybody know where the throttling is happening and > if there's a way to disable it? Failing that, does anybody know if > this is a Java issue or is it something specific to one of the > components of eXist. Any help would be appreciated. > > Best Regards, > Chris |
From: <sb...@we...> - 2004-01-31 09:29:01
|
eXist synchronizes database access on the level of basic operations, e.g. storing a single node, reading an index entry, removing a node. Multi-threaded access is controlled by a pool of NativeBroker instances, which deal with the actual synchronization. The internal API requires every thread to first obtain an instance of NativeBroker and to release it after the operation has finished. If you look at the SOAP interface, every method invocation gets its own NativeBroker instance. Thus, two calls to the SOAP interface with the same session id may be processed by different NativeBroker instances. The SOAP session id just stores the current user and previous result sets. Concurrent access should be possible without restrictions. I thus seems that either the SOAP provider (Axis) or the servlet engine restrict concurrent access. I don't know enough about Axis to understand how it deals with multi-threading. Maybe someone else can help with this? Wolfgang > Fair enough, I'm running: > > eXist 0.9.2 under Tomcat 5.0.16. > The clients are Coldfusion apps talking via SOAP. > > When you run SOAP you have to request a session ID (this may be true for > embedded as well, I'm not sure). I generally request a single ID and run > multiple queries under that ID. What I've found is that doing all my work > under a single ID results in processor utilization that never is greater > than 50%. I've also found that if I split the work up under two ID's, I can > max the processor out at 100% (which is what I want). The same is true if I > run a large XPath via the query interface and I'm running the app at the > same time, I can peg the processor at 100%. > > I realize this is a big question due to the fact that I'm using Tomcat to > run eXist. As I understand it there are many subcomponents of eXist that > could be the issue. Or it may be Tomcat...or a combination of the two. > > Thanks for your time. > > Best, > Chris > > -----Original Message----- > From: Christofer Dutz [mailto:du...@c-...] > Sent: Tuesday, January 27, 2004 6:18 PM > To: Christopher P. Lyman > Subject: Re: [Exist-open] eXist Performance > > > As far as I understand eXist there is no fine granular locking > mechanism. Normaly this should only effect writes (stroing and updates). > So if one process is updating all others may have to wait and don't do > any CPU-time consuming stuff. > > By the way, what are you talking about when you are talking about > "Sessions"? If you were a little more specific with your scenario, mabe > you could get more detailed help. > > Chris > > Christopher P. Lyman wrote: > > > Hello All, > > > > I've got an issue that I could use some help on. I have several > > processes that make intensive uses of eXist, either by storing files, > > XUpdates, or XPath queries and reads. What I've noticed is that > > processor utilization never goes above 50% for a single session. I > > can start a process under a different session and then I can max out > > the processor, but this introduces the overhead of establishing > > multiple sessions. (By the way, I'm using SOAP) > > > > My goal would be to use a single session ID and to max out the > > processors. Does anybody know where the throttling is happening and > > if there's a way to disable it? Failing that, does anybody know if > > this is a Java issue or is it something specific to one of the > > components of eXist. Any help would be appreciated. > > > > Best Regards, > > Chris > > > > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Exist-open mailing list > Exi...@li... > https://lists.sourceforge.net/lists/listinfo/exist-open ______________________________________________________________________________ Erdbeben im Iran: Zehntausende Kinder brauchen Hilfe. UNICEF hilft den Kindern - helfen Sie mit! https://www.unicef.de/spe/spe_03.php |
From: Christofer D. <du...@c-...> - 2004-01-28 16:13:48
|
Christopher P. Lyman wrote: >Fair enough, I'm running: > >eXist 0.9.2 under Tomcat 5.0.16. >The clients are Coldfusion apps talking via SOAP. > >When you run SOAP you have to request a session ID (this may be true for >embedded as well, I'm not sure). I generally request a single ID and run >multiple queries under that ID. What I've found is that doing all my work >under a single ID results in processor utilization that never is greater >than 50%. I've also found that if I split the work up under two ID's, I can >max the processor out at 100% (which is what I want). The same is true if I >run a large XPath via the query interface and I'm running the app at the >same time, I can peg the processor at 100%. > >I realize this is a big question due to the fact that I'm using Tomcat to >run eXist. As I understand it there are many subcomponents of eXist that >could be the issue. Or it may be Tomcat...or a combination of the two. > >Thanks for your time. > >Best, >Chris > >-----Original Message----- >From: Christofer Dutz [mailto:du...@c-...] >Sent: Tuesday, January 27, 2004 6:18 PM >To: Christopher P. Lyman >Subject: Re: [Exist-open] eXist Performance > > >As far as I understand eXist there is no fine granular locking >mechanism. Normaly this should only effect writes (stroing and updates). >So if one process is updating all others may have to wait and don't do >any CPU-time consuming stuff. > >By the way, what are you talking about when you are talking about >"Sessions"? If you were a little more specific with your scenario, mabe >you could get more detailed help. > >Chris > >Christopher P. Lyman wrote: > > > >>Hello All, >> >> I've got an issue that I could use some help on. I have several >>processes that make intensive uses of eXist, either by storing files, >>XUpdates, or XPath queries and reads. What I've noticed is that >>processor utilization never goes above 50% for a single session. I >>can start a process under a different session and then I can max out >>the processor, but this introduces the overhead of establishing >>multiple sessions. (By the way, I'm using SOAP) >> >> My goal would be to use a single session ID and to max out the >>processors. Does anybody know where the throttling is happening and >>if there's a way to disable it? Failing that, does anybody know if >>this is a Java issue or is it something specific to one of the >>components of eXist. Any help would be appreciated. >> >> Have you run a profiling tool yet? Why dont you profile your application (Tomcat ith embeded eXist) and have a look at the results. Here you could see where requests are suspended (which will certainly be the cuase of your bad CPU utilisation). If you want you can just zipup your application with all examples I need and I could do the profiling for you. Chris PS: Did I send my last reply only to you ? I never received my own post ;} >> >>Best Regards, >>Chris >> >> > > > > |
From: Wolfgang M. <me...@if...> - 2004-01-31 00:55:37
|
eXist synchronizes database access on the level of basic operations, e.g. storing a single node, reading an index entry, removing a node. Multi-threaded access is controlled by a pool of NativeBroker instances, which deal with the actual synchronization. The internal API requires every thread to first obtain an instance of NativeBroker and to release it after the operation has finished. If you look at the SOAP interface, every method invocation gets its own NativeBroker instance. Thus, two calls to the SOAP interface with the same session id may be processed by different NativeBroker instances. The SOAP session id just stores the current user and previous result sets. Concurrent access should be possible without restrictions. I thus seems that either the SOAP provider (Axis) or the servlet engine restrict concurrent access. I don't know enough about Axis to understand how it deals with multi-threading. Maybe someone else can help with this? Wolfgang On Wednesday 28 January 2004 16:18, Christopher P. Lyman wrote: > Fair enough, I'm running: > > eXist 0.9.2 under Tomcat 5.0.16. > The clients are Coldfusion apps talking via SOAP. > > When you run SOAP you have to request a session ID (this may be true for > embedded as well, I'm not sure). I generally request a single ID and run > multiple queries under that ID. What I've found is that doing all my work > under a single ID results in processor utilization that never is greater > than 50%. I've also found that if I split the work up under two ID's, I > can max the processor out at 100% (which is what I want). The same is true > if I run a large XPath via the query interface and I'm running the app at > the same time, I can peg the processor at 100%. > > I realize this is a big question due to the fact that I'm using Tomcat to > run eXist. As I understand it there are many subcomponents of eXist that > could be the issue. Or it may be Tomcat...or a combination of the two. > > Thanks for your time. > > Best, > Chris > > -----Original Message----- > From: Christofer Dutz [mailto:du...@c-...] > Sent: Tuesday, January 27, 2004 6:18 PM > To: Christopher P. Lyman > Subject: Re: [Exist-open] eXist Performance > > > As far as I understand eXist there is no fine granular locking > mechanism. Normaly this should only effect writes (stroing and updates). > So if one process is updating all others may have to wait and don't do > any CPU-time consuming stuff. > > By the way, what are you talking about when you are talking about > "Sessions"? If you were a little more specific with your scenario, mabe > you could get more detailed help. > > Chris > > Christopher P. Lyman wrote: > > Hello All, > > > > I've got an issue that I could use some help on. I have several > > processes that make intensive uses of eXist, either by storing files, > > XUpdates, or XPath queries and reads. What I've noticed is that > > processor utilization never goes above 50% for a single session. I > > can start a process under a different session and then I can max out > > the processor, but this introduces the overhead of establishing > > multiple sessions. (By the way, I'm using SOAP) > > > > My goal would be to use a single session ID and to max out the > > processors. Does anybody know where the throttling is happening and > > if there's a way to disable it? Failing that, does anybody know if > > this is a Java issue or is it something specific to one of the > > components of eXist. Any help would be appreciated. > > > > Best Regards, > > Chris > > ------------------------------------------------------- > The SF.Net email is sponsored by EclipseCon 2004 > Premiere Conference on Open Tools Development and Integration > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA. > http://www.eclipsecon.org/osdn > _______________________________________________ > Exist-open mailing list > Exi...@li... > https://lists.sourceforge.net/lists/listinfo/exist-open -- Wolfgang Meier Darmstadt University of Technology Tel.: +49 (0) 06151 16 4458 ICQ: #296595621 http://exist-db.org http://www.sozionet.org |