From: Andreas <and...@gm...> - 2008-03-03 13:58:38
|
Hello. Soap server in Yaws does not handle concurrent processes (request N+1 has to wait for request N to be finished). Non-concurrency doesn't play well with the web services I'm developing, so I've made a fix of my own in yaws_soap_srv. Does it break anything I'm not aware of? My thought process is that the gen_server does only keeps track of the started web services, and doesn't change the state after it is started. Therefor, the actual processing need not be done within the server, and the server is only used to return the server state to the processing code. While returning the state is a non-concurrent gen_server:call(), it is rather quick. /Andreas Hellström NEW CODE: get_state() -> gen_server:call(?SERVER, {get_state}). handle_call( {get_state}, _From, State) -> {reply, State, State}. CHANGED CODE: handler(Args, Id, Payload, SessionValue) -> Headers = Args#arg.headers, SoapAction = yaws_soap_lib:findHeader("SOAPAction", Headers#headers.other), %% Instead of sending the request to gen_server, get the current %% state from gen_server and do the processing outside of the server. State = get_state(), Reply = try case request(State, Id, Payload, SessionValue, SoapAction) of {'EXIT', _} -> cli_error("Model error."); Reply1 -> Reply1 end catch _:_ -> cli_error("Model error.") end, case Reply of %% <TELIA_FIX_END> {ok, XmlDoc, ResCode, undefined} -> {false, XmlDoc, ResCode}; {ok, XmlDoc, ResCode, SessVal} -> {true, 0, SessVal, XmlDoc, ResCode}; {error, _, _} = Error -> Error; false -> false % soap notify end. |
From: Claes W. <kl...@ta...> - 2008-03-03 14:27:17
|
Andreas wrote: > Hello. > > Soap server in Yaws does not handle concurrent processes (request N+1 > has to wait for request N to be finished). Non-concurrency doesn't > play well with the web services I'm developing, so I've made a fix of > my own in yaws_soap_srv. Does it break anything I'm not aware of? > > My thought process is that the gen_server does only keeps track of the > started web services, and doesn't change the state after it is > started. Therefor, the actual processing need not be done within the > server, and the server is only used to return the server state to the > processing code. While returning the state is a non-concurrent > gen_server:call(), it is rather quick. > > /Andreas Hellström > This will copy the entire result of yaws_soap_lib:initModel() for every request. Not good. It would be better to modify the yaws_soap_srv gen_server to have a dynamic set of worker processes. /klacke |
From: Claes W. <kl...@ta...> - 2008-03-03 19:18:26
|
Andreas wrote: > Before I try your suggestion, do you think this is a good idea? I CC the list - I think mor people are interested in my reply. > > - The main soap server inits the web service, starts N worker > processes and hand them the model, and exports a get_worker() function > that returns the pid to an available worker process. When running out > of free workers, another one is started. (With N=1 there is no need to > configure this parameter). > > - The worker process is a modified yaws_soap_srv, just dealing with > requests (and not doing any model init). > > - handler function (called by rpc) asks the main server for a pid, and > sends a gen_server:call() to that pid. Optimal solution is as follows. 1. Let the init/1 function craete a small nuber of workers, say 3. 2. Each worker is ready to receive similar messages as the gen_server is today. 3. Whenever the top proc gets new models they are sent down to all workers. The main idea here is that we want to avoid copying the model data for each req. 4. A request comes in to the top proc, it selects a worker and sends the req to the worker. 5. The worker does it's job and performs an explict reply through gen_server:reply() It needs the full set of params received by the top proc in handle_call() 6. Finally the worker notifies the top proc that it is ready to receive new jobs. If you do this - please make sure I get the result as either a proper patch or optionally a copy of the new yaws_soap_srv.erl file /klacke |
From: AndreasH71 <and...@te...> - 2013-05-03 08:16:36
|
Hi Klacke! This old issue was forgotten five years ago, but better late than never. We have been running an old version of Yaws (1.72) since 2008, with some custom made changes to make the soap server concurrent. A month ago we switched to 1.96 and implemented the changes in that code too. What is the process of adding changes to Yaws? I'm not that familiar with doing a 'proper patch', and would prefer to send the modified modules to someone in the know. In summary, the changes are: - 3 files for the concurrent soap server (yaws_soap_srv + 2 new files) - 3 files for adding a soap_workers gconf param. - 1 file for a bug fix in yaws_stats. Also, we've had an issue where we had to to encode soap data with utf-8 encoding, but I'm not sure how well that change will fit into Yaws? How do I proceed with this? Andreas Claes Wikstrom wrote: > > Andreas wrote: >> Before I try your suggestion, do you think this is a good idea? > > I CC the list - I think mor people are interested in my reply. > >> >> - The main soap server inits the web service, starts N worker >> processes and hand them the model, and exports a get_worker() function >> that returns the pid to an available worker process. When running out >> of free workers, another one is started. (With N=1 there is no need to >> configure this parameter). >> >> - The worker process is a modified yaws_soap_srv, just dealing with >> requests (and not doing any model init). >> >> - handler function (called by rpc) asks the main server for a pid, and >> sends a gen_server:call() to that pid. > > > Optimal solution is as follows. > > 1. Let the init/1 function craete a small nuber of workers, say 3. > > 2. Each worker is ready to receive similar messages as the gen_server > is today. > > 3. Whenever the top proc gets new models they are sent down to all > workers. The main idea here is that we want to avoid copying the > model data for each req. > > 4. A request comes in to the top proc, it selects a worker and sends > the req to the worker. > > 5. The worker does it's job and performs an explict reply through > gen_server:reply() It needs the full set of params received by the > top proc in handle_call() > > 6. Finally the worker notifies the top proc that it is ready to > receive new jobs. > > > If you do this - please make sure I get the result as either > a proper patch or optionally a copy of the new yaws_soap_srv.erl file > > > > /klacke > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Erlyaws-list mailing list > Erl...@li... > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > -- View this message in context: http://old.nabble.com/Soap-server-not-concurrent-tp15804462p35360564.html Sent from the Yaws mailing list archive at Nabble.com. |
From: AndreasH71 <and...@te...> - 2013-05-03 08:15:28
|
Hi Klacke! This old issue was forgotten five years ago, but better late than never. We have been running an old version of Yaws (1.72) since 2008, with some custom made changes to make the soap server concurrent. A month ago we switched to 1.96 and implemented the changes in that code too. What is the process of adding changes to Yaws? I'm not that familiar with doing a 'proper patch', and would prefer to send the modified modules to someone in the know. In summary, the changes are: - 3 files for the concurrent soap server (yaws_soap_srv + 2 new files) - 3 files for adding a soap_workers gconf param. - 1 file for a bug fix in yaws_stats. Also, we've had an issue where we had to to encode soap data with utf-8 encoding, but I'm not sure how well that change will fit into Yaws? How do I proceed with this? Andreas Claes Wikstrom wrote: > > Andreas wrote: >> Before I try your suggestion, do you think this is a good idea? > > I CC the list - I think mor people are interested in my reply. > >> >> - The main soap server inits the web service, starts N worker >> processes and hand them the model, and exports a get_worker() function >> that returns the pid to an available worker process. When running out >> of free workers, another one is started. (With N=1 there is no need to >> configure this parameter). >> >> - The worker process is a modified yaws_soap_srv, just dealing with >> requests (and not doing any model init). >> >> - handler function (called by rpc) asks the main server for a pid, and >> sends a gen_server:call() to that pid. > > > Optimal solution is as follows. > > 1. Let the init/1 function craete a small nuber of workers, say 3. > > 2. Each worker is ready to receive similar messages as the gen_server > is today. > > 3. Whenever the top proc gets new models they are sent down to all > workers. The main idea here is that we want to avoid copying the > model data for each req. > > 4. A request comes in to the top proc, it selects a worker and sends > the req to the worker. > > 5. The worker does it's job and performs an explict reply through > gen_server:reply() It needs the full set of params received by the > top proc in handle_call() > > 6. Finally the worker notifies the top proc that it is ready to > receive new jobs. > > > If you do this - please make sure I get the result as either > a proper patch or optionally a copy of the new yaws_soap_srv.erl file > > > > /klacke > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Erlyaws-list mailing list > Erl...@li... > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > -- View this message in context: http://old.nabble.com/Soap-server-not-concurrent-tp15804462p35360560.html Sent from the Yaws mailing list archive at Nabble.com. |
From: Steve V. <vi...@ie...> - 2013-05-03 12:43:54
|
Hi Andreas, You can send me the diffs, I'll take a look at them. --steve On Fri, May 3, 2013 at 4:15 AM, AndreasH71 < and...@te...> wrote: > > Hi Klacke! > > This old issue was forgotten five years ago, but better late than never. We > have been running an old version of Yaws (1.72) since 2008, with some > custom > made changes to make the soap server concurrent. A month ago we switched to > 1.96 and implemented the changes in that code too. > > What is the process of adding changes to Yaws? I'm not that familiar with > doing a 'proper patch', and would prefer to send the modified modules to > someone in the know. > > In summary, the changes are: > > - 3 files for the concurrent soap server (yaws_soap_srv + 2 new files) > - 3 files for adding a soap_workers gconf param. > - 1 file for a bug fix in yaws_stats. > > Also, we've had an issue where we had to to encode soap data with utf-8 > encoding, but I'm not sure how well that change will fit into Yaws? > > How do I proceed with this? > > Andreas > > > > Claes Wikstrom wrote: > > > > Andreas wrote: > >> Before I try your suggestion, do you think this is a good idea? > > > > I CC the list - I think mor people are interested in my reply. > > > >> > >> - The main soap server inits the web service, starts N worker > >> processes and hand them the model, and exports a get_worker() function > >> that returns the pid to an available worker process. When running out > >> of free workers, another one is started. (With N=1 there is no need to > >> configure this parameter). > >> > >> - The worker process is a modified yaws_soap_srv, just dealing with > >> requests (and not doing any model init). > >> > >> - handler function (called by rpc) asks the main server for a pid, and > >> sends a gen_server:call() to that pid. > > > > > > Optimal solution is as follows. > > > > 1. Let the init/1 function craete a small nuber of workers, say 3. > > > > 2. Each worker is ready to receive similar messages as the gen_server > > is today. > > > > 3. Whenever the top proc gets new models they are sent down to all > > workers. The main idea here is that we want to avoid copying the > > model data for each req. > > > > 4. A request comes in to the top proc, it selects a worker and sends > > the req to the worker. > > > > 5. The worker does it's job and performs an explict reply through > > gen_server:reply() It needs the full set of params received by the > > top proc in handle_call() > > > > 6. Finally the worker notifies the top proc that it is ready to > > receive new jobs. > > > > > > If you do this - please make sure I get the result as either > > a proper patch or optionally a copy of the new yaws_soap_srv.erl file > > > > > > > > /klacke > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Microsoft > > Defy all challenges. Microsoft(R) Visual Studio 2008. > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > _______________________________________________ > > Erlyaws-list mailing list > > Erl...@li... > > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > > > > > > -- > View this message in context: > http://old.nabble.com/Soap-server-not-concurrent-tp15804462p35360560.html > Sent from the Yaws mailing list archive at Nabble.com. > > > > ------------------------------------------------------------------------------ > Get 100% visibility into Java/.NET code with AppDynamics Lite > It's a free troubleshooting tool designed for production > Get down to code-level detail for bottlenecks, with <2% overhead. > Download for free and get started troubleshooting in minutes. > http://p.sf.net/sfu/appdyn_d2d_ap2 > _______________________________________________ > Erlyaws-list mailing list > Erl...@li... > https://lists.sourceforge.net/lists/listinfo/erlyaws-list > |