Thread: [Myghty-users] A more optimized setup...
Brought to you by:
zzzeek
From: Jens H. <jen...@gm...> - 2009-12-20 23:58:37
|
Hi all, I'm not even sure if this mailinglist is still active, I just hope for the best. We have running a (legacy) Mygthy application, which we are unable (and unwilling atm ;) ) to port to something newer, like a WSGI framework or mako. Back when we started deploying the application, we have decided for a mod_python based Apache setup. Unfortunately, this setup is now creating problems, as the mod_python adds tons of memory usage to the Apache processes, which is starting to become a real problem. The background for this: We are using the Myghty application as some sort of management application, where users can edit their data, trigger certain events and so on. The main display is done on a TYPO3 site on the same server, which serves as the "viewing" for the website. Both applications are (unfortunately, due to some bad design decisions in the beginning) coupled rather tightly, so they have to be on the same physical host. So we are basically running the high volume typo3 page on the same Apache as the Myghty application. Observing the Apache processes gives me a rather large memory consumption for each process (around 500 MB), where the PHP application is hosted in fcgi processes, so we are only seeing basic Apache memory and mod_python there (no mod_php). My guess is that at some point the server starts swapping, and from there everything spirals downward. I have seen load values > 150 on that server......and not only once ;) I have tried to find some information if I can put the Myghty application into the same for of fcgi container, the same as the PHP processes, which should lighten the memory usage on the Apache considerably. Unfortunately, the only available documentation points toward mod_python and cgi (a route I'd rather not take ;) ), and the link to a FCGI setup with lighttp is broken. Could somebody point me in the right direction where to look, or maybe give some general hints where to start optimizing? Thanks in advance, Jens |
From: Steve S. (listsin) <li...@in...> - 2009-12-21 02:06:13
|
On Dec 20, 2009, at 6:58 PM, Jens Hoffrichter wrote: > The main display is done on a TYPO3site on the same server, which serves as the "viewing" for the > website. Both applications are (unfortunately, due to some bad design > decisions in the beginning) coupled rather tightly, so they have to be > on the same physical host. So we are basically running the high volume > typo3 page on the same Apache as the Myghty application. This would seem to be the simplest place to "put the screw" as it were. You're going to have to separate them at some point anyway and, once you start, you may find that the bulk of what you're doing in one or the other can be separated more easily than you think they can right now. Probably the best thing to do is bring in a fresh pair of eyes who wasn't there for the initial design and let them see how "tightly coupled" they really are. Once you get the two apps separated, to whatever degree, you can start pinpointing where you can most easily offload to a second, or possibly third server/VM pretty easily id think... S aka; Steve Steiner aka: sst...@gm... |
From: Michael B. <mi...@zz...> - 2009-12-21 02:19:52
|
On Dec 20, 2009, at 6:58 PM, Jens Hoffrichter wrote: > Hi all, > > I'm not even sure if this mailinglist is still active, I just hope for the best. well no, its not really active. But what I'd do in this case is try to use mod_wsgi in conjunction with the WSGI handler. mod_wsgi supports spawning the Python interpreter into a child process, so that your parent Apache can remain light on memory and still be efficient for serving static content. But at the same time, Myghty is really old and never got that far on production systems, so there's also the possibility that its a memory hog. Since we moved the cache/session mechanism of Myghty to Beaker, we've made tons of improvements and fixes in Beaker over the years, none of which are present in Myghty. But at least moving to mod_wsgi should help. > > We have running a (legacy) Mygthy application, which we are unable > (and unwilling atm ;) ) to port to something newer, like a WSGI > framework or mako. > > Back when we started deploying the application, we have decided for a > mod_python based Apache setup. Unfortunately, this setup is now > creating problems, as the mod_python adds tons of memory usage to the > Apache processes, which is starting to become a real problem. > > The background for this: We are using the Myghty application as some > sort of management application, where users can edit their data, > trigger certain events and so on. The main display is done on a TYPO3 > site on the same server, which serves as the "viewing" for the > website. Both applications are (unfortunately, due to some bad design > decisions in the beginning) coupled rather tightly, so they have to be > on the same physical host. So we are basically running the high volume > typo3 page on the same Apache as the Myghty application. > > Observing the Apache processes gives me a rather large memory > consumption for each process (around 500 MB), where the PHP > application is hosted in fcgi processes, so we are only seeing basic > Apache memory and mod_python there (no mod_php). My guess is that at > some point the server starts swapping, and from there everything > spirals downward. I have seen load values > 150 on that > server......and not only once ;) > > I have tried to find some information if I can put the Myghty > application into the same for of fcgi container, the same as the PHP > processes, which should lighten the memory usage on the Apache > considerably. Unfortunately, the only available documentation points > toward mod_python and cgi (a route I'd rather not take ;) ), and the > link to a FCGI setup with lighttp is broken. > > Could somebody point me in the right direction where to look, or maybe > give some general hints where to start optimizing? > > Thanks in advance, > Jens > > ------------------------------------------------------------------------------ > This SF.Net email is sponsored by the Verizon Developer Community > Take advantage of Verizon's best-in-class app development support > A streamlined, 14 day to market process makes app distribution fast and easy > Join now and get one step closer to millions of Verizon customers > http://p.sf.net/sfu/verizon-dev2dev > _______________________________________________ > Myghty-users mailing list > Myg...@li... > https://lists.sourceforge.net/lists/listinfo/myghty-users |
From: Jens H. <jen...@gm...> - 2009-12-21 11:34:44
|
Hi Michael, 2009/12/21 Michael Bayer <mi...@zz...>: >> I'm not even sure if this mailinglist is still active, I just hope for the best. > > well no, its not really active. But what I'd do in this case is try to use mod_wsgi in conjunction with the WSGI handler. mod_wsgi supports spawning the Python interpreter into a child process, so that your parent Apache can remain light on memory and still be efficient for serving static content. mod_wsgi was kinda the way I already thought, as we are using that for a couple of pylons applications already. I just didn't find any examples on how to integrate just myghty into a mod_wsgi - or maybe I just didn't look in the right places. Any hints there? > > But at the same time, Myghty is really old and never got that far on production systems, so there's also the possibility that its a memory hog. Since we moved the cache/session mechanism of Myghty to Beaker, we've made tons of improvements and fixes in Beaker over the years, none of which are present in Myghty. But at least moving to mod_wsgi should help. Yes, I know - this app has been running for 4 years, and I remember writing with you about some caching problems - this was before mako was released, and you pointed me to pylons (which we are happily using since then ;) ) The problem is that a lot of (paid) work went into that app, and it would require a lot of (unpaid) work to port it to one of the newer frameworks - as we used a home-made ORM, form libraries, the whole works. So we are kinda stuck with myghty at the moment :) Thanks already for you help :) Regards, Jens |