From: Damiano M. <dam...@at...> - 2008-07-09 14:31:50
|
Hi all, I'm experimenting webware for the first time to use it in my enterprise. I've developed an XML-RPC service in Python (obviously!) that connect to an Oracle db, make some queries and return the results. Under medium usage (5-10 requests every 5 minutes), the webware app server kill himself and output the following: /home/xxx/yyy/AppServer: line 27: 14440 Killed $PYTHON$PY_OPTS Launch.py ThreadedAppServer $* these are the last lines of the activity log: 92.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 2008,1.49,False 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 2008,2.35,False 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 2008,3.19,False 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 2008,4.06,False while no trace of this error is reported in the errors log. Am I missing something? Thanks for your help, -- Dott. Damiano Morosi Atac S.p.A. - Agenzia per la Mobilità del Comune di Roma Direzione Sistemi - Area Centrale della Mobilità Sviluppo funzioni sistemiche della Centrale 00157 Roma - Via Ostiense, 131/L Tel.: (+39) 0646959361 Mail: dam...@at... Skype: damiano.morosi Web: http://www.atac.roma.it |
From: Jason H. <ja...@pe...> - 2008-07-09 14:43:05
|
Hi Damiano, It appears that it could be a crash/segfault; I would recommend trying to determine whether that is the cause. In my experience using Webware (7 years), any crashes have been due to extension modules, and not to Webware itself. Make sure you are running current versions of extension modules, try to make some educated guesses about which module it could be, try to isolate and reproduce the problem if possible, check upstream bugreports, etc. For example, one tactic would be: * comment out usage of the oracle db, and "fake" the results from the XML-RPC request instead. * run the service under similar load and see if it still dies If it still dies, then you know the crash is not related to the oracle database. If it does not die, then the oracle extension module would become your lead suspect. Good luck! peace, Jason -- Jason Hildebrand PeaceWorks Computer Consulting #2 - 396 Assiniboine Ave, Winnipeg 204 480 0314 --or-- 519 725 7875, ext 620. ----- "Damiano Morosi" <dam...@at...> wrote: > Hi all, > > I'm experimenting webware for the first time to use it in my > enterprise. > I've developed an XML-RPC service in Python (obviously!) that connect > to > an Oracle db, make some queries and return the results. > Under medium usage (5-10 requests every 5 minutes), the webware app > server kill himself and output the following: > > /home/xxx/yyy/AppServer: line 27: 14440 Killed > $PYTHON$PY_OPTS Launch.py ThreadedAppServer $* > > these are the last lines of the activity log: > > 92.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,1.49,False > 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,2.35,False > 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,3.19,False > 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,4.06,False > > while no trace of this error is reported in the errors log. Am I > missing > something? Thanks for your help, > > -- > Dott. Damiano Morosi > Atac S.p.A. - Agenzia per la Mobilità del Comune di Roma > Direzione Sistemi - Area Centrale della Mobilità > Sviluppo funzioni sistemiche della Centrale > 00157 Roma - Via Ostiense, 131/L > Tel.: (+39) 0646959361 > Mail: dam...@at... > Skype: damiano.morosi > Web: http://www.atac.roma.it > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic > lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Webware-devel mailing list > Web...@li... > https://lists.sourceforge.net/lists/listinfo/webware-devel |
From: Ben P. <be...@pa...> - 2008-07-10 03:17:05
|
That looks like the os might have killed the process. Check your syslogs - did you exceed a memory limit? Damiano Morosi wrote on 7/9/08 7:31 AM: > Hi all, > > I'm experimenting webware for the first time to use it in my enterprise. > I've developed an XML-RPC service in Python (obviously!) that connect to > an Oracle db, make some queries and return the results. > Under medium usage (5-10 requests every 5 minutes), the webware app > server kill himself and output the following: > > /home/xxx/yyy/AppServer: line 27: 14440 Killed > $PYTHON$PY_OPTS Launch.py ThreadedAppServer $* > > these are the last lines of the activity log: > > 92.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,1.49,False > 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,2.35,False > 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,3.19,False > 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 > 2008,4.06,False > > while no trace of this error is reported in the errors log. Am I missing > something? Thanks for your help, > > |
From: Damiano M. <dam...@at...> - 2008-07-10 06:29:22
|
Hi, thanks for your suggestions. The only thing prensent in /var/log/messages are the following lines: Jul 9 15:25:14 fe-oracle VMS: failed Jul 9 15:25:14 fe-oracle VMS: succeeded Jul 9 15:28:53 fe-oracle VMS: failed Jul 9 15:28:53 fe-oracle VMS: succeeded Jul 9 15:59:47 fe-oracle VMS: failed Jul 9 15:59:47 fe-oracle VMS: succeeded and I don't know what they mean, seems like the service fail and then succeed each time. Now I'll try to comment out the oracle module and 'fake' the connection to db to determine if this is the cause. Thanks for your support, I'll keep you informed! Ben Parker ha scritto: > That looks like the os might have killed the process. Check your syslogs > - did you exceed a memory limit? > > Damiano Morosi wrote on 7/9/08 7:31 AM: >> Hi all, >> >> I'm experimenting webware for the first time to use it in my enterprise. >> I've developed an XML-RPC service in Python (obviously!) that connect to >> an Oracle db, make some queries and return the results. >> Under medium usage (5-10 requests every 5 minutes), the webware app >> server kill himself and output the following: >> >> /home/xxx/yyy/AppServer: line 27: 14440 Killed >> $PYTHON$PY_OPTS Launch.py ThreadedAppServer $* >> >> these are the last lines of the activity log: >> >> 92.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >> 2008,1.49,False >> 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >> 2008,2.35,False >> 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >> 2008,3.19,False >> 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >> 2008,4.06,False >> >> while no trace of this error is reported in the errors log. Am I missing >> something? Thanks for your help, >> >> > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Webware-devel mailing list > Web...@li... > https://lists.sourceforge.net/lists/listinfo/webware-devel > -- Dott. Damiano Morosi Atac S.p.A. - Agenzia per la Mobilità del Comune di Roma Direzione Sistemi - Area Centrale della Mobilità Sviluppo funzioni sistemiche della Centrale 00157 Roma - Via Ostiense, 131/L Tel.: (+39) 0646959361 Mail: dam...@at... Skype: damiano.morosi Web: http://www.atac.roma.it |
From: Damiano M. <dam...@at...> - 2008-07-10 11:25:37
|
Yes the cause is cx_Oracle. I have installed version 4.4 of the module, that is the last. the code of the method is very simple, and i report it as well: ### CODE START ### def getLoS(self, panel_id): try: connection = cx_Oracle.connect(self.oracle_vms_dsn) cursor = connection.cursor() sql = "select statodiag, diagnosticaled from pannelli where nome='" + panel_id +"'" cursor.execute(sql) r = cursor.fetchall() cursor.close() if r[0][0] >= 10000: return 0.0 else: tot = len(r[0][1]) ok = r[0][1].count('?') return float("%1.2f" % (float(ok)/float(tot))) except cx_Oracle.Error: raise Fault(802, 'Database error: ' + str(sys.exc_info( )[1])) except Exception, e: raise Fault(801, 'Exception: ' + e.args[0] ) ### CODE END ### Did you notice something strange in this? Am I makeing some mistakes? Thanks for your support. Damiano Morosi ha scritto: > Hi, > > thanks for your suggestions. The only thing prensent in > /var/log/messages are the following lines: > > Jul 9 15:25:14 fe-oracle VMS: failed > Jul 9 15:25:14 fe-oracle VMS: succeeded > Jul 9 15:28:53 fe-oracle VMS: failed > Jul 9 15:28:53 fe-oracle VMS: succeeded > Jul 9 15:59:47 fe-oracle VMS: failed > Jul 9 15:59:47 fe-oracle VMS: succeeded > > and I don't know what they mean, seems like the service fail and then > succeed each time. Now I'll try to comment out the oracle module and > 'fake' the connection to db to determine if this is the cause. > Thanks for your support, I'll keep you informed! > > Ben Parker ha scritto: >> That looks like the os might have killed the process. Check your syslogs >> - did you exceed a memory limit? >> >> Damiano Morosi wrote on 7/9/08 7:31 AM: >>> Hi all, >>> >>> I'm experimenting webware for the first time to use it in my enterprise. >>> I've developed an XML-RPC service in Python (obviously!) that connect to >>> an Oracle db, make some queries and return the results. >>> Under medium usage (5-10 requests every 5 minutes), the webware app >>> server kill himself and output the following: >>> >>> /home/xxx/yyy/AppServer: line 27: 14440 Killed >>> $PYTHON$PY_OPTS Launch.py ThreadedAppServer $* >>> >>> these are the last lines of the activity log: >>> >>> 92.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >>> 2008,1.49,False >>> 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >>> 2008,2.35,False >>> 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >>> 2008,3.19,False >>> 192.168.200.252,POST,/VMS/2/service.py,0,service,Wed Jul 9 16:10:54 >>> 2008,4.06,False >>> >>> while no trace of this error is reported in the errors log. Am I missing >>> something? Thanks for your help, >>> >>> >> ------------------------------------------------------------------------- >> Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! >> Studies have shown that voting for your favorite open source project, >> along with a healthy diet, reduces your potential for chronic lameness >> and boredom. Vote Now at http://www.sourceforge.net/community/cca08 >> _______________________________________________ >> Webware-devel mailing list >> Web...@li... >> https://lists.sourceforge.net/lists/listinfo/webware-devel >> > -- Dott. Damiano Morosi Atac S.p.A. - Agenzia per la Mobilità del Comune di Roma Direzione Sistemi - Area Centrale della Mobilità Sviluppo funzioni sistemiche della Centrale 00157 Roma - Via Ostiense, 131/L Tel.: (+39) 0646959361 Mail: dam...@at... Skype: damiano.morosi Web: http://www.atac.roma.it |
From: Christoph Z. <ci...@on...> - 2008-07-10 12:09:29
|
Damiano Morosi schrieb: > Did you notice something strange in this? Am I makeing some mistakes? > Thanks for your support. Nothing that should cause the process to be killed. But there are some things you could improve to make everything more stable and secure: * make the code safe against SQL injection (e.g. excute with params) * properly close connections and cursors in case of failure (use try/finally or the with statement if cx_oracle supports it) * use DBUtils or something similar for connection pooling -- Christoph |
From: Damiano M. <dam...@at...> - 2008-07-10 13:58:33
|
OK all done, my script now is absolutely better (thanks for the suggestions), but the problem persists... However, I have another "piece" of the puzzle; when the error occurs, the system says: *** glibc detected *** free(): invalid pointer: 0x089ebc70 *** maybe a memory leak problem? my configuration is python 2.3.4, GCC 3.4.6, kernel 2.6.9-67.0.20.EL. The dist is a CentOS, and the machine is virtualized under VMWare. Compatibility issues? Christoph Zwerschke ha scritto: > Damiano Morosi schrieb: >> Did you notice something strange in this? Am I makeing some mistakes? >> Thanks for your support. > > Nothing that should cause the process to be killed. But there are some > things you could improve to make everything more stable and secure: > > * make the code safe against SQL injection (e.g. excute with params) > * properly close connections and cursors in case of failure > (use try/finally or the with statement if cx_oracle supports it) > * use DBUtils or something similar for connection pooling > > -- Christoph > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Webware-devel mailing list > Web...@li... > https://lists.sourceforge.net/lists/listinfo/webware-devel > -- Dott. Damiano Morosi Atac S.p.A. - Agenzia per la Mobilità del Comune di Roma Direzione Sistemi - Area Centrale della Mobilità Sviluppo funzioni sistemiche della Centrale 00157 Roma - Via Ostiense, 131/L Tel.: (+39) 0646959361 Mail: dam...@at... Skype: damiano.morosi Web: http://www.atac.roma.it |
From: Christoph Z. <ci...@on...> - 2008-07-10 15:25:25
|
Damiano Morosi schrieb: > maybe a memory leak problem? my configuration is python 2.3.4, GCC > 3.4.6, kernel 2.6.9-67.0.20.EL. The dist is a CentOS, and the machine is > virtualized under VMWare. Compatibility issues? Yes, this looks like a memory allocation problem of cx_oracle; or there is an incompatibility with the Oracle or Python version you are using. I would try to reproduce the problem outside the context of Webware and ask for help on the cx_oracle mailing list. -- Christoph |
From: Damiano M. <dam...@at...> - 2008-07-11 06:34:19
|
SOLVED: there was a problem in threading. It's been necessary to specify the following in the connection string: connection = cx_Oracle.connect(oracle_dsn, threaded=True) Adding the threaded=True parameter, the service is up & running from 14 hours, against the 5 minutes before. This should be something important to say maybe also on the (excellent) WebWare site. Thank you all! Christoph Zwerschke ha scritto: > Damiano Morosi schrieb: >> maybe a memory leak problem? my configuration is python 2.3.4, GCC >> 3.4.6, kernel 2.6.9-67.0.20.EL. The dist is a CentOS, and the machine is >> virtualized under VMWare. Compatibility issues? > > Yes, this looks like a memory allocation problem of cx_oracle; or there > is an incompatibility with the Oracle or Python version you are using. > > I would try to reproduce the problem outside the context of Webware and > ask for help on the cx_oracle mailing list. > > -- Christoph > > ------------------------------------------------------------------------- > Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! > Studies have shown that voting for your favorite open source project, > along with a healthy diet, reduces your potential for chronic lameness > and boredom. Vote Now at http://www.sourceforge.net/community/cca08 > _______________________________________________ > Webware-devel mailing list > Web...@li... > https://lists.sourceforge.net/lists/listinfo/webware-devel > -- Dott. Damiano Morosi Atac S.p.A. - Agenzia per la Mobilità del Comune di Roma Direzione Sistemi - Area Centrale della Mobilità Sviluppo funzioni sistemiche della Centrale 00157 Roma - Via Ostiense, 131/L Tel.: (+39) 0646959361 Mail: dam...@at... Skype: damiano.morosi Web: http://www.atac.roma.it |