From: Greg W. <gwa...@py...> - 2006-06-23 12:54:34
|
Back in May 2002, Ralph Heinkel reported a mysterious error, "This routine cannot be called because another command structure has results pending". Here's his post: http://www.object-craft.com.au/pipermail/python-sybase/2002-May/000034.html He posted a test program that reproduces the problem, and it still reproduces perfectly for me. It appears that any error on a connection renders that connection unusable, or at least un-commit()able and un-rollback()able. I'm connecting to Sybase ASE 11.0.3.3 (using libct.so and libcs.so supplied with the Sybase engine) and python-sybase 0.37. Here's my slight variation on Ralph's test program: -- syberror ------------------------------------------------------------ #!/usr/bin/python # Attempt to reproduce the mysterious "This routine cannot be called # because another command structure has results pending." error. # # Based on code seen in this mailing list post: # http://www.object-craft.com.au/pipermail/python-sybase/2002-May/000034.html import sys import Sybase Sybase._ctx.debug = 1 args = sys.argv[1:] if len(args) != 3: sys.exit("usage: syberror server username password") (server, username, password) = args db = Sybase.connect(server, username, password) print "successfully connected to %s" % server cur = db.cursor() try: # produce an error - THE TABLE 'DUMMY' DOES NOT EXIST cur.execute('select * from dummy') print "WTF?!? select should have failed!" except Sybase.Error, err: print "select failed as expected: %s" % err cur.close() db.commit() ------------------------------------------------------------------------ The most important change is that I added "_ctx.debug = 1", as requested by Dave Cole in a followup. And here is the output I get from running this program (username and password censored): ------------------------------------------------------------------------ ct_con_alloc(ctx0, &conn) -> CS_SUCCEED, conn0 ct_con_props(conn0, CS_SET, CS_USERNAME, "***", CS_NULLTERM, NULL) -> CS_SUCCEEDct_con_props(conn0, CS_SET, CS_PASSWORD, "********", CS_NULLTERM, NULL) -> CS_SUCCEED servermsg_cb ct_connect(conn0, "MIRA321", CS_NULLTERM) -> CS_SUCCEED ct_options(conn0, CS_SET, CS_OPT_CHAINXACTS, 1, CS_UNUSED, NULL) -> CS_SUCCEED successfully connected to MIRA321 Cursor.execute _lock: count -> 1 ct_cmd_alloc(conn0, &cmd) -> CS_SUCCEED, cmd0 _lock: count -> 2 _set_state: _LAZY_IDLE _unlock: count -> 1 ct_command(cmd0, CS_LANG_CMD, "select * from dummy", CS_NULLTERM, CS_UNUSED) -> CS_SUCCEED _set_state: _LAZY_FETCHING ct_send(cmd0) -> CS_SUCCEED _start_results servermsg_cb Msg 156, Level 15, State 1, Line 1 Incorrect syntax near the keyword 'dummy'. ct_results(cmd0, &result) -> CS_SUCCEED, CS_CMD_FAIL select failed as expected: Msg 156, Level 15, State 1, Line 1 Incorrect syntax near the keyword 'dummy'. ct_cmd_alloc(conn0, &cmd) -> CS_SUCCEED, cmd1 ct_command(cmd1, CS_LANG_CMD, "commit transaction", CS_NULLTERM, CS_UNUSED) -> CS_SUCCEED clientmsg_cb Layer: 1, Origin: 1 ct_send(): user api layer: external error: This routine cannot be called because another command structure has results pending.ct_send(cmd1) -> CS_FAIL ct_cancel(conn0, NULL, CS_CANCEL_ALL) -> CS_SUCCEED _unlock: count -> 0 ct_cmd_drop(cmd0) -> CS_SUCCEED Traceback (most recent call last): File "./syberror", line 30, in ? db.commit() File "/usr/local/Intelerad/3rd_Party/python/lib/python2.3/site-packages/Sybase.py", line 971, in commit self.execute('commit transaction') File "/usr/local/Intelerad/3rd_Party/python/lib/python2.3/site-packages/Sybase.py", line 996, in execute fetcher.start(self.arraysize) File "/usr/local/Intelerad/3rd_Party/python/lib/python2.3/site-packages/Sybase.py", line 307, in start status = self._cmd.ct_send() File "/usr/local/Intelerad/3rd_Party/python/lib/python2.3/site-packages/Sybase.py", line 184, in _clientmsg_cb raise DatabaseError(_fmt_client(msg)) Sybase.DatabaseError: Layer: 1, Origin: 1 ct_send(): user api layer: external error: This routine cannot be called because another command structure has results pending. ct_cmd_drop(cmd1) -> CS_SUCCEED ct_con_props(conn0, CS_GET, CS_CON_STATUS, &value, CS_UNUSED, NULL) -> CS_SUCCEED, CS_CONSTAT_CONNECTED ct_close(conn0, CS_FORCE_CLOSE) -> CS_SUCCEED ct_con_drop(conn0) -> CS_SUCCEED ------------------------------------------------------------------------ Does anyone understand what's going on here? For the longest time, I have assumed this was a subtle bug in my code, but now that I have this simple test script I really doubt that. How can one error make it impossible to commit() or rollback() the current transaction? (Same result if the last line is db.rollback(), by the way.) Is this a bug in python-sybase? or in ctlib? or elsewhere? Thanks -- Greg |
From: Greg W. <gwa...@py...> - 2006-07-06 10:02:56
|
On 22 June 2006, I said: > Back in May 2002, Ralph Heinkel reported a mysterious error, "This > routine cannot be called because another command structure has results > pending". Here's his post: > > http://www.object-craft.com.au/pipermail/python-sybase/2002-May/000034.html > > He posted a test program that reproduces the problem, and it still > reproduces perfectly for me. It appears that any error on a connection > renders that connection unusable, or at least un-commit()able and > un-rollback()able. I'm connecting to Sybase ASE 11.0.3.3 (using > libct.so and libcs.so supplied with the Sybase engine) and python-sybase > 0.37. I've done a bit more digging into this problem and discovered a crude workaround. All I have to do is add one line of code after my failed query but before closing the cursor: cur = db.cursor() try: # produce an error - THE TABLE 'DUMMY' DOES NOT EXIST cur.execute('select * from dummy') print "WTF?!? select should have failed!" except Sybase.Error, err: print "select failed as expected: %s" % err cur._fetcher._close() <--- CRUDE WORKAROUND cur.close() db.commit() That suggests to me that the bug lies in python-sybase rather than in my code or in ctlib. I think the bug is that Sybase.py assumes that self._fetcher = None is sufficient to close the fetcher object and release all of its resources, most importantly by calling ct_cancel() on the underlying ctlib command structure. Here's a narrow-minded patch that fixes the particular problem I'm seeing: --- Sybase.py.orig 2005-04-06 18:46:43.000000000 -0400 +++ Sybase.py.hacked 2006-07-05 12:41:55.000000000 -0400 @@ -735,10 +735,12 @@ def close(self): '''DB-API Cursor.close() ''' if self._closed: raise ProgrammingError('cursor is closed') + if self._fetcher: + self._fetcher._close() self._fetcher = None self._closed = 1 def execute(self, sql, params = {}): '''DB-API Cursor.execute() However, I suspect a more appropriate fix would be to factor out a _close_fetcher() method of Cursor: def _close_fetcher(self): if self._fetcher: self._fetcher._close() self._fetcher = None and replace every occurence of "self._fetcher = None" with "self._close_fetcher()". Does this sound close? If so, I'll happily provide a patch. Greg |
From: Greg W. <gwa...@py...> - 2006-07-06 14:23:25
Attachments:
python-sybase-close-fetcher.patch
|
On 05 July 2006, I said: > However, I suspect a more appropriate fix would be to factor out a > _close_fetcher() method of Cursor: > > def _close_fetcher(self): > if self._fetcher: > self._fetcher._close() > self._fetcher = None > > and replace every occurence of "self._fetcher = None" with > "self._close_fetcher()". I have done this and it seems to work. I'm certainly not getting any more mysterious "this routine cannot be called" errors from my Python/sybase scripts, which makes me very happy indeed. I'll attach my patch. Does this look correct? Is it likely to be included in a future release of python-sybase? Do you think I'll get burned if I deploy a version of python-sybase with this patch applied? Greg |
From: Andrew M. <an...@ob...> - 2006-07-06 19:38:59
|
>> However, I suspect a more appropriate fix would be to factor out a >> _close_fetcher() method of Cursor: >> >> def _close_fetcher(self): >> if self._fetcher: >> self._fetcher._close() >> self._fetcher = None >> >> and replace every occurence of "self._fetcher = None" with >> "self._close_fetcher()". > >I have done this and it seems to work. I'm certainly not getting any >more mysterious "this routine cannot be called" errors from my >Python/sybase scripts, which makes me very happy indeed. > >I'll attach my patch. Does this look correct? Is it likely to be >included in a future release of python-sybase? Do you think I'll get >burned if I deploy a version of python-sybase with this patch applied? Dave's tied up with other stuff at the moment, but I've run this by him, and he says your patch looks good. As to whether it make it into a future release, I can certainly apply it to the repository, but neither Dave nor I have a Sybase installation to test it against at the moment (and neither of us can really spare the time). That raises the question - should the project be moved to Sourceforge? I'm not convinced the move would solve anything: it would still require at least one developer with C skills and the time to review and test patches, cut releases, and reply to queries - is there anyone on the list willing to commit to that? -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ |
From: <sab...@gm...> - 2006-12-05 06:22:52
Attachments:
sybase-0.37ssa.diff
|
Hi Andrew, 2006/7/6, Andrew McNamara <an...@ob...>: > ... > As to whether it make it into a future release, I can certainly apply > it to the repository, but neither Dave nor I have a Sybase installation > to test it against at the moment (and neither of us can really spare > the time). > > That raises the question - should the project be moved to Sourceforge? I'= m > not convinced the move would solve anything: it would still require at > least one developer with C skills and the time to review and test patches= , > cut releases, and reply to queries - is there anyone on the list willing > to commit to that? I would like to volunteer to do that. We are using python-sybase in a few tools at the company where I work and so we would like to contribute to this project and ensure it is properly tested. Actually I have already done most of the work: * I merged most of the patches provided on this list * I added some corrections of my own * I added a unit test suite based on the DB API 2.0 driver compliance unit test suite + extra * this unit test suite has been added to our automated testing framework so that it runs frequently on the following platforms and sybase clients: i686-pc-linux-gnu - Sybase ASE 12.5.1 and 15.0 (32bits) sparc-sun-solaris2.10 - Sybase ASE 12.5.1 and 15.0 (32bits and 64bits) sparc-sun-solaris2.8 - Sybase ASE 12.5.1 and 15.0 (32bits) powerpc-ibm-aix5.3.0.0 - Sybase ASE 12.5.1 and 15.0 (32bits) powerpc-ibm-aix5.2.0.0 - Sybase ASE 12.5.1 and 15.0 (32bits) alphaev68-dec-osf5.1 - Sybase ASE 12.5.1 and 15.0 (32bits) I join a patch which includes the modifications and the unit test suite. If that is OK with you and Dave, I would be happy to help you "review and test patches, cut releases, and reply to queries" for coming versions of python-sybase. regards -- S=E9bastien Sabl=E9 SunGard GP3 - Runtime |
From: Greg W. <gwa...@py...> - 2006-07-07 07:44:21
|
On 06 July 2006, Andrew McNamara said: > Dave's tied up with other stuff at the moment, but I've run this by him, > and he says your patch looks good. Yeah, I know how it goes. Nice, stable codebase that's been humming along just fine for years and some yahoo finds a bug. The longer since the last release, the harder it is to find time for the next one. > As to whether it make it into a future release, I can certainly apply > it to the repository, but neither Dave nor I have a Sybase installation > to test it against at the moment (and neither of us can really spare > the time). Knowing that it's checked in is good enough for me. Would be nice to see a 0.38 release someday, but I've no objection if it waits until you guys are good and ready. > That raises the question - should the project be moved to Sourceforge? I'm > not convinced the move would solve anything: Agreed. In my experience, checking in patches is the least time-consuming part of maintaining a stable project. Merely opening checkins to more developers won't help much. Greg |
From: <sk...@po...> - 2006-07-07 08:24:44
|
>> That raises the question - should the project be moved to >> Sourceforge? I'm not convinced the move would solve anything: Greg> Agreed. In my experience, checking in patches is the least Greg> time-consuming part of maintaining a stable project. Merely Greg> opening checkins to more developers won't help much. OTOH, if the Object Craft folks don't have Sybase available to test against, perhaps moving to SF would prompt someone with Sybase access to take over much of the vetting process. Skip |
From: Andrew M. <an...@ob...> - 2006-07-07 17:13:36
|
> Greg> Agreed. In my experience, checking in patches is the least > Greg> time-consuming part of maintaining a stable project. Merely > Greg> opening checkins to more developers won't help much. > >OTOH, if the Object Craft folks don't have Sybase available to test against, >perhaps moving to SF would prompt someone with Sybase access to take over >much of the vetting process. If someone is offering to put pre-release code through it's paces, we can do that without SF (yeah, it's a pretty clunky way of doing it, but we all did somehow get by before the SF, and even IP connectivity 8-) Obviously Greg's patch should go in - are there any other patches vs 0.37 that should be applied? I'll put together a devel snapshot for you to download and test, and if all goes well, we'll call that 0.38. In some ways, this situation is no worse than the linux kernel - multiple repositories, patches being mailed around and being applied without testing (since Andrew Morton and Linus can only ever have a small subset of the hardware available). But if I can find time, I'll have another go at installing Sybase. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ |
From: Greg W. <gwa...@py...> - 2006-07-08 06:33:40
Attachments:
python-sybase-checkready.patch
|
On 07 July 2006, Andrew McNamara said: > If someone is offering to put pre-release code through it's paces, > we can do that without SF (yeah, it's a pretty clunky way of doing it, > but we all did somehow get by before the SF, and even IP connectivity 8-) I'm willing to test out patches posted to this list. I only have access to a 6-year old version of Sybase (11.0.3) though. I suppose that's useful for testing compatibility with old versions. > Obviously Greg's patch should go in - are there any other patches vs > 0.37 that should be applied? I'll put together a devel snapshot for you > to download and test, and if all goes well, we'll call that 0.38. I noticed some copy 'n pasted code, which always rubs me the wrong way; here's a patch to correct it. It's running fine on my development server right now. Greg P.S. patch is relative to my '_close_fetcher()' patch. P.P.S. I just realized that perhaps I should have called it '_closefetcher()' for consistency with the rest of the code. Andrew, please rename it if you feel that's the right thing to do. Stylistic consistency is more important than worrying about hurting some contributor's feelings! |
From: <sk...@po...> - 2006-07-08 09:19:23
|
Greg> I'm willing to test out patches posted to this list. I only have Greg> access to a 6-year old version of Sybase (11.0.3) though. I Greg> suppose that's useful for testing compatibility with old versions. Me too, though I have Sybase 12.5.something at work. Skip |
From: Dave C. <dj...@ob...> - 2006-12-12 23:41:22
|
Sébastien Sablé wrote: > 2006/7/6, Andrew McNamara <an...@ob...>: >> ... >> That raises the question - should the project be moved to Sourceforge? >> I'm not convinced the move would solve anything: it would still require at >> least one developer with C skills and the time to review and test >> patches, cut releases, and reply to queries - is there anyone on the list willing >> to commit to that? > > I would like to volunteer to do that. We are using python-sybase in a > few tools at the company where I work and so we would like to > contribute to this project and ensure it is properly tested. In response to this message we asked Sébastien if he would be prepared to create a sourceforge project and move our code onto the site. Sébastien agreed to create the project on sourceforge and he then converted our CVS module to subversion and loaded it up at: http://sourceforge.net/projects/python-sybase/ We have been hoping someone would make an offer like this for quite some time now, and it is a great relief to see the code be taken over by someone who not only has an ongoing interest in the success of the project, but also has the ability to ensure that success. Thank you Sébastien. - Dave |
From: <sab...@gm...> - 2006-12-13 06:01:53
|
Thank you Dave; I will try to keep python-sybase as good as it is now. I have released a first pre-release 0.38pre1 on sourceforge. It includes most of the patchs provided recently on this mailing-list. I would appreciate if people on this mailing-list could test it on as many environments as possible. You can download it there: http://sourceforge.net/project/showfiles.php?group_id=3D184050 MAJOR CHANGES SINCE 0.37: * This release works with python 2.5 * It also works with sybase 15 * It works with 64bits clients * It can be configured to return native python datetime objects * The bug "This routine cannot be called because another command structure has results pending." which appears in various cases has been corrected * It includes a unitary test suite based on the dbapi2.0 compliance test su= ite 2006/12/12, Dave Cole <dj...@ob...>: > S=E9bastien Sabl=E9 wrote: > > 2006/7/6, Andrew McNamara <an...@ob...>: > >> ... > >> That raises the question - should the project be moved to Sourceforge? > >> I'm not convinced the move would solve anything: it would still requir= e at > >> least one developer with C skills and the time to review and test > >> patches, cut releases, and reply to queries - is there anyone on the l= ist willing > >> to commit to that? > > > > I would like to volunteer to do that. We are using python-sybase in a > > few tools at the company where I work and so we would like to > > contribute to this project and ensure it is properly tested. > > In response to this message we asked S=E9bastien if he would be prepared > to create a sourceforge project and move our code onto the site. > > S=E9bastien agreed to create the project on sourceforge and he then > converted our CVS module to subversion and loaded it up at: > http://sourceforge.net/projects/python-sybase/ > > We have been hoping someone would make an offer like this for quite some > time now, and it is a great relief to see the code be taken over by > someone who not only has an ongoing interest in the success of the > project, but also has the ability to ensure that success. > > Thank you S=E9bastien. > > - Dave > > _______________________________________________ > Python-sybase mailing list > Pyt...@ww... > https://www.object-craft.com.au/cgi-bin/mailman/listinfo/python-sybase > |