python-sybase-misc

[python-sybase] Working on the next release

[Dave C. <dj...@ob...>]

Just thought I should mention what I a working on for the next release
of the module...

I have decided to try switching to callback style error handling
instead of inline error handling.  This will have a couple of benefits;

1) FreeTDS uses callback error handling.

2) I should pick up server error messages generated while executing
   stored procedures.

The biggest problem is in managing the global interpreter lock (GIL).

Consider the case where there are two threads using the module.
Whenever I call a Sybase API in the extension module I release the GIL
to allow other threads to run while the server does its thing.  Once
the API returns I reacquire the lock before returning to the
interpreter.

With callback error handling the Sybase API will be calling a function
in the extension module while the GIL is released.  This presents a
small problem in that I must reacquire the GIL with the correct thread
state before bouncing the callback up into some Python code.  Consider
the following...

GIL:  Thread 1:                         Thread 2:
      PyThreadState *save1;             PyThreadState *save2;
    
1     save1 = PyThreadState_Swap(NULL);
1     PyEval_ReleaseLock();
2                                       save1 = PyThreadState_Swap(NULL);
2                                       PyEval_ReleaseLock();
-   
-     Sybase API                        Sybase API
-   
1     PyEval_AcquireLock();
1     PyThreadState_Swap(save1);
2                                       PyEval_AcquireLock();
2                                       PyThreadState_Swap(save1);

When both threads are executing a Sybase API and the callback function
is invoked I need to be able to reacquire the GIL using the correct
PyThreadState...

The only way I can see to do this is to only allow a single thread to
operate on a connection at a time.  Before calling the Sybase API I
would store the PyThreadState with my connection structure.  When a
callback is performed by the Sybase API I locate my connection
structure and restore the associated PyThreadState before bouncing the
callback up into the Python code registered for that callback.

I am still studying the Sybase documentation to make sure that the
above theory will work for the callbacks I am interested in handling.

- Dave

-- 
http://www.object-craft.com.au

Re: [python-sybase] Working on the next release

[Dave C. <dj...@ob...>]

>>>>> "Dave" == Dave Cole <dj...@ob...> writes:

Dave> The only way I can see to do this is to only allow a single
Dave> thread to operate on a connection at a time.  Before calling the
Dave> Sybase API I would store the PyThreadState with my connection
Dave> structure.  When a callback is performed by the Sybase API I
Dave> locate my connection structure and restore the associated
Dave> PyThreadState before bouncing the callback up into the Python
Dave> code registered for that callback.

Just a heads-up.

I have made the GIL modifications to the sybasect extension module and
things seem to work OK.  With the module compiled with threading
enabled I successfully ran the examples/timeout.py program triggering
a timeout callback.

Now all I have to do is implement cs_config() in the extension module
and convert Sybase.py module to used callback error handling.

At the end of this process I should have a module which works properly
with FreeTDS.

- Dave

-- 
http://www.object-craft.com.au

Re: [python-sybase] Working on the next release

[Dave C. <dj...@ob...>]

>>>>> "Dave" == Dave Cole <dj...@ob...> writes:

Dave> Now all I have to do is implement cs_config() in the extension
Dave> module and convert Sybase.py module to used callback error
Dave> handling.

Slow but steady progress...

I have implemented the cs_config() method for CS_CONTEXT objects.  I
then realised that a whole class of errors were never reaching the
outside (Python) world due to my use of an internal CS_CONTEXT for
doing numeric conversions (among other things).  I decided to fix this
up by placing the creation of the internal CS_CONTEXT under the
control of Python code.

The following is a test program which shows the new features.

- - bugger.py - - - - - - - - - - - - - - - - - - - - - - - - - - -
from sybasect import *

def cslib_cb(ctx, msg):
    print 'cslib_cb:'
    print '  severity ', msg.severity
    print '  msgnumber', msg.msgnumber
    print '  msgstring', msg.msgstring
    print '  osnumber ', msg.osnumber
    print '  osstring ', msg.osstring
    print '  status   ', msg.status
    print '  sqlstate ', msg.sqlstate
    raise ValueError('bugger!')

status, ctx = cs_ctx_alloc(CS_VERSION_110)
set_global_ctx(ctx)
ctx.cs_config(CS_SET, CS_MESSAGE_CB, cslib_cb)
numeric('12a')
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

When I run it this is what happens:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
cslib_cb:
  severity  1
  msgnumber 33816856
  msgstring cs_convert: cslib user api layer: common library error: The conversion/operation was stopped due to a syntax error in the source field.
  osnumber  0
  osstring  
  status    1076857758
  sqlstate  ZZZZZ
Traceback (most recent call last):
  File "bugger.py", line 17, in ?
    numeric('12a')
  File "bugger.py", line 12, in cslib_cb
    raise ValueError('bugger!')
ValueError: bugger!
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Now I am finally ready to convert Sybase.py to callback error handling.

- Dave

-- 
http://www.object-craft.com.au