Re: [Sqlrelay-discussion] Problem with Python cPickle and blobs

[David M. <dav...@fi...>]

It should be ok to use select to fetch a blob.  But, since the blob may
contain NULLs, you need to use both getField() and getFieldLength() to
deal with it.  I wonder what cPickle.loads() does to decide how long the
array of binary data is.  Maybe it stores it in the beginning of the
array or something.

I'll run some tests and see what I get.

Dave
dav...@fi...

On Thu, 2005-07-21 at 16:10 +0100, Ville Silventoinen wrote:
> Hi Dave,
> 
> Python len seems to be working correctly with pickled data.
> 
> I did find out that cPickle.dumps(sdata) works ok. This generates string 
> representation of the pickled data, which doesn't contain a \0 character. 
> (I had a typo in my earlier test, the create procedure used string data, 
> the modify procedure used binary, which is why I thought neither dumps 
> worked.)
> 
> If I use the HIGHEST_PROTOCOL with dumps, it generates binary data. This 
> seems to have a problem with inputBindBlob(), unless I'm still doing 
> something wrong. So if I am right, you should be able to recreate the 
> problem with something like this:
> 
>    sdata={'spam': {'eggs': 'spamspam', 'answers': {1:'', 2:'foo', 3:'bar'}}}
>    ...
>    bdata = cPickle.dumps(sdata, 2)  # uses HIGHEST_PROTOCOL
>    blen = len(bdata)
>    cur.inputBindBlob('colname', bdata, blen)
>    ...
>    # read the data by using normal select
>    ...
>    mydata = cPickle.loads(fields[0])
>    # this throws EOFError (at least in my case)
> 
> I think '' generates \x00 or \x0, which becomes EOF with inputBindBlob. 
> Perhaps it encodes \x00 as \0? Or maybe the problem is in the select?
> Is it ok to use normal select to read blobs?
> 
> Thanks,
> Ville
> 
> 
> On Thu, 21 Jul 2005, Firstworks/4access wrote:
> 
> > Ville,
> >
> > I was actually looking at that last night.  I'm believe you're on track.
> > inputBind() does a strlen() of the string to decide how many characters
> > to store in the database.  strlen() interprets \0's as end-of-string
> > markers, so only part of the data gets stored in the blob.  Then, later
> > when you fetch and unpickle it, only part of it gets unpickled.  I
> > believe the solution is to use inputBindBlob() or inputBindClob() rather
> > than inputBind() to store the pickled data.  inputBindBlob() and
> > inputBindClob() take a length parameter and ignore \0's.
> >
> > Give it a try and let me know whether it works or not.
> >
> > Dave
> > dav...@fi...
> >
> > On Thu, 2005-07-21 at 12:31 +0100, Ville Silventoinen wrote:
> >> Hi,
> >>
> >> I'm using PySQLRClient from sqlrelay-0.36 (with rudiments-0.28.2)
> >> to store blobs in Oracle 9.2.0 database.
> >>
> >> I'm not sure if this has been fixed in the recent sqlrelay release,
> >> but I'm experiencing problems when I store a Python data structure
> >> (dictionary object) to a blob using Pythons cPickle module. Everything
> >> works great until one of the values in the pickled object is an empty
> >> string (''). After that cPickle.load fails to read the data, it raises
> >> EOFError exception. So I wonder if PySQLRClient interprets the pickled
> >> '' as an end-of-file character?
> >>
> >> If I try loading/saving similar data structure to a file, it works ok.
> >>
> >> Thanks for any advice,
> >>
> >> Ville
> >>
> >>
> >>
> >>
> >> -------------------------------------------------------
> >> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> >> from IBM. Find simple to follow Roadmaps, straightforward articles,
> >> informative Webcasts and more! Get everything you need to get up to
> >> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> >> _______________________________________________
> >> Sqlrelay-discussion mailing list
> >> Sql...@li...
> >> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
> >>
> >
> >
> >
> > -------------------------------------------------------
> > SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> > from IBM. Find simple to follow Roadmaps, straightforward articles,
> > informative Webcasts and more! Get everything you need to get up to
> > speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> > _______________________________________________
> > Sqlrelay-discussion mailing list
> > Sql...@li...
> > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
> >
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Sqlrelay-discussion mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
>