Re: [Sqlrelay-discussion] Problem with Python cPickle and blobs
Brought to you by:
mused
Menu
▾
▴
Re: [Sqlrelay-discussion] Problem with Python cPickle and blobs
|
From: David M. <dav...@fi...> - 2005-07-28 02:08:54
|
Ville,
I have run some tests with my development version and the following
script works:
#! /usr/bin/env python
# Copyright (c) 2001 David Muse
# See the file COPYING for more information.
from SQLRelay import PySQLRClient
import sys
import string
import cPickle
con=PySQLRClient.sqlrconnection("localhost",8009,"/tmp/oracle8test.socket","oracle8test","oracle8test")
#con.debugOn()
cur=PySQLRClient.sqlrcursor(con)
cur.sendQuery("drop table testtable");
cur.sendQuery("create table testtable (col1 blob)")
cur.prepareQuery("insert into testtable values(:val1)");
sdata={'spam': {'eggs': 'spamspam', 'answers': {1:'', 2:'foo',
3:'bar'}}}
bdata=cPickle.dumps(sdata)
blen=len(bdata)
cur.inputBindBlob('val1',bdata,blen)
cur.executeQuery()
cur.sendQuery("select * from testtable");
mydata=cPickle.loads(cur.getField(0,0))
print mydata
cur.sendQuery("select * from testtable");
fields=cur.getRow(0)
mydata=cPickle.loads(fields[0])
print mydata
cur.sendQuery("drop table testtable");
It prints out a tuple identical to the one sdata is set to. See if it
doesn't work for you too. If not, then there may be a bug specific to
the version of SQL Relay you're using. What version is that, by the
way?
Dave
On Thu, 2005-07-21 at 16:10 +0100, Ville Silventoinen wrote:
> Hi Dave,
>
> Python len seems to be working correctly with pickled data.
>
> I did find out that cPickle.dumps(sdata) works ok. This generates string
> representation of the pickled data, which doesn't contain a \0 character.
> (I had a typo in my earlier test, the create procedure used string data,
> the modify procedure used binary, which is why I thought neither dumps
> worked.)
>
> If I use the HIGHEST_PROTOCOL with dumps, it generates binary data. This
> seems to have a problem with inputBindBlob(), unless I'm still doing
> something wrong. So if I am right, you should be able to recreate the
> problem with something like this:
>
> sdata={'spam': {'eggs': 'spamspam', 'answers': {1:'', 2:'foo', 3:'bar'}}}
> ...
> bdata = cPickle.dumps(sdata, 2) # uses HIGHEST_PROTOCOL
> blen = len(bdata)
> cur.inputBindBlob('colname', bdata, blen)
> ...
> # read the data by using normal select
> ...
> mydata = cPickle.loads(fields[0])
> # this throws EOFError (at least in my case)
>
> I think '' generates \x00 or \x0, which becomes EOF with inputBindBlob.
> Perhaps it encodes \x00 as \0? Or maybe the problem is in the select?
> Is it ok to use normal select to read blobs?
>
> Thanks,
> Ville
>
>
> On Thu, 21 Jul 2005, Firstworks/4access wrote:
>
> > Ville,
> >
> > I was actually looking at that last night. I'm believe you're on track.
> > inputBind() does a strlen() of the string to decide how many characters
> > to store in the database. strlen() interprets \0's as end-of-string
> > markers, so only part of the data gets stored in the blob. Then, later
> > when you fetch and unpickle it, only part of it gets unpickled. I
> > believe the solution is to use inputBindBlob() or inputBindClob() rather
> > than inputBind() to store the pickled data. inputBindBlob() and
> > inputBindClob() take a length parameter and ignore \0's.
> >
> > Give it a try and let me know whether it works or not.
> >
> > Dave
> > dav...@fi...
> >
> > On Thu, 2005-07-21 at 12:31 +0100, Ville Silventoinen wrote:
> >> Hi,
> >>
> >> I'm using PySQLRClient from sqlrelay-0.36 (with rudiments-0.28.2)
> >> to store blobs in Oracle 9.2.0 database.
> >>
> >> I'm not sure if this has been fixed in the recent sqlrelay release,
> >> but I'm experiencing problems when I store a Python data structure
> >> (dictionary object) to a blob using Pythons cPickle module. Everything
> >> works great until one of the values in the pickled object is an empty
> >> string (''). After that cPickle.load fails to read the data, it raises
> >> EOFError exception. So I wonder if PySQLRClient interprets the pickled
> >> '' as an end-of-file character?
> >>
> >> If I try loading/saving similar data structure to a file, it works ok.
> >>
> >> Thanks for any advice,
> >>
> >> Ville
> >>
> >>
> >>
> >>
> >> -------------------------------------------------------
> >> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> >> from IBM. Find simple to follow Roadmaps, straightforward articles,
> >> informative Webcasts and more! Get everything you need to get up to
> >> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> >> _______________________________________________
> >> Sqlrelay-discussion mailing list
> >> Sql...@li...
> >> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
> >>
> >
> >
> >
> > -------------------------------------------------------
> > SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> > from IBM. Find simple to follow Roadmaps, straightforward articles,
> > informative Webcasts and more! Get everything you need to get up to
> > speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> > _______________________________________________
> > Sqlrelay-discussion mailing list
> > Sql...@li...
> > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
> >
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Sqlrelay-discussion mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
>
|
