Thread: [SQLObject] pickling an SQLObject?
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: sophana <so...@zi...> - 2006-06-09 09:02:00
|
Hi In my webware application, I directly store SQLObjects instances into the session store. After some time, webware store session info disk using (c)Pickle. And there seems to have a problem with Pickle. Just tried, and there seems to be nothing made special for Pickle. Would it be wise to store only the table and id? (and do an update when storing) During load, the object would be retrieved back from the database. Do you think there is a problem implementing the appropriate pickle methods? Regards |
From: Oleg B. <ph...@ma...> - 2006-06-09 09:15:29
|
On Fri, Jun 09, 2006 at 11:01:49AM +0200, sophana wrote: > Would it be wise to store only the table and id? (and do an update when > storing) > During load, the object would be retrieved back from the database. The table is already there, in sqlmeta. You need id and a connection. I am not sure about transactions... Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: sophana <so...@zi...> - 2006-06-09 10:08:55
|
Just the id is needed. tried that. seems to work for me: def __getstate__(self): return self.id def __setstate__(self,id): a=self.get(id) self.__dict__=a.__dict__ Only the id is needed. I don't like the way the object is copied (transfered should I say?). Do you think there is a smarter solution? I tried this. seem to work fine >>> a=Users.get(1) >>> import cPickle >>> b=cPickle.dumps(a) >>> c=cPickle.loads(b) >>> c <Users 1 login='asdf' password='asdfasdf' email="'so...@zi...h...'" validated=True validationString='0sdr8z6kfj'> >>> c.password='qwerqwer' >>> a <Users 1 login='asdf' password='qwerqwer' email="'so...@zi...h...'" validated=True validationString='0sdr8z6kfj'> >>> Oleg Broytmann wrote: >On Fri, Jun 09, 2006 at 11:01:49AM +0200, sophana wrote: > > >>Would it be wise to store only the table and id? (and do an update when >>storing) >>During load, the object would be retrieved back from the database. >> >> > > The table is already there, in sqlmeta. You need id and a connection. I >am not sure about transactions... > >Oleg. > > |
From: sophana <so...@zi...> - 2006-06-09 20:08:23
|
Oleg Broytmann a =E9crit : > On Fri, Jun 09, 2006 at 12:08:35PM +0200, sophana wrote: > =20 >> def __setstate__(self,id): >> a=3Dself.get(id) >> self.__dict__=3Da.__dict__ >> =20 > > This breaks cache - "a" is now in the cache instead of self. See how > .get() is implemented. > > Oleg. > =20 Yes there is a big problem In my previous example: >>> a=3DUsers.get(1) >>> import cPickle >>> b=3DcPickle.dumps(a) >>> c=3DcPickle.loads(b) >>> c <Users 1 login=3D'asdf' password=3D'asdfasdf' email=3D"'so...@zi...h.= ..'"=20 validated=3DTrue validationString=3D'0sdr8z6kfj'> >>> c.password=3D'qwerqwer' >>> a <Users 1 login=3D'asdf' password=3D'qwerqwer' email=3D"'so...@zi...h.= ..'"=20 validated=3DTrue validationString=3D'0sdr8z6kfj'> I did some more test: >>> Obj.get(2).owner is a True >>> Obj.get(2).owner is c False >>> a=3D=3Dc False >>> a is c False Does someone has a suggestion? unfortunately pickle API could have been better for __setstate__ if __setstate__ return something, pickle.load should return the returned=20 object instead of self. |
From: Oleg B. <ph...@ma...> - 2006-06-10 07:20:00
|
On Fri, Jun 09, 2006 at 10:11:55PM +0200, sophana wrote: > >> def __setstate__(self,id): > >> a=self.get(id) > >> self.__dict__=a.__dict__ > > > > This breaks cache - "a" is now in the cache instead of self. See how > > .get() is implemented. > > > Yes there is a big problem > Does someone has a suggestion? You have to model __setstate__ after get() - call self._init(id), put self into the cache... Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Luke O. <lu...@me...> - 2006-06-10 00:15:23
|
I don't recall whether getstate/setstate are necessary - i think so, just for the protocol. the key is overriding new/getnewargs. i also don't recall how generally applicable this is, regarding different connection schemes etc. ## Pickle protocol hacks def __new__(cls, *args, **kw): if len(args) == 1: return cls.getByID(args[0]) return SQLObject.__new__(cls, *args, **kw) def __getnewargs__(self): return (self.id,) def __getstate__(self): return self.id def __setstate__(self, state): ''' handled by __new__/__getnewargs__ ''' pass ## End pickle protocol hacks |
From: sophana <so...@zi...> - 2006-06-12 09:50:59
|
Thanks, it works great, but only for pickle protocol 2: slight modification: cls.get instead of cls.getByID class MySqlObject(SQLObject): def __getstate__(self): return self.id def __setstate__(self,id): pass def __new__(cls, *args, **kw): if len(args) == 1: return cls.get(args[0]) return SQLObject.__new__(cls, *args, **kw) def __getnewargs__(self): return (self.id,) basic test: >>> import cPickle >>> a=Users.get(1) >>> b=cPickle.dumps(a,2) >>> c=cPickle.loads(b) >>> c <Users 1 login='asdf' password='qwerqwer' email="'so...@zi...h...'" validated=True validationString='0sdr8z6kfj'> >>> a is c True >>> a == c True >>> id(a) -1222564404 >>> id(c) -1222564404 >>> Luke Opperman wrote: >I don't recall whether getstate/setstate are necessary - i think so, just for >the protocol. the key is overriding new/getnewargs. i also don't recall how >generally applicable this is, regarding different connection schemes etc. > > ## Pickle protocol hacks > > def __new__(cls, *args, **kw): > if len(args) == 1: > return cls.getByID(args[0]) > return SQLObject.__new__(cls, *args, **kw) > > def __getnewargs__(self): > return (self.id,) > > def __getstate__(self): > return self.id > > def __setstate__(self, state): > ''' handled by __new__/__getnewargs__ ''' > pass > > ## End pickle protocol hacks > > > > >_______________________________________________ >sqlobject-discuss mailing list >sql...@li... >https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > > > |
From: sophana <so...@zi...> - 2006-06-14 08:25:21
|
Oleg Broytmann wrote: >On Mon, Jun 12, 2006 at 11:50:49AM +0200, sophana wrote: > > >>class MySqlObject(SQLObject): >> def __getstate__(self): >> return self.id >> def __setstate__(self,id): >> pass >> def __new__(cls, *args, **kw): >> if len(args) == 1: >> return cls.get(args[0]) >> return SQLObject.__new__(cls, *args, **kw) >> >> def __getnewargs__(self): >> return (self.id,) >> >> > > I think it'd be a good idea to add this to SQLObject. Ian? > >Oleg. > > Just a remark: as I said, this method does not work for the default pickle protocol 0. It would be nice to have a method that works in all protocols based on getstate - setstate only. But I don't think it is possible. There should not be multiple instances of the same row. The problem with setstate is that the object instance is already created with a new id, and cannot be replaced by the already exeisting instance os the cache. |
From: Oleg B. <ph...@ma...> - 2006-06-14 08:44:34
|
On Wed, Jun 14, 2006 at 10:25:00AM +0200, sophana wrote: > It would be nice to have a method that works in all protocols based on > getstate - setstate only. > But I don't think it is possible. I think it's possible - __setstate__() should not call get(), instead it should call _init() and put self into the cache. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ma...> - 2006-06-09 11:34:58
|
On Fri, Jun 09, 2006 at 12:08:35PM +0200, sophana wrote: > def __setstate__(self,id): > a=self.get(id) > self.__dict__=a.__dict__ This breaks cache - "a" is now in the cache instead of self. See how .get() is implemented. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2006-06-12 21:12:16
|
On Mon, Jun 12, 2006 at 11:50:49AM +0200, sophana wrote: > class MySqlObject(SQLObject): > def __getstate__(self): > return self.id > def __setstate__(self,id): > pass > def __new__(cls, *args, **kw): > if len(args) == 1: > return cls.get(args[0]) > return SQLObject.__new__(cls, *args, **kw) > > def __getnewargs__(self): > return (self.id,) I think it'd be a good idea to add this to SQLObject. Ian? Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |