Thread: [SQLObject] default empty dict for picklecol
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: Daniel F. <fet...@go...> - 2009-01-26 04:35:06
|
When using a PickleCol with default={ } the behaviour is not exactly what I would expect: ###################################################################### from sqlobject import * sqlhub.processConnection = connectionForURI( 'sqlite:///:memory:', debug=True ) class t( SQLObject ): x = PickleCol( default={} ) t.createTable( ) row = t( ) print row.x # this will print None ###################################################################### Having a non-empty dict works as expected though. I had a feeling it was related to mutable keyword argument values and the associated problems, but the same problem is there with an empty tuple which is certainly unmutable. Is this a bug or a feature? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-01-26 08:50:07
|
On Sun, Jan 25, 2009 at 08:35:03PM -0800, Daniel Fetchinson wrote: > When using a PickleCol with default={ } the behaviour is not exactly > what I would expect: > > ###################################################################### > from sqlobject import * > sqlhub.processConnection = connectionForURI( 'sqlite:///:memory:', debug=True ) > class t( SQLObject ): > x = PickleCol( default={} ) > t.createTable( ) > row = t( ) > print row.x # this will print None > ###################################################################### > > Having a non-empty dict works as expected though. I had a feeling it > was related to mutable keyword argument values and the associated > problems, but the same problem is there with an empty tuple which is > certainly unmutable. > > Is this a bug or a feature? Seems like a bug related to boolean testing of objects. I.e., somewhere there is a code that does if object:... instead of if object is None:... But a quick glance at PickleCol code didn't reveal a clue. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Daniel F. <fet...@go...> - 2009-01-26 08:58:59
|
>> When using a PickleCol with default={ } the behaviour is not exactly >> what I would expect: >> >> ###################################################################### >> from sqlobject import * >> sqlhub.processConnection = connectionForURI( 'sqlite:///:memory:', >> debug=True ) >> class t( SQLObject ): >> x = PickleCol( default={} ) >> t.createTable( ) >> row = t( ) >> print row.x # this will print None >> ###################################################################### >> >> Having a non-empty dict works as expected though. I had a feeling it >> was related to mutable keyword argument values and the associated >> problems, but the same problem is there with an empty tuple which is >> certainly unmutable. >> >> Is this a bug or a feature? > > Seems like a bug related to boolean testing of objects. I.e., somewhere > there is a code that does > > if object:... > > instead of > > if object is None:... > > But a quick glance at PickleCol code didn't reveal a clue. I thought the same first, but StringCol( default='' ) works as expected, although the empty string also evaluates to False as boolean. And I'd think this part of the code (testing for default) is the same for every col type. So I'm sufficiently confused, I'll keep looking at the code. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Daniel F. <fet...@go...> - 2009-01-26 20:14:48
|
>>> When using a PickleCol with default={ } the behaviour is not exactly >>> what I would expect: >>> >>> ###################################################################### >>> from sqlobject import * >>> sqlhub.processConnection = connectionForURI( 'sqlite:///:memory:', >>> debug=True ) >>> class t( SQLObject ): >>> x = PickleCol( default={} ) >>> t.createTable( ) >>> row = t( ) >>> print row.x # this will print None >>> ###################################################################### >>> >>> Having a non-empty dict works as expected though. I had a feeling it >>> was related to mutable keyword argument values and the associated >>> problems, but the same problem is there with an empty tuple which is >>> certainly unmutable. >>> >>> Is this a bug or a feature? >> >> Seems like a bug related to boolean testing of objects. I.e., >> somewhere >> there is a code that does >> >> if object:... >> >> instead of >> >> if object is None:... >> >> But a quick glance at PickleCol code didn't reveal a clue. > > I thought the same first, but StringCol( default='' ) works as > expected, although the empty string also evaluates to False as > boolean. And I'd think this part of the code (testing for default) is > the same for every col type. So I'm sufficiently confused, I'll keep > looking at the code. I've investigated this further and the problem is with the validator PickleCol wants to use. If I insert a 'print kw' before and after line 1066: kw[name] = dbValue = from_python(value, self._SO_validatorState) in main.py and do the following: ###################################################################### from sqlobject import * sqlhub.processConnection = connectionForURI( 'sqlite:///:memory:' ) class t( SQLObject ): x = PickleCol( default={} ) t.createTable( ) row = t( ) ###################################################################### I'll get: {'x': {}} {'x': None} I.e. the empty dict here gets validated to None. But I couldn't find out the reason for this, PickleCol uses a complicated validator. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-01-26 21:05:38
|
On Mon, Jan 26, 2009 at 12:14:44PM -0800, Daniel Fetchinson wrote: > {'x': {}} > {'x': None} Found it. It is in FormEncode, api.py, class FancyValidator; note if self.is_empty(value): ... return self.empty_value(value) def from_python(self, value, state=None): try: if self.strip and isinstance(value, (str, unicode)): value = value.strip() if not self.accept_python: if self.is_empty(value): if self.not_empty: raise Invalid(self.message('empty', state), value, state) else: return self.empty_value(value) vp = self.validate_python if vp and vp is not self._validate_noop: vp(value, state) fp = self._from_python if fp: value = fp(value, state) vo = self.validate_other if vo and vo is not self._validate_noop: vo(value, state) return value else: if self.is_empty(value): return self.empty_value(value) fp = self._from_python if fp: value = self._from_python(value, state) return value except Invalid: if self.if_invalid_python is NoDefault: raise else: return self.if_invalid_python def is_empty(self, value): # None and '' are "empty" return value is None or value == '' or ( isinstance(value, (list, tuple, dict)) and not value) def empty_value(self, value): return None In newer version of FormEncode compound.All validator got correct is_empty method: def is_empty(self, value): # sub-validators should handle emptiness. return False So it's time to require more recent FormEncode to be used with SQLObject. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Oleg B. <ph...@ph...> - 2009-01-26 21:25:26
|
On Tue, Jan 27, 2009 at 12:05:32AM +0300, Oleg Broytmann wrote: > In newer version of FormEncode compound.All validator got correct > is_empty method: > > def is_empty(self, value): > # sub-validators should handle emptiness. > return False > > So it's time to require more recent FormEncode to be used with SQLObject. I think it is FormEncode version 1.1.1, but I cannot recommend any version until I run the test suite. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Daniel F. <fet...@go...> - 2009-01-26 22:49:00
|
>> In newer version of FormEncode compound.All validator got correct >> is_empty method: >> >> def is_empty(self, value): >> # sub-validators should handle emptiness. >> return False >> >> So it's time to require more recent FormEncode to be used with >> SQLObject. > > I think it is FormEncode version 1.1.1, but I cannot recommend any > version until I run the test suite. > > Oleg. Great, thanks Oleg! Please let us know when it is safe to upgrade to FormEncode 1.2.1. (current version). Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Daniel F. <fet...@go...> - 2009-01-26 23:50:36
|
>>> In newer version of FormEncode compound.All validator got correct >>> is_empty method: >>> >>> def is_empty(self, value): >>> # sub-validators should handle emptiness. >>> return False >>> >>> So it's time to require more recent FormEncode to be used with >>> SQLObject. >> >> I think it is FormEncode version 1.1.1, but I cannot recommend any >> version until I run the test suite. >> >> Oleg. > > Great, thanks Oleg! Please let us know when it is safe to upgrade to > FormEncode 1.2.1. (current version). Just letting you know that I upgraded to FormEncode 1.2.1 and everything works fine with sqlobject 0.10.4 in a pretty large turbogears app (which doesn't mean that everything will be fine for everyone). By the way, are there plans for a 1.0 release? What are your requirements for this? Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-01-27 09:19:26
|
On Mon, Jan 26, 2009 at 03:50:31PM -0800, Daniel Fetchinson wrote: > Just letting you know that I upgraded to FormEncode 1.2.1 and > everything works fine with sqlobject 0.10.4 in a pretty large > turbogears app (which doesn't mean that everything will be fine for > everyone). Thank you! > By the way, are there plans for a 1.0 release? What are your > requirements for this? There are many plans. I have a rather big TODO for 1.0, 1.1 and even 1.2; it's in Russian so I cannot just post it here. But I am very busy at the job, and cannot devote much time to SQLObject. :( Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Daniel F. <fet...@go...> - 2009-01-27 09:24:22
|
>> Just letting you know that I upgraded to FormEncode 1.2.1 and >> everything works fine with sqlobject 0.10.4 in a pretty large >> turbogears app (which doesn't mean that everything will be fine for >> everyone). > > Thank you! > >> By the way, are there plans for a 1.0 release? What are your >> requirements for this? > > There are many plans. I have a rather big TODO for 1.0, 1.1 and even > 1.2; it's in Russian so I cannot just post it here. But I am very busy at > the job, and cannot devote much time to SQLObject. :( Please post the link to the Russian TODO list, google will be able to translate it for everyone not speaking Russian. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-02-02 22:32:29
|
On Mon, Jan 26, 2009 at 02:21:51PM -0800, Daniel Fetchinson wrote: > >> In newer version of FormEncode compound.All validator got correct > >> is_empty method: > >> > >> def is_empty(self, value): > >> # sub-validators should handle emptiness. > >> return False > >> > >> So it's time to require more recent FormEncode to be used with > >> SQLObject. > > > > I think it is FormEncode version 1.1.1, but I cannot recommend any > > version until I run the test suite. > > > Great, thanks Oleg! Please let us know when it is safe to upgrade to > FormEncode 1.2.1. (current version). The test suite passed with FormEncode 1.2.1. I added a proper test to test_validation.py and set the lowest version of FormEncode in setup.py. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Daniel F. <fet...@go...> - 2009-02-03 02:24:46
|
>> >> In newer version of FormEncode compound.All validator got correct >> >> is_empty method: >> >> >> >> def is_empty(self, value): >> >> # sub-validators should handle emptiness. >> >> return False >> >> >> >> So it's time to require more recent FormEncode to be used with >> >> SQLObject. >> > >> > I think it is FormEncode version 1.1.1, but I cannot recommend any >> > version until I run the test suite. >> > >> Great, thanks Oleg! Please let us know when it is safe to upgrade to >> FormEncode 1.2.1. (current version). > > The test suite passed with FormEncode 1.2.1. I added a proper test to > test_validation.py and set the lowest version of FormEncode in setup.py. Thanks a lot Oleg! Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |
From: Oleg B. <ph...@ph...> - 2009-02-02 22:38:03
|
On Tue, Jan 27, 2009 at 01:24:18AM -0800, Daniel Fetchinson wrote: > >> By the way, are there plans for a 1.0 release? What are your > >> requirements for this? > > > > There are many plans. I have a rather big TODO for 1.0, 1.1 and even > > 1.2 That's it: In dbconnection.py in ConsoleWriter do not save self.logfile in __init__ in case sys.stdout will be replaced later. Choose SQLite driver: sqlite://...?backend=(sqlite|sqlite3|pysqlite2) alternateMethodName for all unique fields, not only alternateID. tableParamSQL: class MyTable(SQLObject): class sqlmeta: tableParamSQL = 'ENGINE InnoDB' tableParamSQL = {'mysql': 'ENGINE InnoDB'} delColumn('foreignKey'), delColumn('foreignKeyID') TimestampCol() - make default=None. Proper %-encoded URI. SET client_encoding for PostgreSQL. IntervalCol. All signal handlers must receive 'self' as the first argument. __version__.py Always use .lazyIter(). Test 'dirty' and other attributes for collisions. Release 1.0. Invert tests 'isinstance(obj, (tuple, list))' to 'not isinstance(obj, basestr)' to allow any iterable. More decorators especially @classmethod. Generators instead of loops (fetchall => fetchone). List databases in the connection. List tables in the DB. The query in MySQL is 'SHOW TABLES' in SQLite it is "SELECT name FROM sqlite_master WHERE type='table' ORDER BY name" and in Postgres it is something like "SELECT c.relname FROM pg_class c, pg_type t WHERE c.reltype = t.oid AND t.typname = 'table'" Release 1.1. Stop supporting Python 2.4: use 'with lock'. Release 1.2. ---------- RelatedJoin.hasOther(otherObject[.id]) .select(), .selectBy() and .by*() must pass the value(s) through .from_python(); then the hack with UnicodeCol could be removed. Calculated columns - in SELECT but not in INSERT/UPDATE. Something like this: class MyClass(SQLObject): function1 = CalculatedCol(func.my_function(MyClass.q.col1)) function2 = CalculatedCol('my_function') All-unicode - one encoding per DB/connection. Optimize Iteration.next() - use cursor.fetchmany(). More documentation. Wiki. Trac. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: Daniel F. <fet...@go...> - 2009-02-03 02:32:13
|
>> >> By the way, are there plans for a 1.0 release? What are your >> >> requirements for this? >> > >> > There are many plans. I have a rather big TODO for 1.0, 1.1 and even >> > 1.2 > > That's it: Thanks for this Oleg, it's good to know where sqlobject is heading. And it's even better to know that no major issues are there only minor things, at least nothing on your list stands out as extraordinary. This means that sqlobject is already a pretty mature project, even before 1.0. By the way, some weeks ago I decided to switch to sqlalchemy, but after looking at the documentation and some correspondence on the mailing list, I decided not to do it. Sqlobject for my purposes is really great, works perfectly and sqlalchemy seems so super complicated without any obvious gain for my (admittedly, mid-sized) projects. Long live sqlobject! Cheers, Daniel > In dbconnection.py in ConsoleWriter do not save self.logfile in __init__ in > case sys.stdout will be replaced later. > > > Choose SQLite driver: sqlite://...?backend=(sqlite|sqlite3|pysqlite2) > > > alternateMethodName for all unique fields, not only alternateID. > > > tableParamSQL: > > class MyTable(SQLObject): > class sqlmeta: > tableParamSQL = 'ENGINE InnoDB' > tableParamSQL = {'mysql': 'ENGINE InnoDB'} > > > delColumn('foreignKey'), delColumn('foreignKeyID') > > > TimestampCol() - make default=None. > > > Proper %-encoded URI. > > > SET client_encoding for PostgreSQL. > > > IntervalCol. > > > All signal handlers must receive 'self' as the first argument. > > > __version__.py > > > Always use .lazyIter(). > > > Test 'dirty' and other attributes for collisions. > > > Release 1.0. > > > Invert tests 'isinstance(obj, (tuple, list))' to 'not isinstance(obj, > basestr)' > to allow any iterable. > > > More decorators especially @classmethod. > > > Generators instead of loops (fetchall => fetchone). > > > List databases in the connection. > > > List tables in the DB. The query in MySQL is 'SHOW TABLES' > > in SQLite it is > "SELECT name FROM sqlite_master WHERE type='table' ORDER BY name" > > and in Postgres it is something like > "SELECT c.relname FROM pg_class c, pg_type t > WHERE c.reltype = t.oid AND t.typname = 'table'" > > > Release 1.1. > > > Stop supporting Python 2.4: use 'with lock'. > > > Release 1.2. > > > ---------- > > > RelatedJoin.hasOther(otherObject[.id]) > > > .select(), .selectBy() and .by*() must pass the value(s) through > .from_python(); then the hack with UnicodeCol could be removed. > > > Calculated columns - in SELECT but not in INSERT/UPDATE. > Something like this: > > class MyClass(SQLObject): > function1 = CalculatedCol(func.my_function(MyClass.q.col1)) > function2 = CalculatedCol('my_function') > > > All-unicode - one encoding per DB/connection. > > > Optimize Iteration.next() - use cursor.fetchmany(). > > > More documentation. Wiki. Trac. > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > -- Psss, psss, put it down! - http://www.cafepress.com/putitdown |