Thread: [SQLObject] Some question about fromDatabase
SQLObject is a Python ORM.
Brought to you by:
ianbicking,
phd
From: gary n. <li...@ga...> - 2005-10-21 02:44:31
|
Hi, I rely heavily on fromDatabase and find it missing something. I have already submitted some patches so more validators can be guessed from database info. However, I notice one thing that is not very convenient. The validators created are by default with "NOT NULL" equivalent as the validators created don't have if_empty=None keyword. So if I use these validators to validate some input which is empty, all will fail even though my table definition may not specify NOT NULL which means empty/None is valid content. This I believe is in general not the case of table definition. In order to do this properly some changes are needed to the whole col.py file so all column type can take one more argument for the column validator creation(if_empty). Another is of course change formencode so all validators assume if_empty=None. Just want to know what people think about this issue. |
From: gary n. <li...@ga...> - 2005-10-21 04:08:40
|
Another thing I notice is that the validators in col.py inherit from Validator, not FancyValidator and defines : to_python rather than _to_python. The result seems to be that it takes over everything in formencode tree and thus the if_empty thing just don't work at all even if I specify it. Is this intended ? "gary ng" <li...@ga...> wrote in message news:dj9khe$a0q$1...@se...... > Hi, > > I rely heavily on fromDatabase and find it missing something. I have already > submitted some patches so more validators can be guessed from database info. > However, I notice one thing that is not very convenient. The validators > created are by default with "NOT NULL" equivalent as the validators created > don't have if_empty=None keyword. So if I use these validators to validate > some input which is empty, all will fail even though my table definition may > not specify NOT NULL which means empty/None is valid content. > > This I believe is in general not the case of table definition. In order to > do this properly some changes are needed to the whole col.py file so all > column type can take one more argument for the column validator > creation(if_empty). Another is of course change formencode so all validators > assume if_empty=None. > > Just want to know what people think about this issue. > > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 08:08:54
|
Hi! On Fri, Oct 21, 2005 at 10:40:02AM +0800, gary ng wrote: > more validators can be guessed from database info. Column types? > However, I notice one thing that is not very convenient. The validators > created are by default with "NOT NULL" equivalent I do not understand this. My be you mean not validators but columns? SOCol and Col classes? > as the validators created > don't have if_empty=None keyword. What is if_empty? I've never used it. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: gary n. <li...@ga...> - 2005-10-21 09:22:51
|
"Oleg Broytmann" <ph...@ma...> wrote in message news:200...@ph...... > Hi! > > On Fri, Oct 21, 2005 at 10:40:02AM +0800, gary ng wrote: > > more validators can be guessed from database info. > > Column types? > > > However, I notice one thing that is not very convenient. The validators > > created are by default with "NOT NULL" equivalent > > I do not understand this. My be you mean not validators but columns? > SOCol and Col classes? No I mean validators. The current fromDatabase code will create columns and attach a validator to it, sort of mimicking as if I hand code it. The change I sent yesterday for postgresql basically now create all the right validators for most column types. However, the validators defined in col.py while do subclass from FormEncode, they do it in a very crude way and take over the validator features in formencode ones. > > > as the validators created > > don't have if_empty=None keyword. > > What is if_empty? I've never used it. This is one of the feature of validator in formencode where I can define if_empty=my_value. What it does is that if the value is being validated is empty, the whole validation process would be short circuited and return my_value(None in this case) instead. This has everything to do with the table. As if I don't give a "NOT NULL" for a column, it means I can insert rows with no value for this column which I assume is None in SQLObject speak. Consider a scenario where I create a HTML form and get the post back value. I can validate them against the validators in SQLObject which is a first line data type checking(and validity, say wrong format of date). With its current form, anything that is not filled(especially none string fields) would be considered as error by the validator because of this lack of if_empty but my database definition clearly allows it unless I said NOT NULL. I briefly trace a bit and it seems that this validator thing is defined but not used at all in the operation ? > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 09:40:03
|
On Fri, Oct 21, 2005 at 05:17:57PM +0800, gary ng wrote: > The current fromDatabase code will create columns and > attach a validator to it Where?! I do not see this in the code! > This is one of the feature of validator in formencode where I can define > if_empty=my_value. What it does is that if the value is being validated is > empty, the whole validation process would be short circuited and return > my_value(None in this case) instead. Isn't Col(default=my_value) what you want? > With its current form, anything that is not filled(especially none string > fields) would be considered as error by the validator Again, I do not understand (probably a difference in terms). Let me try to guess what you mean. class MyTable(SQLObject): name = StringCol(length=255) row = MyTable() # error - the call lacks required column "name" Is it what you mean? If I guessed it right, if_empty in validators would not help you - the error is triggered in SQLObject._create(), no validators have been triggered yet. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: gary n. <li...@ga...> - 2005-10-21 10:07:59
|
"Oleg Broytmann" <ph...@ma...> wrote in message news:200...@ph...... > On Fri, Oct 21, 2005 at 05:17:57PM +0800, gary ng wrote: > > The current fromDatabase code will create columns and > > attach a validator to it > > Where?! I do not see this in the code! Inside postgresql/dbconnection.py > > > This is one of the feature of validator in formencode where I can define > > if_empty=my_value. What it does is that if the value is being validated is > > empty, the whole validation process would be short circuited and return > > my_value(None in this case) instead. > > Isn't Col(default=my_value) what you want? That is if I define the column in SQLObject. I am using fromDatabase so I don't need to do this as I have already done it in the database definition. > > > With its current form, anything that is not filled(especially none string > > fields) would be considered as error by the validator > > Again, I do not understand (probably a difference in terms). Let me try > to guess what you mean. > > class MyTable(SQLObject): > name = StringCol(length=255) > > row = MyTable() # error - the call lacks required column "name" > > Is it what you mean? If I guessed it right, if_empty in validators would > not help you - the error is triggered in SQLObject._create(), no validators > have been triggered yet. More like this: class MyTable(SQLObject): tdate = DateCol() amt=FloatCol() row = MyTable(tdate="",amt="") But anyway, I believe the validator feature in col.py is there but not used by SQLObject. > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 10:22:25
|
On Fri, Oct 21, 2005 at 06:01:13PM +0800, gary ng wrote: > > > The current fromDatabase code will create columns and > > > attach a validator to it > > > > Where?! I do not see this in the code! > Inside postgresql/dbconnection.py Line(s)? > > Isn't Col(default=my_value) what you want? > That is if I define the column in SQLObject. I am using fromDatabase so I > don't need to do this as I have already done it in the database definition. So patch columnsFromSchema() to pass the defaults. Actuall there is a code for this in postgres/pgconnection.py, so what is the problem. > class MyTable(SQLObject): > tdate = DateCol() > amt=FloatCol() > row = MyTable(tdate="",amt="") class MyTable(SQLObject): tdate = DateCol() amt=FloatCol() def _create(self, id, **kw): for name, value in {"tdate": defaultDate, "amt": defaultAmount}.items(): if (name not in kw) or (not kw[name]): kw[name] = value super(MyTable, self)._create(id, **kw) row = MyTable(amt="") Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: gary n. <li...@ga...> - 2005-10-21 10:45:01
|
"Oleg Broytmann" <ph...@ma...> wrote in message news:200...@ph...... > > Inside postgresql/dbconnection.py > > Line(s)? 232: colClass, kw = self.guessClass(t,notnull) This would make the right colClass defined which would automatically has a validator(that part is in col.py). > So patch columnsFromSchema() to pass the defaults. Actuall there is a > code for this in postgres/pgconnection.py, so what is the problem. Yes. That fix the SQLObject part of the problem but what I want is to use the validators to : 1. validate against my web page input for data type validity(I don't want to try to update the database for this) 2. then validate for logical validity 3. update the database. Beside, the SQLObject column default is only useful if I do something like : row=MyTable() but not row=MyTable(tdate="") > > > class MyTable(SQLObject): > > tdate = DateCol() > > amt=FloatCol() > > row = MyTable(tdate="",amt="") > > class MyTable(SQLObject): > tdate = DateCol() > amt=FloatCol() > > def _create(self, id, **kw): > for name, value in {"tdate": defaultDate, "amt": defaultAmount}.items(): > if (name not in kw) or (not kw[name]): > kw[name] = value > super(MyTable, self)._create(id, **kw) > > row = MyTable(amt="") That again means I need to implement things that I don't have to as the database definiton clearly has all the necessary information and SQLObject also create the right things(except the not null part). If I have 30 tables and 20 fields each, that means lots of unnecessary code. > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 10:56:06
|
On Fri, Oct 21, 2005 at 06:38:13PM +0800, gary ng wrote: > colClass, kw = self.guessClass(t,notnull) > > This would make the right colClass defined which would automatically has a > validator(that part is in col.py). That's what I've said - it creates column definitions, not validators. > > So patch columnsFromSchema() to pass the defaults. Actuall there is a > > code for this in postgres/pgconnection.py, so what is the problem. > Yes. That fix the SQLObject part of the problem but what I want is to use > the validators to : > > 1. validate against my web page input for data type validity(I don't want to > try to update the database for this) > 2. then validate for logical validity > 3. update the database. I dont understand how "if_empty" will help your web app. > That again means I need to implement things that I don't have to as the > database definiton clearly has all the necessary information and SQLObject > also create the right things(except the not null part). If I have 30 tables > and 20 fields each, that means lots of unnecessary code. Not so much: class Base(SQLObject): defaults = {} def _create(self, id, **kw): for name, value in self.defaults.items(): if (name not in kw) or (not kw[name]): kw[name] = value super(MyTable, self)._create(id, **kw) class MyTable1(Base): tdate = DateCol() amt=FloatCol() defaults = {"tdate": default1Date, "amt": default1Amount} class MyTable2(Base): tdate = DateCol() amt=FloatCol() defaults = {"tdate": default2Date, "amt": default2Amount} Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: gary n. <li...@ga...> - 2005-10-21 11:38:05
|
> That's what I've said - it creates column definitions, not validators. My mistake. > > I dont understand how "if_empty" will help your web app. my_value=dict(fields from web) db_schema = formencode.Schema(**dict(validators from sql.meta.columns)) my_validated_value = db_schema.to_python(my_value) Now my_validated_value contains completely valid content(data type wise, None filled if as said by the table definition). Then I can go on with my logic check like if the date is within range etc. Without doing the type checking again. Without the if_empty, I need to trap the error then check each field that is flagged invalid and see if it is really invalid or just empty string(all return from web are in this form if not filled) then check the sql.meta.columns defintion to see if I should or should not replace None(or other value). It can be done, just like your work around of _create but I just think it is not necessary when 98% of it is already there. Beside, as I said, your solution requires me to do an insert/update(I need to check what _create do, it seems not mentioned in the manual). > > > That again means I need to implement things that I don't have to as the > > database definiton clearly has all the necessary information and SQLObject > > also create the right things(except the not null part). If I have 30 tables > > and 20 fields each, that means lots of unnecessary code. > > Not so much: > > class Base(SQLObject): > defaults = {} > > def _create(self, id, **kw): > for name, value in self.defaults.items(): > if (name not in kw) or (not kw[name]): > kw[name] = value > super(MyTable, self)._create(id, **kw) > > class MyTable1(Base): > tdate = DateCol() > amt=FloatCol() > > defaults = {"tdate": default1Date, "amt": default1Amount} > > class MyTable2(Base): > tdate = DateCol() > amt=FloatCol() > > defaults = {"tdate": default2Date, "amt": default2Amount} > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 12:11:31
|
On Fri, Oct 21, 2005 at 07:33:23PM +0800, gary ng wrote: > my_value=dict(fields from web) > db_schema = formencode.Schema(**dict(validators from sql.meta.columns)) > my_validated_value = db_schema.to_python(my_value) > > Now my_validated_value contains completely valid content(data type wise, > None filled if as said by the table definition). Then I can go on with my > logic check like if the date is within range etc. Without doing the type > checking again. Aha, I'm starting to understand. You want to reuse validators, to avoid creating 2 sets of validators - for SQLObject and for web. Right? > Beside, as I said, your > solution requires me to do an insert/update(I need to check what _create do, > it seems not mentioned in the manual). ._create() is being called on INSERTs. For UPDATEs you would override .set(), ._SO_setValue() or individual attribute setters. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: gary n. <li...@ga...> - 2005-10-21 12:48:49
|
"Oleg Broytmann" <ph...@ma...> wrote in message news:200...@ph...... > On Fri, Oct 21, 2005 at 07:33:23PM +0800, gary ng wrote: > > my_value=dict(fields from web) > > db_schema = formencode.Schema(**dict(validators from sql.meta.columns)) > > my_validated_value = db_schema.to_python(my_value) > > > > Now my_validated_value contains completely valid content(data type wise, > > None filled if as said by the table definition). Then I can go on with my > > logic check like if the date is within range etc. Without doing the type > > checking again. > > Aha, I'm starting to understand. You want to reuse validators, to avoid > creating 2 sets of validators - for SQLObject and for web. Right? Exactly. The schema thing is very powerful as they can be chained. So the first in my chain would be data type validity, as defined either in the database or SQLObject. I still need to create my validators for logic, but not data type as that is already handled by the column definition validators, which works 98% except the default empty is None(or whatever) situation. > > > Beside, as I said, your > > solution requires me to do an insert/update(I need to check what _create do, > > it seems not mentioned in the manual). > > ._create() is being called on INSERTs. For UPDATEs you would override > .set(), ._SO_setValue() or individual attribute setters. Thanks. So this is effectly triggers in SQLObject. > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 13:28:58
|
On Fri, Oct 21, 2005 at 08:44:50PM +0800, gary ng wrote: > that is already handled by the column definition > validators, which works 98% except the default empty is None(or whatever) > situation. I see now. It seems you need to patch col.py. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
From: gary n. <li...@ga...> - 2005-10-21 14:23:50
|
I have. But the change is not as trivial as the last patch about guess. So I am not sure if you would like it. I would post it when I feel it is good enough but as I said, it seems that the validator that defined right now is not in use by the SQLObject core code. "Oleg Broytmann" <ph...@ma...> wrote in message news:200...@ph...... > On Fri, Oct 21, 2005 at 08:44:50PM +0800, gary ng wrote: > > that is already handled by the column definition > > validators, which works 98% except the default empty is None(or whatever) > > situation. > > I see now. It seems you need to patch col.py. > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Power Architecture Resource Center: Free content, downloads, discussions, > and more. http://solutions.newsforge.com/ibmarch.tmpl |
From: Oleg B. <ph...@ma...> - 2005-10-21 15:08:15
|
On Fri, Oct 21, 2005 at 10:11:25PM +0800, gary ng wrote: > the change is not as trivial as the last patch about guess. So I > am not sure if you would like it. Add more tests, at least. Oleg. -- Oleg Broytmann http://phd.pp.ru/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |