Thread: [Modeling-users] API cleanup proposal
Status: Abandoned
Brought to you by:
sbigaret
Menu
▾
▴
modeling-users
|
From: Sebastien B. <sbi...@us...> - 2003-06-02 18:12:10
|
Hi all,
Here is a proposal to change some of the most commonly used methods;
the general idea is to improve developers' experience with the framework
by providing better, cleaner, shorter names for existing functionalities.
[Note on "most commonly used methods": I do not plan to change every
method's name when it's too long: they are mostly for the framework's
use, or at best for advanced usage]
EditingContext (remember: an EC can be thought as a transaction)
--------------
- insert() instead of insertObject()
- delete() instead of deleteObject()
- fetching: remove the need to import FetchSpecification and
Qualifier, and propose an alternate for
objectsWithFetchSpecification()
def fetch(self, entityName,
qualifier=3DNone, # either a Qualifier instance or a string
orderBy=3DNone, # either a SortOrdering or a string
isDeep=3D0, # should subentities be fetched as well?
refresh=3D0, # unsupported yet
lock=3D0, # unsupported yet
limit=3DNone, # \
page=3DNone, # > slice parameters, not in core yet
offset=3DNone, # / (page/offset: mutually exclusive)
)
=20=20=20=20
Typical usage:
ec.fetch('Writer', 'lastName ilike "hu?o"')
ec.fetch('Writer', 'age>=3D80"')
ec.fetch('Writer', 'author.pygmalion.lastName caseInsensitiveLike "r*=
"')
ec.fetch('Writer', orderBy=3D'firstName asc, lastName desc', isDeep=
=3D1)
ec.fetch('Books', orderBy=3D'title', limit=3D10, page=3D1)
- replace objectsCountWithFetchSpecification() with fetchCount() ?
[same API as fetch()]
- replace (set)propagatesInsertionForRelatedObjects() with
autoInsertion() and setAutoInsertion()
CustomObject
------------
I don't see any methods to change here. Maybe we could add globalID()?
However, the name itself 'CustomObject' could be changed if you have
some ideas (I admit it is not very representative of what it does,
i.e. defining generic methods for an object to be bound w/ the
framework)
Validation
----------
Do we need to change anything there? validateValueForKey() is the
longest one, however I think this is not widely used, is it?
KeyValueCoding
--------------
As Mario repeatedly pointed it out some time ago, this module needs to
be cleaned.
Core methods:
| [public access] | [private access]
------+---------------------------+----------------------------------
[get] |valueForKey() | storedValueForKey()
|valueForKeyPath() |
------+---------------------------+----------------------------------
[set] |takeValueForKey() | takeStoredValueForKey()
|takeValueForKeyPath() |
------+---------------------------+----------------------------------
[dict]|takeValuesFromDictionary() | takeStoredValuesFromDictionary()
=20=20
Quick reminder: KVC is intensively used by the framework to get & set
an object's values without knowing where it is
actually stored (for example, attribute 'name' can be
stored in name, _name, and/or can be accessed/set via
getters/setters name/getName()/setName())
The framework *always* uses the so-called "private" access
methods. The reason is that e.g. a public setter can have some
side-effect that should be triggered from a user's point of view (at
creation or modification time), but shouldn't be triggered when
object's data are fetched from the db (this is not very common,
though, hence I think we need a simplified and more efficient KVC for
the common cases --off-topic here).
This is my proposal:
* getProp(name, path):
getProp(key) replaces valueForKey(key)
getProp(path=3DkeyPath) replaces valueForKeyPath(keyPath)
* setProp(name, path):
setProp(key) replaces takeValueForKey(key)
setProp(path=3DkeyPath) replaces takeValueForKeyPath(keyPath)
* same for getStoredProp() and setStoredProp() which replace
(take)storedValueForKey
=20=20
Notes: - parameters name & path are obviously mutually exclusive,
- get/setStoredProp shouldn't be needed from a developer's point
of view.
* updateProps() replaces takeValuesFromDictionary()
updateStoredProps() replaces takeValuesFromStoredDictionary()
Along with that, we also need an other module/class name for
KeyValueCoding.
Mario proposed PropValueAccess. Any ideas?
GenericAccess? PropHandling?
RelationshipManipulation
------------------------
Any thoughts about how addObjectToBothSidesOfRelationshipWithKey() and
removeObjectFromBothSidesOfRelationshipWithKey() can be renamed?=20
=3D> An other solution could be: gather KVC and RelationshipManipulation
in a single module, since they both address the same thing: generic
manipulation of an object, given its definition in the model. What do
you think?
BTW this is not a definitive list, just the things I know that need to
be changed. If you feel like changing other names please tell.
Your feedback is greatly appreciated!-)
-- S=E9bastien.
|
|
From: Yannick G. <ygi...@yg...> - 2003-06-03 02:45:56
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Monday 02 June 2003 14:10, Sebastien Bigaret wrote: > [Note on "most commonly used methods": I do not plan to change every > method's name when it's too long: they are mostly for the framework's > use, or at best for advanced usage] Please make aliases for backward compatibility. : ) The proposal sounds really fair to me ! - -- Yannick Gingras Coder for OBB : Obscenely Base Bottlegrass http://OpenBeatBox.org -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE+3Avarhy5Fqn/MRARAqX8AJsHZcGE2QW+Uf+N8su6mCkkCJeCSACgjhEe Ad0Tl6Gqiw9xV3J4lUdjDyA= =fNcy -----END PGP SIGNATURE----- |
|
From: Sebastien B. <sbi...@us...> - 2003-06-04 19:27:16
|
Yannick Gingras <ygi...@yg...> wrote: > On Monday 02 June 2003 14:10, Sebastien Bigaret wrote: > > [Note on "most commonly used methods": I do not plan to change every > > method's name when it's too long: they are mostly for the framework's > > use, or at best for advanced usage] > > Please make aliases for backward compatibility. >=20 > : ) >=20 > The proposal sounds really fair to me ! A quick answer --sorry, I'm quite busy these days. When I wrote: "method_1() replaces by method_2()", obviously I meant *is an alias for*. I just want to make it clear that I pay a careful attention to backward compatibility and that it won't be broken in any ways after the API "changes"/upgrades. I'll comment further on the proposal and answers later, probably on sunday. -- S=E9bastien. |
|
From: Mario R. <ma...@ru...> - 2003-06-03 08:06:55
|
> Hi all,
>
> Here is a proposal to change some of the most commonly used methods;
> the general idea is to improve developers' experience with the=20
> framework
> by providing better, cleaner, shorter names for existing=20
> functionalities.
I would also add "by indicating clearly the small subset of methods=20
intended
for use by client code, and the stability level".
> [Note on "most commonly used methods": I do not plan to change every
> method's name when it's too long: they are mostly for the =
framework's
> use, or at best for advanced usage]
> EditingContext (remember: an EC can be thought as a transaction)
> --------------
>
> - insert() instead of insertObject()
>
> - delete() instead of deleteObject()
OK. What about commit() for saveChanges()?
> - fetching: remove the need to import FetchSpecification and
> Qualifier, and propose an alternate for
> objectsWithFetchSpecification()
>
> def fetch(self, entityName,
> qualifier=3DNone, # either a Qualifier instance or a=20=
> string
> orderBy=3DNone, # either a SortOrdering or a string
> isDeep=3D0, # should subentities be fetched as=20=
> well?
> refresh=3D0, # unsupported yet
> lock=3D0, # unsupported yet
> limit=3DNone, # \
> page=3DNone, # > slice parameters, not in core =
yet
> offset=3DNone, # / (page/offset: mutually =
exclusive)
> )
>
> Typical usage:
>
> ec.fetch('Writer', 'lastName ilike "hu?o"')
> ec.fetch('Writer', 'age>=3D80"')
> ec.fetch('Writer', 'author.pygmalion.lastName=20
> caseInsensitiveLike "r*"')
> ec.fetch('Writer', orderBy=3D'firstName asc, lastName desc',=20
> isDeep=3D1)
> ec.fetch('Books', orderBy=3D'title', limit=3D10, page=3D1)
Yes, this is much easier than using FetchSpecification and=20
QualifierFormats...
I feel that some things (even if no one has requested them yet ;) are=20
missing...
for example, should one ever need to take advantage of sql "group by"=20
and "having".
Access to these sql clauses may be added later, without breaking this=20
API,
which is OK. Also, such manips may be done in the middle code, but
that would be very inefficient.
Another issue is that even with all the keyword options offered, one=20
still
has to "dip into" small pieces of raw sql very quickly, as shown by your
examples. This may be an acceptable compromise, but then client code
should be allowed to pass any SQL it wants...
Another detail is that the object loads all (direct) properties, every=20=
time --
there may be no workaround for this, given the way the framework is=20
built.
The real problem with this is that to request the result of an sql=20
function,
of which count() is an example, additional api functionality is needed.
But what happens if I want other functions, such as SUM or MAX or=20
AVERAGE
over a few columns? Each of these functions may take complex parameters.
Again, the functionality may be replicated in the middle code, but this=20=
would
not only be a waste of development time, but also be very inefficient.
I propose either a generalization of "select" (which may be too=20
complicated
in its implications), or an addition of a func keyword option, e.g.
func=3D'count'
func=3D('funcName', param1, param2, ...)
func=3D( ('funcName', param1, param2, ...), ('funcname2') )
func=3D('sum','age')
The question is then how is this information returned?
I suggest as a tuple...
Which reminds me that it would be nice to have a convenience
"resultset" option, that when true returns results as a db resulset.
This implies that one should not mix, in the same non-resultset fetch,
requests for real objects and for additional "virtual" properties,
such as counts, sums, etc. (Or stated differently, the presence
of func will cause data to be returned as a tuple.)
> - replace objectsCountWithFetchSpecification() with fetchCount() ?
> [same API as fetch()]
I propose use fetch() as descibed above:
ec.fetch('Writer', qualifier=3D'age>=3D80"', func=3D'count')
=3D> (3)
but also:
ec.fetch('Writer', qualifier=3D'age>=3D80"',=20
func=3D(('count'),('max','age'),('average','age')) )
=3D> (3,91,85)
[ or else we can stick to raw sql:
ec.fetch('Writer', qualifier=3D'age>=3D80"', func=3D'count(*), max(age),=20=
average(age)' )
which may be simpler for everyone... in this case might as well change=20=
'func' to 'select',
which will always assume a 'resulset' response ]
> - replace (set)propagatesInsertionForRelatedObjects() with
> autoInsertion() and setAutoInsertion()
I prefer setAutoInsertion()
>
> CustomObject
> ------------
>
> I don't see any methods to change here. Maybe we could add=20
> globalID()?
> However, the name itself 'CustomObject' could be changed if you have
> some ideas (I admit it is not very representative of what it does,
> i.e. defining generic methods for an object to be bound w/ the
> framework)
CustomObject will inherit the changes to the KeyValueCoding API,
so it should be OK.
CustomObject as name is not so bad. Other names that come to
mind are MappedObject, ManagedObject, MorfedObject ;)
PersistedObjected
> Validation
> ----------
>
> Do we need to change anything there? validateValueForKey() is the
> longest one, however I think this is not widely used, is it?
I suggest validateProperty() instead of validateValueForKey().
>
> KeyValueCoding
> --------------
>
> As Mario repeatedly pointed it out some time ago, this module needs=20=
> to
> be cleaned.
>
> Core methods:
>
>
> | [public access] | [private access]
> =20
> ------+---------------------------+----------------------------------
> [get] |valueForKey() | storedValueForKey()
> |valueForKeyPath() |
> =20
> ------+---------------------------+----------------------------------
> [set] |takeValueForKey() | takeStoredValueForKey()
> |takeValueForKeyPath() |
> =20
> ------+---------------------------+----------------------------------
> [dict]|takeValuesFromDictionary() |=20
> takeStoredValuesFromDictionary()
>
> Quick reminder: KVC is intensively used by the framework to get & =
set
> an object's values without knowing where it is
> actually stored (for example, attribute 'name' can =
be
> stored in name, _name, and/or can be accessed/set =
via
> getters/setters name/getName()/setName())
>
> The framework *always* uses the so-called "private" access
> methods. The reason is that e.g. a public setter can have some
> side-effect that should be triggered from a user's point of view (at
> creation or modification time), but shouldn't be triggered when
> object's data are fetched from the db (this is not very common,
> though, hence I think we need a simplified and more efficient KVC =
for
> the common cases --off-topic here).
>
> This is my proposal:
>
> * getProp(name, path):
>
> getProp(key) replaces valueForKey(key)
> getProp(path=3DkeyPath) replaces valueForKeyPath(keyPath)
I like the use of path instead of another method. However does
this imply an extra check per call? Also, when specified path must
include name... is this better than path must never include name?
> * setProp(name, path):
>
> setProp(key) replaces takeValueForKey(key)
> setProp(path=3DkeyPath) replaces takeValueForKeyPath(keyPath)
>
> * same for getStoredProp() and setStoredProp() which replace
> (take)storedValueForKey
>
> Notes: - parameters name & path are obviously mutually exclusive,
> - get/setStoredProp shouldn't be needed from a developer's=20
> point
> of view.
>
> * updateProps() replaces takeValuesFromDictionary()
> updateStoredProps() replaces takeValuesFromStoredDictionary()
OK for this, but I would in these case prefer to use the full word=20
(respecting
the naming convention so far...), i.e.:
co.getProperty()
co.setProperty()
co.getStoredProperty()
co.setStoredProperty()
co.updateProperties()
co.updateStoredProperties()
> Along with that, we also need an other module/class name for
> KeyValueCoding.
>
> Mario proposed PropValueAccess. Any ideas?
> GenericAccess?=20
> PropHandling?
>
I would be in favour of combining with RelationshipManipulation...
> RelationshipManipulation
> ------------------------
>
> Any thoughts about how addObjectToBothSidesOfRelationshipWithKey()=20=
> and
> removeObjectFromBothSidesOfRelationshipWithKey() can be renamed?
How about addViaRelation() and removeViaRelation()
> =3D> An other solution could be: gather KVC and =
RelationshipManipulation
> in a single module, since they both address the same thing: generic
> manipulation of an object, given its definition in the model. What=20=
> do
> you think?
Yes. How about MappedObject PR?
For Properties and Relations of course ;)
Module name could be "mopr" or simply "opr".
> BTW this is not a definitive list, just the things I know that need to
> be changed. If you feel like changing other names please tell.
>
>
> Your feedback is greatly appreciated!-)
>
>
> -- S=E9bastien.
mario
|
|
From: Sebastien B. <sbi...@us...> - 2003-06-12 14:35:32
|
Hi all,
Some more comments about the API change proposal; I concentrate here on
EditingContext, and will post another message for CustomObject and
related APIs.
Note: I also received two private answers.
* Again: methods will be aliased, not replaced,
* EC.insert() and EC.delete() are okay to everyone.
* autoInsertion()/setAutoInsertion(): the former is a getter, the
latter, a setter
Mario> OK. What about commit() for saveChanges()?
* about commit(): that was initially in my proposal. I removed
it. Remember I'm working on integrating the ZODB transaction and
observation mechanisms into the modeling? Well, then, commit() clashes
with py2.1 zodb's transaction protocol (and has a very different
semantic than saveChanges(): it is equivalent to objectWillChange()).
Hence I'd prefer to let it apart from the proposal for the moment
being.
* EC.fetch():
The initial proposal is accepted by everyone so we'll keep it as a basis.
Mario> Another issue is that even with all the keyword options offered, one
Mario> still has to "dip into" small pieces of raw sql very quickly, as sho=
wn
Mario> by your examples.
Here I strongly disagree with you. It /seems/ that you dip into small
pieces of raw sql, but it's not the case at all. Qualifiers and
FetchSpecification properties offers a generic command language;
consider this:
ec.fetch('Book', 'author.pygmalion.lastName ilike "r*"')
-> this is far from the generated sql:
SELECT t0.id, t0.title, t0.FK_WRITER_ID, t0.PRICE=20
FROM BOOK t0=20
INNER JOIN ( WRITER t1=20
INNER JOIN WRITER t2=20
ON t1.FK_WRITER_ID=3Dt2.ID )
ON t0.FK_WRITER_ID=3Dt1.ID
WHERE UPPER(t2.LAST_NAME) LIKE UPPER('r%');
In the framework we generally make the strong assumption that raw sql
is taken out of the python code. Even if some of the keywords (such as
'like'/'ilike' for qualifiers, or 'asc'/'desc' for ordering) are
actually the same than the corresponding sql keywords, they are indeed
decorrelated. See again the example above: the 'ilike' keyword is not
used, rather we compare UPPER()s (this might change in the future, now
that 'ilike' is a sql keyword accepted by a lot of db-servers ;)
Mario> This may be an acceptable compromise, but then client code should be
Mario> allowed to pass any SQL it wants...
That's an other problem, to which I completely agree: we should have a
mean to execute complete raw sql statements, and get the (raw) result
back. Moreover, we should be able to transform the returned raw rows
into real objects if necessary (and if possible).
> Another detail is that the object loads all (direct) properties, every
> time -- there may be no workaround for this, given the way the
> framework is built.
(also related to your 'resultset' proposal)
Last, we should also be able to tell the framework to return the raw
rows instead of fully initialized objects (which can later be reverted
to real objects): there's a real need for that; sometimes you want to
present a (very) long list of objects in a summary page/widget, but
you do not need the full objects, not even every attributes, but a
subset to present the user. Then the users selects one or more of this
rows and that's where you'll transform the raw rows to real objects.
Note that the framework architecture will have no problem to support
this. Some of the needed APIs are already present but not implemented
(such as DatabaseContext.faultForRawRow()).
I thought this was on the todo list but it's not --I'll add that,
since I've been thinking about these points for quite a long time.
Impact on the API:
when this is implemented I suggest we add the following parameters to
fetch():
rawRows -- (default: false) return raw rows instead of objects
sql -- execute the sql statement (and ignore all other parameters
except entityName and rawRows --both optional in that case)
BTW I also suggest that the unsupported features in the fetch API are
removed until they are implemented (such as limit/page/offset/etc).
> I feel that some things (even if no one has requested them yet ;) are
> missing...
> for example, should one ever need to take advantage of sql "group by" and
> "having".
> Access to these sql clauses may be added later, without breaking this API,
> which is OK. Also, such manips may be done in the middle code, but
> that would be very inefficient.
[...]
>=20
> The real problem with this is that to request the result of an sql functi=
on,
> of which count() is an example, additional api functionality is needed.
> But what happens if I want other functions, such as SUM or MAX or AVERAGE
> over a few columns? Each of these functions may take complex parameters.
> Again, the functionality may be replicated in the middle code, but this w=
ould
> not only be a waste of development time, but also be very inefficient.
>=20
> I propose either a generalization of "select" (which may be too complicat=
ed
> in its implications), or an addition of a func keyword option, e.g.
>=20
> func=3D'count'
> func=3D('funcName', param1, param2, ...)
> func=3D( ('funcName', param1, param2, ...), ('funcname2') )
>=20
> func=3D('sum','age')
>=20
> The question is then how is this information returned?
> I suggest as a tuple...
[snipped]
That's a very interesting idea, but this will take too much efforts
for it to be shortly developped. Let me explain that: if we do this
just as it sounds, then we will have *pieces* of raw sql in the middle
of generic command patterns --I don't like that. Do not misunderstand
me: it should be possible to abstract this in a certain way, but I
really wonder if it's worth the effort. And there's more problems,
consider this:
- either you simply want to sum()/avg()/max()/etc. on a table and
its attributes, and then I guess it's probably sufficient to offer
the possibility to fetch(sql=3D'select max(age) from author'), as
stated above;
- or you want to use this along with the automatic generation of
complex queries (e.g. with multiple joins): okay, but then you
must be able to say which attributes you want, and it's definitely
not sufficient to tell which table it belongs to: relations can be
reflexive (such as the 'pygmalion' relationship in the author/book
model), and in such cases you have two possibilities for
Author.age: table alias t1 or t2 (referring to the sql statements
above). This also means that this expression needs to be bound to
the automatic generation of sql statements.
I can't think of a straightforward way to do this by now. Again, I'm
not saying this is impossible: I'm just playing with the interesting
idea and explaining the difficulties I can foresee, wondering whether
such an advanced functionality would be worth the effort. That's an
open question, and for the moment being I suggest we do not take this
into account _as far as the API change proposal is concerned_. This
could be discussed in a separate thread, and it would help a lot if we
had some real-life examples showing where this could be very handy.
--> Same for 'group by' and 'having' statements by the way, since I'm not
really familiar with them either.
Mario> I would also add "by indicating clearly the small subset of methods
Mario> intended for use by client code, and the stability level".
Right. I'm still looking for a way to include this in the docstrings
so that the generation of the API can eat it (FYI we use epydoc).
-- S=E9bastien.
|
|
From: Mario R. <ma...@ru...> - 2003-06-17 18:32:10
|
On jeudi, juin 12, 2003, at 16:34 Europe/Amsterdam, Sebastien Bigaret=20
wrote:
>
> Hi all,
>
> Some more comments about the API change proposal; I concentrate here =
on
> EditingContext, and will post another message for CustomObject and
> related APIs.
>
> Note: I also received two private answers.
>
> * Again: methods will be aliased, not replaced,
>
> * EC.insert() and EC.delete() are okay to everyone.
>
> * autoInsertion()/setAutoInsertion(): the former is a getter, the
> latter, a setter
>
> Mario> OK. What about commit() for saveChanges()?
>
> * about commit(): that was initially in my proposal. I removed
> it. Remember I'm working on integrating the ZODB transaction and
> observation mechanisms into the modeling? Well, then, commit()=20
> clashes
> with py2.1 zodb's transaction protocol (and has a very different
> semantic than saveChanges(): it is equivalent to =
objectWillChange()).
> Hence I'd prefer to let it apart from the proposal for the moment
> being.
OK.
> * EC.fetch():
>
> The initial proposal is accepted by everyone so we'll keep it as a=20=
> basis.
>
> Mario> Another issue is that even with all the keyword options=20
> offered, one
> Mario> still has to "dip into" small pieces of raw sql very quickly,=20=
> as shown
> Mario> by your examples.
>
> Here I strongly disagree with you. It /seems/ that you dip into =
small
> pieces of raw sql, but it's not the case at all. Qualifiers and
> FetchSpecification properties offers a generic command language;
> consider this:
>
> ec.fetch('Book', 'author.pygmalion.lastName ilike "r*"')
>
> -> this is far from the generated sql:
>
> SELECT t0.id, t0.title, t0.FK_WRITER_ID, t0.PRICE
> FROM BOOK t0
> INNER JOIN ( WRITER t1
> INNER JOIN WRITER t2
> ON t1.FK_WRITER_ID=3Dt2.ID )
> ON t0.FK_WRITER_ID=3Dt1.ID
> WHERE UPPER(t2.LAST_NAME) LIKE UPPER('r%');
>
> In the framework we generally make the strong assumption that raw =
sql
> is taken out of the python code. Even if some of the keywords (such=20=
> as
> 'like'/'ilike' for qualifiers, or 'asc'/'desc' for ordering) are
> actually the same than the corresponding sql keywords, they are=20
> indeed
> decorrelated. See again the example above: the 'ilike' keyword is=20=
> not
> used, rather we compare UPPER()s (this might change in the future,=20=
> now
> that 'ilike' is a sql keyword accepted by a lot of db-servers ;)
OK, sorry I am not appreciating enough the high-levelness of the query=20=
api...
Your example certainly is a nice generic separation, and simplification.
However by "bits" of sql i intended really little bits, such as the age=20=
param
and the orderBy param, below:
ec.fetch('Writer', 'age>=3D80"')
ec.fetch('Writer', orderBy=3D'firstName asc, lastName desc', isDeep=3D1)
>
> Mario> This may be an acceptable compromise, but then client code=20
> should be
> Mario> allowed to pass any SQL it wants...
>
> That's an other problem, to which I completely agree: we should have=20=
> a
> mean to execute complete raw sql statements, and get the (raw) =
result
> back. Moreover, we should be able to transform the returned raw rows
> into real objects if necessary (and if possible).
Very good. For "casting" a raw row to an object, probably a simple
generic utility function could be enough. The only problem is the
order of the tuple, and the mapping onto the attributes.
>> Another detail is that the object loads all (direct) properties, =
every
>> time -- there may be no workaround for this, given the way the
>> framework is built.
> (also related to your 'resultset' proposal)
>
> Last, we should also be able to tell the framework to return the raw
> rows instead of fully initialized objects (which can later be=20
> reverted
> to real objects): there's a real need for that; sometimes you want =
to
> present a (very) long list of objects in a summary page/widget, but
> you do not need the full objects, not even every attributes, but a
> subset to present the user. Then the users selects one or more of=20
> this
> rows and that's where you'll transform the raw rows to real objects.
>
> Note that the framework architecture will have no problem to support
> this. Some of the needed APIs are already present but not =
implemented
> (such as DatabaseContext.faultForRawRow()).
>
> I thought this was on the todo list but it's not --I'll add that,
> since I've been thinking about these points for quite a long time.
OK
> Impact on the API:
>
> when this is implemented I suggest we add the following parameters =
to
> fetch():
>
> rawRows -- (default: false) return raw rows instead of objects
>
> sql -- execute the sql statement (and ignore all other parameters
> except entityName and rawRows --both optional in that case)
Ok for rawRows on fetch().
However, come to think of it, sql should be part of a different=20
function,
with a different interface, and that always returns raw rows.
Something like fetchSql().
As it is very nice to have a full query interface that is completely
db independent, it is also convenient to be able to access specific
db query api features.
This could also supercede fetchCount() and other such
possibilities. In this way, fetch() can be guaranteed
to be db-independent, while fetchSql() may not be.
>
> BTW I also suggest that the unsupported features in the fetch API =
are
> removed until they are implemented (such as limit/page/offset/etc).
OK.
>
>> I feel that some things (even if no one has requested them yet ;) are
>> missing...
>> for example, should one ever need to take advantage of sql "group by"=20=
>> and
>> "having".
>> Access to these sql clauses may be added later, without breaking this=20=
>> API,
>> which is OK. Also, such manips may be done in the middle code, but
>> that would be very inefficient.
> [...]
>>
>> The real problem with this is that to request the result of an sql=20
>> function,
>> of which count() is an example, additional api functionality is=20
>> needed.
>> But what happens if I want other functions, such as SUM or MAX or=20
>> AVERAGE
>> over a few columns? Each of these functions may take complex=20
>> parameters.
>> Again, the functionality may be replicated in the middle code, but=20
>> this would
>> not only be a waste of development time, but also be very =
inefficient.
>>
>> I propose either a generalization of "select" (which may be too=20
>> complicated
>> in its implications), or an addition of a func keyword option, e.g.
>>
>> func=3D'count'
>> func=3D('funcName', param1, param2, ...)
>> func=3D( ('funcName', param1, param2, ...), ('funcname2') )
>>
>> func=3D('sum','age')
>>
>> The question is then how is this information returned?
>> I suggest as a tuple...
> [snipped]
>
> That's a very interesting idea, but this will take too much efforts
> for it to be shortly developped. Let me explain that: if we do this
> just as it sounds, then we will have *pieces* of raw sql in the=20
> middle
> of generic command patterns --I don't like that. Do not =
misunderstand
> me: it should be possible to abstract this in a certain way, but I
> really wonder if it's worth the effort.
This is certainly not high priority. It is a good point to consider
now to know that you will not be stuck with a limited API.
> And there's more problems,
> consider this:
>
> - either you simply want to sum()/avg()/max()/etc. on a table and
> its attributes, and then I guess it's probably sufficient to=20
> offer
> the possibility to fetch(sql=3D'select max(age) from author'), =
as
> stated above;
>
> - or you want to use this along with the automatic generation of
> complex queries (e.g. with multiple joins): okay, but then you
> must be able to say which attributes you want, and it's=20
> definitely
> not sufficient to tell which table it belongs to: relations can=20=
> be
> reflexive (such as the 'pygmalion' relationship in the=20
> author/book
> model), and in such cases you have two possibilities for
> Author.age: table alias t1 or t2 (referring to the sql =
statements
> above). This also means that this expression needs to be bound =
to
> the automatic generation of sql statements.
Yes, this second one is certainly difficult. For now, separation of
possibility to make raw sql fetches should be enough.
> I can't think of a straightforward way to do this by now. Again, I'm
> not saying this is impossible: I'm just playing with the interesting
> idea and explaining the difficulties I can foresee, wondering =
whether
> such an advanced functionality would be worth the effort. That's an
> open question, and for the moment being I suggest we do not take =
this
> into account _as far as the API change proposal is concerned_. This
> could be discussed in a separate thread, and it would help a lot if=20=
> we
> had some real-life examples showing where this could be very handy.
> --> Same for 'group by' and 'having' statements by the way, since=20
> I'm not
> really familiar with them either.
>
>
> Mario> I would also add "by indicating clearly the small subset of=20
> methods
> Mario> intended for use by client code, and the stability level".
>
> Right. I'm still looking for a way to include this in the docstrings
> so that the generation of the API can eat it (FYI we use epydoc).
OK, great.
mario
> -- S=E9bastien.
|
|
From: Sebastien B. <sbi...@us...> - 2003-06-12 16:22:55
|
And now for CustomObject and related APIs!
I've thought about it quite a lot since the first proposal. Answers I
got, including two private ones, helped me a lot. Let me summarize all
this.
* Backward compatibility is of great concern. This means in particular
that KeyValueCoding will remain a mix-in for CustomObject, and that
KeyValueCoding will almost remain as-is. I say almost, because we
should deprecate (not remove) the methods: setValueForKey(),
setValueForKeyPath() and setStoredValueForKey(). As Mario already
wrote in a previous post, their only presence makes it harder to
understand the API and I fully agree.
I don't remember when I added this but this was not a brilliant idea.
Deprecation: means that they will issue a warning (python module
warnings) when they are used, and they will be removed at a given
release, to be determined (after 0.9). Note that the framework does
not use them at all.
What do you think?
* I think I made a big mistake when exposing the KeyValueCoding so early
in the User's Guide --again, Mario already pointed that out. This part
should be moved in an 'advanced techniques' part in the guide. I
suspect that this is /one/ of the main reasons why people tends to
focus on this api, despite the fact that it is definitely not needed
at first sight.
* What about the new API: I've come to the (still-to-be-discussed)
conclusion that it should not be mixed-in to CustomObject. While
making aliases in the framework's core is no problem, adding more
methods to CustomObject can induce some problem because we would then
add methods to the users' classes (this was pointed out by one of the
private mail I got). I tend to agree with that.
However what we can do is make an alternate module offering both the
new methods and a mix-in class that people could use in their own
classes when they prefer it. It seems that this solution addresses the
request for changing the name while preventing the set of users
classes' methods to grow unwillingly with aliased methods (no
functionality added).
* Last, I must say that the other private msg I got supports the KVC and
considers it better than the new props. As far as I'm concerned, I'm
not opposed to aliases, but I must admit that my preference goes to
the original KVC api, well, just like any former NeXTStep or EOF
developper I guess ;)
CustomObject
------------
We'll add globalID() in CustomObject if there is no argument
against. That would be a handy shortcut for:
obj.editingContext().globalIDForObject(self)
which is a common idiom for people willing to access the primary keys'
values while not making them class properties (this question is a
recurrent one)
Validation
----------
> > Do we need to change anything there? validateValueForKey() is the
> > longest one, however I think this is not widely used, is it?
>=20
> I suggest validateProperty() instead of validateValueForKey().
Here again, backward compatibility means that the old name will be
kept. Again, this adds an alias and no functionality to
CustomObject. Moreover, I suspect this is all but a commonly used
method. I hate saying that but I feel like going one step backward and
remove the proposal. Please shout at me if you disagree ;)
Back on the new KVC:
--------------------
Module's name: I'd strongly prefer if it could be explicit rather than
an acronym (opr). GenericAccess still has my preference but that's
no strong pref., I'll probably never use it so basically _you_ have
the lead!
Methods names: I'm okay to use the full names, but then they will
sensibly be of the same length than the original ones, compare:
co.getProperty()
--valueForKey--
co.setProperty()
--setValueForKey--
co.getStoredProperty()
--storedValueForKey--
co.setStoredProperty()
--setStoredValueForKey--
co.updateProperties()
--takeValuesFromDictionary--
co.updateStoredProperties()
--takeStoredValuesFromDictionary--
Up to you, however.
BTW I'll probably extend KVC with valuesForKeys(), counterpart for
takeValueForKey(). I was surprised to realize it was not there.
This also means to add something like 'properties()' to the new API
(which is maybe problematic, I've the feeling that this particular
name could easily clash with another one defined elsewhere).
However *if* we agree on what I said above, i.e. that the new KVC will
not be a mix-in for CustomObject by default, then choosing the new api
can be decorrelated to the changes made to the core. These changes
would then only be:
+ CustomObject.globalID()
+ KeyValueCoding.
(plus the changes made to EditingContext)
What do you think?
With best regards,
-- S=E9bastien.
|
|
From: Ernesto R. <er...@si...> - 2003-06-12 20:12:25
|
Don't take me too serious, I'm new to this list. I agree that, especially for summaries, like statistics (counts, sums, average, etc.) raw SQL access may be needed, I saw direct SQL calls in some of existing products. But they return raw rows, i.e. an array. No middletier functionality. On the otherhand, it may be interesting to encapsulate this for later enhacenments. In one of the papers I studied on this theme (http://www.ambysoft.com/persistenceLayer.pdf, pg 7, point 8.) I saw the use of proxy objects which are light weight objects with the most basic attributes (some user-key, name and few more). This idea could be extended, telling the fetch machinery to return objects with some additional, user requested attributes. On the other hand, we could have summary objects, because it's very usual to get sums, counts and averages. The idea is that the resultset would be an encapsulated a collection of summary objects: objects=ec.fetchSummary(sourceEntityName="Book", what="count(*), avg(price), sum(toInvoiceLines.amount)", where="author.name like 'John*'", groupBy="author, genre", resultEntityName="BookSummary") This may seem to be a SQL (OQL is very near to), but it has advantages over SQL: * the dotted notation is allowed (get away from ugly inner joins). * it would process inheritance trees * don't have to repeat the grouping fields in select clause, they are added automatically The only purpose of the entityName is to give an 'entry entity' which the attributes are related to. The result could be just an instance of 'SummaryObject', or perhaps an instance of a special BookSummary which encapsulates special behaviour. Should the result be a flat list or a tree-like list or dictionary? Erny |
|
From: Sebastien B. <sbi...@us...> - 2003-06-12 23:56:38
|
"Ernesto Revilla" <er...@si...> wrote: > Don't take me too serious, I'm new to this list. >=20 > I agree that, especially for summaries, like statistics (counts, sums, > average, etc.) raw SQL access may be needed, I saw direct SQL calls in so= me > of existing products. But they return raw rows, i.e. an array. No middlet= ier > functionality. On the otherhand, it may be interesting to encapsulate this > for later enhacenments. >=20 > In one of the papers I studied on this theme > (http://www.ambysoft.com/persistenceLayer.pdf, pg 7, point 8.) I saw the = use > of proxy objects which are light weight objects with the most basic > attributes (some user-key, name and few more). This idea could be extende= d, > telling the fetch machinery to return objects with some additional, user > requested attributes. The idea of proxies is quite interesting and notably pretty straightforward to implement. Suppose the ability to fetch raw rows is implemented, the common idiom would be (not speaking about the fetch() shortcut) something like this: >>> qual=3DqualifierWithQualifierFormat('title ilike "%t%"') >>> fs=3DFetchSpecification('Book', qual) >>> fs.setFetchesRawRows(true) >>> fs.setRawRowsKeyPath('id', 'title', 'author.lastName') >>> rows=3Dec.objectsWithFetchSpecification(fs) >>> print rows ({'id': 1, 'title': 'Gargantua', 'author_lastName': 'Rabelais'}, {'id': 2, 'title': 'Bouge ton pied...', 'author_lastName': 'Dard'}, {'id': 4, 'title': "T'assieds pas...", 'author_lastName': 'Dard'}, ) (note: I consider that dicts are better than arrays as a result set) Then it simply a matter of declaring:=20 >>> class GenericProxy: pass after which the creation and initialization of the proxies is just: >>> proxies=3D[GenericProxy() for i in range(len(rows))] >>> [proxies[i].__dict__.update(rows[i]) for i in range(len(rows))] That's all! And that's a really nice idea. > On the other hand, we could have summary objects, because it's very usual= to > get sums, counts and averages. The idea is that the resultset would be an > encapsulated a collection of summary objects: >=20 > objects=3Dec.fetchSummary(sourceEntityName=3D"Book", what=3D"count(*), av= g(price), > sum(toInvoiceLines.amount)", where=3D"author.name like 'John*'", > groupBy=3D"author, genre", resultEntityName=3D"BookSummary") >=20 > This may seem to be a SQL (OQL is very near to), but it has advantages > over SQL: > * the dotted notation is allowed (get away from ugly inner joins). > * it would process inheritance trees > * don't have to repeat the grouping fields in select clause, they are add= ed > automatically > The only purpose of the entityName is to give an 'entry entity' which the > attributes are related to. The result could be just an instance of > 'SummaryObject', or perhaps an instance of a special BookSummary which > encapsulates special behaviour. * You make me realize that my previous answer to Mario saying ''you'll have to be able to distinguish between table alias t1 and table alias t2 when declaring attribute author.age and that's a problem bla bla bla'' was stupid: the dotted notation obviously solves this naturally (the distinction between author.age and author.pygmalion.age is clear). * You're partially answering to the question of real-life examples --although I still need to familiarize myself with 'group by' and 'having' statements. * This brings a different light to what Mario formerly proposed (different api but the very same idea as far as I can see) --just because we here use a different method (fetchSummmary instead of fetch); sorry, sometimes I can't see the forest for the trees because of such details :| Now that the concept (summaries) is clearer to me I'm beginning to see the ideas behind. And I guess I'm not far from thinking that, okay, fetchSummary() does contain raw sql pieces, but that it's maybe not such a big deal if this covers common needs. * could this be made clearer: despite the fact that I cannot really see its implication for the moment being, I have the feeling that 'group by' and multiple fetches (walking through the inheritance tree) is not only a matter of getting the results and assembling them one after the other, is it? > Should the result be a flat list or a tree-like list or dictionary? I cannot answer that: as I said I'm not familiar enough with this kind of queries to see what form would be the most accurate, or whether it could depend on the query itself. I'd need different examples and your opinion here. BTW I suggest that we do not consider this as an element of the current API change proposal, rather as a new feature request. Last, you wrote: > On the other hand, we could have summary objects, because it's very > usual to get sums, counts and averages. I'd like to hear more about those usual cases, if you could give some examples that would help --help me, at least ;) since my personal approach to that kind of things would be e.g. to treat the raw rows on the python side rather than to ask the rdbms to do the job. -- S=E9bastien. |
|
From: Ernesto R. <er...@si...> - 2003-06-13 01:18:58
|
Examples for totals (sums) and summaries:
As we study economic data to decide if a company goes right, here are some
cases:
* Total revenues by month (group by year-month)
select month(year) as mes, sum(totalAmount) where date between
'2003-01-01' and '2003-12-31'
group by month (date)
for more than a year:
select year(date) as ano, month(year) as mes, sum(totalAmount) where
date between '2003-01-01' and '2003-12-31'
group by year(month), month(date)
The database does this in 2 seconds for 20.000 records
* costs, revenues and difference (benefit) by month
* customers who bought prodcuts with a total amount more than X (send them
some gifts)
select customer,sum(totalAmount) as Total from invoices where date
between.... group by customer having total>10.000
* costs to produce each order of the month (hours, materials, etc.)
* we use them a lot if we have to compare numbers as a whole or in groups,
for example, accounting with invoices (because invoice generate assets, but
user change invoices after assets have been done, and elsewhere change any
another data, so we check if the accounting is more or less ok, so the
company can pay VAT to the government.)
* consistency checks, e.g. see if anybody really has one address:
select person.id from person inner join address on
person.id=address.FK_person group by person.id having count(*)>1
* best-sellers:
select article,count(*) As numberSold from productLine group by article
having numberSold>20.000
Ok this could all be done in Python, but I think this is a bit slower. One
of our customers has something like 50.000 delivery notes a year, and this
is not much as bigger companies (we only have 'small' customers) can have
hundreds of thousands of documents a year. It's amazing how fast some
databases are. They only take 2 or 3 seconds to answer those questions. I
think Python takes at least 10 times more and consumes a lot of memory.
The implication on vertical inheritence mapping is that we have to retrieve
subtotals for each table and total them in python at the final step.
Although there are other operations like average, max, min, etc. but these
are easy done. To really get knowledge out of the data it doesn't make sense
that the result is too big, because we are only human. Perhaps 100-200 rows
max? So if each table summary is 100 records, we have to sum them for each
class of the inheritance tree, so we would have to process <1000 numbers in
python which is fast enough. (In real-life queries, we'll have often queries
much smaller with <100 numbers to process.)
The idea is that the query has not raw sql, as we need to process the
summary operations at both levels, first SQL than Python. The 'having'
operation should also be done at both levels (so db returns less data, and
Python only has to filter out the ultimate result.) Although 'where'
condition could be raw, Modeling already processes it the right way, so
nothing has to be done.
The SQL 'GROUP BY' returns a tuple for each combination of the the grouping
values, but it really expresses a tree (list of lists, or dictionary of
dictionaries): e.g. how many products of each has bought every customer on a
per month basis?
cust1 product1 month1 100
cust1 product1 month2 130
cust1 product1 month4 90 # 0 for month3 doesn't appear using
SQL
....
cust1 product2 month1 150
cust1 product2 month7 130
cust1 product3 month2 10
....
cust4 product1 month3 10
....
If this was a multi-dimensional structure (customer, product, date, units
bought), every summary is a projection into a space with fewer dimensions.
This is just was OLAP and data-warehounsing does: it picks up data from
several tables, several databases, etc. and puts it into a multidimensional
structure (getting effectively hypercubes) and throws projections to answer
business strategic questions.
This is not for the 0.9 release (nor perhaps for the 1.0 release, I don't
know). Better a bit slower but more solid. (We'll need at least one year
more for our software, although we have to pick up all the basic components
now: PyGTK & XML-RPC (client) - Modeling & Webware or Zope / XML-RPC
(middle-tier) - postgres (back-end) - reportlab (pdf - generation) -
cheetah, etc.)
Erny
----- Original Message -----
From: "Sebastien Bigaret" <sbi...@us...>
To: "modeling-users" <mod...@li...>
Sent: Friday, June 13, 2003 1:55 AM
Subject: Re: [Modeling-users] Summary objects and Proxy objects
"Ernesto Revilla" <er...@si...> wrote:
> Don't take me too serious, I'm new to this list.
>
> I agree that, especially for summaries, like statistics (counts, sums,
> average, etc.) raw SQL access may be needed, I saw direct SQL calls in
some
> of existing products. But they return raw rows, i.e. an array. No
middletier
> functionality. On the otherhand, it may be interesting to encapsulate this
> for later enhacenments.
>
> In one of the papers I studied on this theme
> (http://www.ambysoft.com/persistenceLayer.pdf, pg 7, point 8.) I saw the
use
> of proxy objects which are light weight objects with the most basic
> attributes (some user-key, name and few more). This idea could be
extended,
> telling the fetch machinery to return objects with some additional, user
> requested attributes.
The idea of proxies is quite interesting and notably pretty
straightforward to implement. Suppose the ability to fetch raw rows is
implemented, the common idiom would be (not speaking about the fetch()
shortcut) something like this:
>>> qual=qualifierWithQualifierFormat('title ilike "%t%"')
>>> fs=FetchSpecification('Book', qual)
>>> fs.setFetchesRawRows(true)
>>> fs.setRawRowsKeyPath('id', 'title', 'author.lastName')
>>> rows=ec.objectsWithFetchSpecification(fs)
>>> print rows
({'id': 1, 'title': 'Gargantua', 'author_lastName': 'Rabelais'},
{'id': 2, 'title': 'Bouge ton pied...', 'author_lastName': 'Dard'},
{'id': 4, 'title': "T'assieds pas...", 'author_lastName': 'Dard'},
)
(note: I consider that dicts are better than arrays as a result set)
Then it simply a matter of declaring:
>>> class GenericProxy: pass
after which the creation and initialization of the proxies is just:
>>> proxies=[GenericProxy() for i in range(len(rows))]
>>> [proxies[i].__dict__.update(rows[i]) for i in range(len(rows))]
That's all!
And that's a really nice idea.
> On the other hand, we could have summary objects, because it's very usual
to
> get sums, counts and averages. The idea is that the resultset would be an
> encapsulated a collection of summary objects:
>
> objects=ec.fetchSummary(sourceEntityName="Book", what="count(*),
avg(price),
> sum(toInvoiceLines.amount)", where="author.name like 'John*'",
> groupBy="author, genre", resultEntityName="BookSummary")
>
> This may seem to be a SQL (OQL is very near to), but it has advantages
> over SQL:
> * the dotted notation is allowed (get away from ugly inner joins).
> * it would process inheritance trees
> * don't have to repeat the grouping fields in select clause, they are
added
> automatically
> The only purpose of the entityName is to give an 'entry entity' which the
> attributes are related to. The result could be just an instance of
> 'SummaryObject', or perhaps an instance of a special BookSummary which
> encapsulates special behaviour.
* You make me realize that my previous answer to Mario saying ''you'll
have to be able to distinguish between table alias t1 and table alias
t2 when declaring attribute author.age and that's a problem bla bla
bla'' was stupid: the dotted notation obviously solves this naturally
(the distinction between author.age and author.pygmalion.age is
clear).
* You're partially answering to the question of real-life examples
--although I still need to familiarize myself with 'group by' and
'having' statements.
* This brings a different light to what Mario formerly proposed
(different api but the very same idea as far as I can see) --just
because we here use a different method (fetchSummmary instead of
fetch); sorry, sometimes I can't see the forest for the trees because
of such details :|
Now that the concept (summaries) is clearer to me I'm beginning to see
the ideas behind. And I guess I'm not far from thinking that, okay,
fetchSummary() does contain raw sql pieces, but that it's maybe not
such a big deal if this covers common needs.
* could this be made clearer: despite the fact that I cannot really see
its implication for the moment being, I have the feeling that 'group
by' and multiple fetches (walking through the inheritance tree) is not
only a matter of getting the results and assembling them one after the
other, is it?
> Should the result be a flat list or a tree-like list or dictionary?
I cannot answer that: as I said I'm not familiar enough with this kind
of queries to see what form would be the most accurate, or whether it
could depend on the query itself. I'd need different examples and your
opinion here.
BTW I suggest that we do not consider this as an element of the
current API change proposal, rather as a new feature request.
Last, you wrote:
> On the other hand, we could have summary objects, because it's very
> usual to get sums, counts and averages.
I'd like to hear more about those usual cases, if you could give some
examples that would help --help me, at least ;) since my personal
approach to that kind of things would be e.g. to treat the raw rows
on the python side rather than to ask the rdbms to do the job.
-- Sébastien.
-------------------------------------------------------
This SF.NET email is sponsored by: eBay
Great deals on office technology -- on eBay now! Click here:
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
_______________________________________________
Modeling-users mailing list
Mod...@li...
https://lists.sourceforge.net/lists/listinfo/modeling-users
|
|
From: Sebastien B. <sbi...@us...> - 2003-06-12 16:24:32
|
And now for CustomObject and related APIs!
I've thought about it quite a lot since the first proposal. Answers I
got, including two private ones, helped me a lot. Let me summarize all
this.
* Backward compatibility is of great concern. This means in particular
that KeyValueCoding will remain a mix-in for CustomObject, and that
KeyValueCoding will almost remain as-is. I say almost, because we
should deprecate (not remove) the methods: setValueForKey(),
setValueForKeyPath() and setStoredValueForKey(). As Mario already
wrote in a previous post, their only presence makes it harder to
understand the API and I fully agree.
I don't remember when I added this but this was not a brilliant idea.
Deprecation: means that they will issue a warning (python module
warnings) when they are used, and they will be removed at a given
release, to be determined (after 0.9). Note that the framework does
not use them at all.
What do you think?
* I think I made a big mistake when exposing the KeyValueCoding so early
in the User's Guide --again, Mario already pointed that out. This part
should be moved in an 'advanced techniques' part in the guide. I
suspect that this is /one/ of the main reasons why people tends to
focus on this api, despite the fact that it is definitely not needed
at first sight.
* What about the new API: I've come to the (still-to-be-discussed)
conclusion that it should not be mixed-in to CustomObject. While
making aliases in the framework's core is no problem, adding more
methods to CustomObject can induce some problem because we would then
add methods to the users' classes (this was pointed out by one of the
private mail I got). I tend to agree with that.
However what we can do is make an alternate module offering both the
new methods and a mix-in class that people could use in their own
classes when they prefer it. It seems that this solution addresses the
request for changing the name while preventing the set of users
classes' methods to grow unwillingly with aliased methods (no
functionality added).
* Last, I must say that the other private msg I got supports the KVC and
considers it better than the new props. As far as I'm concerned, I'm
not opposed to aliases, but I must admit that my preference goes to
the original KVC api, well, just like any former NeXTStep or EOF
developper I guess ;)
CustomObject
------------
We'll add globalID() in CustomObject if there is no argument
against. That would be a handy shortcut for:
obj.editingContext().globalIDForObject(self)
which is a common idiom for people willing to access the primary keys'
values while not making them class properties (this question is a
recurrent one)
Validation
----------
> > Do we need to change anything there? validateValueForKey() is the
> > longest one, however I think this is not widely used, is it?
>=20
> I suggest validateProperty() instead of validateValueForKey().
Here again, backward compatibility means that the old name will be
kept. Again, this adds an alias and no functionality to
CustomObject. Moreover, I suspect this is all but a commonly used
method. I hate saying that but I feel like going one step backward and
remove the proposal. Please shout at me if you disagree ;)
Back on the new KVC:
--------------------
Module's name: I'd strongly prefer if it could be explicit rather than
an acronym (opr). GenericAccess still has my preference but that's
no strong pref., I'll probably never use it so basically _you_ have
the lead!
Methods names: I'm okay to use the full names, but then they will
sensibly be of the same length than the original ones, compare:
co.getProperty()
--valueForKey--
co.setProperty()
--setValueForKey--
co.getStoredProperty()
--storedValueForKey--
co.setStoredProperty()
--setStoredValueForKey--
co.updateProperties()
--takeValuesFromDictionary--
co.updateStoredProperties()
--takeStoredValuesFromDictionary--
Up to you, however.
BTW I'll probably extend KVC with valuesForKeys(), counterpart for
takeValueForKey(). I was surprised to realize it was not there.
This also means to add something like 'properties()' to the new API
(which is maybe problematic, I've the feeling that this particular
name could easily clash with another one defined elsewhere).
However *if* we agree on what I said above, i.e. that the new KVC will
not be a mix-in for CustomObject by default, then choosing the new api
can be decorrelated to the changes made to the core. These changes
would then only be:
+ CustomObject.globalID()
+ KeyValueCoding.
(plus the changes made to EditingContext)
What do you think?
-- S=E9bastien.
|
|
From: Sebastien B. <sbi...@us...> - 2003-06-24 09:44:36
|
Hi,
Time to summarize the changes we discussed here:
EditingContext
--------------
- insert() is an alias for insertObject()
- delete() is an alias for deleteObject()
- fetching: remove the need to import FetchSpecification and
Qualifier, and propose an alternate for
objectsWithFetchSpecification()
def fetch(self, entityName,
qualifier=3DNone, # either a Qualifier instance or a string
isDeep=3D0, # should subentities be fetched as well?
)
Note that parameters orderBy, limit/page/offset, lock and rawRows
have been removed since they are unsupported yet (they will be
introduced when support is available).
- make fetchCount() an alias for objectsCountWithFetchSpecification()
[same API as fetch()]
- make (set)autoInsertion() aliases for
(set)propagatesInsertionForRelatedObjects()
=20=20=20=20
CustomObject
------------
- add globalID()
KeyValueCoding
--------------
- deprecate methods setValueForKey(), setValueForKeyPath() and
setStoredValueForKey()
I propose to set the removal time for these deprecated methods at
version 0.9.1
- add valuesForKeys(), counterpart for takeValuesFromDictionary()
Additionally, I propose to move the chapter dealing with KVC to an
other part in the User's Guide, in some 'advanced techniques' chapter,
so that it is not exposed as it is today.
Validation
----------
No changes (because of backward compatibility issues, mainly)
About the ``new'' KVC module:
-----------------------------
Do we agree that it can be separatedly defined so that users can use
it as a mix-in for their classes, at their will? If so, does anyone
want to take the lead for this? (meaning at least continuing the
discussion and coming to a decision)
Future enhancements
-------------------
Interesting future features (NOT included in this proposal) have been
proposed in this thread, including:
- fetchSQL(): the possibility to pass a raw sql query to build
objects,
- the ability to fetch raw rows (i.e. raw dict) rather than
fully-initialized objects,
- fetchSummary(): adds access to sum()/avg()/... and group by/having
to the fetching API
We'll probably need to discuss some of these points a bit more,
however if you think you'll need one or more of these for the coming
releases please fill in a RFE on sourceforge's page and announce it
here --this is definitely the best way to make it happen sooner.
I'd like to validate the API change proposal for the end of the week, if
possible, so I'd appreciate if you could comment on this in the coming
days. And BTW thanks a lot for the attention you already paid to this
topic.
-- S=E9bastien.
|
|
From: Sebastien B. <sbi...@us...> - 2003-07-01 19:21:21
|
Hi all,
Got no answer for this; I guess this has already been enough
discussed. Since the proposed changes is a subset of the initial one
that everybody aggreed on, I'll apply them on thursday evening (ie in
two days) --unless somebody comments further on this.
I'll then make a new release.
-- S=E9bastien.
Sebastien Bigaret <sbi...@us...> writes:
> Hi,
>=20
> Time to summarize the changes we discussed here:
>=20
> EditingContext
> --------------
>=20
> - insert() is an alias for insertObject()
>=20
> - delete() is an alias for deleteObject()
>=20
> - fetching: remove the need to import FetchSpecification and
> Qualifier, and propose an alternate for
> objectsWithFetchSpecification()
>=20
> def fetch(self, entityName,
> qualifier=3DNone, # either a Qualifier instance or a str=
ing
> isDeep=3D0, # should subentities be fetched as wel=
l?
> )
>=20
> Note that parameters orderBy, limit/page/offset, lock and rawRows
> have been removed since they are unsupported yet (they will be
> introduced when support is available).
>=20
> - make fetchCount() an alias for objectsCountWithFetchSpecification()
> [same API as fetch()]
>=20
> - make (set)autoInsertion() aliases for
> (set)propagatesInsertionForRelatedObjects()
>=20=20=20=20=20
>=20
> CustomObject
> ------------
>=20
> - add globalID()
>=20
> KeyValueCoding
> --------------
>=20
> - deprecate methods setValueForKey(), setValueForKeyPath() and
> setStoredValueForKey()
>=20
> I propose to set the removal time for these deprecated methods at
> version 0.9.1
>=20
> - add valuesForKeys(), counterpart for takeValuesFromDictionary()
>=20
> Additionally, I propose to move the chapter dealing with KVC to an
> other part in the User's Guide, in some 'advanced techniques' chapter,
> so that it is not exposed as it is today.
>=20
> Validation
> ----------
>=20
> No changes (because of backward compatibility issues, mainly)
>=20
>=20
> About the ``new'' KVC module:
> -----------------------------
>=20
> Do we agree that it can be separatedly defined so that users can use
> it as a mix-in for their classes, at their will? If so, does anyone
> want to take the lead for this? (meaning at least continuing the
> discussion and coming to a decision)
>=20
>=20
> Future enhancements
> -------------------
>=20
> Interesting future features (NOT included in this proposal) have been
> proposed in this thread, including:
>=20
> - fetchSQL(): the possibility to pass a raw sql query to build
> objects,
>=20
> - the ability to fetch raw rows (i.e. raw dict) rather than
> fully-initialized objects,
>=20
> - fetchSummary(): adds access to sum()/avg()/... and group by/having
> to the fetching API
>=20
> We'll probably need to discuss some of these points a bit more,
> however if you think you'll need one or more of these for the coming
> releases please fill in a RFE on sourceforge's page and announce it
> here --this is definitely the best way to make it happen sooner.
>=20
>=20
> I'd like to validate the API change proposal for the end of the week, if
> possible, so I'd appreciate if you could comment on this in the coming
> days. And BTW thanks a lot for the attention you already paid to this
> topic.
>=20
>=20
> -- S=E9bastien.
>=20
>=20
>=20
> -------------------------------------------------------
> This SF.Net email is sponsored by: INetU
> Attention Web Developers & Consultants: Become An INetU Hosting Partner.
> Refer Dedicated Servers. We Manage Them. You Get 10% Monthly Commission!
> INetU Dedicated Managed Hosting http://www.inetu.net/partner/index.php
> _______________________________________________
> Modeling-users mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modeling-users
|
|
From: Mario R. <ma...@ru...> - 2003-07-02 08:06:12
|
Hi,
sorry about this reply being so late... I agree with most of
what is summarized below, but have a few comments.
A general comment is about the "new" method being an alias
for the "old" method (this may only be a lingo issue). It seems
sensible to me to give the "new" or preferred method the
code, and other methods that need to be supported for
bw compatibility are the aliases (plus an appropriate docstring).
This also helps clarify which methods are to be standardized on.
On mardi, juin 24, 2003, at 11:43 Europe/Amsterdam, Sebastien Bigaret=20
wrote:
>
> Hi,
>
> Time to summarize the changes we discussed here:
>
> EditingContext
> --------------
>
> - insert() is an alias for insertObject()
>
> - delete() is an alias for deleteObject()
>
> - fetching: remove the need to import FetchSpecification and
> Qualifier, and propose an alternate for
> objectsWithFetchSpecification()
>
> def fetch(self, entityName,
> qualifier=3DNone, # either a Qualifier instance or a=20=
> string
> isDeep=3D0, # should subentities be fetched as=20=
> well?
> )
>
> Note that parameters orderBy, limit/page/offset, lock and rawRows
> have been removed since they are unsupported yet (they will be
> introduced when support is available).
OK.
> - make fetchCount() an alias for =
objectsCountWithFetchSpecification()
> [same API as fetch()]
Hmmn, I would not introduce such a specialized method (will become
clutter even in the short term). As suggested earlier, it may be more
flexible to just have a fetchSql() open method for now (that is not
necessarily db-independent) and that returns whatever the dbapi
returns. For me nothing is gained by having fetchCount('customobjname')
over fetchSql('select count(*) from tblname') -- it is jn fact less=20
flexibe.
When the time comes to generailize this to be db-independant, either
the fetch() method will gain new parameters to allow this, or other
generic methods introduced, such as fetchRaw, fetchTuples, fetchDicts,
or whatever. fetchSql can always have a use, though, so it it not likely
to become clutter.
> - make (set)autoInsertion() aliases for
> =
(set)propagatesInsertionForRelatedObjects()
OK
> CustomObject
> ------------
>
> - add globalID()
Yes. The scope of gloabl here is the EditingContext (and children)
or all EditingContexts in this python session, or the current model,
the db, ...
>
> KeyValueCoding
> --------------
>
> - deprecate methods setValueForKey(), setValueForKeyPath() and
> setStoredValueForKey()
>
> I propose to set the removal time for these deprecated methods at
> version 0.9.1
OK.
> - add valuesForKeys(), counterpart for takeValuesFromDictionary()
>
> Additionally, I propose to move the chapter dealing with KVC to an
> other part in the User's Guide, in some 'advanced techniques'=20
> chapter,
> so that it is not exposed as it is today.
OK, we can discuss this more (where to move this to in the guide).
> Validation
> ----------
>
> No changes (because of backward compatibility issues, mainly)
>
>
> About the ``new'' KVC module:
> -----------------------------
>
> Do we agree that it can be separatedly defined so that users can use
> it as a mix-in for their classes, at their will? If so, does anyone
> want to take the lead for this? (meaning at least continuing the
> discussion and coming to a decision)
I was under the impression that you said it will **not** be a mix-in=20
(as per:
http://sourceforge.net/mailarchive/message.php?msg_id=3D5250043 ).
The interface described in the archive message above is OK for me.
However, if it is to be a mix-in, discussion on this can continue=20
independently
of these changes? In any case, I will be happy to continue to be a part=20=
of
this discussion.
> Future enhancements
> -------------------
>
> Interesting future features (NOT included in this proposal) have =
been
> proposed in this thread, including:
>
> - fetchSQL(): the possibility to pass a raw sql query to build
> objects,
>
> - the ability to fetch raw rows (i.e. raw dict) rather than
> fully-initialized objects,
>
> - fetchSummary(): adds access to sum()/avg()/... and group=20
> by/having
> to the fetching API
>
> We'll probably need to discuss some of these points a bit more,
> however if you think you'll need one or more of these for the coming
> releases please fill in a RFE on sourceforge's page and announce it
> here --this is definitely the best way to make it happen sooner.
OK, except for fetchSQL. I think this should be not so difficult to add=20=
as
a generic method, and will take off the pressure on other premature
additions.
> I'd like to validate the API change proposal for the end of the week,=20=
> if
> possible, so I'd appreciate if you could comment on this in the coming
> days. And BTW thanks a lot for the attention you already paid to this
> topic.
>
>
> -- S=E9bastien.
Also, what about the renaming of =20
addObjectToBothSidesOfRelationshipWithKey()
and removeObjectFromBothSidesOfRelationshipWithKey(), respectively with
addViaRelation() and removeViaRelation() ?
And, yes I think RelationshipManipulation should be combined with KVC
(or whatever this is renamed to -- how about "Access" module
within a CustomObject sub-package ?)
mario
|
|
From: Jerome K. <Jer...@fi...> - 2003-07-02 09:22:39
|
Mario Ruggier wrote:
Hum .. A general note about aliasing methods: In fact
I use a couple of tools for test (SQLObject / MiddleKit/
and Modeling) so as MiddleKit and Modeling try to
monkey EOF, it use the same method names like
insertObject() .. so for me staying closer to EOF is a good
thing .
>> - make fetchCount() an alias for objectsCountWithFetchSpecification()
>> [same API as fetch()]
>
>
> Hmmn, I would not introduce such a specialized method (will become
> clutter even in the short term). As suggested earlier, it may be more
> flexible to just have a fetchSql() open method for now (that is not
> necessarily db-independent) and that returns whatever the dbapi
> returns. For me nothing is gained by having fetchCount('customobjname')
> over fetchSql('select count(*) from tblname') -- it is jn fact less
> flexibe.
> When the time comes to generailize this to be db-independant, either
> the fetch() method will gain new parameters to allow this, or other
> generic methods introduced, such as fetchRaw, fetchTuples, fetchDicts,
> or whatever. fetchSql can always have a use, though, so it it not likely
> to become clutter.
I guess you miss something Mario.
Let's take a example .
qualifier=qualifierWithQualifierFormat('lastName=="Hugo"')
fetchSpec=FetchSpecification(entityName='Writer', qualifier=qualifier)
fetchCount(fetchSpec) will return the number of writer with lastname =
'Hugo' ..
and so on this is not so trivial as a simple 'select count(*) ..'
so the answer is no : fetchCount() and fetchSql() haven't the same
purpose, even
is you can use fetSQL to do a fetchCount() or anything else.
And i really like the OO approach of modeling, and i really think that
even fetchSQL() ( which for me should be fetchSQLObject() ) should
alway return a object.
Take a look at this:
- in the model you describe the object mapping (which is not a real object
but only a "result object" .
for example:
CustomSearch
- searchString()
- books relation one-> many
- ...
- I thinks we should find a way to do
ec.objectsWithFetchSpecification('CustomSearch, ..searchString = 'Toto') ..
and so provide a method to do some special SQL in the CustomSearch() which
return CustomSearch object.
But this way you can do something like
search = CustomSearch(...)
books = search.getBooks() ( which return some books object .. not raw SQL)
>> Future enhancements
>> -------------------
>>
>> Interesting future features (NOT included in this proposal) have been
>> proposed in this thread, including:
>>
>> - fetchSQL(): the possibility to pass a raw sql query to build
>> objects,
>
To build Objects or raw SQL ?
Perhaps we can take a look on how SQLObject do that ?
>>
>> - the ability to fetch raw rows (i.e. raw dict) rather than
>> fully-initialized objects,
>>
yeah ..
Bye Bye ..
|
|
From: Mario R. <ma...@ru...> - 2003-07-02 09:35:03
|
On mercredi, juil 2, 2003, at 11:05 Europe/Amsterdam, Jerome Kerdreux
wrote:
> Mario Ruggier wrote:
>
>
> Hum .. A general note about aliasing methods: In fact
> I use a couple of tools for test (SQLObject / MiddleKit/
> and Modeling) so as MiddleKit and Modeling try to
> monkey EOF, it use the same method names like
> insertObject() .. so for me staying closer to EOF is a good
> thing .
That would be fine. My point was that whatever method
name is preferred, then this should be the "master" and
others that do same thing are aliases. This is irrespective of
whether the preferred method name is the same as in the
EOF or not.
>>> - make fetchCount() an alias for
>>> objectsCountWithFetchSpecification()
>>> [same API as fetch()]
>>
>>
>> Hmmn, I would not introduce such a specialized method (will become
>> clutter even in the short term). As suggested earlier, it may be more
>> flexible to just have a fetchSql() open method for now (that is not
>> necessarily db-independent) and that returns whatever the dbapi
>> returns. For me nothing is gained by having
>> fetchCount('customobjname')
>> over fetchSql('select count(*) from tblname') -- it is jn fact less
>> flexibe.
>> When the time comes to generailize this to be db-independant, either
>> the fetch() method will gain new parameters to allow this, or other
>> generic methods introduced, such as fetchRaw, fetchTuples, fetchDicts,
>> or whatever. fetchSql can always have a use, though, so it it not
>> likely
>> to become clutter.
>
>
> I guess you miss something Mario.
> Let's take a example .
> qualifier=qualifierWithQualifierFormat('lastName=="Hugo"')
> fetchSpec=FetchSpecification(entityName='Writer', qualifier=qualifier)
>
> fetchCount(fetchSpec) will return the number of writer with lastname =
> 'Hugo' ..
> and so on this is not so trivial as a simple 'select count(*) ..'
>
> so the answer is no : fetchCount() and fetchSql() haven't the same
> purpose, even
> is you can use fetSQL to do a fetchCount() or anything else.
Fair enough. fetchSQL is an open-ended convenience hack, that
allows more time to think over what a more generalized API for
fetch() could be -- so that it would handle in an OO way queries
such as qualified counts, sums, etc. I suspect that defining
a fetchCount will lead to other unnecessary definitions of other
"dedicated" methods (that could all be addressed with a well-thought
out, and simpler, fetch+ interface).
> And i really like the OO approach of modeling, and i really think that
> even fetchSQL() ( which for me should be fetchSQLObject() ) should
> alway return a object.
Yes, agreed. To achieve this, there needs to be a way to allow
"generic custom objects" to be returned by fetches, i.e. objects
that do not necessarily correspond to a class defined in a model
(such as for count, sum, ...), or, as you nicely describe below,
object that 'is not a real object but only a "result object"'.
>
> Take a look at this:
> - in the model you describe the object mapping (which is not a real
> object
> but only a "result object" . for example:
> CustomSearch
> - searchString()
> - books relation one-> many
> - ...
> - I thinks we should find a way to do
> ec.objectsWithFetchSpecification('CustomSearch, ..searchString =
> 'Toto') ..
> and so provide a method to do some special SQL in the CustomSearch()
> which
> return CustomSearch object.
>
> But this way you can do something like
> search = CustomSearch(...)
> books = search.getBooks() ( which return some books object .. not raw
> SQL)
Absolutely. It would be wonderful to have fetch API also be able
to handle this. The question is how to generalize the API to achieve
all these features (plus be easily extensible) while keeping it as
simple as possible.
mario
|
|
From: Sebastien B. <sbi...@us...> - 2003-07-02 21:59:03
|
Hi,
Mario Ruggier <ma...@ru...> wrote:
> On mercredi, juil 2, 2003, at 11:05 Europe/Amsterdam, Jerome Kerdreux wro=
te:
> > Mario Ruggier wrote:
> >
> > A general comment is about the "new" method being an alias for the "old"
> > method (this may only be a lingo issue). It seems sensible to me to give
> > the "new" or preferred method the code, and other methods that need to =
be
> > supported for bw compatibility are the aliases (plus an appropriate
> > docstring). This also helps clarify which methods are to be standardiz=
ed
> > on.
> > Hum .. A general note about aliasing methods: In fact
> > I use a couple of tools for test (SQLObject / MiddleKit/
> > and Modeling) so as MiddleKit and Modeling try to
> > monkey EOF, it use the same method names like
> > insertObject() .. so for me staying closer to EOF is a good
> > thing .
>=20
> That would be fine. My point was that whatever method
> name is preferred, then this should be the "master" and
> others that do same thing are aliases. This is irrespective of
> whether the preferred method name is the same as in the
> EOF or not.
I see your points here, both; I have no definitive answer, my point of
view is a bit different.
- real aliases, i.e. methods w/ different name but w/ the very same
functionality as the aliased methods (insert(), delete(),
setAutoInsertion), all these have in my mind the very same status.
In the code point of view, they'll be made by simple assignments:
'insert=3DinsertObject'; they'll actually *be* the very same method,
sharing code and docstrings.
In a developer's point of view, I guess the preference will go here
or there; if you're used to sql, 'insert' will probably be natural,
while 'insertObject' will come up more easily if you already know
SQLObject or the EOF.
So I won't say there will be a preference in itself between these,
only a personal preference. Even if they appear in different
circumstances, historical or at users' request, they are, and probably
will remain, identical.
- Framework's enhancements: fetch(), fetchCount()
These are not just aliases: you can think of the fa=E7ade design
pattern. They alleviate the developer's work by offering a simpler and
handier API to fetch objects.
Here again, I won't say that there will be a preferred one in the
absolute. fetch() is a real enhancement and I'm glad it's showing up,
because having to import modules, then instanciate a Qualifier and a
FetchSpecification just to fetch some objects is just more painful
than simply call fetch with the needed parameters.
However, the "old way" remains meaningful --and different, because
it offers plain control over the different objects collaborating
within the framework at some point (here, when fetching). It will
eventually be moved into an 'advanced techniques' section, of course,
and handier methods will be introduced first; after all, the framework
is supposed to make it easy to access to objects stored in a db!
An other slighty-different question to which Mario was probably
referring to is what are the methods you need to know at first sight,
and what are those that you can safely ignore if you do not want to
fine-tune the framework, or if you don't have time to dive too deeply
into it. This is an important issue I'm aware of. I already said here
that I'm looking for a solution to distinguish the one from the others
in the documentation. I haven't spent on it, still, but this *will* be
addressed in the future.
Sorry, I leave the other interesting points pending --mostly because
they are out of the scope of the actual API changes, and because I
need some rest now ;) But we'll get back to these for sure, and maybe
Soif could elaborate a bit further on his CustomSearch example? It
would be interesting to see how this compares to the other approaches
we already discussed here.
Cheers,
-- S=E9bastien.
|
|
From: Jerome K. <Jer...@fi...> - 2003-07-02 11:20:09
|
Mario Ruggier wrote:
Hum .. A general note about aliasing methods: In fact
I use a couple of tools for test (SQLObject / MiddleKit/
and Modeling) so as MiddleKit and Modeling try to
monkey EOF, it use the same method names like
insertObject() .. so for me staying closer to EOF is a good
thing .
>> - make fetchCount() an alias for objectsCountWithFetchSpecification()
>> [same API as fetch()]
>
>
> Hmmn, I would not introduce such a specialized method (will become
> clutter even in the short term). As suggested earlier, it may be more
> flexible to just have a fetchSql() open method for now (that is not
> necessarily db-independent) and that returns whatever the dbapi
> returns. For me nothing is gained by having fetchCount('customobjname')
> over fetchSql('select count(*) from tblname') -- it is jn fact less
> flexibe.
> When the time comes to generailize this to be db-independant, either
> the fetch() method will gain new parameters to allow this, or other
> generic methods introduced, such as fetchRaw, fetchTuples, fetchDicts,
> or whatever. fetchSql can always have a use, though, so it it not likely
> to become clutter.
I guess you miss something Mario.
Let's take a example .
qualifier=qualifierWithQualifierFormat('lastName=="Hugo"')
fetchSpec=FetchSpecification(entityName='Writer', qualifier=qualifier)
fetchCount(fetchSpec) will return the number of writer with lastname =
'Hugo' ..
and so on this is not so trivial as a simple 'select count(*) ..'
so the answer is no : fetchCount() and fetchSql() haven't the same
purpose, even
is you can use fetSQL to do a fetchCount() or anything else.
And i really like the OO approach of modeling, and i really think that
even fetchSQL() ( which for me should be fetchSQLObject() ) should
alway return a object.
Take a look at this:
- in the model you describe the object mapping (which is not a real object
but only a "result object" .
for example:
CustomSearch
- searchString()
- books relation one-> many
- ...
- I thinks we should find a way to do
ec.objectsWithFetchSpecification('CustomSearch, ..searchString = 'Toto') ..
and so provide a method to do some special SQL in the CustomSearch() which
return CustomSearch object.
But this way you can do something like
search = CustomSearch(...)
books = search.getBooks() ( which return some books object .. not raw SQL)
>Future enhancements
>-------------------
> Interesting future features (NOT included in this proposal) have been
>proposed in this thread, including:
>- fetchSQL(): the possibility to pass a raw sql query to build
> objects,
To build Objects or raw SQL ?
Perhaps we can take a look on how SQLObject do that ?
> - the ability to fetch raw rows (i.e. raw dict) rather than
> fully-initialized objects,
yeah ..
Bye Bye
Bye Bye ..
|
Thanks for helping keep SourceForge clean.
X