Re: [Modeling-users] API cleanup proposal: EditingContext

[Mario R. <ma...@ru...>]

On jeudi, juin 12, 2003, at 16:34 Europe/Amsterdam, Sebastien Bigaret=20
wrote:
>
>         Hi all,
>
> Some more comments about the API change proposal; I concentrate here =
on
> EditingContext, and will post another message for CustomObject and
> related APIs.
>
> Note: I also received two private answers.
>
> * Again: methods will be aliased, not replaced,
>
> * EC.insert() and EC.delete() are okay to everyone.
>
> * autoInsertion()/setAutoInsertion(): the former is a getter, the
>   latter, a setter
>
> Mario> OK. What about commit() for saveChanges()?
>
> * about commit(): that was initially in my proposal. I removed
>   it. Remember I'm working on integrating the ZODB transaction and
>   observation mechanisms into the modeling? Well, then, commit()=20
> clashes
>   with py2.1 zodb's transaction protocol (and has a very different
>   semantic than saveChanges(): it is equivalent to =
objectWillChange()).
>   Hence I'd prefer to let it apart from the proposal for the moment
>   being.

OK.

> * EC.fetch():
>
>   The initial proposal is accepted by everyone so we'll keep it as a=20=

> basis.
>
> Mario> Another issue is that even with all the keyword options=20
> offered, one
> Mario> still has to "dip into" small pieces of raw sql very quickly,=20=

> as shown
> Mario> by your examples.
>
>   Here I strongly disagree with you. It /seems/ that you dip into =
small
>   pieces of raw sql, but it's not the case at all. Qualifiers and
>   FetchSpecification properties offers a generic command language;
>   consider this:
>
>     ec.fetch('Book', 'author.pygmalion.lastName ilike "r*"')
>
>   -> this is far from the generated sql:
>
>         SELECT t0.id, t0.title, t0.FK_WRITER_ID, t0.PRICE
>         FROM BOOK t0
>           INNER JOIN  ( WRITER t1
>                         INNER JOIN WRITER t2
>                         ON t1.FK_WRITER_ID=3Dt2.ID )
>           ON t0.FK_WRITER_ID=3Dt1.ID
>         WHERE UPPER(t2.LAST_NAME) LIKE UPPER('r%');
>
>   In the framework we generally make the strong assumption that raw =
sql
>   is taken out of the python code. Even if some of the keywords (such=20=

> as
>   'like'/'ilike' for qualifiers, or 'asc'/'desc' for ordering) are
>   actually the same than the corresponding sql keywords, they are=20
> indeed
>   decorrelated.  See again the example above: the 'ilike' keyword is=20=

> not
>   used, rather we compare UPPER()s (this might change in the future,=20=

> now
>   that 'ilike' is a sql keyword accepted by a lot of db-servers ;)

OK, sorry I am not appreciating enough the high-levelness of the query=20=

api...
Your example certainly is a nice generic separation, and simplification.
However by "bits" of sql i intended really little bits, such as the age=20=

param
and the orderBy param, below:

ec.fetch('Writer', 'age>=3D80"')
ec.fetch('Writer', orderBy=3D'firstName asc, lastName desc', isDeep=3D1)

>
> Mario> This may be an acceptable compromise, but then client code=20
> should be
> Mario> allowed to pass any SQL it wants...
>
>   That's an other problem, to which I completely agree: we should have=20=

> a
>   mean to execute complete raw sql statements, and get the (raw) =
result
>   back. Moreover, we should be able to transform the returned raw rows
>   into real objects if necessary (and if possible).

Very good. For "casting" a raw row to an object, probably a simple
generic utility function could be enough. The only problem is the
order of the tuple, and the mapping onto the attributes.

>> Another detail is that the object loads all (direct) properties, =
every
>> time -- there may be no workaround for this, given the way the
>> framework is built.
> (also related to your 'resultset' proposal)
>
>   Last, we should also be able to tell the framework to return the raw
>   rows instead of fully initialized objects (which can later be=20
> reverted
>   to real objects): there's a real need for that; sometimes you want =
to
>   present a (very) long list of objects in a summary page/widget, but
>   you do not need the full objects, not even every attributes, but a
>   subset to present the user. Then the users selects one or more of=20
> this
>   rows and that's where you'll transform the raw rows to real objects.
>
>   Note that the framework architecture will have no problem to support
>   this. Some of the needed APIs are already present but not =
implemented
>   (such as DatabaseContext.faultForRawRow()).
>
>   I thought this was on the todo list but it's not --I'll add that,
>   since I've been thinking about these points for quite a long time.

OK

> Impact on the API:
>
>   when this is implemented I suggest we add the following parameters =
to
>   fetch():
>
>     rawRows -- (default: false) return raw rows instead of objects
>
>     sql -- execute the sql statement (and ignore all other parameters
>            except entityName and rawRows --both optional in that case)

Ok for rawRows on fetch().
However, come to think of it, sql should be part of a different=20
function,
with a different interface, and that always returns raw rows.
Something like fetchSql().

As it is very nice to have a full query interface that is completely
db independent, it is also convenient to be able to access specific
db query api features.

This could also supercede fetchCount() and other such
possibilities. In this way, fetch() can be guaranteed
to be db-independent, while fetchSql() may not be.

>
>   BTW I also suggest that the unsupported features in the fetch API =
are
>   removed until they are implemented (such as limit/page/offset/etc).

OK.

>
>> I feel that some things (even if no one has requested them yet ;) are
>> missing...
>> for example, should one ever need to take advantage of sql "group by"=20=

>> and
>> "having".
>> Access to these sql clauses may be added later, without breaking this=20=

>> API,
>> which is OK. Also, such manips may be done in the middle code, but
>> that would be very inefficient.
> [...]
>>
>> The real problem with this is that to request the result of an sql=20
>> function,
>> of which count() is an example, additional api functionality is=20
>> needed.
>> But what happens if I want other functions, such as SUM or MAX or=20
>> AVERAGE
>> over a few columns? Each of these functions may take complex=20
>> parameters.
>> Again, the functionality may be replicated in the middle code, but=20
>> this would
>> not only be a waste of development time, but also be very =
inefficient.
>>
>> I propose either a generalization of "select" (which may be too=20
>> complicated
>> in its implications), or an addition of a func keyword option, e.g.
>>
>> func=3D'count'
>> func=3D('funcName', param1, param2, ...)
>> func=3D( ('funcName', param1, param2, ...), ('funcname2') )
>>
>> func=3D('sum','age')
>>
>> The question is then how is this information returned?
>> I suggest as a tuple...
> [snipped]
>
>   That's a very interesting idea, but this will take too much efforts
>   for it to be shortly developped. Let me explain that: if we do this
>   just as it sounds, then we will have *pieces* of raw sql in the=20
> middle
>   of generic command patterns --I don't like that. Do not =
misunderstand
>   me: it should be possible to abstract this in a certain way, but I
>   really wonder if it's worth the effort.

This is certainly not high priority. It is a good point to consider
now to know that you will not be stuck with a limited API.

> And there's more problems,
>   consider this:
>
>     - either you simply want to sum()/avg()/max()/etc. on a table and
>       its attributes, and then I guess it's probably sufficient to=20
> offer
>       the possibility to fetch(sql=3D'select max(age) from author'), =
as
>       stated above;
>
>     - or you want to use this along with the automatic generation of
>       complex queries (e.g. with multiple joins): okay, but then you
>       must be able to say which attributes you want, and it's=20
> definitely
>       not sufficient to tell which table it belongs to: relations can=20=

> be
>       reflexive (such as the 'pygmalion' relationship in the=20
> author/book
>       model), and in such cases you have two possibilities for
>       Author.age: table alias t1 or t2 (referring to the sql =
statements
>       above). This also means that this expression needs to be bound =
to
>       the automatic generation of sql statements.

Yes, this second one is certainly difficult. For now, separation of
possibility to make raw sql fetches should be enough.

>   I can't think of a straightforward way to do this by now. Again, I'm
>   not saying this is impossible: I'm just playing with the interesting
>   idea and explaining the difficulties I can foresee, wondering =
whether
>   such an advanced functionality would be worth the effort. That's an
>   open question, and for the moment being I suggest we do not take =
this
>   into account _as far as the API change proposal is concerned_. This
>   could be discussed in a separate thread, and it would help a lot if=20=

> we
>   had some real-life examples showing where this could be very handy.
>   --> Same for 'group by' and 'having' statements by the way, since=20
> I'm not
>   really familiar with them either.
>
>
> Mario> I would also add "by indicating clearly the small subset of=20
> methods
> Mario> intended for use by client code, and the stability level".
>
>   Right. I'm still looking for a way to include this in the docstrings
>   so that the generation of the API can eat it (FYI we use epydoc).

OK, great.

mario


> -- S=E9bastien.