modeling-users — For users: questions, support, announcements, etc.

Re: [Modeling-users] Foreign key required for 1:1 relations Schema update thru differences

[Sebastien B. <sbi...@us...>]

        Hi Erny and all,

"Ernesto Revilla" <er...@si...> wrote:
> I started modeling with the ZModelingTool and the following issues arose:
>=20
> * When I modeled
>             1                                    0..*
>          addresses                         person
> Person <---------------------------->>address
>=20
> although I told in the relation in entity address that it had to map '1:1'
> to Person, it didn't set the FK_Person to required. During model validati=
on
> I got the warning :
> - relationship is mandatory  but source attribute direccion.FK_PERSONA is
> not required.
>
> All in all, I see the tool a bit redundant, because:
> * the former issue should be done automatically

I guess you declared the FK, than the relationship, and then set the
relationship's multiplicity to 1..1.

Note: If you had used the 'Add relationship' stuff on the model's
      properties page, the FK would have been created and set to
      required, as expected.

That's completely normal --and that's why the validation is here. I
understand you have the feeling that this is redundant, but in fact it's
not (even if most of the time it is). The idea here is that it's just a
warning, moreover it's sometimes exactly what you want: for example, you
do not want the generated DB schema to include the 'NOT NULL' stuff for
a given field if you know you'll populate this with existing data that
are not fully correct (so we don't want to change the FK property
automatically).

The best way of handling this would be to check the FK's properties when
a relationship's multiplicity changes ; then ask the user if he wants to
update it as well. But you must understand that if we're going that way,
then I think there are a lot of such warnings/suggestions to users to
issue at almost each possible changes. This would complicate the tool a
lot, and I have the feeling that a web interface is not really
appropriate for this.

  Here's the philosophy currently underlying the ZModeler: design your
  model, check the model, correct it when applicable, iterate ;)

> * I would like to map all python string to varchar (w/o limit) or text (in
> fact, they should all be mapped to the same (perhaps system dependent) DB
> type

Right, the ZModeler does not handle this for the moment being. See below
the discussion on PyModels.

> * typeName is required but not used

It's not required, it's just an 'info' message. But okay, since it's not
used for the moment being it might be dropped (and from the interface as
well). For the curious, this field originally came from a previous
version of the framework (never publicly released), where it was based
on ZODB+ZCatalog (no rdbms at all), and the field was used to generate
python and/or cmf zproducts.

> * name, className, moduleName and externalName have all to be specified,
> perhaps the latter three should be defaulted to the first if not specifie=
d.

When you create a class within the tool, they are automatically
specified, ain't they?

> Sorry, this should not sound too destructive. I really appreciate a lot a=
ll
> the work done. I just think that the XML-file is too explicit (with all t=
he
> required fields). Perhaps the semantics could be loosened a bit, making al
> lot of the attributes optional (like moduleName, packageName, externalTyp=
e,
> etc.)

No need to worry. We really need to know what annoys new users --most of
us here are far too accustomed to the way it reacts to be able to detect
those things.

About the XML file: yes, it is very explicit indeed. That's why the
ZModelizationTool was built anyway, because it's almost impossible to
build a xml-model from scratch.

  You already pointed that out: even the defaults (applied when a field
  is not present in the xml file) are not clearly stated. The fact is
  that the xml model should be thought as a complete snapshot for a
  model (serialization of a model). This sounds reasonable, but I must
  admit that the very reason for this situation is that the classes
  Model, Entity, Attribute and Relationship were the starting point of
  the project; if you look at their code, you'll probably notice that
  they are messier than others' --they need refactoring, but it's never
  been done since the beginning (they lived through two major rewrite of
  the framework before the current one), they've just grown from what
  they were to what they are and obviously they lack clear support for
  things like clear definition of defaults.

Now that applications are deployed, it's complicated to change this,
just because some people are using the 'observed' defaults which should
not be changed from one version to another. Say it differently:
refactoring these classes requires an exhaustive review of every single
defaulting behaviour. And that's not a high-priority item... (same for
the ZModelizationTool however, whose code I'm pretty ashamed of, <g>,
ugly one)

  Have you looked at the PyModel proposal (implementation has begun and
will probably be integrated into the main trunk in a near future) ??

The original post by Mario is at:
https://sourceforge.net/mailarchive/forum.php?thread_id=3D1702463&forum_id=
=3D10674
then the full thread is at:
https://sourceforge.net/mailarchive/forum.php?thread_id=3D1703927&forum_id=
=3D10674
and
https://sourceforge.net/mailarchive/forum.php?thread_id=3D1755270&forum_id=
=3D10674

(sorry about the thread being splitted in three parts in the archives,
 my fault: my mailer was not correctly configured at that time--I can
 digest and send the whole stuff to you if you wish)

--> the PyModels design was initiated because of the lack of readability
    and clarity of the xml model. They contains exactly what you're
    looking for: defaults (so you can tell that every string attribute
    are TEXT), and a wide range of verbosity, allowing you to declare
    the minimal amount of required stuff up to the whole set of possible
    properties).

If you want to give a try, you can get the cvs branch tagged
'brch-0_9pre7-1-PyModel'

> If I can help, please tell me how. (I think I really need this project for
> our own (ERP).)

Reporting all these things already helps. If you feel like patching the
ZModeler for the FK problem you reported, go for it! Or you can simply
gather all annoyances you notice using the ZModeler and then submit a
feature request. I have the feeling that after the PyModel are released,
they could benefit a lot to the Ztool.

  Do not misunderstand my answers in general: I always try to make it
  explicit why things are like this or that, even when there's nothing
  to be proud of :/  When I say that sth is on the TODO list but
  low-priority, I basically mean that I already have too much to do with
  high-priority items, so *I* probably won't have the time to
  concentrate on low-priority items. Obviously, I'll always be happy to
  help anybody willing to change/enhance things.  Collaborations can
  take various forms (up to transfer of responsability for a given
  module, say, the ZModeler:), and as a general rule discussing these
  things generally tends to augment their priority level.

Again, thanks for reporting, and keep it up, we need fresh eyes!

        Cheers,

-- S=E9bastien.

Re: [Modeling-users] Summary objects and Proxy objects

[Ernesto R. <er...@si...>]

Examples for totals (sums) and summaries:

As we study economic data to decide if a company goes right, here are some
cases:
* Total revenues by month (group by year-month)
    select month(year) as mes, sum(totalAmount) where date between
'2003-01-01' and '2003-12-31'
    group by month (date)
    for more than a year:
    select year(date) as ano, month(year) as mes, sum(totalAmount) where
date between '2003-01-01' and '2003-12-31'
    group by year(month), month(date)
    The database does this in 2 seconds for 20.000 records

* costs, revenues and difference (benefit) by month

* customers who bought prodcuts with a total amount more than X (send them
some gifts)
   select customer,sum(totalAmount) as Total from invoices where date
between.... group by customer having total>10.000

* costs to produce each order of the month (hours, materials, etc.)

* we use them a lot if we have to compare numbers as a whole or in groups,
for example, accounting with invoices (because invoice generate assets, but
user change invoices after assets have been done, and elsewhere change any
another data, so we check if the accounting is more or less ok, so the
company can pay VAT to the government.)

* consistency checks, e.g. see if anybody really has one address:
select person.id from person inner join address on
person.id=address.FK_person group by person.id having count(*)>1

* best-sellers:
select article,count(*) As numberSold from productLine group by article
having numberSold>20.000

Ok this could all be done in Python, but I think this is a bit slower. One
of our customers has something like 50.000 delivery notes a year, and this
is not much as bigger companies (we only have 'small' customers) can have
hundreds of thousands of documents a year. It's amazing how fast some
databases are. They only take 2 or 3 seconds to answer those questions. I
think Python takes at least 10 times more and consumes a lot of memory.

The implication on vertical inheritence mapping is that we have to retrieve
subtotals for each table and total them in python at the final step.
Although there are other operations like average, max, min, etc. but these
are easy done. To really get knowledge out of the data it doesn't make sense
that the result is too big, because we are only human. Perhaps 100-200 rows
max? So if each table summary is 100 records, we have to sum them for each
class of the inheritance tree, so we would have to process <1000 numbers in
python which is fast enough. (In real-life queries, we'll have often queries
much smaller with <100 numbers to process.)

The idea is that the query has not raw sql, as we need to process the
summary operations at both levels, first SQL than Python. The 'having'
operation should also be done at both levels (so db returns less data, and
Python only has to filter out the ultimate result.) Although 'where'
condition could be raw, Modeling already processes it the right way, so
nothing has to be done.

The SQL 'GROUP BY' returns a tuple for each combination of the the grouping
values, but it really expresses a tree (list of lists, or dictionary of
dictionaries): e.g. how many products of each has bought every customer on a
per month basis?

cust1    product1    month1    100
cust1    product1    month2    130
cust1    product1    month4    90        # 0 for month3 doesn't appear using
SQL
....
cust1    product2    month1    150
cust1    product2    month7    130
cust1    product3    month2    10
....
cust4    product1    month3    10
....

If this was a multi-dimensional structure (customer, product, date, units
bought), every summary is a projection into a space with fewer dimensions.
This is just was OLAP and data-warehounsing does: it picks up data from
several tables, several databases, etc. and puts it into a multidimensional
structure (getting effectively hypercubes) and throws projections to answer
business strategic questions.

This is not for the 0.9 release (nor perhaps for the 1.0 release, I don't
know). Better a bit slower but more solid. (We'll need at least one year
more for our software, although we have to pick up all the basic components
now: PyGTK & XML-RPC (client)  -  Modeling & Webware or Zope / XML-RPC
(middle-tier) - postgres (back-end) - reportlab (pdf - generation) -
cheetah, etc.)

Erny


----- Original Message -----
From: "Sebastien Bigaret" <sbi...@us...>
To: "modeling-users" <mod...@li...>
Sent: Friday, June 13, 2003 1:55 AM
Subject: Re: [Modeling-users] Summary objects and Proxy objects



"Ernesto Revilla" <er...@si...> wrote:
> Don't take me too serious, I'm new to this list.
>
> I agree that, especially for summaries, like statistics (counts, sums,
> average, etc.) raw SQL access may be needed, I saw direct SQL calls in
some
> of existing products. But they return raw rows, i.e. an array. No
middletier
> functionality. On the otherhand, it may be interesting to encapsulate this
> for later enhacenments.
>
> In one of the papers I studied on this theme
> (http://www.ambysoft.com/persistenceLayer.pdf, pg 7, point 8.) I saw the
use
> of proxy objects which are light weight objects with the most basic
> attributes (some user-key, name and few more). This idea could be
extended,
> telling the fetch machinery to return objects with some additional, user
> requested attributes.

The idea of proxies is quite interesting and notably pretty
straightforward to implement. Suppose the ability to fetch raw rows is
implemented, the common idiom would be (not speaking about the fetch()
shortcut) something like this:

  >>> qual=qualifierWithQualifierFormat('title ilike "%t%"')
  >>> fs=FetchSpecification('Book', qual)
  >>> fs.setFetchesRawRows(true)
  >>> fs.setRawRowsKeyPath('id', 'title', 'author.lastName')
  >>> rows=ec.objectsWithFetchSpecification(fs)
  >>> print rows
  ({'id': 1, 'title': 'Gargantua', 'author_lastName': 'Rabelais'},
   {'id': 2, 'title': 'Bouge ton pied...', 'author_lastName': 'Dard'},
   {'id': 4, 'title': "T'assieds pas...", 'author_lastName': 'Dard'},
  )

  (note: I consider that dicts are better than arrays as a result set)

  Then it simply a matter of declaring:
  >>> class GenericProxy: pass
  after which the creation and initialization of the proxies is just:
  >>> proxies=[GenericProxy() for i in range(len(rows))]
  >>> [proxies[i].__dict__.update(rows[i]) for i in range(len(rows))]

  That's all!

  And that's a really nice idea.

> On the other hand, we could have summary objects, because it's very usual
to
> get sums, counts and averages. The idea is that the resultset would be an
> encapsulated a collection of summary objects:
>
> objects=ec.fetchSummary(sourceEntityName="Book", what="count(*),
avg(price),
>   sum(toInvoiceLines.amount)", where="author.name like 'John*'",
>   groupBy="author, genre", resultEntityName="BookSummary")
>
> This may seem to be a  SQL  (OQL is very near to), but it has advantages
> over SQL:
> * the dotted notation is allowed (get away from ugly inner joins).
> * it would process inheritance trees
> * don't have to repeat the grouping fields in select clause, they are
added
> automatically
> The only purpose of the entityName is to give an 'entry entity' which the
> attributes are related to. The result could be just an instance of
> 'SummaryObject', or perhaps an instance of a special BookSummary which
> encapsulates special behaviour.

* You make me realize that my previous answer to Mario saying ''you'll
  have to be able to distinguish between table alias t1 and table alias
  t2 when declaring attribute author.age and that's a problem bla bla
  bla'' was stupid: the dotted notation obviously solves this naturally
  (the distinction between author.age and author.pygmalion.age is
  clear).

* You're partially answering to the question of real-life examples
  --although I still need to familiarize myself with 'group by' and
  'having' statements.

* This brings a different light to what Mario formerly proposed
  (different api but the very same idea as far as I can see) --just
  because we here use a different method (fetchSummmary instead of
  fetch); sorry, sometimes I can't see the forest for the trees because
  of such details :|

  Now that the concept (summaries) is clearer to me I'm beginning to see
  the ideas behind. And I guess I'm not far from thinking that, okay,
  fetchSummary() does contain raw sql pieces, but that it's maybe not
  such a big deal if this covers common needs.

* could this be made clearer: despite the fact that I cannot really see
  its implication for the moment being, I have the feeling that 'group
  by' and multiple fetches (walking through the inheritance tree) is not
  only a matter of getting the results and assembling them one after the
  other, is it?

> Should the result be a flat list or a tree-like list or dictionary?

  I cannot answer that: as I said I'm not familiar enough with this kind
  of queries to see what form would be the most accurate, or whether it
  could depend on the query itself. I'd need different examples and your
  opinion here.

    BTW I suggest that we do not consider this as an element of the
    current API change proposal, rather as a new feature request.


  Last, you wrote:
> On the other hand, we could have summary objects, because it's very
> usual to get sums, counts and averages.

I'd like to hear more about those usual cases, if you could give some
examples that would help --help me, at least ;) since my personal
approach to that kind of things would be e.g. to treat the raw rows
on the python side rather than to ask the rdbms to do the job.

-- Sébastien.



-------------------------------------------------------
This SF.NET email is sponsored by: eBay
Great deals on office technology -- on eBay now! Click here:
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
_______________________________________________
Modeling-users mailing list
Mod...@li...
https://lists.sourceforge.net/lists/listinfo/modeling-users

Re: [Modeling-users] Summary objects and Proxy objects

[Sebastien B. <sbi...@us...>]

"Ernesto Revilla" <er...@si...> wrote:
> Don't take me too serious, I'm new to this list.
>=20
> I agree that, especially for summaries, like statistics (counts, sums,
> average, etc.) raw SQL access may be needed, I saw direct SQL calls in so=
me
> of existing products. But they return raw rows, i.e. an array. No middlet=
ier
> functionality. On the otherhand, it may be interesting to encapsulate this
> for later enhacenments.
>=20
> In one of the papers I studied on this theme
> (http://www.ambysoft.com/persistenceLayer.pdf, pg 7, point 8.) I saw the =
use
> of proxy objects which are light weight objects with the most basic
> attributes (some user-key, name and few more). This idea could be extende=
d,
> telling the fetch machinery to return objects with some additional, user
> requested attributes.

The idea of proxies is quite interesting and notably pretty
straightforward to implement. Suppose the ability to fetch raw rows is
implemented, the common idiom would be (not speaking about the fetch()
shortcut) something like this:

  >>> qual=3DqualifierWithQualifierFormat('title ilike "%t%"')
  >>> fs=3DFetchSpecification('Book', qual)
  >>> fs.setFetchesRawRows(true)
  >>> fs.setRawRowsKeyPath('id', 'title', 'author.lastName')
  >>> rows=3Dec.objectsWithFetchSpecification(fs)
  >>> print rows
  ({'id': 1, 'title': 'Gargantua', 'author_lastName': 'Rabelais'},
   {'id': 2, 'title': 'Bouge ton pied...', 'author_lastName': 'Dard'},
   {'id': 4, 'title': "T'assieds pas...", 'author_lastName': 'Dard'},
  )

  (note: I consider that dicts are better than arrays as a result set)

  Then it simply a matter of declaring:=20
  >>> class GenericProxy: pass
  after which the creation and initialization of the proxies is just:
  >>> proxies=3D[GenericProxy() for i in range(len(rows))]
  >>> [proxies[i].__dict__.update(rows[i]) for i in range(len(rows))]

  That's all!

  And that's a really nice idea.

> On the other hand, we could have summary objects, because it's very usual=
 to
> get sums, counts and averages. The idea is that the resultset would be an
> encapsulated a collection of summary objects:
>=20
> objects=3Dec.fetchSummary(sourceEntityName=3D"Book", what=3D"count(*), av=
g(price),
>   sum(toInvoiceLines.amount)", where=3D"author.name like 'John*'",
>   groupBy=3D"author, genre", resultEntityName=3D"BookSummary")
>=20
> This may seem to be a  SQL  (OQL is very near to), but it has advantages
> over SQL:
> * the dotted notation is allowed (get away from ugly inner joins).
> * it would process inheritance trees
> * don't have to repeat the grouping fields in select clause, they are add=
ed
> automatically
> The only purpose of the entityName is to give an 'entry entity' which the
> attributes are related to. The result could be just an instance of
> 'SummaryObject', or perhaps an instance of a special BookSummary which
> encapsulates special behaviour.

* You make me realize that my previous answer to Mario saying ''you'll
  have to be able to distinguish between table alias t1 and table alias
  t2 when declaring attribute author.age and that's a problem bla bla
  bla'' was stupid: the dotted notation obviously solves this naturally
  (the distinction between author.age and author.pygmalion.age is
  clear).

* You're partially answering to the question of real-life examples
  --although I still need to familiarize myself with 'group by' and
  'having' statements.

* This brings a different light to what Mario formerly proposed
  (different api but the very same idea as far as I can see) --just
  because we here use a different method (fetchSummmary instead of
  fetch); sorry, sometimes I can't see the forest for the trees because
  of such details :|

  Now that the concept (summaries) is clearer to me I'm beginning to see
  the ideas behind. And I guess I'm not far from thinking that, okay,
  fetchSummary() does contain raw sql pieces, but that it's maybe not
  such a big deal if this covers common needs.

* could this be made clearer: despite the fact that I cannot really see
  its implication for the moment being, I have the feeling that 'group
  by' and multiple fetches (walking through the inheritance tree) is not
  only a matter of getting the results and assembling them one after the
  other, is it?

> Should the result be a flat list or a tree-like list or dictionary?

  I cannot answer that: as I said I'm not familiar enough with this kind
  of queries to see what form would be the most accurate, or whether it
  could depend on the query itself. I'd need different examples and your
  opinion here.

    BTW I suggest that we do not consider this as an element of the
    current API change proposal, rather as a new feature request.


  Last, you wrote:
> On the other hand, we could have summary objects, because it's very
> usual to get sums, counts and averages.

I'd like to hear more about those usual cases, if you could give some
examples that would help --help me, at least ;) since my personal
approach to that kind of things would be e.g. to treat the raw rows
on the python side rather than to ask the rdbms to do the job.

-- S=E9bastien.

Re: [Modeling-users] varchar and text

[Sebastien B. <sbi...@us...>]

Ernesto Revilla <er...@si...> wrote:
> Going thru the PostgreSQL doc version 7.3.2 it says:
> * varchar w/o upper limit is not standard SQL
> * text is postgres specific
> * varchar w/o limit and text have no performance penalities compared with
> varchar with limit
> * a lot of modern dbms systems support unlimited text.
>=20
> What's about mySQL? I would like to see if the the width requirement of
> varchar would be loosened (perhaps a warning, or ignored). I don't know h=
ow
> other database system treat this.

MySQL: http://www.mysql.com/doc/en/CHAR.html, no CHAR/VARCHAR w/o width
SQLite: accepts varchar without limit

  ModelValidation issues an error when a varchar column is declared
  without a width, however that does not prevent you to
  mdl_generate_DB_schema: if the db underneath supports it, everything
  will be fine :)

I agree, however, that ModelValidation should reflect the db
specificities when possible. This will be changed in a near future so
that it issues a message at the 'info' level (rather than 'error') for
rdbms supporting it.

  See also the note I recently added to the User's Guide about the
  'width' parameter (will be online for the next release), depicting the
  role played by 'width' at runtime:

<< Note: The field width is used at runtime when validating a string
   attribute's value (see 3.4, ``Validation''), before objects are saved
   in the database. SQL data types CHAR and VARCHAR require that width
   is specified; on the other hand, TEXT -when supported by the
   database- does not accept that width is set. If you set it on an
   attribute, it will be ignored when the database schema is generated
   (i.e. a TEXT field my_text with width=3D30 won't be declared as
   TEXT(30), but simply as TEXT), but it will be checked at runtime and
   validation for this attribute will fail if its value's length exceeds
   the given width. Note that the same goal can be achieved by writing a
   specific validation method for the attribute (see 3.4.2).
>>

-- S=E9bastien.

[Modeling-users] Foreign key required for 1:1 relations Schema update thru differences

[Ernesto R. <er...@si...>]

I started modeling with the ZModelingTool and the following issues arose:

* When I modeled
            1                                    0..*
         addresses                         person
Person <---------------------------->>address

although I told in the relation in entity address that it had to map '1:1'
to Person, it didn't set the FK_Person to required. During model validation
I got the warning :
- relationship is mandatory  but source attribute direccion.FK_PERSONA is
not required.

All in all, I see the tool a bit redundant, because:
* the former issue should be done automatically
* I would like to map all python string to varchar (w/o limit) or text (in
fact, they should all be mapped to the same (perhaps system dependent) DB
type
* typeName is required but not used
* name, className, moduleName and externalName have all to be specified,
perhaps the latter three should be defaulted to the first if not specified.

Sorry, this should not sound too destructive. I really appreciate a lot all
the work done. I just think that the XML-file is too explicit (with all the
required fields). Perhaps the semantics could be loosened a bit, making al
lot of the attributes optional (like moduleName, packageName, externalType,
etc.)

If I can help, please tell me how. (I think I really need this project for
our own (ERP).)

Thanx, Erny

[Modeling-users] varchar and text

[Ernesto R. <er...@si...>]

Going thru the PostgreSQL doc version 7.3.2 it says:
* varchar w/o upper limit is not standard SQL
* text is postgres specific
* varchar w/o limit and text have no performance penalities compared with
varchar with limit
* a lot of modern dbms systems support unlimited text.

What's about mySQL? I would like to see if the the width requirement of
varchar would be loosened (perhaps a warning, or ignored). I don't know how
other database system treat this.

Erny

[Modeling-users] Summary objects and Proxy objects

[Ernesto R. <er...@si...>]

Don't take me too serious, I'm new to this list.

I agree that, especially for summaries, like statistics (counts, sums,
average, etc.) raw SQL access may be needed, I saw direct SQL calls in some
of existing products. But they return raw rows, i.e. an array. No middletier
functionality. On the otherhand, it may be interesting to encapsulate this
for later enhacenments.

In one of the papers I studied on this theme
(http://www.ambysoft.com/persistenceLayer.pdf, pg 7, point 8.) I saw the use
of proxy objects which are light weight objects with the most basic
attributes (some user-key, name and few more). This idea could be extended,
telling the fetch machinery to return objects with some additional, user
requested attributes.

On the other hand, we could have summary objects, because it's very usual to
get sums, counts and averages. The idea is that the resultset would be an
encapsulated a collection of summary objects:

objects=ec.fetchSummary(sourceEntityName="Book", what="count(*), avg(price),
  sum(toInvoiceLines.amount)", where="author.name like 'John*'",
  groupBy="author, genre", resultEntityName="BookSummary")

This may seem to be a  SQL  (OQL is very near to), but it has advantages
over SQL:
* the dotted notation is allowed (get away from ugly inner joins).
* it would process inheritance trees
* don't have to repeat the grouping fields in select clause, they are added
automatically

The only purpose of the entityName is to give an 'entry entity' which the
attributes are related to. The result could be just an instance of
'SummaryObject', or perhaps an instance of a special BookSummary which
encapsulates special behaviour.

Should the result be a flat list or a tree-like list or dictionary?
Erny

Re: [Modeling-users] API cleanup proposal: CustomObject and related

[Sebastien B. <sbi...@us...>]

And now for CustomObject and related APIs!

  I've thought about it quite a lot since the first proposal. Answers I
  got, including two private ones, helped me a lot. Let me summarize all
  this.

* Backward compatibility is of great concern. This means in particular
  that KeyValueCoding will remain a mix-in for CustomObject, and that
  KeyValueCoding will almost remain as-is. I say almost, because we
  should deprecate (not remove) the methods: setValueForKey(),
  setValueForKeyPath() and setStoredValueForKey(). As Mario already
  wrote in a previous post, their only presence makes it harder to
  understand the API and I fully agree.

  I don't remember when I added this but this was not a brilliant idea.

  Deprecation: means that they will issue a warning (python module
  warnings) when they are used, and they will be removed at a given
  release, to be determined (after 0.9). Note that the framework does
  not use them at all.
                            What do you think?

* I think I made a big mistake when exposing the KeyValueCoding so early
  in the User's Guide --again, Mario already pointed that out. This part
  should be moved in an 'advanced techniques' part in the guide. I
  suspect that this is /one/ of the main reasons why people tends to
  focus on this api, despite the fact that it is definitely not needed
  at first sight.

* What about the new API: I've come to the (still-to-be-discussed)
  conclusion that it should not be mixed-in to CustomObject. While
  making aliases in the framework's core is no problem, adding more
  methods to CustomObject can induce some problem because we would then
  add methods to the users' classes (this was pointed out by one of the
  private mail I got). I tend to agree with that.

  However what we can do is make an alternate module offering both the
  new methods and a mix-in class that people could use in their own
  classes when they prefer it. It seems that this solution addresses the
  request for changing the name while preventing the set of users
  classes' methods to grow unwillingly with aliased methods (no
  functionality added).

* Last, I must say that the other private msg I got supports the KVC and
  considers it better than the new props. As far as I'm concerned, I'm
  not opposed to aliases, but I must admit that my preference goes to
  the original KVC api, well, just like any former NeXTStep or EOF
  developper I guess ;)

CustomObject
------------
  We'll add globalID() in CustomObject if there is no argument
  against. That would be a handy shortcut for:
      obj.editingContext().globalIDForObject(self)
  which is a common idiom for people willing to access the primary keys'
  values while not making them class properties (this question is a
  recurrent one)

Validation
----------
> >   Do we need to change anything there? validateValueForKey() is the
> >   longest one, however I think this is not widely used, is it?
>=20
> I suggest validateProperty() instead of validateValueForKey().

  Here again, backward compatibility means that the old name will be
  kept. Again, this adds an alias and no functionality to
  CustomObject. Moreover, I suspect this is all but a commonly used
  method. I hate saying that but I feel like going one step backward and
  remove the proposal. Please shout at me if you disagree ;)


Back on the new KVC:
--------------------

  Module's name: I'd strongly prefer if it could be explicit rather than
    an acronym (opr). GenericAccess still has my preference but that's
    no strong pref., I'll probably never use it so basically _you_ have
    the lead!

  Methods names: I'm okay to use the full names, but then they will
    sensibly be of the same length than the original ones, compare:

    co.getProperty()
     --valueForKey--

    co.setProperty()
     --setValueForKey--

    co.getStoredProperty()
     --storedValueForKey--

    co.setStoredProperty()
     --setStoredValueForKey--

    co.updateProperties()
     --takeValuesFromDictionary--

    co.updateStoredProperties()
     --takeStoredValuesFromDictionary--

  Up to you, however.


BTW I'll probably extend KVC with valuesForKeys(), counterpart for
    takeValueForKey(). I was surprised to realize it was not there.
    This also means to add something like 'properties()' to the new API
    (which is maybe problematic, I've the feeling that this particular
    name could easily clash with another one defined elsewhere).


  However *if* we agree on what I said above, i.e. that the new KVC will
  not be a mix-in for CustomObject by default, then choosing the new api
  can be decorrelated to the changes made to the core. These changes
  would then only be:

    + CustomObject.globalID()
    + KeyValueCoding.

  (plus the changes made to EditingContext)

        What do you think?


-- S=E9bastien.

Re: [Modeling-users] API cleanup proposal: CustomObject and related

[Sebastien B. <sbi...@us...>]

And now for CustomObject and related APIs!

  I've thought about it quite a lot since the first proposal. Answers I
  got, including two private ones, helped me a lot. Let me summarize all
  this.

* Backward compatibility is of great concern. This means in particular
  that KeyValueCoding will remain a mix-in for CustomObject, and that
  KeyValueCoding will almost remain as-is. I say almost, because we
  should deprecate (not remove) the methods: setValueForKey(),
  setValueForKeyPath() and setStoredValueForKey(). As Mario already
  wrote in a previous post, their only presence makes it harder to
  understand the API and I fully agree.

  I don't remember when I added this but this was not a brilliant idea.

  Deprecation: means that they will issue a warning (python module
  warnings) when they are used, and they will be removed at a given
  release, to be determined (after 0.9). Note that the framework does
  not use them at all.
                            What do you think?

* I think I made a big mistake when exposing the KeyValueCoding so early
  in the User's Guide --again, Mario already pointed that out. This part
  should be moved in an 'advanced techniques' part in the guide. I
  suspect that this is /one/ of the main reasons why people tends to
  focus on this api, despite the fact that it is definitely not needed
  at first sight.

* What about the new API: I've come to the (still-to-be-discussed)
  conclusion that it should not be mixed-in to CustomObject. While
  making aliases in the framework's core is no problem, adding more
  methods to CustomObject can induce some problem because we would then
  add methods to the users' classes (this was pointed out by one of the
  private mail I got). I tend to agree with that.

  However what we can do is make an alternate module offering both the
  new methods and a mix-in class that people could use in their own
  classes when they prefer it. It seems that this solution addresses the
  request for changing the name while preventing the set of users
  classes' methods to grow unwillingly with aliased methods (no
  functionality added).

* Last, I must say that the other private msg I got supports the KVC and
  considers it better than the new props. As far as I'm concerned, I'm
  not opposed to aliases, but I must admit that my preference goes to
  the original KVC api, well, just like any former NeXTStep or EOF
  developper I guess ;)

CustomObject
------------
  We'll add globalID() in CustomObject if there is no argument
  against. That would be a handy shortcut for:
      obj.editingContext().globalIDForObject(self)
  which is a common idiom for people willing to access the primary keys'
  values while not making them class properties (this question is a
  recurrent one)

Validation
----------
> >   Do we need to change anything there? validateValueForKey() is the
> >   longest one, however I think this is not widely used, is it?
>=20
> I suggest validateProperty() instead of validateValueForKey().

  Here again, backward compatibility means that the old name will be
  kept. Again, this adds an alias and no functionality to
  CustomObject. Moreover, I suspect this is all but a commonly used
  method. I hate saying that but I feel like going one step backward and
  remove the proposal. Please shout at me if you disagree ;)


Back on the new KVC:
--------------------

  Module's name: I'd strongly prefer if it could be explicit rather than
    an acronym (opr). GenericAccess still has my preference but that's
    no strong pref., I'll probably never use it so basically _you_ have
    the lead!

  Methods names: I'm okay to use the full names, but then they will
    sensibly be of the same length than the original ones, compare:

    co.getProperty()
     --valueForKey--

    co.setProperty()
     --setValueForKey--

    co.getStoredProperty()
     --storedValueForKey--

    co.setStoredProperty()
     --setStoredValueForKey--

    co.updateProperties()
     --takeValuesFromDictionary--

    co.updateStoredProperties()
     --takeStoredValuesFromDictionary--

  Up to you, however.


BTW I'll probably extend KVC with valuesForKeys(), counterpart for
    takeValueForKey(). I was surprised to realize it was not there.
    This also means to add something like 'properties()' to the new API
    (which is maybe problematic, I've the feeling that this particular
    name could easily clash with another one defined elsewhere).


  However *if* we agree on what I said above, i.e. that the new KVC will
  not be a mix-in for CustomObject by default, then choosing the new api
  can be decorrelated to the changes made to the core. These changes
  would then only be:

    + CustomObject.globalID()
    + KeyValueCoding.

  (plus the changes made to EditingContext)

        What do you think?

  With best regards,

-- S=E9bastien.

Re: [Modeling-users] API cleanup proposal: EditingContext

[Sebastien B. <sbi...@us...>]

        Hi all,

Some more comments about the API change proposal; I concentrate here on
EditingContext, and will post another message for CustomObject and
related APIs.

Note: I also received two private answers.

* Again: methods will be aliased, not replaced,

* EC.insert() and EC.delete() are okay to everyone.

* autoInsertion()/setAutoInsertion(): the former is a getter, the
  latter, a setter

Mario> OK. What about commit() for saveChanges()?

* about commit(): that was initially in my proposal. I removed
  it. Remember I'm working on integrating the ZODB transaction and
  observation mechanisms into the modeling? Well, then, commit() clashes
  with py2.1 zodb's transaction protocol (and has a very different
  semantic than saveChanges(): it is equivalent to objectWillChange()).
  Hence I'd prefer to let it apart from the proposal for the moment
  being.

* EC.fetch():

  The initial proposal is accepted by everyone so we'll keep it as a basis.

Mario> Another issue is that even with all the keyword options offered, one
Mario> still has to "dip into" small pieces of raw sql very quickly, as sho=
wn
Mario> by your examples.

  Here I strongly disagree with you. It /seems/ that you dip into small
  pieces of raw sql, but it's not the case at all. Qualifiers and
  FetchSpecification properties offers a generic command language;
  consider this:

    ec.fetch('Book', 'author.pygmalion.lastName ilike "r*"')

  -> this is far from the generated sql:

        SELECT t0.id, t0.title, t0.FK_WRITER_ID, t0.PRICE=20
        FROM BOOK t0=20
          INNER JOIN  ( WRITER t1=20
                        INNER JOIN WRITER t2=20
                        ON t1.FK_WRITER_ID=3Dt2.ID )
          ON t0.FK_WRITER_ID=3Dt1.ID
        WHERE UPPER(t2.LAST_NAME) LIKE UPPER('r%');

  In the framework we generally make the strong assumption that raw sql
  is taken out of the python code. Even if some of the keywords (such as
  'like'/'ilike' for qualifiers, or 'asc'/'desc' for ordering) are
  actually the same than the corresponding sql keywords, they are indeed
  decorrelated.  See again the example above: the 'ilike' keyword is not
  used, rather we compare UPPER()s (this might change in the future, now
  that 'ilike' is a sql keyword accepted by a lot of db-servers ;)

Mario> This may be an acceptable compromise, but then client code should be
Mario> allowed to pass any SQL it wants...

  That's an other problem, to which I completely agree: we should have a
  mean to execute complete raw sql statements, and get the (raw) result
  back. Moreover, we should be able to transform the returned raw rows
  into real objects if necessary (and if possible).

> Another detail is that the object loads all (direct) properties, every
> time -- there may be no workaround for this, given the way the
> framework is built.
(also related to your 'resultset' proposal)

  Last, we should also be able to tell the framework to return the raw
  rows instead of fully initialized objects (which can later be reverted
  to real objects): there's a real need for that; sometimes you want to
  present a (very) long list of objects in a summary page/widget, but
  you do not need the full objects, not even every attributes, but a
  subset to present the user. Then the users selects one or more of this
  rows and that's where you'll transform the raw rows to real objects.

  Note that the framework architecture will have no problem to support
  this. Some of the needed APIs are already present but not implemented
  (such as DatabaseContext.faultForRawRow()).

  I thought this was on the todo list but it's not --I'll add that,
  since I've been thinking about these points for quite a long time.

Impact on the API:

  when this is implemented I suggest we add the following parameters to
  fetch():

    rawRows -- (default: false) return raw rows instead of objects

    sql -- execute the sql statement (and ignore all other parameters
           except entityName and rawRows --both optional in that case)

  BTW I also suggest that the unsupported features in the fetch API are
  removed until they are implemented (such as limit/page/offset/etc).

> I feel that some things (even if no one has requested them yet ;) are
> missing...
> for example, should one ever need to take advantage of sql "group by" and
> "having".
> Access to these sql clauses may be added later, without breaking this API,
> which is OK. Also, such manips may be done in the middle code, but
> that would be very inefficient.
[...]
>=20
> The real problem with this is that to request the result of an sql functi=
on,
> of which count() is an example, additional api functionality is needed.
> But what happens if I want other functions, such as SUM or MAX or AVERAGE
> over a few columns? Each of these functions may take complex parameters.
> Again, the functionality may be replicated in the middle code, but this w=
ould
> not only be a waste of development time, but also be very inefficient.
>=20
> I propose either a generalization of "select" (which may be too complicat=
ed
> in its implications), or an addition of a func keyword option, e.g.
>=20
> func=3D'count'
> func=3D('funcName', param1, param2, ...)
> func=3D( ('funcName', param1, param2, ...), ('funcname2') )
>=20
> func=3D('sum','age')
>=20
> The question is then how is this information returned?
> I suggest as a tuple...
[snipped]

  That's a very interesting idea, but this will take too much efforts
  for it to be shortly developped. Let me explain that: if we do this
  just as it sounds, then we will have *pieces* of raw sql in the middle
  of generic command patterns --I don't like that. Do not misunderstand
  me: it should be possible to abstract this in a certain way, but I
  really wonder if it's worth the effort. And there's more problems,
  consider this:

    - either you simply want to sum()/avg()/max()/etc. on a table and
      its attributes, and then I guess it's probably sufficient to offer
      the possibility to fetch(sql=3D'select max(age) from author'), as
      stated above;

    - or you want to use this along with the automatic generation of
      complex queries (e.g. with multiple joins): okay, but then you
      must be able to say which attributes you want, and it's definitely
      not sufficient to tell which table it belongs to: relations can be
      reflexive (such as the 'pygmalion' relationship in the author/book
      model), and in such cases you have two possibilities for
      Author.age: table alias t1 or t2 (referring to the sql statements
      above). This also means that this expression needs to be bound to
      the automatic generation of sql statements.

  I can't think of a straightforward way to do this by now. Again, I'm
  not saying this is impossible: I'm just playing with the interesting
  idea and explaining the difficulties I can foresee, wondering whether
  such an advanced functionality would be worth the effort. That's an
  open question, and for the moment being I suggest we do not take this
  into account _as far as the API change proposal is concerned_. This
  could be discussed in a separate thread, and it would help a lot if we
  had some real-life examples showing where this could be very handy.
  --> Same for 'group by' and 'having' statements by the way, since I'm not
  really familiar with them either.


Mario> I would also add "by indicating clearly the small subset of methods
Mario> intended for use by client code, and the stability level".

  Right. I'm still looking for a way to include this in the docstrings
  so that the generation of the API can eat it (FYI we use epydoc).


-- S=E9bastien.

[Modeling-users] Attention psycopg users: never-ending transactions / additional info

[Sebastien B. <sbi...@us...>]

Remember I wrote on Sat Feb 22 2003:
> [...]
> Important (I almost forgot, silly me): if you're using the framework in
> --------- a long-standing process (being alive for more than 24 hours),
>           I strongly suggest that you upgrade your copy and that you do
>           NOT set MDL_PERMANENT_DB_CONNECTION (see changelog below) ; I
>           have experienced strange behaviour with long-standing
>           processes (w/ python2.1 & Zope 2.6, psycopg v1.0.14 and
>           1.0.15, and PG server 7.2), such as committed changes being
>           correctly issued by the psycopg adaptor but never committed
>           within the postgresql server, hence causing the corresponding
>           transaction to be *lost* (when this happens, you see a
>           postgresql 'idle transaction in progress', never ending until
>           the python process itself dies)
>=20
>   While the exact reason for this problem is not completely clear at the
>   moment, it was solved by the new behaviour (which closes the
>   connection to the db as soon as no more adaptor channels are opened).

Current status: even with psycopg 1.1.4 and MDL_PERMANENT_DB_CONNECTION
unset, the problem still appears on one project using modeling and zope
(despite what I wrote at that time, saying it could be resolved).

This *only* affects modeling/psycopg in a multi-threaded environment
(such as with zope).

  Other users also reported the problem, for example:
    http://lists.initd.org/pipermail/psycopg/2002-August/001308.html
    http://lists.initd.org/pipermail/psycopg/2003-June/002079.html
  My initial report is at:
    http://lists.initd.org/pipermail/psycopg/2003-March/thread.html#1885

A few days ago I worked again on this and I may have found the reason
for this, along with a test prg. reproducing the pb. and a possible
correction; I'm still waiting for comments from the psycopg
mailing-list. May it be of some interest to you, you'll find my analysis
and proposals at:
    http://lists.initd.org/pipermail/psycopg/2003-June/002086.html

  In brief: until an other solution is validated by the psycopg guys, I
just removed the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS
statements in cursor.c:_psyco_curs_execute() and the problem seems to
disappear (note that by doing this, if you have some sql queries taking
a long time to execute, all the other threads will be *blocked* until the
query finishes).

  Please keep in mind that for the moment being this is /my/ analysis
  and definitely not an authoritative answer to the bug.

I'll report here when additional infos are available. I've thought you'd
appreciate to be kept informed.

-- S=E9bastien.

Re: [Modeling-users] Questions and suggestions

[Sebastien B. <sbi...@us...>]

I wrote:
> >>> q =3D qualifierWithQualifierFormat('toAddresses.zipCode like "4%"')

Ooops... such a qualifier should normally match '4\%', NOT '4' and any
characters after.

Hence:
  1. to match character '4' followed by any characters, the qualifier string
     should be: 'toAddresses.zipCode like "4*"'
=20
  2. this was a bug in the postgresql adaptor layer: postgresql interprets
     backslashes, so escaping '%' should be like: '\\%', NOT '\%'.
     This bug is now corrected on CVS and will be in next release.
     (it affects neither mysql nor sqlite)

-- S=E9bastien.

Re: [Modeling-users] Request for specific adaptors?

[Sebastien B. <sbi...@us...>]

Sebastien Bigaret <sbi...@us...> wrote:
> Mario> S=E9bastien, do you think you will integrate the SQLite adapter
> Mario> you had announced some time ago, for 0.9?
>=20
>   Yes, it will be integrated in 0.9 (and possibly before).

  The new adaptor layer for SQLite has been added to cvs today, and will
  participate in the next release.

-- S=E9bastien.

Re: [Modeling-users] Questions and suggestions

[Sebastien B. <sbi...@us...>]

Ernesto Revilla <er...@si...> wrote:
>  Inheritance:
>=20
> I'm absolutely astonished. In your example:
>=20
> >>> q =3D qWQF('toAddresses.zipCode like "4%"')
> >>> fs =3D FS('Employee', qualifier=3Dq)
> >>> fs_deep =3D FS('Employee', qualifier=3Dq, deepFlag=3D1) # Note the de=
epFlag
> >>> ec.objectsWithFetchSpecification(fs)
>=20
> how this is done at SQL level? Do you execute 3 (one for each subclass of
> the specified class) SQL queries appending it's result objects to a list?

That's exactly the way this is done: any deep fetch is made against the
whole inheritance hierarchy below the specified entity (here, Employee)
--this is DatabaseContext's responsability, you can have a look at
objectsWithFetchSpecification().

This is the reason why the PK automatic generation support only uses one
source per inheritance tree, either a sequence (postgresql) or a
dedicated table (mysql, sqlite): this is required if we want that a
relationship points to different rows in different tables.

> Can you tell us a little bit about your experience, e.g. performance
> penalties, complications when using inheritance in real world application=
s?

Performance penalty:

  mostly what you would expect, i.e. as many fetchs as there are classes
  in the inheritance tree below a given entity. Not that big since in
  any case you'll have to do this by hand if it was not automatically
  handled.

  However the framework does not stop after finding the first
  row/object. The reason for this is that if it can find more than an
  object for a given GlobalID that's a serious error, at least at the
  framework's eyes, and this error should not be hidden. So all fetchs
  are executed --whatever their result. But unless your inheritance tree
  is really big, some additional fetchs returning no row have no
  significant impact.

Complications in real world applications:

  I can't think of any complications :) If you think reliability:
  inheritance support is used in two important projects I know, it has
  been extensively used and tested for months.

  Okay, sometimes it happens that you just want A in relation with B,
  not with any of B's subentities: in such a case, you'll have to deal
  with the extra fetchs the framework makes (there's no way to express
  this in a model, but this can be enforced by custom business-logic
  validation), but again that's no big deal unless you have a big
  inheritance hierarchy.

BTW: you can see the exact SQL statements that the framework executes by
     setting the environment varibale MDL_ENABLE_DATABASE_LOGGING to any
     value (except non-empty string).

> The example you specified (address<<---->employee) is good, could also be:
> travelling guide, novel, historic novel, etc. to match you book-author
> example. It would be nice if you could copy and paste it into the
> inheritance section (quick and dirty), perhaps it could be expanded in the
> next version.

You're right here, I realize this section definitely needs to be
written, and a raw copy/paste or a summary of this thread should be
sufficient for a first draft; I'll do this for the next release.

  Deriving the book/author model with travelling guides, novels etc. is
  also a good idea --I might reuse it in the coming tutorial (I was
  looking for a better model than store/employee to demonstrate
  inheritance, and this one looks actually better ;)

-- S=E9bastien.

Fw: [Modeling-users] Questions and suggestions

[Ernesto R. <er...@si...>]

Mail I wanted to send to the list instead of sending it directly to
Sébastien

Erny
----- Original Message -----
From: "Ernesto Revilla" <er...@si...>
To: "Sebastien Bigaret" <sbi...@us...>
Sent: Tuesday, June 10, 2003 8:09 PM
Subject: Re: [Modeling-users] Questions and suggestions


 Hello again,

 Inheritance:

I'm absolutely astonished. In your example:

>>> q = qWQF('toAddresses.zipCode like "4%"')
>>> fs = FS('Employee', qualifier=q)
>>> fs_deep = FS('Employee', qualifier=q, deepFlag=1) # Note the deepFlag
>>> ec.objectsWithFetchSpecification(fs)

how this is done at SQL level? Do you execute 3 (one for each subclass of
the specified class) SQL queries appending it's result objects to a list?

Can you tell us a little bit about your experience, e.g. performance
penalties, complications when using inheritance in real world applications?

The example you specified (address<<---->employee) is good, could also be:
travelling guide, novel, historic novel, etc. to match you book-author
example. It would be nice if you could copy and paste it into the
inheritance section (quick and dirty), perhaps it could be expanded in the
next version.

Erny


> ----- Original Message -----
> From: "Sebastien Bigaret" <sbi...@us...>
> To: "Ernesto Revilla" <er...@si...>
> Cc: <mod...@li...>
> Sent: Tuesday, June 10, 2003 7:30 PM
> Subject: Re: [Modeling-users] Questions and suggestions
>
>
>
> Ernesto Revilla <er...@si...> wrote:
> > Hi,
> >
> > for Sebastién:
> > * I can only do a comparison matrix with the best-known Python
OR-Mappers
> > because I haven't done any serious work with them. The major difference
is
> > that the other ORMs have no query language. In Middlekit, e.g., you can
> use
> > SQL conditions, but the may not be db-independent.
>
> Ok.
>
> > Strange that there is no implementation of OQL.
>
> Well, it's not *that* known (at least to me;). If you have some good
> pointers
> for it I'd have a closer look --I've never played with that.
>
> > To all:
> > after working through the docs (v.0.9-pre7) (still didn't try the
package)
> > here are some
> > points:
> >
> > * Dependencies: The PyPgSQL and PyGreSQL names are interchanged.  Also,
> > should be pypgsql
>
> Yes, this has been changed with 0.9pre8 --hopefully this is now okay. If
you
> find more occurences of such errors, please report again.
>
> > * What functions are used from 4Suite? (It's 6MB to download for
Windows.)
> > Isn't it enough to use PyXML?
>
> Yes it is! 0.9pre8 removed 4Suite from its dependencies --but I forgot to
> include that in the announcement.
>
> > * Installation is a bit complicated, because of the dependencies. What
> > chances are there to create a complete install file (perhaps excluding
> > db-adaptors) for some of the mile-stone versions?
>
> There are some chances, if only I could find a script we formerly used to
> install the whole stuff. Maybe some of us here still have it at hand?
(hey,
> these ones know who they are ;)
>
>
> > * Although the inheritance mapping is described, I can't see out of the
> docs
> > if a relations may map to different tables (say Animals and Humans).
This
> is
> > because the class has to be stored with the foreign key. (See ids.)
> >
> > About ids and inheritance:
> > * Would it be a good idea to specify all object pointers as type
ObjectId
> in
> > the XML-Model file and tell how this is mapped to the DB in the adaptor?
> > (integer is enough?)
> >
> > * May be ObjectIds should be unique throughout the database? (A seperate
> > procedure would dispatch new objectids, using some high:low algorithm.)
> See
> > next point. Also good for replication.
> >
> > * Either the foreign class (id) has to be stored with each foreign key,
or
> > the object id contains the class id as part of it (MiddleKit does this).
> >
> > * How would this be treated in queries? Would a query return objects of
> > several tables? (This may require extra queries. A simple INNER JOIN
> > wouldn't do it.)
> >
> > Inheritance makes it real complicated, and in practice, may not be so
> > important, depending on the object-model design.
>
> I'm not sure to fully understand what you mean here, pardon me if my
answer
> is
> off-topic.
>
>   1. If you're looking for an 'ObjectId', globalID() is for you. A
GlobalID
>      uniquely identifies an object. You can get it with
>      ec.globalIDForObject(object)
>
>      It answers to: entityName() (mapped to a class in the model)
>                     keyValues (dictionary with pks as keys and their
>                                corresponding values)
>
>      In particular, if two different objects refer to the same row in the
>      database, they have the same GlobalID.
>
>   2. Suppose you have the following model:
>
>       Address <<-toAddresses---toEmployee-> Employee
>                                                ^
>                                               / \
>                                                T
>                                                |
>                                        +--------------+
>                                        |              |
>                                    Executive      SalesClerk
>
>       * An address object can be linked to either an Employee, an
Executive
> or
>         a SalesClerk instance
>
>       * An address.toEmployee() will retrieve the right object (being an
>         instances of one of those three classes)
>
>       * when fetching, you can specify whether you want to fetch a single
>         class or its inheritance tree as well. Compare this, based on the
> test
>         database and model StoreEmployees:
>
> >>> from StoreEmployees import Address
> >>> from Modeling.FetchSpecification import FetchSpecification as FS
> >>> from Modeling.EditingContext import EditingContext as EC
> >>> from Modeling.Qualifier import qualifierWithQualifierFormat as qWQF
> >>>
> >>> ec = EC()
> >>> q = qWQF('toAddresses.zipCode like "4%"')
> >>> fs = FS('Employee', qualifier=q)
> >>> fs_deep = FS('Employee', qualifier=q, deepFlag=1) # Note the deepFlag
> >>> ec.objectsWithFetchSpecification(fs)
> []
> >>> [(o.getFirstName(),o.getLastName())
> ...  for o in ec.objectsWithFetchSpecification(fs_deep)]
> [('John Jr.', 'Cleese'), ('Jeanne', 'Cleese'), ('John', 'Cleese')]
>
>
>   So the answer to your question:
>
> > * How would this be treated in queries? Would a query return objects of
> > several tables? (This may require extra queries. A simple INNER JOIN
> > wouldn't do it.)
>
>   is clearly yes!
>
>   However you're right, the documentation lacks even a basic example and
>   explanantions for this. Does this answers your questions? Feel free to
ask
>   for more.
>
> -- Sébastien.
>
>
>

[Modeling-users] ADV: Reciprocal Links Service * Increase Your Website's Link Popularity

[cindy <ci...@st...>]

To remove your email address click the link below:
http://www.StarLinker.com/remove.asp?2043hg2w098y2
-------------------------------------------------------------------------------------------

The best way to drive traffic to your website is by using www.StarLinker.com.

They offer a FREE links page management solution which will also increase your link popularity.

Sign up for FREE.

3 Simple Set Up Steps

1. Set up your link account.
2. Select link partners.
3. Put your linking code on your links page.
You're Done!

Sign Up FREE Now at http://www.StarLinker.com?9


***

Re: [Modeling-users] Questions and suggestions

[Jerome K. <Jer...@fi...>]

On Tue, Jun 10, 2003 at 07:30:10PM +0200, Sebastien Bigaret wrote:

> > * Installation is a bit complicated, because of the dependencies. What
> > chances are there to create a complete install file (perhaps excluding
> > db-adaptors) for some of the mile-stone versions?
> 
> There are some chances, if only I could find a script we formerly used to
> install the whole stuff. Maybe some of us here still have it at hand? (hey,
> these ones know who they are ;)
> 

I think this is for me :) .. 
And yes, i have an old copy (which i have never test) 
I sent it off-list 

Bye Bye 

Re: [Modeling-users] Questions and suggestions

[Sebastien B. <sbi...@us...>]

Ernesto Revilla <er...@si...> wrote:
> Hi,
>=20
> for Sebasti=E9n:
> * I can only do a comparison matrix with the best-known Python OR-Mappers
> because I haven't done any serious work with them. The major difference is
> that the other ORMs have no query language. In Middlekit, e.g., you can u=
se
> SQL conditions, but the may not be db-independent.=20

Ok.=20

> Strange that there is no implementation of OQL.

Well, it's not *that* known (at least to me;). If you have some good pointe=
rs
for it I'd have a closer look --I've never played with that.

> To all:
> after working through the docs (v.0.9-pre7) (still didn't try the package)
> here are some
> points:
>=20
> * Dependencies: The PyPgSQL and PyGreSQL names are interchanged.  Also,
> should be pypgsql

Yes, this has been changed with 0.9pre8 --hopefully this is now okay. If you
find more occurences of such errors, please report again.

> * What functions are used from 4Suite? (It's 6MB to download for Windows.)
> Isn't it enough to use PyXML?

Yes it is! 0.9pre8 removed 4Suite from its dependencies --but I forgot to
include that in the announcement.

> * Installation is a bit complicated, because of the dependencies. What
> chances are there to create a complete install file (perhaps excluding
> db-adaptors) for some of the mile-stone versions?

There are some chances, if only I could find a script we formerly used to
install the whole stuff. Maybe some of us here still have it at hand? (hey,
these ones know who they are ;)


> * Although the inheritance mapping is described, I can't see out of the d=
ocs
> if a relations may map to different tables (say Animals and Humans). This=
 is
> because the class has to be stored with the foreign key. (See ids.)
>=20
> About ids and inheritance:
> * Would it be a good idea to specify all object pointers as type ObjectId=
 in
> the XML-Model file and tell how this is mapped to the DB in the adaptor?
> (integer is enough?)
>=20
> * May be ObjectIds should be unique throughout the database? (A seperate
> procedure would dispatch new objectids, using some high:low algorithm.) S=
ee
> next point. Also good for replication.
>=20
> * Either the foreign class (id) has to be stored with each foreign key, or
> the object id contains the class id as part of it (MiddleKit does this).
>=20
> * How would this be treated in queries? Would a query return objects of
> several tables? (This may require extra queries. A simple INNER JOIN
> wouldn't do it.)
>=20
> Inheritance makes it real complicated, and in practice, may not be so
> important, depending on the object-model design.

I'm not sure to fully understand what you mean here, pardon me if my answer=
 is
off-topic.

  1. If you're looking for an 'ObjectId', globalID() is for you. A GlobalID
     uniquely identifies an object. You can get it with
     ec.globalIDForObject(object)

     It answers to: entityName() (mapped to a class in the model)
                    keyValues (dictionary with pks as keys and their
                               corresponding values)

     In particular, if two different objects refer to the same row in the
     database, they have the same GlobalID.

  2. Suppose you have the following model:

      Address <<-toAddresses---toEmployee-> Employee
                                               ^
                                              / \
                                               T
                                               |
                                       +--------------+
                                       |              |
                                   Executive      SalesClerk

      * An address object can be linked to either an Employee, an Executive=
 or
        a SalesClerk instance

      * An address.toEmployee() will retrieve the right object (being an
        instances of one of those three classes)

      * when fetching, you can specify whether you want to fetch a single
        class or its inheritance tree as well. Compare this, based on the t=
est
        database and model StoreEmployees:

>>> from StoreEmployees import Address
>>> from Modeling.FetchSpecification import FetchSpecification as FS
>>> from Modeling.EditingContext import EditingContext as EC
>>> from Modeling.Qualifier import qualifierWithQualifierFormat as qWQF
>>>=20
>>> ec =3D EC()
>>> q =3D qWQF('toAddresses.zipCode like "4%"')
>>> fs =3D FS('Employee', qualifier=3Dq)
>>> fs_deep =3D FS('Employee', qualifier=3Dq, deepFlag=3D1) # Note the deep=
Flag
>>> ec.objectsWithFetchSpecification(fs)
[]
>>> [(o.getFirstName(),o.getLastName())
...  for o in ec.objectsWithFetchSpecification(fs_deep)]
[('John Jr.', 'Cleese'), ('Jeanne', 'Cleese'), ('John', 'Cleese')]


  So the answer to your question:

> * How would this be treated in queries? Would a query return objects of
> several tables? (This may require extra queries. A simple INNER JOIN
> wouldn't do it.)

  is clearly yes!

  However you're right, the documentation lacks even a basic example and
  explanantions for this. Does this answers your questions? Feel free to ask
  for more.

-- S=E9bastien.

Re: [Modeling-users] patch : Entity.externalNameForInternalName()

[Sebastien B. <sbi...@us...>]

You're absolutely right. I applied your patch on the main trunk, it will
be in next release. Thanks for reporting!

-- S=E9bastien.

Yannick Gingras <yan...@sa...> wrote:

> Hi,=20
>   I use Entity.externalNameForInternalName() and
> Entity.nameForExternalName() extensivly to reload phpMyAdmin XML dumps of=
 my=20
> database.  Both should be 100% reversible so I submit this patch :=20
>=20
>=20
> --- Entity.py.orig      2003-06-09 15:11:57.000000000 -0400
> +++ Entity.py   2003-06-09 15:12:16.000000000 -0400
> @@ -70,7 +70,7 @@
>    #If 'returnCaps' is set to '0', the returned string consists of lower =
case
>    #letters, instead of upper case ones.
>    #if useCaps:
>  _regexp=3Dre.compile('([a-z])([A-Z])')
> +  _regexp=3Dre.compile('([a-z0-9])([A-Z])')
>    _results=3D_regexp.sub('\\1'+separatorString+'\\2', aName)
>    #else:
>    #  _results=3DaName
>=20
>=20
> In short, when the word is broken after a number, the '_' is not restored=
 by=20
> externalNameForInternalName().
>=20
> ex :=20
>   db2_id -> db2Id -> db2id
>=20
> --=20
> Yannick Gingras
> Byte Gardener, Savoir-faire Linux inc.
> (514) 276-5468
>=20
>=20
>=20
>=20
> -------------------------------------------------------
> This SF.net email is sponsored by:  Etnus, makers of TotalView, The best
> thread debugger on the planet. Designed with thread debugging features
> you've never dreamed of, try TotalView 6 free at www.etnus.com.
> _______________________________________________
> Modeling-users mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modeling-users

Re: [Modeling-users] Request for specific adaptors?

[Sebastien B. <sbi...@us...>]

        Hi,

Mario> Yes, in fact SQLite or Gadfly are certainly better choices for the
Mario> cases I described...
Mario>=20
Mario> S=E9bastien, do you think you will integrate the SQLite adapter
Mario> you had announced some time ago, for 0.9?

  Yes, it will be integrated in 0.9 (and possibly before).

Soif> If fact the only problem for me is that all of this requires some
Soif> special DBApi adapter in C on the server. As i host some website
Soif> on a friend box, this avoid me to use some C adapter (same issue
Soif> on python-hosting.com for example) . That's why gadfly is=20
Soif> interesting no ?=20

  Well, in a way you're right, however it lacks a lot of
  functionalities, for example: no NULL, no Date, Time, and Interval
  data types, no LIKE string predicate.  (cf. gadfly.sf.net/sql.html)

  Plus another thing I do not understand (or do I?)

<< Cursor based updates and deletes
     (justification: if you really need them the db design is flawed,
      and it's possible to use python instead).
>>

  --> does it really mean you cannot INSERT/UPDATE on a cursor??

  BTW, making a gadfly adaptor means that every non-supported feature
  should be detected and reported as such; I don't want to do this by
  now but if anyone feels like starting it I'll help for sure.

  You're also mentioning python-hosting.com: "plan 1" lets you use
  firebird, I guess it's probably a better choice there. If there is
  interest for that one I'll make the effort.

        Cheers,

-- S=E9bastien.

[Modeling-users] patch : Entity.externalNameForInternalName()

[Yannick G. <yan...@sa...>]

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi,=20
  I use Entity.externalNameForInternalName() and
Entity.nameForExternalName() extensivly to reload phpMyAdmin XML dumps of m=
y=20
database.  Both should be 100% reversible so I submit this patch :=20


=2D --- Entity.py.orig      2003-06-09 15:11:57.000000000 -0400
+++ Entity.py   2003-06-09 15:12:16.000000000 -0400
@@ -70,7 +70,7 @@
   #If 'returnCaps' is set to '0', the returned string consists of lower ca=
se
   #letters, instead of upper case ones.
   #if useCaps:
=2D -  _regexp=3Dre.compile('([a-z])([A-Z])')
+  _regexp=3Dre.compile('([a-z0-9])([A-Z])')
   _results=3D_regexp.sub('\\1'+separatorString+'\\2', aName)
   #else:
   #  _results=3DaName


In short, when the word is broken after a number, the '_' is not restored b=
y=20
externalNameForInternalName().

ex :=20
  db2_id -> db2Id -> db2id

=2D --=20
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE+5N14rhy5Fqn/MRARAkYbAJoCWoCnI+cvJcQwdDTROvoL3NNLagCdEeAG
c8O+RMQ11iS0QIh/DoT9wqs=3D
=3D1GcE
=2D----END PGP SIGNATURE-----

[Modeling-users] Questions and suggestions

[Ernesto R. <er...@si...>]

Hi,

for Sebastién:
* I can only do a comparison matrix with the best-known Python OR-Mappers
because I haven't done any serious work with them. The major difference is
that the other ORMs have no query language. In Middlekit, e.g., you can use
SQL conditions, but the may not be db-independent. Strange that there is no
implementation of OQL.

To all:
after working through the docs (v.0.9-pre7) (still didn't try the package)
here are some
points:

* Dependencies: The PyPgSQL and PyGreSQL names are interchanged.  Also,
should be pypgsql

* What functions are used from 4Suite? (It's 6MB to download for Windows.)
Isn't it enough to use PyXML?

* Installation is a bit complicated, because of the dependencies. What
chances are there to create a complete install file (perhaps excluding
db-adaptors) for some of the mile-stone versions?

* Although the inheritance mapping is described, I can't see out of the docs
if a relations may map to different tables (say Animals and Humans). This is
because the class has to be stored with the foreign key. (See ids.)

About ids and inheritance:
* Would it be a good idea to specify all object pointers as type ObjectId in
the XML-Model file and tell how this is mapped to the DB in the adaptor?
(integer is enough?)

* May be ObjectIds should be unique throughout the database? (A seperate
procedure would dispatch new objectids, using some high:low algorithm.) See
next point. Also good for replication.

* Either the foreign class (id) has to be stored with each foreign key, or
the object id contains the class id as part of it (MiddleKit does this).

* How would this be treated in queries? Would a query return objects of
several tables? (This may require extra queries. A simple INNER JOIN
wouldn't do it.)

Inheritance makes it real complicated, and in practice, may not be so
important, depending on the object-model design.

More to come,

Erny

Re: [Modeling-users] Request for specific adaptors?

[<so...@la...>]

On Fri, Jun 06, 2003 at 11:17:35AM +0200, Mario Ruggier wrote:
> >
> >Perhaps we should look at gadfly ? (or ZODB + SQL )
>=20
> Yes, in fact SQLite or Gadfly are certainly better choices for the
> cases I described...
>=20
> S=E9bastien, do you think you will integrate the SQLite adapter
> you had announced some time ago, for 0.9?


If fact the only problem for me is that all of this requires some
special DBApi adapter in C on the server. As i host some website
on a friend box, this avoid me to use some C adapter (same issue
on python-hosting.com for example) . That's why gadfly is=20
interesting no ?=20


bye Bye .:)=20

Re: [Modeling-users] Request for specific adaptors?

[Mario R. <ma...@ru...>]

>> I have recently felt that an adaptor to a non-relational db server=20
>> would
>> be very nice -- useful for simple applications (and installations of)
>> and/or
>> quick prototyping, but will allow that the same application be easily
>> "scaled-up"
>> to a relational server (with no changes to the data model).
>> This does not equate to simple object persistence (e.g. any of =
several
>> such schemes available, from pickle to zodb, etc.). It is also not=20
>> about
>> performance ;-!. It is about giving a very high-level separation=20
>> between
>> the application objects, and the back-end, that will also allow
>> switching
>> between SQL and non-SQL storage. I am in particular thinking of
>> something like bsddb, with string values (rows) that are simply
>> repr(dict).
>> Then searching and filtering can still be achieved without modifying
>> any client code (albeit very inefficiently!). The advantage is that
>> small applications that do not do so much searching and filtering
>> do not suffer very much, and they can be installed a lot simpler
>> (requiring only to make sure to have write access to some directory,
>> as opposed to setting up an sql server). But, the power of the model
>> is still at hand, on which the application can build...
>>
>> mario
>
>
> This is a good idea, since I usually don't use modeling for small =
stuff
> as i need to use a big SQL server w/. PySQLITE is really a great stuff
> i think my next dev about modeling will use this.
>
> Perhaps we should look at gadfly ? (or ZODB + SQL )

Yes, in fact SQLite or Gadfly are certainly better choices for the
cases I described...

S=E9bastien, do you think you will integrate the SQLite adapter
you had announced some time ago, for 0.9?

Cheers, mario

Re: [Modeling-users] API cleanup proposal

[Sebastien B. <sbi...@us...>]

Yannick Gingras <ygi...@yg...> wrote:
> On Monday 02 June 2003 14:10, Sebastien Bigaret wrote:
> >  [Note on "most commonly used methods": I do not plan to change every
> >   method's name when it's too long: they are mostly for the framework's
> >   use, or at best for advanced usage]
>
> Please make aliases for backward compatibility.
>=20
>   : )
>=20
> The proposal sounds really fair to me !

  A quick answer --sorry, I'm quite busy these days.

  When I wrote: "method_1() replaces by method_2()", obviously I meant
*is an alias for*. I just want to make it clear that I pay a careful
attention to backward compatibility and that it won't be broken in any
ways after the API "changes"/upgrades.

  I'll comment further on the proposal and answers later, probably
  on sunday.

-- S=E9bastien.