modeling-users — For users: questions, support, announcements, etc.

[Modeling-users] Time for a Murphys...

[Mario R. <ma...@ru...>]

After looking at the Modeling sourceforge page so many times, the
initials of MORBY (approximately), for Modeling OR-Bridge for Python,
start to stand-out... but, this could be not such a bad unique name (in
lowercase, of course) for the framework. However, this also makes
other, cuter, names come to mind, such as Morfy (where the bridge
becomes a framework) that in turn brings to mind the action of
morphing (relational data to objects, and back), which points to
another possible name, Morphy, for Modeling, Object-Relational
Phramework  for Python ;-) Which, in turn, would beg that an
OR model be called a MetaMorphy... Hmmn, mario

Re: [Modeling-users] MySQL TEXT

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 06 March 2003 06:37 pm, Sebastien Bigaret wrote:
> (SQL types should be lower-cased)
>
>
>   This should work ; please report and I'll apply the patch when you
>   confirm that this is ok.

Noop but almost :
######### patch beg ##########
- --- old_MySQLAdaptorLayer/MySQLSQLExpression.py  2003-03-07 10:19:57.000000000
- -0500
+++ MySQLSQLExpression.py       2003-02-01 06:59:13.000000000 -0500
@@ -33,7 +33,7 @@

 __version__='$Revision: 1.2 $'[11:-2]

- -from Modeling.SQLExpression import SQLExpression, DateType, CharacterType
+from Modeling.SQLExpression import SQLExpression, DateType

 class MySQLSQLExpression(SQLExpression):
   """
@@ -77,5 +77,5 @@
     """
     values=SQLExpression.valueTypeForExternalTypeMapping.im_func(self)
     del values['timestamp']
- -    values.update({ 'datetime': DateType , 'text': CharacterType})
+    values.update({ 'datetime': DateType })
     return values
######### patch end ##########

I just need to import CharacterType and it works perfectly.

Thanks for the super fast answer !

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+aLsvrhy5Fqn/MRARAhJIAJsH8EAmpSr9X6DITvcy/oiOqLdUyQCfXKI5
3OHlHnh32aHsv8AqUh1WjOE=
=acp6
-----END PGP SIGNATURE-----

[Modeling-users] Bugfix proposal for MySQL-related bug #614261

[Sebastien B. <sbi...@us...>]

        Hi,

I've just checked-in a bugfix for MySQLSQLExpression to support arbitra=
rily
long qualifiers when selecting objects.

  A few words on what happened: a fetch specified that way was failing:

    >>> from Modeling.Qualifier import qualifierWithQualifierFormat as =
qWQF
    >>> q=3DqWQF('(age<100) AND pygmalion.books.title like "G*"')
    >>>
    >>> fetchSpec=3DFetchSpecification(entityName=3D'Writer', qualifier=
=3Dq)
    >>> objects=3Dec.objectsWithFetchSpecification(fetchSpec)

  The default Modeling.SQLExpression produces the following statement:
  (extracted from test_11_allQualifierOperators)

      SELECT t0.ID, t0.FIRST_NAME, t0.LAST_NAME, t0.AGE, t0.BIRTHDAY,
             t0.FK_WRITER_ID
      FROM WRITER t0=20
      INNER JOIN  ( WRITER t1=20
                    INNER JOIN BOOK t2=20
                    ON t1.ID=3Dt2.FK_WRITER_ID )
      ON t0.FK_WRITER_ID=3Dt1.ID=20
      WHERE (t0.AGE < 100 AND t2.title LIKE 'G%')

  --> this led to a _mysql_exceptions.ProgrammingError/1064, "You have =
an
      error in your SQL syntax near ...", every time a keypath with 2 or
      more entities was inserted in a qualifier (such as in:
      'pygmalion.books.title')

  See: http://www.mysql.com/doc/en/JOIN.html, the first comment in
       ''User Comments'' (bottom of the page) describes this.

Now MySQLSQLExpression overrides this default behaviour so that element=
s are
produced in the correct order for MySQL, i.e.:

      SELECT t0.ID, t0.FIRST_NAME, t0.LAST_NAME, t0.AGE, t0.BIRTHDAY,=20
             t0.FK_WRITER_ID
      FROM ( BOOK t2=20
             INNER JOIN WRITER t1
             ON ( t1.ID=3Dt2.FK_WRITER_ID )=20=20
           )=20=20
      INNER JOIN WRITER t0
      ON ( t0.FK_WRITER_ID=3Dt1.ID )
      WHERE (t0.AGE < 100 AND t2.title LIKE 'G%')


  I did not close the bug-item, however: since I do not use MySQL at al=
l,
  all what I can say is that it now passes all the tests. I'd prefer th=
at
  users using mysql plays a bit with that and report success or failure.

  The corrected MySQLSQLExpression can be found at:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/modeling/ProjectModeling=
/Modeling/DatabaseAdaptors/MySQLAdaptorLayer/MySQLSQLExpression.py?rev=
=3D1.3&content-type=3Dtext/vnd.viewcvs-markup

  and the updated test_EditingContext_Global.py at:

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/modeling/ProjectModeling=
/Modeling/tests/test_EditingContext_Global.py?rev=3D1.17&content-type=
=3Dtext/vnd.viewcvs-markup

        Happy fetching :)


-- S=E9bastien.

Re: [Modeling-users] MySQL TEXT

[Sebastien B. <sbi...@us...>]

> Yannick Gingras <yan...@sa...> wrote:
> > the user guide section 2.2.4 talk about CHAR and VARCHAR external t=
ype
> > support but there is no mention about the support for TEXT.  I have
> > really strange errors when using TEXT.
> >=20
> >   File "<stdin>", line 92, in insertElement
> >   File "/usr/lib/python2.2/site-packages/Modeling/EditingContext.py=
", line=20
> > 678, in saveChanges
> >     self.parentObjectStore().saveChangesInEditingContext(self)
> >   File "/usr/lib/python2.2/site-packages/Modeling/ObjectStoreCoordi=
nator.py",=20
> > line 559, in saveChangesInEditingContext
> >     raise RuntimeError, _error
> > RuntimeError: performChanges() failed on=20
> > <Modeling.DatabaseContext.DatabaseContext instance at 0x88dd2ec>:=20
> > exceptions.ValueError:Unknown value type: None for externalType: te=
xt=20
> > (attribute: JOBI18N.description)
> > [...]
>=20
> Yes, this is because TEXT is an unknown sql type for the MySQLAdaptor.
> Didn't the validation of the model indicate it ?-)
>=20
> (I agree the message is not very informative, I added bug #699046)

  =3D> fixed in CVS repository, SQLExpression v1.14

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/modeling/ProjectModeling=
/Modeling/SQLExpression.py?rev=3D1.14&only_with_tag=3DHEAD&content-type=
=3Dtext/vnd.viewcvs-markup

  The message is now clear enough I think:
InvalidSQLTypeError: SQL type: TEXT (attribute: JOBI18N.description) is=
 invalid for the current adaptor (valueTypeForExternalType returned: No=
ne)


-- S=E9bastien.

Re: [Modeling-users] Warning: possible problem with Zope, maybe MT-related

[Sebastien B. <sbi...@us...>]

        Hi,

>   Just a quick note to warn you about the fact that I currently have a
>   very annoying bug revealing itself on a zope-based application using
>   the modeling.
>=20
>   It is still not clear where the bug is, but it has been reported
>   that it was also observed with zope and ZPsycopgDA alone.
>=20
> It might be a MT-issue, or just a Zope-issue (this is just my guess at
> this point).
>=20
>   Description of the problem: at some point, the psycopg adaptor stops
>   adding BEGIN and END or ABORT at the begin/end of the txn.  From
>   that point, none of the sql statements sent to the pg server is
>   committed anymore, hence they are lost.

It seems that the problem is triggered when the same psycopg cursor is
used in two different threads.

  I submitted and fixed bug #699272 for the modeling framework: when
  calling EditingContext.objectsCountWithFetchSpecification() the
  underlying adaptorChannel was not closed at the end. Hence, even if
  MDL_PERMANENT_DB_CONNECTION was unset, the corresponding
  (psycopg-)connection to the database was not closed. This makes it
  possible for the same cursor to be reused by an other request,
  especially in a different thread (when used within zope e.g.) and as a
  consequence it made the bug more likely to occur.

--> If you are using the framework in a MT environment, please update
--> your copy of DatabaseChannel w/ v.1.10 or apply the patch attached
--> below. Additionally, please remember that you are strongly advised
--> against setting MDL_PERMANENT_DB_CONNECTION in a multi-threaded
--> environment.

  This does not guarantee that the problem won't show up, but it will
  dramatically reduce its frequency. Hopefully a complete resolution for
  the bug will be found, I'll keep you informed.

        Regards,

-- S=E9bastien.



------------------------------------------------------------------------
retrieving revision 1.9
retrieving revision 1.10
diff -r1.9 -r1.10
32c32
<     $Id: DatabaseChannel.py,v 1.9 2003/02/20 11:59:52 sbigaret Exp $
---
>     $Id: DatabaseChannel.py,v 1.10 2003/03/07 09:27:18 sbigaret Exp $
36c36
< __version__=3D'$Revision: 1.9 $'[11:-2]
---
> __version__=3D'$Revision: 1.10 $'[11:-2]
292c292
<       self.__isFetchInProgress=3D0
---
>       self.cancelFetch()
------------------------------------------------------------------------

Re: [Modeling-users] MySQL TEXT

[Sebastien B. <sbi...@us...>]

        Hi,

Yannick Gingras <yan...@sa...> wrote:
> the user guide section 2.2.4 talk about CHAR and VARCHAR external type
> support but there is no mention about the support for TEXT.  I have
> really strange errors when using TEXT.
>=20
>   File "<stdin>", line 92, in insertElement
>   File "/usr/lib/python2.2/site-packages/Modeling/EditingContext.py",=
 line=20
> 678, in saveChanges
>     self.parentObjectStore().saveChangesInEditingContext(self)
>   File "/usr/lib/python2.2/site-packages/Modeling/ObjectStoreCoordina=
tor.py",=20
> line 559, in saveChangesInEditingContext
>     raise RuntimeError, _error
> RuntimeError: performChanges() failed on=20
> <Modeling.DatabaseContext.DatabaseContext instance at 0x88dd2ec>:=20
> exceptions.ValueError:Unknown value type: None for externalType: text=
=20
> (attribute: JOBI18N.description)
>=20
> Is it possible I am positively certain that my default value is not
> None and that I did put a value in JOBI18N.description before insert.
>=20
> Anyone have a clue about the cause ?

Yes, this is because TEXT is an unknown sql type for the MySQLAdaptor.
Didn't the validation of the model indicate it ?-)

(I agree the message is not very informative, I added bug #699046)

  It is raised by SQLExpression.formatValueForAttribute(), which got an
  'None' type from valueTypeForExternalType().

If TEXT is a valid mysql format mapped to a python string, then modify
MySQLSQLExpression.valueTypeForExternalTypeMapping() in
DatabaseAdaptors.MySQLAdaptorLayer:

diff -c -r1.2 MySQLSQLExpression.py
*** MySQLSQLExpression.py    1 Feb 2003 11:59:13 -0000        1.2
--- MySQLSQLExpression.py    6 Mar 2003 22:18:13 -0000
***************
*** 77,81 ****
      """
      values=3DSQLExpression.valueTypeForExternalTypeMapping.im_func(se=
lf)
      del values['timestamp']
!     values.update({ 'datetime': DateType })
      return values
--- 77,82 ----
      """
      values=3DSQLExpression.valueTypeForExternalTypeMapping.im_func(se=
lf)
      del values['timestamp']
!     values.update({ 'datetime': DateType,
!                     'text': CharacterType })
      return values


(SQL types should be lower-cased)

=20=20
  This should work ; please report and I'll apply the patch when you
  confirm that this is ok.

        Regards,


-- S=E9bastien.

[Modeling-users] MySQL TEXT

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi all, 

the user guide section 2.2.4 talk about CHAR and VARCHAR external type
support but there is no mention about the support for TEXT.  I have
really strange errors when using TEXT.

  File "<stdin>", line 92, in insertElement
  File "/usr/lib/python2.2/site-packages/Modeling/EditingContext.py", line 
678, in saveChanges
    self.parentObjectStore().saveChangesInEditingContext(self)
  File "/usr/lib/python2.2/site-packages/Modeling/ObjectStoreCoordinator.py", 
line 559, in saveChangesInEditingContext
    raise RuntimeError, _error
RuntimeError: performChanges() failed on 
<Modeling.DatabaseContext.DatabaseContext instance at 0x88dd2ec>: 
exceptions.ValueError:Unknown value type: None for externalType: text 
(attribute: JOBI18N.description)

Is it possible I am positively certain that my default value is not
None and that I did put a value in JOBI18N.description before insert.

Anyone have a clue about the cause ?

Regards, 

- -- 
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+Z6Dsrhy5Fqn/MRARAqzdAKCT3TVzNpxfcfb+mIaaFQOGerMGgQCghwmr
IJ/8GJflHISProw37iiQGAc=
=CI3Y
-----END PGP SIGNATURE-----

[Modeling-users] Warning: possible problem with Zope, maybe MT-related

[Sebastien B. <sbi...@us...>]

        Hi all,

  Just a quick note to warn you about the fact that I currently have a very
  annoying bug revealing itself on a zope-based application using the
  modeling.

  It is still not clear where the bug is, but it has been reported that it was
  also observed with zope and ZPsycopgDA alone.

It might be a MT-issue, or just a Zope-issue (this is just my guess at this
point).

  Description of the problem: at some point, the psycopg adaptor stops adding
  BEGIN and END or ABORT at the begin/end of the txn.
  From that point, none of the sql statements sent to the pg server is
  committed anymore, hence they are lost.

It do not have any other solution than monitoring the application and
restarting it when it happens --ok, this is not a solution, rather an ugly
workaround.

My initial post is at:
http://lists.initd.org/pipermail/psycopg/2003-March/001885.html

  I'll post here if and when a solution will be found.

-- Sebastien.

Re: [Modeling-users] Pythonic, and non-XML, Model description

[Mario R. <ma...@ru...>]

Hi,

Thanks for the ModelSet clarification. Further comments below...

> About: Models' being mutable at runtime
>> In general this is not what anyone woudl want to do, and I was trying
>> to think of times when this would be useful. Can't really, but i
>> *suspect* it might come in handy. But, in fact, imagine some
>> application that is stocking lots of incoming unknown data into a
>> db... let's say the data is XML, but each XML data-schema is unknown a
>> priori. You can create a py model to correspond, at runtime (but saved
>> for subsequent accessing), create the tables, and stock the unknown
>> XML in the db directly as real db tables....
>
> Okay, I now can be more precise here: adding an entity at runtime is
> definitely not a problem. But mutating an entity (changing its
> attributes for example, or its name), or removing an entity should 
> *not*
> be done after an object of the corresponding class has been registered
> within an EditingContext.
>
> (this should be a FAQ)

OK, understood, and probbaly a good idea to mention it as an FAQ.

...

>> What is wrong with a python list?
>> multiplicity = [0,1]
>> multiplicity = [1,-1]
>
> Well, I understand your point quite well and I'm ok with a list. [1,-1]
> seems however obscure, if we go for a python list what do you think of
> [0,'*'] or maybe better: [0,]  ?

Absolutely fine with me (used -1 to keep it as an integer.) I think i 
prefer
the None version of this, which could also be explicit, i.e [1,None]

>>>   - Same for external type's width, precision and scale, such as in:
>>>     'NUMERIC(12,2)' or 'VARCHAR(200)'
>>
>> Same thing here. This would also make it difficult to provide defaults
>> separately for dbtype, and for width or precision.
>
> I disagree: default would be treated exactly the same way: if the width
> or precision/scale cannot be found in the parsed string then the 
> defaults
> apply.

Ah, so you mean that assignem,ent will just map such an initial value 
onto the
concerned properties, and then work with the separated value sfrom then 
on.
Well, OK. But may lead to confusion if you for the value of type, and 
get
something else than what you specified (e.g. only 'NUMERIC' without the 
qualifiers).

>> So, I would vote to keep them separate (and tyo change the name of
>> externalType to dbType). I think it will be easier to work with,
>> and clearer for everyone.
>
> I'd really like to keep the former possibility, though. It can co-exist
> with yours, I've no problem with that.
>
> I won't however go crazy if we do not support it ! I suggest we forget
> about my proposal for 'VARCHAR(20)' and that we only consider explicit
> parameters width/precision/scale. Since it's not a hard thing to add, 
> we
> can safely forget it for the moment and add it later if this reveals to
> be a users' request.

I would agree with proceeding this way.

>   Now that I think of it, there's an underlying point that should be
>   discussed for clarification. For example a string/VARCHAR won't get
>   any defaults in the conversion process, unless explicitly stated in
>   the model itself (yours AString.default['width']) --same for NUMERIC,
>   etc. This way it makes it clear that a String requires a width, that 
> a
>   float requires precision and scale, etc., since we would get an
>   error if neither a default nor an explicit parameter is provided.

Yes, this syntax will make that association clearer. But the validation 
should
also point out if these are missing.

>   What do you think about this? In fact, I do not even know what a good
>   default for width, scale or precision can be. Different DB have
>   different defaults, and I don't think it's worth registering/tracking
>   them.

Have no idea what reasonable defaults should be...
I feel their main purpose is to make things less error-prone, especially
in the beginning. Programs should probably specify them anyway,
when they know what is best for the program (and, should probably
not rely on default values set by the framework.)

>>>   - Again, same for an entity's parent which could be specified
>>>     along with the entity's name: 'Executive(Employee)'
>>
>> Same. I vote to keep separate, and to rename parentEntity to isAlso,
>> e.g.  Entity('Executive', isAlso='Employee', ...
>
> I know this is a detail, but I would have sponteanously shortened it to
> 'parent'. 'isAlso' vaguely recalls me something, but I don't know
> what... It sounds familiar. Is this UML jargon?

Not UML -- it is just what i think of (isAlso) when i think of 
inheritance...
But parent is just fine.

>>>   - What about the possibility to add a '*' to an attribute's name
>>> when it's
>>>     required? (may be same for a relationship, equivalent to lower
>>> bound==1)
>>
>> Same for adding '*' to att name
>
> Ok, again, let's forget it, I added it for completeness but didn't
> really like it either. The explicit parameter 'required' is enough.
>
> Speaking of this term, maybe it's not clear enough: I observed people
> being surprised that '' (empty string) is a valid value for a field 
> that
> is required because they misunderstood it, while it only tells whether
> the attribute can be None (python) / NULL (SQL). In the Attribute API
> there's also the counterpart 'allowsNone', what do you think?

Hmmn, i feel isRequired is more logical, while allowsNone is more
implementation-oriented -- but quite clear. The problem with it is that
it is a double negative (required => allowsNone = No) which i always
find irritating. Plus, should one day a formal expression of what values
are allowed for an attribute are also specified in the model, then 
isRequired
will lose the confusion potential you mention, e.g. if the model 
supports
a regexp for an attribute (to which all values must match) then 
isRequired
will clearly mean that a value respecting this constraint must be 
present.

Which is a good opportunity to propose the feature request of being 
able to
specify a regexp for an attribute, against which the framework will
validate all values to be assigned to the attribute (and which does not
exclude providing other declarations to use for data validation for any
attribute, such as type).

...

>> See the defaults for sourceAttribute and destinationAttribute in
>> RToOne and RToMany... Also, see the explicit declaration of the
>> foreign keys (with the possibility of automatic the name for it).
>
> I'm afraid the naming scheme you propose won't make it. In fact, I was
> thinking of automating the simple case, where only one association
> exists between two entities. In this case, the foreign key only needs 
> to
> be named 'fk<destinationEntityName>' ; this makes it easy for one
> relationship and its possible inverse to actually refer to the same
> source/destination attributes in their joins.
>
> Now a more complex example: suppose you want to model something like
> this:
>
>       A <------->> B
>         <------->>
>
>   (this comes from a "real-life" example, two relationships from A to B
>    having a different semantics)
>
> Suppose we first process A's relationships: this will create two
> different FK in B pointing to A's PK. Now we process entity B: there is
> no way to identify which relationship coming from A is the inverse of a
> toOne from B. Hence, this can be automated *if and only if* we also 
> have
> the keyword 'inverse' in the declaration of a relationship.
>
>   BTW, this is where I think automation will not be a real gain and 
> will
>   not be worth the effort, because that sort of model is really not
>   showing up frequently, and because when it shows up you usually want
>   to have names for foreign keys more explicit than
>   fk<destinationEntityName><count>
>
>   (You stated:
>    fk<destinationEntityName><destinationAttributeName><count> but since
>    the destAttr. will always be the same for a given dest. entity (the
>    PK), I removed it here)
>
>>> This could be as handy as the feature you submitted in your proposal:
>>> automatic definition parent properties which are not overriden in
>>> sub-entities.
>>
>> Yes. Auto generation of foreign keys from a relationship definition 
>> does
>> make sense in fact... but the possibility to declare them explicitly
>> (with
>> the desired values for their attributes) should always be there. Since
>> such explicit declaration can be very simple, I prefer to leave it as
>> must be explicit.
>
> My opinion here is:
>
>   - make it possible to automate the simple case (PK and FK), as
>     explained above,
>
>   - always keep the possibility to explicitly write every single
>     properties in models, entities, etc.

Here i think the second point is more important than the first, and will
not limit unneccessarily what the framework can handle. The first point
is nice, maybe, but since these would be so simple to declare, there is
not so much to gain.

> You might think that I'm definitely reluctant to impose explicit
> declarations (such as for the FK). I think I am but it's not a
> religion ;)

Well, i can understand that in an ideal world the "db details" are
all handled automatically, and inded you cannot get to them. But unless
full automation can be guaranteed, then going this route may cause
more problems than it solves. And, will not handle existent databases.
"Optional automation" is OK, but i feel one should always be able to
override it.

On the same line, i also feel that an object id should optionally be
manually set by the client code.In 90% (or more) of the time you just 
want
it automatic, but sometimes you may want to control it (for db 
optimization
or whatever other reasons).

> In fact, it's really a matter of usual practices in E-R modeling. This
> is not only /my/ practice, but the way most people seem to deal with
> entity-relationship modeling --they do not care about the db-schema
> PK/FK details unless they need to, for example when defining a model 
> for
> an existing model, or when some complex design is required (just like
> the one I described above).
>
>   Obviously this observation was not made on that framework (there are
> still not enough people using it for such an observation), but it is
> what I observed from the community using the original Apple's 
> Enterprise
> Object Framework(tm).
>
> I'm possibly wrong and maybe I just think that people do not care
> because most of the times *I* usually don't care. In the ZModeler I
> added the ability to make the two relationships necessary for a
> one-to-many association at a single-click reach for that very reason:
> it's a valuable feature to my own eyes, and I think it may be for
> other's.

I agree that *most* of the time you want all these things to be managed
automatically. But, again, as long as full automation cannot be 
guaranteed,
one should always allow the possibility to do whatever the db allows you
to, as long as one takes responsibility for it. This is also similar to 
python's
way of doing things (that of not "forcing" that variables are private 
for example)
and has turned out to be one of the strong points of the language (that 
whatever
unforeseen situation may arise, you are not locked in).


>   Last comment on this: I do not really like the idea of imposing the 
> FK
>   to be declared while having a implicit default for the PK, it sounds
>   inconsistent to me.

Yes, here i agree. I am fine with not requiring that FKs are declared
explicitly, and the framework handles their definition automatically.
But the framework should also recognize if the FKs are declared and
and create them accordingly.

...

>> Oh, and some other changes I have not mentioned here are listed in
>> the top of PyModel_3_mr.py, but mostly name changes, and having
>> entities have only one list, that mixes attributes and relations...
>> (your opinion?).
>
> * deleteRule -> delete: ok
>
> * multiplicity lower/upper bound -> multiplicity as a python list: ok
>
> * entity only has attributes that may also be relations: I'm ok for the
>   principle, however i would prefer an distinct term for this, 
> relations
>   being attributes could be a source of confusion. Maybe 'properties' ?

OK for me.

>   More comment on this: having attributes and relations mixed in a
>     common list will require extra-checks when analysing the model to
>     perform the possibly needed automated steps. This is because we 
> will
>     need to have all the entities and their attributes loaded and
>     processed before we can make a decision on whether a declared
>     relation needs automatic generation of FK. We will need to separate
>     attributes from relations anyway.

When processing atts and rels, the logic would be to assign them to 2 
different
arrays, and work with those arrays.

>   I do not mean this is complex: it's not. However I'm not sure this
>   enhance readability. In the sample model your clearly first declared
>   attributes, than relationships, and I bet everyone will do it that
>   way, because it would be a real mess to have some attrs, than rels,
>   than attrs, etc.

Well, some may prefer to group in some other logical way than by type...
But, my main reason for this is that have to deal with extra list 
brackets inside
the Entity constructor may be avoided, making the code a little more
readable.

> * misc.: at some point in the discussion the FK were made class
>          properties in the py-model, but they should not!

Ooops, mistake.

>
> Regarding PyModel_3_mr.py:
>
>   - I'm not sure we want a default for entity.adaptorName

You mean model? OK, no problem.

>   - I'd add a APrimaryKey subclass of Attribute

OK.

The updated sample (4) is below, but you can also get to it, along with 
the
new PyModel version (4) from http://ruggier.dyndns.org:8080/PyModel/

Cheers, mario

'''
A sample Pythonic OO-RDB Model
(re-expressing testPackages/StoreEmployees/model_StoreEmployees.xml) --
A PyModel must define a global variable called 'model', that is of type 
Model
'''

from PyModel import *

##
# Set preferred defaults for this model (when different from
# standard defaults, or if we want to make things explicit)

AInteger.defaults['precision'] = 10
AString.defaults['width'] = 20

RToOne.defaults['delete'] = 'cascade'
RToOne.defaults['multiplicity'] = [0,1]
RToOne.defaults['sourceAttribute'] = RToOne.attNameFromRel # 
'fk'+destEnt+destAtt+count
RToOne.defaults['destinationAttribute'] = 'id'

RToMany.defaults['delete'] = 'deny'
RToMany.defaults['multiplicity'] = [0,None]
RToMany.defaults['sourceAttribute'] = 'id'
RToMany.defaults['destinationAttribute'] = RToMany.attNameFromRel # 
fk+destEnt+sourceAtt+count

# Note that Relation.attNameFromRel is a callable, that calculates the 
att name
# from the indicated pieces (where count to distinguish between 
multiple relations
# between same source and target


Entity.defaults['properties'] = [
     APrimaryKey('id', isClassProperty=0, isRequired=1)
]

##

_connDict = {}
model = Model('StoreEmployees',connDict=_connDict)
model.entities = [

     #
     Entity('Store',
         properties=[
             AString('corporateName', isRequired=1),
             RToMany('employees', 'Employee')
         ]
     )

     # Employee and its subclasses SalesClerk and Executive
     Entity('Employee',
         properties=[
             AString('lastName', isRequired=1, usedForLocking=1),
             AString('firstName', isRequired=1, width=50, 
usedForLocking=1),
             AForeignKey(Attribute.fk('Store'), isClassProperty=0),
             RToMany('toAddresses', 'Address', delete='cascade'),
             RToOne('toStore', 'Store')
         ]
     )

     Entity('SalesClerk', parent='Employee',
         properties=[
             AString('storeArea')
         ]
     ),

     Entity('Executive', parent='Employee',
         properties=[
             AString('officeLocation', width=5),
             RToMany('marks', 'Mark', delete='cascade')
         ]
     ),

     #
     Entity('Address',
         properties=[
             AString('street', width=80),
             AString('zipCode'),
             AString('town', width=80),
             AForeignKey(Attribute.fk('Employee'), isClassProperty=0),
             RToMany('toEmployee', 'Employee', delete='deny')
         ]
     ),

     #
     Entity('Mark',
         properties=[
             AInteger('month', isRequired=1),
             AInteger('mark', isRequired=1),
             AForeignKey(Attribute.fk('Executive'), isClassProperty=0),
             RToOne('executive', 'Executive' ]
         ]
     )

]

if __name__ == '__main__':
     print model.validate()
     print model.toXML()
     # plus whatever ...

##

Re: [Modeling-users] Limit

[Jerome K. <Jer...@fi...>]

On Sat, Mar 01, 2003 at 02:15:07PM +0000, Sebastien Bigaret wrote:

[snip]

> * MySQL offers the very same syntax
>   http://www.mysql.com/doc/en/SELECT.html
> 
>   even if it's not clear to me if an ORDER BY clause is also required for
>   consistency between consecutive queries, but a few tries will show (I'll
>   appreciate if someone with a mysql server at hand could try and report)

This is right in fact in mysql if you don't have a 'ORDER BY' clause the server 
will use the insert ordering.

[snip]

 
> Back to the fetchLimit(): what do you all think about the proposal?
> 
> Implementing it for Postgresql and MySQL wouldn't be that hard, except that
> it will require support for SortOrderings on the SQL side --not done yet.

Yeah this sound great. 


Bye Bye :)

Re: [Modeling-users] Re: About Database Schema Designer

[Sebastien B. <sbi...@us...>]

        Hi,

> >   The most annoying problem I see is the complete lack of inheritan=
ce --
> >   there seem to be no way to model it. Okay this is quite normal fo=
r a tool
> >   dedicated to db-schema, but we'll need it somehow.
>=20
> I confirm that I saw no way to model it, it could simply become a
> specialized form of relation.

Oh, well, this should be changed, somehow, if at some point we want to =
point
to this tool for use by the framework's users.

> > - about the script: yes, there are some default values, but they are
> > easily identified hence this is not really a problem changing these=
 (I
> > noticed you really want to have your primary key as class propertie=
s :)
>=20
> We could load customs defaults from a .dd2pmrc or allow a command line
> argument.  As I said in a former email, we need to cross a XML-RPC br=
idge
> and the only way I see to keep objects identity is through primary ke=
y.  I
> found the snapshot() method quite useful

Ok, I see. Have you considered using the informations enclosed within e=
very
GlobalID?

  The GlobalID of an object is its identifier within the framework ; yo=
u can
  get it from an EditingContext by using the message 'objectForGlobalID=
'.
  Every object that exists in the database is assigned a unique KeyGlob=
alID,
  whose keyValues() method returns a dictionary mapping the primary key=
's
  name with its value.

> >   I was really surprised by the way relations are mapped to
> >   relationships. If I'm not wrong, you're mapping them to 'to-one'
> >   relationship, unconditionally. However, when I tried the tool, I =
came to
> >   the opposite opinion, because of the 2nd screenshot:
> >   http://www.danny.cz/screenshots/datadesigner1.jpg
> >
> >   Can you explain your practices wrt the relations? Do you design to
> >   relations, one being the inverse of the other, when you design a =
model
> >   with the db-designer, and edit the xml file afterwards to change =
one of
> >   these from to-one to to-many?
>=20
> To date I only used to-one but to-many can be modeled and should not
> pose any problems to convert.

Ok.

> When is Zope going to support Python 2.2 ?

I think 2.7 will officially support 2.2, but I've not checked. I've read
that 2.6 already (but not officially) supports py2.2.

> >   - defaults in mutateEntity() & mutateEntAttr() are directly defin=
ed as
> >     dictionaries, while 'mutateRelation()' uses makeDefaults()
>=20
> I got tired of quoting dict keys so I used a quick hack I saw in the
> Python Cookbook.  I agree that we should use only one way to do it all
> across the script, which one do you find cleaner ?
>=20
> I prefer makeDefaults()

So do I, and I'll remember that idiom for sure, really handy.

About PyDBDesigner:
> Ehhh... this tool is far from being mature...

Unfortunately, yes...

> By the way how do you achieve this funny indentation ?

Well, emacs almost does all the work! (M-q, fill-paragraph)

> > Last, just curiosity: was the zmodeler not an option for you becaus=
e of
> > zope itself or do you find/use more of the db-designer xml file tha=
t the
> > zmodeler (and/or the modeling framework's model xml file) does not
> > offer?
>=20
> Zope was not a problem since we already had a Zope up and running but=
 I'm
> not sure if we would have installed one just to test the framework.  =
The
> main problem with the ZModeler is the lack of support for various com=
ments
> but also the lack of visual presentation of the model.

  I'm afraid there's nothing useful to be done as far as the ZModeler a=
nd
  its lack of visual presentation are concerned. Your other remark abou=
t the
  possibility to add comments is interesting. Where would you ideally a=
dd
  these comments in the model?

        Regards,

-- S=E9bastien.

[Modeling-users] [Core] v0.9-pre-3 released

[Sebastien B. <sbi...@us...>]

        Hi all,

  I have just released 0.9pre3. This is the release candidate for 0.9.

It includes bug-fixes for bugs #690224 "Moved model raises
UnboundLocalError" and #695671 "Validation does not detect INTEGER(widt=
h)"
=20
  It also contains documentation for the new feature #617997: "Nested
  EditingContexts" introduced in 0.9-pre-2 --shortly put, you can think=
 of
  nested EditingContext as a transaction at the object level.

I'd really appreciate your feedback on this new section.

  Best regards,

-- S=E9bastien.

Re: [Modeling-users] fetching problem

[Sebastien B. <sbi...@us...>]

> > > But I can't fetch any value, the following code :
> > >
> > >  from Modeling.EditingContext import EditingContext
> > >  from Modeling.FetchSpecification import FetchSpecification
> > >  from Autogen.PROJECT import PROJECT
> >=20
> > I found the problem, the import statement was edited for the email,=
 it
> > really was from core.utils.Autogen import PROJECT.  The if I flatten
> > my hierarchy a bit to REALLY have from Autogen.PROJECT import PROJE=
CT,
> > it works perfectly.
> >=20
> > Do you plan to support sub-package import in future releases ?
>=20
> Okay, I think I see your problem: you generated the python code
> templates with the model you supplied in your previous mail, than you
> put the generated package in package core.utils.
>=20
> If true, the only thing you need to change is attribute 'packageName'=
 of
> xml-entity <model>: change it from 'Autogen' to 'core.utils.Autogen',
> and you'll see that your entities can now find their own classes (you=
'll
> probably want to modify both model_Autogen.xml and model_Autogen.py)
>=20
>   BTW this question is addressed in section 2.6 / FAQ of the User's
>   Guide ;) but if you read it and did not understand that way, please
>   tell me
>=20
> However, you did find a bug here: it shouldn't raise UnboundLocalError
> but at least something more informative! Could you please take some t=
ime
> and fill in a bug report in sourceforge's tracker?

Fixed in CVS repository.

  The exception raised now looks like the following, hopefully more
  informative:

Traceback (most recent call last):
  File "./test_EntityClassDescription.py", line 60, in test_02_classFor=
Entity_incorrect_packageName
    EntityClassDescription.classForEntity(e)
  File "Modeling/EntityClassDescription.py", line 504, in classForEntity
    raise ImportError, err_msg
ImportError: Unable to locate class Dummy.StoreEmployees.Employee.Emplo=
yee corresponding to entity 'Employee'.
It is possible that the model's packageName, or the entity's moduleName=
 or
className do not correspond to where the module is installed --for exam=
ple,
you might have moved it to a sub-package. You can solve this easily by
updating your model so that 'packageName.moduleName.className' points to
the exact location where the class is installed

Original exception was: ImportError: No module named Dummy


-- S=E9bastien.

Re: [Modeling-users] mdl_generate_DB_schema: table creation problem

[Sebastien B. <sbi...@us...>]

> Yes it is exactly it.
> I have entered bug num 695671

Fixed in CVS repository.

Note: INTEGER(#) is an invalid SQL data type (not postgres nor mysql sp=
ecific)

>=20
>   It must be because INTEGER(20) is invalid -- I get the same error w=
ith
>   postgresql until I replaced it with INTEGER alone.
>=20
>   --> Just remove any 'width' for id and hopefully it will be ok.
>=20
> I am feeding a simple (and with only one table) model to
> mdl_generate_DB_schema, that validates and generates python code OK. =
It
> fails with the log below.
>=20
>   Please fill in a bug report: the validation should detect that.
>=20
>   Can someone with a mysql at hand confirm that INTEGER(#) is also in=
valid for
>   mysql?
>=20
>=20
> -- S=E9bastien.

Re: [Modeling-users] Limit

[Mario R. <ma...@ru...>]

> NB: Postgresql documentation says that neither LIMIT nor OFFSET are  
> part of
>     SQL92, and that there is no ANSI definition for these features.
>
> By the way, I finally searched to find out how this could be done,  
> since
> this request shows up persistently from time to time --I guess the  
> idea is
> to make it possible to present different pages in batches in a GUI.
>
> * Postgresql has  LIMIT # [OFFSET #]
> http://www.postgresql.org/docs/view.php?version=7.3&idoc=1&file=sql- 
> select.html

Ah, great, i was not aware of these 2... i'm pretty sure they are not  
supported
by mssql, but have no idea about others.

>   requiring that ORDER BY is set if you want consistent batches  
> between two
>   selects
>
> * MySQL offers the very same syntax
>   http://www.mysql.com/doc/en/SELECT.html
>
>   even if it's not clear to me if an ORDER BY clause is also required  
> for
>   consistency between consecutive queries, but a few tries will show  
> (I'll
>   appreciate if someone with a mysql server at hand could try and  
> report)

I would say yes, as these would be logically separate queries (possibly  
with
other different queries ove the same connection.

> * Oracle defines the pseudo-column ROWNUM
>
> http://forums.devshed.com/archive/5/2002/11/3/47157
> http://www.orafaq.net/glossary/faqglosr.htm
> http://gethelp.devx.com/techtips/oracle_pro/10min/10min1200/10min1200- 
> 3.asp
>
>   but it has a different semantics, since the pseudo-column is filled  
> in
>   *before* any orderings --the last url gives an easy solution for  
> solving
>   this.
>
> [and there are some other comments at
>  http://www.experts-exchange.com/Databases/Q_20307165.html]
>
>
> So it should be possible to modify the FetchSpecification API:
>
>   instead of (unimplemented) setFetchLimit(self, limit) we could have
>   setFetchLimit(self, limit, offset=0). Then you'll be on your own to
>   iterate on the result sets given that you previously queried the  
> select
>   count.

Yes, this would be fine, but will not work or Oracle?
Or an Oracle layer will automatically manage limit and offset, mapping
them to some thing like "where rownum>x and rownum<y" ?

> Mario answered:
>> [...] would be really nice to have a generic declarative way for this
>> (essentially cursors, or paging, but as far the client app code is
>> concerned, there should be no worries about maintaining state
>> information).
>
>   Yes it would be nice, and not very hard to code. However I can't see  
> any
>   object in the framework where it could make sense to add such a
>   feature. It would be the responsability for an object on top of an
>   EditingContext, but there's none for the moment being.

Well, i think proposal above is very close to this. But, to clarify, I  
was just
imagining some qualifier extension to be able to set a chunkBySize=N
and then be able to say getNextChunk or getPreviousChunk or  
getChuckNum...
A setFetchLimit (or setFetchScope or setFetchSlice ...) will provide the
basics for this.

BTW, how is the ORDER BY functionality exposed vis the fetch and
qualifier interface?

>
> Back to the fetchLimit(): what do you all think about the proposal?
>
> Implementing it for Postgresql and MySQL wouldn't be that hard, except  
> that
> it will require support for SortOrderings on the SQL side --not done  
> yet.

It would be very good to have, but maybe with a different name (limit  
is too limited ;),,
and adding an page, such as setFetchSlice(self, limit, offset=0,  
page=0). This
will allow not only getting a chunk, but also jumping to subsequent (or  
previous)
chunks --> page would be the chunk index, thus, if we ask for

setFetchSlice(self, 10, offset=7, page=2)

on a resultset of total 100 items, we will get the 10 items in the 3rd  
page (if
we alas start from a page 0), i.e. from 27 to 36.

Cheers, mario

Re: [Modeling-users] mdl_generate_DB_schema: table creation problem

[Mario R. <ma...@ru...>]

Yes it is exactly it.
I have entered bug num 695671

Many thanks, mario



>   It must be because INTEGER(20) is invalid -- I get the same error=20
> with
>   postgresql until I replaced it with INTEGER alone.
>
>   --> Just remove any 'width' for id and hopefully it will be ok.
>
>> I am feeding a simple (and with only one table) model to
>> mdl_generate_DB_schema, that validates and generates python code OK.=20=

>> It
>> fails with the log below.
>
>   Please fill in a bug report: the validation should detect that.
>
>   Can someone with a mysql at hand confirm that INTEGER(#) is also=20
> invalid for
>   mysql?
>
>
> -- S=E9bastien.

Re: [Modeling-users] Pythonic, and non-XML, Model description

[Sebastien B. <sbi...@us...>]

        Hi,

> > (same for generating an xml-model, thus mapping a py-model to an=20
> > xml-model
> >  will be straightforward --cf. ModelSet.getXMLDOMForModelNamed() and
> >  getXMLStreamForModelNamed())
>=20
> Yes, should be the same idea, except that there'd be no need to have
> methods with "forModelNamed" in their name, as this would be implied
> by the object on which the method is called:
>=20
> modelset.getXML()
> modelset.getXMLDOM()
> model.getXML()
> entity.getXML()
> ...

Yes, we're meaning the same thing here. I just wanted to point out where
it was implemented. And it was stupid, I should have pointed the
existing methods Model.getXMLDOM() and .saveModelAsXMLFile() instead.

> The ModelSet concept is not yet included in the PyModel description
> (mostly becuase it is not fully clear to me how it is used).=20

I'm afraid there's no real documentation on that. Let's go for a short
one: the framework can only deal with one ModelSet at a time, the one
returned by defaultModelSet() in module ModelSet. The module & class
main responsabilities are:

  1. making sure that all the entities defined in a ModelSet instance h=
ave
     distinct names: a ModelSet identifies its models and entities by n=
ame.
     Hence, object defaultModelSet() can be asked at runtime for models=
 and
     entities, given their names

  2. registering the object returned by defaultModelSet() as the default
     receiver for the notification
     ClassDescriptionNeededForEntityNameNotification posted by
     classDescriptionForName() in module ClassDescription.

(I added this to the module's docstring)

> How would you extend this description to include it?

I won't ;! Since the framework only deals with one ModelSet at a time, I
can't see any reason to add it in the model (there's no model set in the
xml model either).

About: key&locking attrs defined on atts and rels rather than entities
> [...] But, see how it makes things easier [and] consistent [...]

I saw ;) and I'm convinced.

About: Models' being mutable at runtime
> In general this is not what anyone woudl want to do, and I was trying
> to think of times when this would be useful. Can't really, but i
> *suspect* it might come in handy. But, in fact, imagine some
> application that is stocking lots of incoming unknown data into a
> db... let's say the data is XML, but each XML data-schema is unknown a
> priori. You can create a py model to correspond, at runtime (but saved
> for subsequent accessing), create the tables, and stock the unknown
> XML in the db directly as real db tables....

Okay, I now can be more precise here: adding an entity at runtime is
definitely not a problem. But mutating an entity (changing its
attributes for example, or its name), or removing an entity should *not*
be done after an object of the corresponding class has been registered
within an EditingContext.

(this should be a FAQ)

> >> And, speed of course, as i see no reason why this model object
> >> would also not serve as the model object in memory at runtime (but
> >> that's for Sebastien to confirm) -- thus loading the model is
> >> equivalent to "from MyModel import model".
> >
> > Well, it will need a few more lines of code to actually load the
> > model into the so-called defaultModelSet (cf. Modeling.ModelSet and
> > the generated __init__.py), but that's it.
>=20
> Yes, but i was leaving all that to you ;)

No problem!

> But, details of how such a py model would integrate with the rest of
> the framework are all to be decided, and please point out any problems
> that only you can foresee!

Integrating a py-model is just a matter of transforming a pymodel to a
Modeling.Model. Hence, all possible problems are really conversion
problems.  And I keep my eyes opened!

> in general i do not like 'special format strings' as values to
> things. This adds possible errors, requires special documentation,
> makes checking more difficult, and in general is less pythonic... I
> think keeping the info units separate would be easier to work with,
> and clearer. The overhead of having to specify the same parameters
> many times over can be reduced by standard defaults and the
> possibility to set defaults only for this model (see new sample...)
>=20
> >   - maybe we'll gain more readability if relationships' multiplicity
> >     lower- and upper bounds where encoded in strings, such as: '0-1=
',
> >     '1-1', '0-*' or '2-16'.
>=20
> Yes, but not as strings. What is wrong with a python list?
> multiplicity =3D [0,1]
> multiplicity =3D [1,-1]

Well, I understand your point quite well and I'm ok with a list. [1,-1]
seems however obscure, if we go for a python list what do you think of
[0,'*'] or maybe better: [0,]  ?

> >   - Same for external type's width, precision and scale, such as in:
> >     'NUMERIC(12,2)' or 'VARCHAR(200)'
>=20
> Same thing here. This would also make it difficult to provide defaults
> separately for dbtype, and for width or precision.

I disagree: default would be treated exactly the same way: if the width
or precision/scale cannot be found in the parsed string then the defaul=
ts
apply.=20

> So, I would vote to keep them separate (and tyo change the name of
> externalType to dbType). I think it will be easier to work with,
> and clearer for everyone.

I'd really like to keep the former possibility, though. It can co-exist
with yours, I've no problem with that.

I won't however go crazy if we do not support it ! I suggest we forget
about my proposal for 'VARCHAR(20)' and that we only consider explicit
parameters width/precision/scale. Since it's not a hard thing to add, we
can safely forget it for the moment and add it later if this reveals to
be a users' request.

  Now that I think of it, there's an underlying point that should be
  discussed for clarification. For example a string/VARCHAR won't get
  any defaults in the conversion process, unless explicitly stated in
  the model itself (yours AString.default['width']) --same for NUMERIC,
  etc. This way it makes it clear that a String requires a width, that a
  float requires precision and scale, etc., since we would get an
  error if neither a default nor an explicit parameter is provided.

  What do you think about this? In fact, I do not even know what a good
  default for width, scale or precision can be. Different DB have
  different defaults, and I don't think it's worth registering/tracking
  them.

> >   - Again, same for an entity's parent which could be specified
> >     along with the entity's name: 'Executive(Employee)'
>=20
> Same. I vote to keep separate, and to rename parentEntity to isAlso,
> e.g.  Entity('Executive', isAlso=3D'Employee', ...

I know this is a detail, but I would have sponteanously shortened it to
'parent'. 'isAlso' vaguely recalls me something, but I don't know
what... It sounds familiar. Is this UML jargon?

> >   - What about the possibility to add a '*' to an attribute's name=
=20
> > when it's
> >     required? (may be same for a relationship, equivalent to lower=
=20
> > bound=3D=3D1)
>=20
> Same for adding '*' to att name

Ok, again, let's forget it, I added it for completeness but didn't
really like it either. The explicit parameter 'required' is enough.

Speaking of this term, maybe it's not clear enough: I observed people
being surprised that '' (empty string) is a valid value for a field that
is required because they misunderstood it, while it only tells whether
the attribute can be None (python) / NULL (SQL). In the Attribute API
there's also the counterpart 'allowsNone', what do you think?


> but definately yes to have constraints forced down as a result from
> relations, e.g. if a one-to-one relation is required, then the related
> attributes must also be required -- but this is automatic and taken
> into account by the validation.

Currently designing one-to-one is not possible, but we can automatically
make a choice on which entity the foreign key should be dropped in.  For
one-to-many this is straightforward (almost, see below comments on FK
names)

> >   - Allow litterals instead of the equivalent integers in the xml:
> >     'CASCADE' for the delete rule is more explicit than int(2)!
>=20
> Definately. Also, in lowercase! (Hate being screamed at, which is what
> uppercase seems to be always doing  \-)

This is an old habit for declaring constants I inherited from the years
where I was a C programmer! So lowercase, ok.

> > Now that I get used to the idea of a python-model, I'm also thinking
> > of some extents to your proposal:
> >
> >   - We could have subclasses for your Attribute: PrimaryKey, Foreig=
nKey
> >     (defaults for both would be: int, not class property, etc.), St=
ring
> >     (with a default external size/width), Integer, Numeric, ... You=
=20
> >     get the idea.
> >=20=20=20=20=20
>=20
> Great! There could be some standard sub-classes, but a user is
> ofcourse allowed to make his own. What about the naming scheme propoes
> by the example below, that Att subs start with the letter 'A' and rel
> subs start with the letter 'R' ? (to avoid unnecessarily long names)

I easily got used to it, if there's no other objections let's keep it

> > Last, I'm thinking of some automatic processing which is already
> > coded in the zmodeler and that could be done at model-time to reduce
> > verbosity in a significant manner:
> >
> >   - have a primary key 'id' automatically declared if not set,
>=20
> In general "magical" behaviour is more trouble than gain... how about
> if we have the possibility to define a default attribute on an Entity
> description class?

Fine, we then have standard defaults along with the ability to specify
specific default for a model, all in one. +1 on this, definitely.

> >   - have foreign keys automatically set for relationships, using the
> >   same defaults the zmodeler already uses (e.g. FKEmployeeId for a
> >   to-many relationship pointing to the entity Employee). It would
> >   need some additional checks but it's definitely possible.
>=20
> Again, I like simplifying the management of all this, but i do not
> like being forced to accept decisions imposed by the framework, such
> as the names of my columns (what if I want to provide a model for an
> existing db?)
> So, providing a default scheme, that the user may redefine (or not
> use at all), seems more reasonable to me.=20

Ok -- what I wrote was not clear enough, I meant that too.

> See the defaults for sourceAttribute and destinationAttribute in
> RToOne and RToMany... Also, see the explicit declaration of the
> foreign keys (with the possibility of automatic the name for it).

I'm afraid the naming scheme you propose won't make it. In fact, I was
thinking of automating the simple case, where only one association
exists between two entities. In this case, the foreign key only needs to
be named 'fk<destinationEntityName>' ; this makes it easy for one
relationship and its possible inverse to actually refer to the same
source/destination attributes in their joins.

Now a more complex example: suppose you want to model something like
this:

      A <------->> B
        <------->>

  (this comes from a "real-life" example, two relationships from A to B
   having a different semantics)

Suppose we first process A's relationships: this will create two
different FK in B pointing to A's PK. Now we process entity B: there is
no way to identify which relationship coming from A is the inverse of a
toOne from B. Hence, this can be automated *if and only if* we also have
the keyword 'inverse' in the declaration of a relationship.

  BTW, this is where I think automation will not be a real gain and will
  not be worth the effort, because that sort of model is really not
  showing up frequently, and because when it shows up you usually want
  to have names for foreign keys more explicit than
  fk<destinationEntityName><count>

  (You stated:
   fk<destinationEntityName><destinationAttributeName><count> but since
   the destAttr. will always be the same for a given dest. entity (the
   PK), I removed it here)

> > This could be as handy as the feature you submitted in your proposa=
l:
> > automatic definition parent properties which are not overriden in
> > sub-entities.
>=20
> Yes. Auto generation of foreign keys from a relationship definition d=
oes
> make sense in fact... but the possibility to declare them explicitly=
=20
> (with
> the desired values for their attributes) should always be there. Since
> such explicit declaration can be very simple, I prefer to leave it as
> must be explicit.

My opinion here is:

  - make it possible to automate the simple case (PK and FK), as
    explained above,

  - always keep the possibility to explicitly write every single
    properties in models, entities, etc.

You might think that I'm definitely reluctant to impose explicit
declarations (such as for the FK). I think I am but it's not a
religion ;)=20

In fact, it's really a matter of usual practices in E-R modeling. This
is not only /my/ practice, but the way most people seem to deal with
entity-relationship modeling --they do not care about the db-schema
PK/FK details unless they need to, for example when defining a model for
an existing model, or when some complex design is required (just like
the one I described above).

  Obviously this observation was not made on that framework (there are
still not enough people using it for such an observation), but it is
what I observed from the community using the original Apple's Enterprise
Object Framework(tm).

I'm possibly wrong and maybe I just think that people do not care
because most of the times *I* usually don't care. In the ZModeler I
added the ability to make the two relationships necessary for a
one-to-many association at a single-click reach for that very reason:
it's a valuable feature to my own eyes, and I think it may be for
other's.

  Last comment on this: I do not really like the idea of imposing the FK
  to be declared while having a implicit default for the PK, it sounds
  inconsistent to me.

Please do not misunderstand what I'm saying here, my english is not good
enough to be subtle. I'm just trying to make my opinions and positions
clearer by being more explicit :)

> I have taken your simplified model, and evolved it to correspond to my
> comments above. Correspondingly, i have also updated the PyModel
> module to indicate how it would change to support these changes.
> [...]
> Oh, and some other changes I have not mentioned here are listed in
> the top of PyModel_3_mr.py, but mostly name changes, and having
> entities have only one list, that mixes attributes and relations...=20
> (your opinion?).

* deleteRule -> delete: ok

* multiplicity lower/upper bound -> multiplicity as a python list: ok

* entity only has attributes that may also be relations: I'm ok for the
  principle, however i would prefer an distinct term for this, relations
  being attributes could be a source of confusion. Maybe 'properties' ?

  More comment on this: having attributes and relations mixed in a
    common list will require extra-checks when analysing the model to
    perform the possibly needed automated steps. This is because we will
    need to have all the entities and their attributes loaded and
    processed before we can make a decision on whether a declared
    relation needs automatic generation of FK. We will need to separate
    attributes from relations anyway.=20

  I do not mean this is complex: it's not. However I'm not sure this
  enhance readability. In the sample model your clearly first declared
  attributes, than relationships, and I bet everyone will do it that
  way, because it would be a real mess to have some attrs, than rels,
  than attrs, etc.

* misc.: at some point in the discussion the FK were made class
         properties in the py-model, but they should not!


Regarding PyModel_3_mr.py:

  - I'm not sure we want a default for entity.adaptorName

  - I'd add a APrimaryKey subclass of Attribute

> >   I really have the feeling that we will succeed in designing a very
> > nice python model. Let's go for it now that Mario brought some light
> > on the path!
>=20
> That would not be XPath, would it ;-?

Ooooh no, it wouldn't ;!

        Cheers,

-- S=E9bastien.

Re: [Modeling-users] mdl_generate_DB_schema: table creation problem

[Sebastien B. <sbi...@us...>]

        Hi,

  It must be because INTEGER(20) is invalid -- I get the same error with
  postgresql until I replaced it with INTEGER alone.

  --> Just remove any 'width' for id and hopefully it will be ok.

> I am feeding a simple (and with only one table) model to
> mdl_generate_DB_schema, that validates and generates python code OK. =
It
> fails with the log below.

  Please fill in a bug report: the validation should detect that.

  Can someone with a mysql at hand confirm that INTEGER(#) is also inva=
lid for
  mysql?


-- S=E9bastien.


Mario Ruggier <ma...@ru...> wrote:
> Hi,
>=20
> I am feeding a simple (and with only one table) model to=20=20
> mdl_generate_DB_schema,
> that validates and generates python code OK. It fails with the log=20=
=20
> below.
> This is running on OSX, with postgres 7.3, pypgsql 2.3, osx 10.2.4 (n=
ew=20=20
> update,
> that has broken some other things...), and modeling 0.9-pre-2.
> Any ideas?
>=20
> Cheers, mario
>=20
>=20
> % mdl_generate_DB_schema.py -T cwText.xml
> Couldn't evaluate expression CREATE TABLE Text (
>    fr VARCHAR(400) ,
>    en VARCHAR(400) ,
>    nl VARCHAR(400) ,
>    name VARCHAR(40) NOT NULL,
>    pt VARCHAR(400) ,
>    de VARCHAR(400) ,
>    id INTEGER(20) NOT NULL,
>    es VARCHAR(400) ,
>    it VARCHAR(400) ). Reason: libpq.OperationalError:ERROR:  parser:=
=20=20
> parse error at or near "(" at character 168
>=20
> FAILED to execute: CREATE TABLE Text (
>    fr VARCHAR(400) ,
>    en VARCHAR(400) ,
>    nl VARCHAR(400) ,
>    name VARCHAR(40) NOT NULL,
>    pt VARCHAR(400) ,
>    de VARCHAR(400) ,
>    id INTEGER(20) NOT NULL,
>    es VARCHAR(400) ,
>    it VARCHAR(400) )
> Reason:
> Modeling.Adaptor.GeneralAdaptorException
> Couldn't evaluate expression CREATE TABLE Text (
>    fr VARCHAR(400) ,
>    en VARCHAR(400) ,
>    nl VARCHAR(400) ,
>    name VARCHAR(40) NOT NULL,
>    pt VARCHAR(400) ,
>    de VARCHAR(400) ,
>    id INTEGER(20) NOT NULL,
>    es VARCHAR(400) ,
>    it VARCHAR(400) ). Reason: libpq.OperationalError:ERROR:  parser:=
=20=20
> parse error at or near "(" at character 168
>=20
>    File "/usr/bin/mdl_generate_DB_schema.py", line 105, in=20=20
> databaseSchemaWithOptions
>      channel.evaluateExpression(sqlExpr)
>    File=20=20
> "/usr/lib/python2.2/site-packages/Modeling/DatabaseAdaptors/=20
> AbstractDBAPI2AdaptorLayer/AbstractDBAPI2AdaptorChannel.py", line 164=
,=20=20
> in evaluateExpression
>      raise GeneralAdaptorException, msg
>=20
>=20
>=20
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Modeling-users mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modeling-users

[Modeling-users] Re: About Database Schema Designer

[Yannick G. <ygi...@yg...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 27 February 2003 19:50, Sebastien Bigaret wrote:
> - about the tool itself: it is really focused on db-schemas ; it has some
>   features I cannot understand, such as the 'type' of a relation
>   (identifying, identifying or informational), or the referential integrity
>   properties for the child/destinationEntity (insert/update props). May be
>   you can comment on these?

Referential Integrity: cascade the delete with references on foreign
                       keys, don't allow inserts with a non-existing
                       foreign key.  Or something close to that...

I simply ignore those informations in the conversion for now.

>   The most annoying problem I see is the complete lack of inheritance --
>   there seem to be no way to model it. Okay this is quite normal for a tool
>   dedicated to db-schema, but we'll need it somehow.

I confirm that I saw no way to model it, it could simply become a
specialized form of relation.

> - about the script: yes, there are some default values, but they are easily
>   identified hence this is not really a problem changing these (I noticed
>   you really want to have your primary key as class properties :)

We could load customs defaults from a .dd2pmrc or allow a command line
argument.  As I said in a former email, we need to cross a XML-RPC
bridge and the only way I see to keep objects identity is through
primary key.  I found the snapshot() method quite useful

>   I was really surprised by the way relations are mapped to
>   relationships. If I'm not wrong, you're mapping them to 'to-one'
>   relationship, unconditionally. However, when I tried the tool, I came to
>   the opposite opinion, because of the 2nd screenshot:
>   http://www.danny.cz/screenshots/datadesigner1.jpg
>
>   Can you explain your practices wrt the relations? Do you design to
>   relations, one being the inverse of the other, when you design a model
>   with the db-designer, and edit the xml file afterwards to change one of
>   these from to-one to to-many?

To date I only used to-one but to-many can be modeled and should not
pose any problems to convert.

> I do not fully understand what are for: parent's update policy, as
> well as child's insert & update policy. If someone could bring in a
> light...

I can't...

>   - on db-designer 0.4, there's a 'default' for attribute that can be used
>     for defaultValue --to be added to mutateEntAttrType(). 'int' and
>     'integer' expected by the scripts seem to have disappeared and replaced
>     by int2, int4 and int8 (same for float -> float4 & float8) -- to be
>     updated in DD_PM_TYPE_MAP

No Problem.

>   - 'class ModelingDDImporter:' simply not making it derive from 'object'
>     makes the script available for py2.1 as well.

No Problem either.

When is Zope going to support Python 2.2 ?

>   - defaults in mutateEntity() & mutateEntAttr() are directly defined as
>     dictionaries, while 'mutateRelation()' uses makeDefaults()

I got tired of quoting dict keys so I used a quick hack I saw in the
Python Cookbook.  I agree that we should use only one way to do it all
across the script, which one do you find cleaner ?

I prefer makeDefaults()

> Any comments from your experience will be helpful to fully appreciate the
> db-designer. Some features could be added easily (such as a checkbox for
> isClassProperty in attributes and relations), other would be more difficult
> (such as a support for inheritance). Are you by any chance in contact with
> the maintainers?

There are some part of the interface that are a bit annoying.  You
can't edit an attribute by double-clicking on it from the entity-tree
and minor stuff like that.  A really nice feature is that you can put
remarks everywhere.

>
>   Another tool that may be useful is PyDBDesigner at
>   http://pydbdesigner.sourceforge.net/   (thank you Soaf for the link)
>
>   It's python-based, only at an alpha stage for the moment but definitely
>   worth a try. I currently do not have any time to study both of these
> tools in details but if anyone feels like doing it I'll be interested in
> their comments. Having a stand-alone modeler (i.e. out of zope) is a real
> concern as far as I'm concerned and since time is a concern as well |: I
> would for sure prefer using or may be forking an existing project, if
> possible, rather than coding it from scratch.

Ehhh... this tool is far from being mature...

By the way how do you achieve this funny indentation ?

> Last, just curiosity: was the zmodeler not an option for you because of
> zope itself or do you find/use more of the db-designer xml file that the
> zmodeler (and/or the modeling framework's model xml file) does not offer?

Zope was not a problem since we already had a Zope up and running but
I'm not sure if we would have installed one just to test the
framework.  The main problem with the ZModeler is the lack of support
for various comments but also the lack of visual presentation of the
model.

We tried data-designer and it happened to be really easy to import the
schema into a model, easier that fixing the ZModeler so we opted for
an external editor.

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE+YMterhy5Fqn/MRARAvNEAJ9OhsDDVR0J9k87S3iSCyfJzuECUwCfYaRD
mjgxDd0aH3KMZr/XAuv1shQ=
=iW8w
-----END PGP SIGNATURE-----

[Modeling-users] mdl_generate_DB_schema: table creation problem

[Mario R. <ma...@ru...>]

Hi,

I am feeding a simple (and with only one table) model to  
mdl_generate_DB_schema,
that validates and generates python code OK. It fails with the log  
below.
This is running on OSX, with postgres 7.3, pypgsql 2.3, osx 10.2.4 (new  
update,
that has broken some other things...), and modeling 0.9-pre-2.
Any ideas?

Cheers, mario


% mdl_generate_DB_schema.py -T cwText.xml
Couldn't evaluate expression CREATE TABLE Text (
   fr VARCHAR(400) ,
   en VARCHAR(400) ,
   nl VARCHAR(400) ,
   name VARCHAR(40) NOT NULL,
   pt VARCHAR(400) ,
   de VARCHAR(400) ,
   id INTEGER(20) NOT NULL,
   es VARCHAR(400) ,
   it VARCHAR(400) ). Reason: libpq.OperationalError:ERROR:  parser:  
parse error at or near "(" at character 168

FAILED to execute: CREATE TABLE Text (
   fr VARCHAR(400) ,
   en VARCHAR(400) ,
   nl VARCHAR(400) ,
   name VARCHAR(40) NOT NULL,
   pt VARCHAR(400) ,
   de VARCHAR(400) ,
   id INTEGER(20) NOT NULL,
   es VARCHAR(400) ,
   it VARCHAR(400) )
Reason:
Modeling.Adaptor.GeneralAdaptorException
Couldn't evaluate expression CREATE TABLE Text (
   fr VARCHAR(400) ,
   en VARCHAR(400) ,
   nl VARCHAR(400) ,
   name VARCHAR(40) NOT NULL,
   pt VARCHAR(400) ,
   de VARCHAR(400) ,
   id INTEGER(20) NOT NULL,
   es VARCHAR(400) ,
   it VARCHAR(400) ). Reason: libpq.OperationalError:ERROR:  parser:  
parse error at or near "(" at character 168

   File "/usr/bin/mdl_generate_DB_schema.py", line 105, in  
databaseSchemaWithOptions
     channel.evaluateExpression(sqlExpr)
   File  
"/usr/lib/python2.2/site-packages/Modeling/DatabaseAdaptors/ 
AbstractDBAPI2AdaptorLayer/AbstractDBAPI2AdaptorChannel.py", line 164,  
in evaluateExpression
     raise GeneralAdaptorException, msg

Re: [Modeling-users] Limit

[Sebastien B. <sbi...@us...>]

        Hi,

Soaf wrote:
> >   More on this: if someone knows some standard SQL for doing this, =
I'm
> >   interested, it would for sure make the release-time for such a
> >   feature, if requested, closer than it actually is.
>=20
> After some googling, i found nothing. In fact 'LIMIT' is a SQL92 keyw=
ord,
> but i found nowhere the SQL92 def of this :( , anyways i use it on a
> couple of DB before and i 've never see something that doesn't suppor=
t it.


NB: Postgresql documentation says that neither LIMIT nor OFFSET are par=
t of
    SQL92, and that there is no ANSI definition for these features.

By the way, I finally searched to find out how this could be done, since
this request shows up persistently from time to time --I guess the idea=
 is
to make it possible to present different pages in batches in a GUI.

* Postgresql has  LIMIT # [OFFSET #]
http://www.postgresql.org/docs/view.php?version=3D7.3&idoc=3D1&file=3Ds=
ql-select.html

  requiring that ORDER BY is set if you want consistent batches between=
 two
  selects

* MySQL offers the very same syntax
  http://www.mysql.com/doc/en/SELECT.html

  even if it's not clear to me if an ORDER BY clause is also required f=
or
  consistency between consecutive queries, but a few tries will show (I=
'll
  appreciate if someone with a mysql server at hand could try and repor=
t)

* Oracle defines the pseudo-column ROWNUM

http://forums.devshed.com/archive/5/2002/11/3/47157
http://www.orafaq.net/glossary/faqglosr.htm
http://gethelp.devx.com/techtips/oracle_pro/10min/10min1200/10min1200-3=
.asp

  but it has a different semantics, since the pseudo-column is filled in
  *before* any orderings --the last url gives an easy solution for solv=
ing
  this.

[and there are some other comments at
 http://www.experts-exchange.com/Databases/Q_20307165.html]


So it should be possible to modify the FetchSpecification API:

  instead of (unimplemented) setFetchLimit(self, limit) we could have
  setFetchLimit(self, limit, offset=3D0). Then you'll be on your own to
  iterate on the result sets given that you previously queried the sele=
ct
  count.

Mario answered:
> [...] would be really nice to have a generic declarative way for this
> (essentially cursors, or paging, but as far the client app code is
> concerned, there should be no worries about maintaining state
> information).

  Yes it would be nice, and not very hard to code. However I can't see =
any
  object in the framework where it could make sense to add such a
  feature. It would be the responsability for an object on top of an
  EditingContext, but there's none for the moment being.


Back to the fetchLimit(): what do you all think about the proposal?

Implementing it for Postgresql and MySQL wouldn't be that hard, except =
that
it will require support for SortOrderings on the SQL side --not done ye=
t.


-- S=E9bastien.

Re: About Database Schema Designer [was: Re: [Modeling-users] Pythonic, and non-XML, Model description]

[Sebastien B. <sbi...@us...>]

Replying to my message:
>   I was really surprised by the way relations are mapped to
>   relationships. If I'm not wrong, you're mapping them to 'to-one'
>   relationship, unconditionally. However, when I tried the tool, I ca=
me to
>   the opposite opinion, because of the 2nd screenshot:
>   http://www.danny.cz/screenshots/datadesigner1.jpg
>=20
>   Given that "firmy" has an attribute "forma_id" which is also the PK=
 of
>   entity "forma" and that the former entity has a relation to the lat=
ter, my
>   first conclusion was that the relations were intended to model to-m=
any
>   relationships... But czech is greek to me ;) so this is nothing mor=
e than
>   an intuition.

Did I really write that ?-: Now I'm surprised again... Looks like my br=
ain
was in stanby-mode for this one.  Just forget the stupid remark: my fir=
st
conclusion should have been "relations model to-one rels", just as expe=
cted
given the DB-schema described here.


-- S=E9bastien.

Re: [Modeling-users] Pythonic, and non-XML, Model description

[Mario R. <ma...@ru...>]

Hello,

thanks for the detailed feedback, and I like most of it very much.
I comment on each point (needing comment) below...

...

>> The behaviour in general would stay the same as the XML way of
>> doing things, namely form this model you would generate the classes
>> and the db schemas.
>
> These features are available for any Modeling.Model object, so the 
> "only"
> thing to do is build it from the py-model, just like it goes for the 
> xml.
>
> (same for generating an xml-model, thus mapping a py-model to an 
> xml-model
>  will be straightforward --cf. ModelSet.getXMLDOMForModelNamed() and
>  getXMLStreamForModelNamed())

Yes, should be the same idea, except that there'd be no need to have 
methods
with "forModelNamed" in their name, as this would be implied by the 
object on
which the method is called:

modelset.getXML()
modelset.getXMLDOM()
model.getXML()
entity.getXML()
...

The ModelSet concept is not yet included in the PyModel description 
(mostly
becuase it is not fully clear to me how it is used). How would you 
extend this
description to include it?

>> Some differences are choice of default values, and that key and 
>> locking
>> attributes are defined on attributes and relations rather than 
>> entities
>> ;)
>
> Ok, I guess we won't argue any further on this by now ;)

But, see how it makes things easier! You declare a key on an att, you 
use
the att, defining also teh key on the using entity -- makes things 
consistent,
and less wordy. If you want that an id in one entity is a key but not 
in another,
then you are forced to define 2 different id 'types', which helps make 
the
difference obvious to anyone looking in.

>> As an exercise, I have taken the sample model in the distribution:
>> testPackages/StoreEmployees/model_StoreEmployees.xml and re-expressed 
>> it
>> in this pythonic way.
>
> Thanks, this made things even clearer.
>>
>> Apart from readability and losing of some verbosity, other gains are
>> "executability" and dynamicity -- it could be very easy to modify a 
>> model
>> at runtime, if you would ever want to do that.
>
> Oh oh, I can't believe you said that! Models' mutability at runtime is 
> an
> other subject we'd better discuss apart from this thread. It *is* 
> possible,
> under some conditions I certainly need to gather before answering. If 
> you
> need this sort of feature in short-term, you'd better picked me on 
> wildly!!

In general this is not what anyone woudl want to do, and I was trying 
to think
of times when this would be useful. Can't really, but i *suspect* it 
might come
in handy. But, in fact, imagine some application that is stocking lots 
of
incoming unknown data into a db... let's say the data is XML, but each 
XML
data-schema is unknown a priori. You can create a py model to 
correspond,
at runtime (but saved for subsequent accessing), create the tables, and
stock the unknown XML in the db directly as real db tables....

>>  Also, given that a model now becomes a module in and of itself, it 
>> gains
>> from what modules have to offer, such as self-tests.
>
> Ok.
>
>> And, speed of course, as i see no reason why this model object would 
>> also
>> not serve as the model object in memory at runtime (but that's for
>> Sebastien to confirm) -- thus loading the model is equivalent to "from
>> MyModel import model".
>
> Well, it will need a few more lines of code to actually load the model 
> into
> the so-called defaultModelSet (cf. Modeling.ModelSet and the generated
> __init__.py), but that's it.

Yes, but i was leaving all that to you ;)
But, details of how such a py model would integrate with the rest of
the framework are all to be decided, and please point out any problems
that only you can foresee!

>> Can you please take a look at the 2 linked files below (as temporary
>> URLs, as to attach they are too big);
>> - PyModel.py -- defines the classes (signatures for) for, and
>> documents the rules for, a PyModel instance
>> - sample_PyModel.py -- re-expresses the StoreEmployees model
>
> Ok, now let's take a look at the big stuff ;)
>
> Your python code is sound and clear. My first comments are:

A general comment to the comments below -- in general i do not like
'special format strings' as values to things. This adds possible errors,
requires special documentation, makes checking more difficult, and
in general is less pythonic... I think keeping the info units separate
would be easier to work with, and clearer. The overhead of having
to specify the same parameters many times over can be reduced
by standard defaults and the possibility to set defaults only for
this model (see new sample...)

>   - maybe we'll gain more readability if relationships' multiplicity 
> lower-
>     and upper bounds where encoded in strings, such as: '0-1', '1-1', 
> '0-*'
>     or '2-16'.

Yes, but not as strings. What is wrong with a python list?
multiplicity = [0,1]
multiplicity = [1,-1]

>   - Same for external type's width, precision and scale, such as in:
>     'NUMERIC(12,2)' or 'VARCHAR(200)'

Same thing here. This would also make it difficult to provide defaults
separately for dbtype, and for width or precision.
So, I would vote to keep them separate (and tyo change the name of
externalType to dbType). I think it will be easier to work with,
and clearer for everyone.

>   - Again, same for an entity's parent which could be specified along 
> with
>     the entity's name: 'Executive(Employee)'

Same. I vote to keep separate, and to rename parentEntity to isAlso, 
e.g.
Entity('Executive', isAlso='Employee', ...

>   - What about the possibility to add a '*' to an attribute's name 
> when it's
>     required? (may be same for a relationship, equivalent to lower 
> bound==1)

Same for adding '*' to att name, but definately yes to have constraints 
forced
down as a result from relations, e.g. if a one-to-one relation is 
required, then
the related attributes must also be required -- but this is automatic 
and taken
into account by the validation.

>   - Allow litterals instead of the equivalent integers in the xml: 
> 'CASCADE'
>     for the delete rule is more explicit than int(2)!

Definately. Also, in lowercase! (Hate being screamed at, which is what
uppercase seems to be always doing  \-)

>   - I think I would have made 'name' instead 'columnName' the default 
> for
>     displayLabel :)

Yes, it is probably better.

> Now that I get used to the idea of a python-model, I'm also thinking 
> of some
> extents to your proposal:
>
>   - We could have subclasses for your Attribute: PrimaryKey, ForeignKey
>     (defaults for both would be: int, not class property, etc.), String
>     (with a default external size/width), Integer, Numeric, ... You 
> get the
>     idea.

Great! There could be some standard sub-classes, but a user is
ofcourse allowed to make his own. What about the naming scheme propoes
by the example below, that Att subs start with the letter 'A' and rel 
subs start
with the letter 'R' ? (to avoid unnecessarily long names)

>   - Relation can also be sub-classed to 'ToOne' and 'ToMany' (default 
> would
>     be '0-*' for the latter)

Yes. Also, all defaults may be set for a specific model as necessary...

> Last, I'm thinking of some automatic processing which is already coded 
> in
> the zmodeler and that could be done at model-time to reduce verbosity 
> in a
> significant manner:
>
>   - have a primary key 'id' automatically declared if not set,

In general "magical" behaviour is more trouble than gain... how about if
we have the possibility to define a default attribute on an Entity 
description class?

>   - have foreign keys automatically set for relationships, using the 
> same
>     defaults the zmodeler already uses (e.g. FKEmployeeId for a to-many
>     relationship pointing to the entity Employee). It would need some
>     additional checks but it's definitely possible.

Again, I like simplifying the management of all this, but i do not like 
being
forced to accept decisions imposed by the framework, such as the names 
of
my columns (what if I want to provide a model for an existing db?)
So, providing a default scheme, that the user may redefine (or
not use at all), seems more reasonable to me. See the defaults for 
sourceAttribute
and destinationAttribute in RToOne and RToMany... Also, see the explicit
declaration of the foreign keys (with the possibility of automatic the 
name
for it).

> This could be as handy as the feature you submitted in your proposal:
> automatic definition parent properties which are not overriden in
> sub-entities.

Yes. Auto generation of foreign keys from a relationship definition does
make sense in fact... but the possibility to declare them explicitly 
(with
the desired values for their attributes) should always be there. Since
such explicit declaration can be very simple, I prefer to leave it as
must be explicit.

> I wrote all these items with something in mind, actually. You know, I'm
> basically lazing, most of the time I do not take about DB-Schemas 
> details,
> and I will be delighted if it was possible to write the same 
> StoreEmployee
> model really simply. This is a good illustration of the automatic 
> processing
> I described above because this model (like every test models) were 
> designed
> using the zmodeler and its functionalities.
>
>   In fact, I'm thinking that something simple like that could be 
> written:
> (this would imply some changes to your proposed API, such as making
>  'destinationEntity' the second argument for the Relationship's 
> initializer).

Ah, very good -- destinationEntity must in fact be specified everytime, 
so
this is better this way.

I have taken your simplified model, and evolved it to correspond to my
comments above. Correspondingly, i have also updated the PyModel
module to indicate how it would change to support these changes.
I have put the two files (PyModel_3_mr.py and sample_ PyModel_3_mr.py) 
at:
http://ruggier.dyndns.org:8080/PyModel/
But for convenience, I am also pasting below the sample model...

Oh, and some other changes I have not mentioned here are listed in
the top of PyModel_3_mr.py, but mostly name changes, and having
entities have only one list, that mixes attributes and relations... 
(your opinion?).

> Of course this would need some additional cpu-time to load a model, 
> but I
> bet it would be far quicker than parsing the xml!
>
>   This of course still needs to be discussed and refined. I did not 
> have any
> time to try & implement my proposal, but I guess that at some point of 
> the
> discussion we'll need to see the words take shape --at this point this 
> could
> naturally be made in a dev-branch if several of us are on this.

OK.

>   I really have the feeling that we will succeed in designing a very 
> nice
>   python model. Let's go for it now that Mario brought some light on 
> the
>   path!

That would not be XPath, would it ;-?

Cheers, mario

ps: sample_ PyModel_3_mr.py
'''
A sample Pythonic OO-RDB Model
(re-expressing testPackages/StoreEmployees/model_StoreEmployees.xml) --
A PyModel must define a global variable called 'model', that is of type 
Model
'''

from PyModel import *

##
# Set preferred defaults for this model (when different from
# standard defaults, or if we want to make things explicit)

AInteger.defaults['precision'] = 10
AString.defaults['width'] = 20

RToOne.defaults['delete'] = 'cascade'
RToOne.defaults['multiplicity'] = [0,1]
RToOne.defaults['sourceAttribute'] = RToOne.attNameFromRel # 
'fk'+destEnt+destAtt+count
RToOne.defaults['destinationAttribute'] = 'id'

RToMany.defaults['delete'] = 'deny'
RToMany.defaults['multiplicity'] = [0,*]
RToMany.defaults['sourceAttribute'] = 'id'
RToMany.defaults['destinationAttribute'] = RToMany.attNameFromRel # 
fk+destEnt+sourceAtt+count

# Note that Relation.attNameFromRel is a callable, that calculates the 
att name
# from the indicated pieces (where count to distinguish between 
multiple relations
# between same source and target

Entity.defaults['attributes'] = [
     AInteger('id', key=1, isClassProperty=0, isRequired=1)
]

##

_connDict = {}
model = Model('StoreEmployees',connDict=_connDict)
model.entities = [

     #
     Entity('Store',
         attributes=[
             AString('corporateName', isRequired=1),
             RToMany('employees', 'Employee')
         ]
     )

     # Employee and its subclasses SalesClerk and Executive
     Entity('Employee',
         attributes=[
             AString('lastName', isRequired=1, usedForLocking=1),
             AString('firstName', isRequired=1, width=50, 
usedForLocking=1),
             AForeignKey(Attribute.fk('Store'), isClassProperty=0),
             RToMany('toAddresses', 'Address', delete='cascade'),
             RToOne('toStore', 'Store')
         ]
     )

     Entity('SalesClerk', isAlso='Employee',
         attributes=[
             AString('storeArea')
         ]
     ),

     Entity('Executive', isAlso='Employee',
         attributes=[
             AString('officeLocation', width=5),
             RToMany('marks', 'Mark', delete='cascade')
         ]
     ),

     #
     Entity('Address',
         attributes=[
             AString('street', width=80),
             AString('zipCode'),
             AString('town', width=80),
             AForeignKey(Attribute.fk('Employee'), isClassProperty=1),
             RToMany('toEmployee', 'Employee', delete='deny')
         ]
     ),

     #
     Entity('Mark',
         attributes=[
             AInteger('month', isRequired=1),
             AInteger('mark', isRequired=1),
             AForeignKey(Attribute.fk('Executive'), isClassProperty=1),
             RToOne('executive', 'Executive' ]
         ]
     )

]

if __name__ == '__main__':
     print model.validate()
     print model.toXML()
     # plus whatever ...

##

Re: [Modeling-users] Pythonic, and non-XML, Model description

[Sebastien B. <sbi...@us...>]

        Hi Mario,

> following comments about the XML model (on and off the list) and
> how it can be improved or redone altogether, i started thinking about=
=20
> it...
> And (to be able to get it out of my mind and get to some other work ;=
-)
> here is a proposal for a pure python description of the model. It=20
> follows
> the XML closely, and in fact may always retain compatibility (at least
> generation of XML from the python model, which will be trivial).

I agree: the pymodel must have the same power of expression than the
existing xml-file, i.e. every properties for Entities, Attributes
etc. should be settable in the py-model just like they are in the xml.=
=20=20
Your proposal holds the complete kick-off material.

> The behaviour in general would stay the same as the XML way of
> doing things, namely form this model you would generate the classes
> and the db schemas.

These features are available for any Modeling.Model object, so the "onl=
y"
thing to do is build it from the py-model, just like it goes for the xm=
l.

(same for generating an xml-model, thus mapping a py-model to an xml-mo=
del
 will be straightforward --cf. ModelSet.getXMLDOMForModelNamed() and
 getXMLStreamForModelNamed())

> Some differences are choice of default values, and that key and locki=
ng
> attributes are defined on attributes and relations rather than entiti=
es=20
> ;)

Ok, I guess we won't argue any further on this by now ;)

> As an exercise, I have taken the sample model in the distribution:
> testPackages/StoreEmployees/model_StoreEmployees.xml and re-expressed=
 it
> in this pythonic way.

Thanks, this made things even clearer.

> Apart from readability and losing of some verbosity, other gains are
> "executability" and dynamicity -- it could be very easy to modify a m=
odel
> at runtime, if you would ever want to do that.

Oh oh, I can't believe you said that! Models' mutability at runtime is =
an
other subject we'd better discuss apart from this thread. It *is* possi=
ble,
under some conditions I certainly need to gather before answering. If y=
ou
need this sort of feature in short-term, you'd better picked me on wild=
ly!!

>  Also, given that a model now becomes a module in and of itself, it g=
ains
> from what modules have to offer, such as self-tests.

Ok.

> And, speed of course, as i see no reason why this model object would =
also
> not serve as the model object in memory at runtime (but that's for
> Sebastien to confirm) -- thus loading the model is equivalent to "from
> MyModel import model".

Well, it will need a few more lines of code to actually load the model =
into
the so-called defaultModelSet (cf. Modeling.ModelSet and the generated
__init__.py), but that's it.

> Can you please take a look at the 2 linked files below (as temporary
> URLs, as to attach they are too big);
> - PyModel.py -- defines the classes (signatures for) for, and
> documents the rules for, a PyModel instance
> - sample_PyModel.py -- re-expresses the StoreEmployees model

Ok, now let's take a look at the big stuff ;)

Your python code is sound and clear. My first comments are:

  - maybe we'll gain more readability if relationships' multiplicity lo=
wer-
    and upper bounds where encoded in strings, such as: '0-1', '1-1', '=
0-*'
    or '2-16'.
=20=20
  - Same for external type's width, precision and scale, such as in:
    'NUMERIC(12,2)' or 'VARCHAR(200)'

  - Again, same for an entity's parent which could be specified along w=
ith
    the entity's name: 'Executive(Employee)'

  - What about the possibility to add a '*' to an attribute's name when=
 it's
    required? (may be same for a relationship, equivalent to lower boun=
d=3D=3D1)

  - Allow litterals instead of the equivalent integers in the xml: 'CAS=
CADE'
    for the delete rule is more explicit than int(2)!

  - I think I would have made 'name' instead 'columnName' the default f=
or
    displayLabel :)

Now that I get used to the idea of a python-model, I'm also thinking of=
 some
extents to your proposal:

  - We could have subclasses for your Attribute: PrimaryKey, ForeignKey
    (defaults for both would be: int, not class property, etc.), String
    (with a default external size/width), Integer, Numeric, ... You get=
 the
    idea.

  - Relation can also be sub-classed to 'ToOne' and 'ToMany' (default w=
ould
    be '0-*' for the latter)


Last, I'm thinking of some automatic processing which is already coded =
in
the zmodeler and that could be done at model-time to reduce verbosity i=
n a
significant manner:

  - have a primary key 'id' automatically declared if not set,

  - have foreign keys automatically set for relationships, using the sa=
me
    defaults the zmodeler already uses (e.g. FKEmployeeId for a to-many
    relationship pointing to the entity Employee). It would need some
    additional checks but it's definitely possible.

This could be as handy as the feature you submitted in your proposal:
automatic definition parent properties which are not overriden in
sub-entities.

I wrote all these items with something in mind, actually. You know, I'm
basically lazing, most of the time I do not take about DB-Schemas detai=
ls,
and I will be delighted if it was possible to write the same StoreEmplo=
yee
model really simply. This is a good illustration of the automatic proce=
ssing
I described above because this model (like every test models) were desi=
gned
using the zmodeler and its functionalities.

  In fact, I'm thinking that something simple like that could be writte=
n:
(this would imply some changes to your proposed API, such as making
 'destinationEntity' the second argument for the Relationship's initial=
izer).

-----------------------------------------------------------------------=
-----
from PyModel import Attribute,Relation,Entity,Model
_connDict =3D {}
model =3D Model('StoreEmployees',connDict=3D_connDict)
model.entities =3D [

    Entity('Store',
      attributes=3D[ String('corporateName *', width=3D30) ],
      relations=3D[  ToMany('employees', 'Employee', delete=3D'deny') ])

    # Employee and its subclasses SalesClerk and Executive

    Entity('Employee',
      attributes=3D[  String('lastName *', width=3D20, isUsedForLocking=
=3D1 ],=20
                    String['firstName *', width=3D50, isUsedForLocking=
=3D1 ], ],
      relations=3D[   ToMany('toAddresses', 'Address', delete=3D'cascad=
e' ],=20
                    ToOne('toStore', 'Store')  ])

    Entity('SalesClerk(Employee)',=20
      attributes=3D[ String('storeArea', width=3D20) ]),

    Entity('Executive(Employee)',
                                          ## width declared in external=
Type
      attributes=3D[  String('officeLocation', externalType=3D'VARCHAR(=
5)') ],
      relations=3D[   ToMany('marks', 'Mark', delete=3D'cascade' ]),

    #
    Entity('Address',
      attributes=3D[ String('street', width=3D80),=20
                   String('zipCode', width=3D80),=20
                   String('town', width=3D80), ],
      relations=3D[ ToMany('toEmployee', 'Employee', delete=3D'deny') ]=
),

    #
    Entity('Mark',
      attributes=3D[ Integer('month *'),=20
                   Integer('mark *'), ],
      relations=3D[  ToOne('executive', 'Executive' ]  ])

]

 if __name__ =3D=3D '__main__':
    print model.validate()
    print model.toXML()
-----------------------------------------------------------------------=
-----


  (this example was written with the intention of being strictly equiva=
lent
   to the original one)=20

Of course this would need some additional cpu-time to load a model, but=
 I
bet it would be far quicker than parsing the xml!

  This of course still needs to be discussed and refined. I did not hav=
e any
time to try & implement my proposal, but I guess that at some point of =
the
discussion we'll need to see the words take shape --at this point this =
could
naturally be made in a dev-branch if several of us are on this.

  I really have the feeling that we will succeed in designing a very ni=
ce
  python model. Let's go for it now that Mario brought some light on the
  path!

        Cheers,

-- S=E9bastien.

About Database Schema Designer [was: Re: [Modeling-users] Pythonic, and non-XML, Model description]

[Sebastien B. <sbi...@us...>]

        Hi Yannick,

  Thanks for the script. I've not played a lot with db designer, but re=
ad
  your code with a db-designer model at hand and converted a simple
  model. That inspired me the following comments:

- about the tool itself: it is really focused on db-schemas ; it has so=
me
  features I cannot understand, such as the 'type' of a relation
  (identifying, identifying or informational), or the referential integ=
rity
  properties for the child/destinationEntity (insert/update props). May=
 be
  you can comment on these?

  The most annoying problem I see is the complete lack of inheritance --
  there seem to be no way to model it. Okay this is quite normal for a =
tool
  dedicated to db-schema, but we'll need it somehow.

- about the script: yes, there are some default values, but they are ea=
sily
  identified hence this is not really a problem changing these (I notic=
ed
  you really want to have your primary key as class properties :)

  I was really surprised by the way relations are mapped to
  relationships. If I'm not wrong, you're mapping them to 'to-one'
  relationship, unconditionally. However, when I tried the tool, I came=
 to
  the opposite opinion, because of the 2nd screenshot:
  http://www.danny.cz/screenshots/datadesigner1.jpg

  Given that "firmy" has an attribute "forma_id" which is also the PK of
  entity "forma" and that the former entity has a relation to the latte=
r, my
  first conclusion was that the relations were intended to model to-many
  relationships... But czech is greek to me ;) so this is nothing more =
than
  an intuition.

  Can you explain your practices wrt the relations? Do you design to
  relations, one being the inverse of the other, when you design a model
  with the db-designer, and edit the xml file afterwards to change one =
of
  these from to-one to to-many?

- Unused stuff in relations' props that might be used when mapping a mo=
del:

    - Parent (source entity) & Child (destination entity) props:
       mandatory/non-mandatory: this can be used to set the lower
       multiplicity bound to 1 or 0, respectively.

    - In 'Referential Integrity': it seems that parent's delete policies
      'none', 'restrict', 'cascade' & 'set NULL' maps to DELETE_NOACTIO=
N,
      DELETE_DENY, DELETE_CASCADE and DELETE_NULLIFY, respectively.

      I do not fully understand what are for: parent's update policy, as
      well as child's insert & update policy. If someone could bring in=
 a
      light...


- misc.:

  - on db-designer 0.4, there's a 'default' for attribute that can be u=
sed
    for defaultValue --to be added to mutateEntAttrType(). 'int' and
    'integer' expected by the scripts seem to have disappeared and repl=
aced
    by int2, int4 and int8 (same for float -> float4 & float8) -- to be
    updated in DD_PM_TYPE_MAP

  - 'class ModelingDDImporter:' simply not making it derive from 'objec=
t'
    makes the script available for py2.1 as well.

  - defaults in mutateEntity() & mutateEntAttr() are directly defined as
    dictionaries, while 'mutateRelation()' uses makeDefaults()


Any comments from your experience will be helpful to fully appreciate t=
he
db-designer. Some features could be added easily (such as a checkbox for
isClassProperty in attributes and relations), other would be more diffi=
cult
(such as a support for inheritance). Are you by any chance in contact w=
ith
the maintainers?


  Another tool that may be useful is PyDBDesigner at
  http://pydbdesigner.sourceforge.net/   (thank you Soaf for the link)

  It's python-based, only at an alpha stage for the moment but definite=
ly
  worth a try. I currently do not have any time to study both of these =
tools
  in details but if anyone feels like doing it I'll be interested in th=
eir
  comments. Having a stand-alone modeler (i.e. out of zope) is a real
  concern as far as I'm concerned and since time is a concern as well |=
: I
  would for sure prefer using or may be forking an existing project, if
  possible, rather than coding it from scratch.

Last, just curiosity: was the zmodeler not an option for you because of=
 zope
itself or do you find/use more of the db-designer xml file that the zmo=
deler
(and/or the modeling framework's model xml file) does not offer?

           Cheers,

-- S=E9bastien.


Yannick Gingras <yan...@sa...> wrote:
> On Wednesday 26 February 2003 03:28 pm, Sebastien Bigaret wrote:
> > Would you mind sharing your conversion scripts from db-designer to
> > xml-modeling? I'm thinking of adding a few notes on this tool in the
> > framework's documentation, and having the conversion script would be
> > really handy.
>=20
> The script is really sketchy at the moment with a lot of default
> values hard-coded but it would not be hard to modularize it enough to
> make it useful for the general public.
>=20
> I let you look at it, tell me how we could make it more general and
> I'll see what we can do.
>=20
> Fell free to ask questions about the implementation.
>=20
> --=20
> Yannick Gingras
> Byte Gardener, Savoir-faire Linux inc.
> (514) 276-5468