modeling-users — For users: questions, support, announcements, etc.

Re: [Modeling-users] Reply-to header

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 23, 2003 02:53 pm, Jerome Kerdreux wrote:
> Hum It's work fine except that after I post something
> in a thread the default reply to will be the list not the
> user . So it works for me but this will break the default
> policy .

Noop, your reply-to is kept to yourself, Mailman probably restore
it...

KMail has an insant filter for this : right-click on a message from the
mailing list -> create folter on mailing list -> action -> set Reply-To
Header -> modeling-users...

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HuNPrhy5Fqn/MRARApD8AJ42wHEKltN6wJdK+IVEix7L1yPtfwCeM0JR
Gj/XN9AWeaIYlI7/whI6haQ=
=RUtY
-----END PGP SIGNATURE-----

Re: [Modeling-users] Reply-to header

[Jerome K. <Jer...@fi...>]

Jerome Kerdreux wrote:

> Sebastien Bigaret wrote:
>
>> After a few more clicks, I've got a little more on reply-to munging:
>> from an original post at:
>> http://info.ccone.at/INFO/Mail-Archives/redhat/Apr-2002/msg01860.html
>>
>> I'm pretty sure that something like this should work (untested here
>> though, so if it works for you, please report):
>>
>> ------------------------------------------------------------------------
>> # modeling-users list
>> :0
>> * ^List-Id:.*modeling-users.lists.sourceforge.net
>>  {
>>    :0f
>>    | formail -bfi 'Reply-To: modeling-users 
>> <mod...@li...>'
>>  }
>> ------------------------------------------------------------------------
>>
>>
>
> So this is test for the procmail ..
>
Hum It's work fine except that after I post something
in a thread the default reply to will be the list not the
user . So it works for me but this will break the default
policy .

Re: [Modeling-users] Reply-to header

[Jerome K. <Jer...@fi...>]

Sebastien Bigaret wrote:

>After a few more clicks, I've got a little more on reply-to munging:
>from an original post at:
>http://info.ccone.at/INFO/Mail-Archives/redhat/Apr-2002/msg01860.html
>
>I'm pretty sure that something like this should work (untested here
>though, so if it works for you, please report):
>
>------------------------------------------------------------------------
># modeling-users list
>:0
>* ^List-Id:.*modeling-users.lists.sourceforge.net
>  {
>    :0f
>    | formail -bfi 'Reply-To: modeling-users <mod...@li...urc=
eforge.net>'
>  }
>------------------------------------------------------------------------
>
>(I guess -i is better than -A, sionce, according to 'man formail', -i
> preserves the original field, if any, by prepending 'Old-' to the name)
>
>  Be sure you've read
>  https://sourceforge.net/docman/display_doc.php?docid=3D6693&group_id=3D=
1
>  and the links it holds before making up your mind ;)
>
>-- S=E9bastien.
>
> =20
>

So this is test for the procmail ..

Re: [Modeling-users] Reply-to header

[Sebastien B. <sbi...@us...>]

After a few more clicks, I've got a little more on reply-to munging:
from an original post at:
http://info.ccone.at/INFO/Mail-Archives/redhat/Apr-2002/msg01860.html

I'm pretty sure that something like this should work (untested here
though, so if it works for you, please report):

------------------------------------------------------------------------
# modeling-users list
:0
* ^List-Id:.*modeling-users.lists.sourceforge.net
  {
    :0f
    | formail -bfi 'Reply-To: modeling-users <mod...@li...urcefo=
rge.net>'
  }
------------------------------------------------------------------------

(I guess -i is better than -A, sionce, according to 'man formail', -i
 preserves the original field, if any, by prepending 'Old-' to the name)

  Be sure you've read
  https://sourceforge.net/docman/display_doc.php?docid=3D6693&group_id=3D1
  and the links it holds before making up your mind ;)

-- S=E9bastien.

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Okay, so:

  1. when you're saving the db into xml, you want to avoid saving objects in
     relations (I presume that's why 'i18n' e.g. is hard-coded),

  2. when restoring, you have to find back the objects in relations.

About 1: I saw you're using CustomObject.snapshot(). Note that since
fetchRawRow() exists, you also have a method CustomObject.snapshot_raw()
at hand which returns the row corresponding to the object (and this one
is exactly the same one than the one you'd get with fetch and parameter
rawRows=3D1). So, as expected, you get the PK and the FKs defined for the
corresp. entity, *no matter* whether they are class properties or not.

  (And yes, it should be documented :)

  Now for restoring, I must be possible to do something like this:

  - initialize and populate each object, store them in a dictionary with
    something like=20

       rootEntityName=3Dentity.rootEntity().name() #this makes it easier
                                                 #if you use inheritance
       d[rootEntityName+'_%i'%pk]=3Dobject

  - then iterate on your objects and fill the relations, just as you do,
    except that you'll avoid the slow xpath query. Instead, you'll
    iterate on your i18ns and find the corresponding master object with
    a simple lookup in the dictionary, because the master will be:

      rootEntityName_master=3D...
      master=3Dd[rootEntityName_master+'_%i'%i18n._fk_master]

    given that the fk is stored in the i18n._fk_master

> So skiping PKs would be much more simple

  In fact, your problem here is not exactly to set a PK you generate
  yourself, but to match an object's FK w/ an other object's PK, which
  is not exactly the same pb. and cannot be easily solved.

> but having the framework
> doing the dump and restoration would be even beter.  I do the dump
> like this (phpMyAdmin is full Latin-1) :

  In fact, yes, this would be better, for example with serialization of
  EditingContext. Not done however...

Last, I don't know why you insist on dumping/restoring your db w/ the
framework, but that's not the most efficient way of doing this, is it?-)

Maybe you should consider dumping and restoring raw sql (postgresql can
dump a whole database in a text file). In this case, the only thing
you'll have to take care of wrt the framework is that sequences
supporting PK generation (you're using postgresql, right?) get the right
value before trying to insert new objects (the sequence should start at
max(id)+1 for a given table, or, if you're using inheritance, to the
max(id) of the different tables involved in an inheritance tree --there
is one and only one sequence used for generating the pks per root
entity).


-- S=E9bastien.


Yannick Gingras <yan...@sa...> wrote:
> Ehh humm like this :
>=20
>             exec ("global %s" % childClassName)
>             exec (importCmd % (childClassName, childClassName))
>=20=20=20=20=20=20=20=20=20
>             posElem =3D xpath.Evaluate(("./%s/text()" % fkStr), elem)[0]
>             curPos =3D int(posElem.nodeValue)
>=20
>             i18nPath =3D '/db_dump/%s[%s/text()=3D"%d"]' % ( childClassNa=
me,
>                                                          fkStr,
>                                                          curPos )
>             i18nElems =3D xpath.Evaluate(i18nPath, self.i18nDoc)
>=20=20=20=20=20=20=20=20=20=20=20=20=20
>             # create i18ns
>             for i18nElem in i18nElems:
>                 i18nObj =3D eval("%s()" % childClassName)
>                 self.fillObject(i18nObj, i18nElem)
>                 obj.addObjectToBothSidesOfRelationshipWithKey(i18nObj, 'i=
18n')
>                 self.ec.insertObject(i18nObj)
>             self.ec.insertObject(obj)
>=20
>=20
> I process the table dumps 2 by 2, one for the master, one for i18ns.
> I must admit that I don't know how I will restore the more complex
> relations.  The way I do it now fix the missing PKs problem because I
> find the matching ones in the i18n dump with i18nPath but it's a hack,
> a really slow hack.  I have to do a XPath query on a 1400 nodes file
> 700 time : 300 secs avg.
>=20
> So skiping PKs would be much more simple but having the framework
> doing the dump and restoration would be even beter.  I do the dump
> like this (phpMyAdmin is full Latin-1) :
>=20
> if __name__ =3D=3D "__main__":
>     if not len(sys.argv) =3D=3D 2:
>         print "USAGE : dump_xml.py <entity-name>"
>         sys.exit(1)
>=20
>     sys.path.append("..")
>=20=20=20=20=20
>     entityName =3D sys.argv[1]
>     ec =3D EditingContext()
>     mm =3D ModelManager(ec)
>=20
>     importCmd =3D "from Autogen.%s import %s" % (entityName, entityName)
>     exec(importCmd)
>=20
>     objs =3D mm.fetch(entityName)
>=20
>     # get the primary key name
>     key =3D ec.globalIDForObject(objs[0]).keyValues().keys()[0]
>=20=20=20=20=20
>     objs =3D map(lambda obj:mm.snapshot(obj), objs)
>=20
>     objs =3D sortById(objs, key)
>=20
>     domDoc =3D getDomDoc()
>     for obj in objs:
>         elem =3D domDoc.createElement(entityName)
>         for key in obj.keys():
>             tagName =3D externalNameForInternalName(key).lower()
>             property =3D domDoc.createElement(tagName)
>             if key !=3D "i18n":
>                 elem.appendChild(property)
>                 dataStr =3D toStr(obj[key])
>                 data =3D domDoc.createTextNode(dataStr)
>                 property.appendChild(data)
>         xpath.Evaluate("/db_dump", domDoc)[0].appendChild(elem)
>=20=20=20=20=20
>     PrettyPrint(domDoc)
>=20
>=20
> Yes, i18n is hardcoded... : \
>=20
> --=20
> Yannick Gingras
> Byte Gardener, Savoir-faire Linux inc.
> (514) 276-5468

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 23, 2003 10:09 am, Sebastien Bigaret wrote:
> > But if I have some of the records that have been
> > deleted, the PKs are not in a continuous sequence so FKs become out of
> > synch.
>
> Ok. But could you tell how you are doing this exactly? Are you
> building/inserting-in-ec new objects, then saving changes, or are you
> using some of the low-level functionalities to execute raw sql inserts,
> or something else? Asking this, because I'm trying to understand at
> which point exactly you need to provide the PK, where it is stored, etc.

Ehh humm like this :

            exec ("global %s" % childClassName)
            exec (importCmd % (childClassName, childClassName))

            posElem = xpath.Evaluate(("./%s/text()" % fkStr), elem)[0]
            curPos = int(posElem.nodeValue)

            i18nPath = '/db_dump/%s[%s/text()="%d"]' % ( childClassName,
                                                         fkStr,
                                                         curPos )
            i18nElems = xpath.Evaluate(i18nPath, self.i18nDoc)

            # create i18ns
            for i18nElem in i18nElems:
                i18nObj = eval("%s()" % childClassName)
                self.fillObject(i18nObj, i18nElem)
                obj.addObjectToBothSidesOfRelationshipWithKey(i18nObj, 'i18n')
                self.ec.insertObject(i18nObj)
            self.ec.insertObject(obj)


I process the table dumps 2 by 2, one for the master, one for i18ns.
I must admit that I don't know how I will restore the more complex
relations.  The way I do it now fix the missing PKs problem because I
find the matching ones in the i18n dump with i18nPath but it's a hack,
a really slow hack.  I have to do a XPath query on a 1400 nodes file
700 time : 300 secs avg.

So skiping PKs would be much more simple but having the framework
doing the dump and restoration would be even beter.  I do the dump
like this (phpMyAdmin is full Latin-1) :

if __name__ == "__main__":
    if not len(sys.argv) == 2:
        print "USAGE : dump_xml.py <entity-name>"
        sys.exit(1)

    sys.path.append("..")

    entityName = sys.argv[1]
    ec = EditingContext()
    mm = ModelManager(ec)

    importCmd = "from Autogen.%s import %s" % (entityName, entityName)
    exec(importCmd)

    objs = mm.fetch(entityName)

    # get the primary key name
    key = ec.globalIDForObject(objs[0]).keyValues().keys()[0]

    objs = map(lambda obj:mm.snapshot(obj), objs)

    objs = sortById(objs, key)

    domDoc = getDomDoc()
    for obj in objs:
        elem = domDoc.createElement(entityName)
        for key in obj.keys():
            tagName = externalNameForInternalName(key).lower()
            property = domDoc.createElement(tagName)
            if key != "i18n":
                elem.appendChild(property)
                dataStr = toStr(obj[key])
                data = domDoc.createTextNode(dataStr)
                property.appendChild(data)
        xpath.Evaluate("/db_dump", domDoc)[0].appendChild(elem)

    PrettyPrint(domDoc)


Yes, i18n is hardcoded... : \

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/Hq2brhy5Fqn/MRARAgH1AJwLLuIHR5WxlGkRv8vjXuvveMH2TACeLpZi
EuosOQLR5OrqkhwSJU13TTw=
=+ReG
-----END PGP SIGNATURE-----

[Modeling-users] Reply-to header

[Sebastien B. <sbi...@us...>]

        Hi all,

  Some of you have asked if the reply-to header could be automatically
  set. I already replied that I won't, mainly (& shortly) because I
  agree with the why-it-is-considered-harmful arguments.

  However, there's no reason why you should not set it yourself if you
  want to. It seems that a very simple procmail rule can do the trick,
  see for example:

https://listman.redhat.com/archives/fisher-list/2001-February/msg00391.html
https://mail.fukt.bth.se/pipermail/crux/2003-June/000588.html
http://www.wecs.com/replytorc.htm

  I've never used procmail myself so I'm not able to post here the exact
  rule for the list, but if some of you try makes this successfully it
  might be a good idea to share with others. I'll probably add it to the
  welcome message of the mailing-list, then.

        Regards,

-- S=E9bastien.

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Yannick Gingras <yan...@sa...> wrote:
> There is a nice feature in phpMyAdmin to dump a database to XML.
> There is no nice feature in phpMyAdmin to import a XML dump.  So I
> wrote a little script to import this XML dump using the framework.
> This provide me RDBMS independance so I can re-import my dump in
> Postgres in no time.

Nice.

> But if I have some of the records that have been
> deleted, the PKs are not in a continuous sequence so FKs become out of
> synch.

Ok. But could you tell how you are doing this exactly? Are you
building/inserting-in-ec new objects, then saving changes, or are you
using some of the low-level functionalities to execute raw sql inserts,
or something else? Asking this, because I'm trying to understand at
which point exactly you need to provide the PK, where it is stored, etc.

-- S=E9bastien.

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 23, 2003 09:04 am, you wrote:
> > Since PKs are automatic, it would be really nice to be able to skip
> > some ids in the sequence.  Mostly when restoring a backup with deleted
> > records using the framework instead of a plain SQL dump (why I do this
> > is pure mystery, even me).
>
>   Could you be more explicit on what you're doing exactly? I suspect
>   that what you need here is the more general feature where you could
>   provide your own pk values.

There is a nice feature in phpMyAdmin to dump a database to XML.
There is no nice feature in phpMyAdmin to import a XML dump.  So I
wrote a little script to import this XML dump using the framework.
This provide me RDBMS independance so I can re-import my dump in
Postgres in no time.  But if I have some of the records that have been
deleted, the PKs are not in a continuous sequence so FKs become out of
synch.

So yes, I'm looking for a way to provide my PK values.

=2D --=20
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HpJ3rhy5Fqn/MRARAjPNAJ403ltRwkRO4jdoXqNRAWLRar+0WACgiQRW
P+nBLzqyLzEcemBLJbjHezA=3D
=3Dek7I
=2D----END PGP SIGNATURE-----

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Yannick Gingras <yan...@sa...> wrote:
> Since PKs are automatic, it would be really nice to be able to skip
> some ids in the sequence.  Mostly when restoring a backup with deleted
> records using the framework instead of a plain SQL dump (why I do this
> is pure mystery, even me).

  Could you be more explicit on what you're doing exactly? I suspect
  that what you need here is the more general feature where you could
  provide your own pk values.

-- S=E9bastien.

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Yannick Gingras <yan...@sa...> wrote:
> On July 21, 2003 11:41 am, Sebastien Bigaret wrote:
> > > > Is there a way to do it without breaking anything ?
> > >
> > > Of course there is a way, just set the foreign key to be class proper=
ty.
> > >
> > > Forgive me to post before proper re-caffeination !
> >
> > Oh oh, here you can get into very serious troubles. Demonstration, using
> > test package AuthorBooks where I made Book's FK_Writer_Id a class
>=20
> You're just too right !  re-caffeination is so slow after those lan
> parties...

He he

[...]
> > That's an illustration of why PKs and FKs should be considered
> > *read-only* when they are made class properties, as stated in the faq
> > [http://modeling.sf.net/UserGuide/faq-change-class-location.html].
>=20
> I might be nice to raise when trying to set a read-only property.

While this could be done, I'm 90% sure this is not a good idea ;)

I'm thinking of two reasons:

  1. it would add additional cpu-times for a feature most users do not
     need,

  2. when it becomes possible to provide your own generation scheme for
     generating PKs, it is possible that you'll use this attribute for
     storing the id to be used at saveChanges() time.


-- S=E9bastien.

Re: [Modeling-users] unloading CustomObjects

[Sebastien B. <sbi...@us...>]

Mario Ruggier <ma...@ru...> wrote:
[About EditingContext.refaultObject()]
> I can see this being very useful, My question is how do you
> keep track of which objects need to be freed? Do you do the
> bookkeeping separately? It might be nice to have EC's that
> automate this according to some criteria, such as, automatically
> drop (unfetch?) objects not accessed since 1 hour, or automatically
> unfetch all the least recently accessed objects beyond N, where N is
> some positive number (i.e. keep only the most recently accessed N
> objects).

This is the second step, right, the automation of the process will be
made separately. This API only allows you to make it by hand.

  However, rethinking of this, I'm wondering if the way it is done will
  actually free as much memory as expected... Here are details on the
  process exposed in the patch:

    - deincrement the snapshot reference count for an object in the
      database's cache. If the refaulted object is the last one
      referencing the snapshot, the snapshot is freed.

    - revert the object into a fault. Doing this basically means: call
      __init__() to reset its value to their defaults, and mark it as a
      fault by storing an instance of AccessFaultHandler in it. This
      step does not free a lot of memory, although your objects'
      attributes have a big memory footprint: basically, you still have
      the reinitialized object in the EC.

  So the most significant impact on memory (unless, I repeat, your
  objects are quite big and some of their attributes contains instances,
  such as FixedPoint instances) will occur when forgetting the
  snapshots.=20

Hmmm... This needs to be checked in details. I'll probably also compare
with ZODB ghosting mechanisms and its impact on memory. Probably other
approaches might be taken, such as:

  - having a shared EditingContext holding mostly read-only objects that
    are used by an application: this would make it possible to
    manipulate these objects in <n> different ECs while having only one
    (instead of <n>) instance for each on them,

  - changing the object's class into FaultHandler, instead of
    reinitializing its values and storing a FaultHandler instance in
    it.

To be continued...

-- S=E9bastien.

[Modeling-users] Working on a 'bench' script

[Sebastien B. <sbi...@us...>]

        Hi all,

  Following Mario's original idea, I'm currently working on a script for
  keeping track of the time spent on particular actions of the
  framework, in particular situations. The script will be added in the
  tests/, and the results will be publicly announced for each new
  version. As Mario wrote:

> It would be interesting to keep an eye on these values, for a particular
> setup, thus when changes to the system are made, unexpected performance
> side effects may still be observed.

  (for the curious, I'm using python2.3's new module timeit)

  I have already added these:

    - addObjectToBothSidesOfRelationshipWithKey (toMany)
    - addObjectToBothSidesOfRelationshipWithKey (toOne)
    - fetching Persons (5000 simple objects)
    - fetching a raw row
    - fetching 5000 simple raw rows
    - fetching 5000 raw rows when all objects are modified

  Given what was already tried and reported here, I'll also add:

    - refetching Persons (5000 simple objects) after they all have been
      loaded,
    - fetching 5000 raw rows when all objects are already fetched
    - building qualifiers from strings (qualifierWithQualifierFormat)
    - clearing the toMany faults on 5000 objects,
    - batch-fetching toMany faults for 5000 objects (when this is added
      in the core)

  I will also add the same tests for fetches, with more complex objects
  (say, with 10 attributes+PK+FK)

  Do you have in my mind additional situations you'd like to be tracked?

-- S=E9bastien.

Re: [Modeling-users] unloading CustomObjects

[Mario R. <ma...@ru...>]

Sebastien Bigaret wrote:
> Yannick Gingras <yan...@sa...> wrote:
>> Hi !
>>
>> Once the object is loaded in an EditingContext, is there a way to  
>> unload it ?
>>
>> I don't want to kick it from the DB, just free some RAM...  There is
>> forgetObject() but the doc told not to use it.
>
>
> First: you're completely right, forgetObject() is for internal use  
> only.
>
> You need EditingConext.refaultObject(). Don't search your local copy,  
> it
> does not have it; I was sure this was implemented, and it is... but not
> where I first expected to find it.
>
> Check the ml archives:; (this was in march)
> https://sourceforge.net/mailarchive/ 
> forum.php?thread_id=1884326&forum_id=10674
>
>   I've quickly checked the implementation & at the test I proposed
>   there, so here is my suggestion: you can use it as-is, given that:
>
>     - you're not applying it on deleted, inserted or modified objects,
>       (BTW: it will become an error to refault a deleted or an inserted
>       object)
>
>     - you stick to EC.refaultObject and do not try to call
>       DBContext.refaultObject
>
>     - you do not use it with a nested ec.
>
>   (I'm not saying this won't work, it probably will... but it needs to
>    be tested before I can say this is supported).
>
> This request comes up on a regular basis, I guess I should consider
> integrating it in the core. Any other vote for this, anyone?

I can see this being very useful, My question is how do you
keep track of which objects need to be freed? Do you do the
bookkeeping separately? It might be nice to have EC's that
automate this according to some criteria, such as, automatically
drop (unfetch?) objects not accessed since 1 hour, or automatically
unfetch all the least recently accessed objects beyond N, where N is
some positive number (i.e. keep only the most recently accessed N
objects).

mario

Re: [Modeling-users] unloading CustomObjects

[Sebastien B. <sbi...@us...>]

Yannick Gingras <yan...@sa...> wrote:
> Hi !
>=20
> Once the object is loaded in an EditingContext, is there a way to unload =
it ?
>=20
> I don't want to kick it from the DB, just free some RAM...  There is
> forgetObject() but the doc told not to use it.


First: you're completely right, forgetObject() is for internal use only.

You need EditingConext.refaultObject(). Don't search your local copy, it
does not have it; I was sure this was implemented, and it is... but not
where I first expected to find it.

Check the ml archives:; (this was in march)
https://sourceforge.net/mailarchive/forum.php?thread_id=3D1884326&forum_id=
=3D10674

  I've quickly checked the implementation & at the test I proposed
  there, so here is my suggestion: you can use it as-is, given that:

    - you're not applying it on deleted, inserted or modified objects,
      (BTW: it will become an error to refault a deleted or an inserted
      object)

    - you stick to EC.refaultObject and do not try to call
      DBContext.refaultObject

    - you do not use it with a nested ec.

  (I'm not saying this won't work, it probably will... but it needs to
   be tested before I can say this is supported).

This request comes up on a regular basis, I guess I should consider
integrating it in the core. Any other vote for this, anyone?

-- S=E9bastien.

[Modeling-users] unloading CustomObjects

[Yannick G. <yan...@sa...>]

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi !

Once the object is loaded in an EditingContext, is there a way to unload it=
 ?

I don't want to kick it from the DB, just free some RAM...  There is
forgetObject() but the doc told not to use it.



Thanks in advance !

=2D --=20
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468


=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HYKOrhy5Fqn/MRARAiR0AKCQ+Fi4Ico43tGfGzbjDg9gzJqvKQCgjzdA
qHy8gi7QPvvTAIb2g5AZ69U=3D
=3DW7vE
=2D----END PGP SIGNATURE-----

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Sorry, my fault, just a typo: the test should read:

 if master.isFault() and ec.parentObjectStore().__class__!=EditingContext:
                                                        ^^^^

Yannick Gingras <yan...@sa...> wrote:
> On July 22, 2003 02:52 am, you wrote:
> > these restrictions, it can accelerate the change:
> > >>> i18n = I18N()
> > >>> ec.insert(i18n)
> > >>> master=ec.faultForRawRow(masterSnapshot)
> > >>> if master.isFault() and
> > >>> ec.parentObjectStore().__class__==EditingContext:
> >
> > ...    i18n.setMaster(master)
> > ... else:
> > ...    i18n.addObjectToBothSidesOfRelationshipWithKey(master, 'master')
> 
> Why does my EditingContext's parentObjectStore() is not an EditingContext ?
> 
>   >>> print ec.parentObjectStore().__class__
>   Modeling.ObjectStoreCoordinator.ObjectStoreCoordinator
> 
> it's created with :
> 
>   ec = EditingContext()
> 
> What have I done wrong ?
> 
>  : \
> 
> -- 
> Yannick Gingras
> Byte Gardener, Savoir-faire Linux inc.
> (514) 276-5468

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 22, 2003 02:52 am, you wrote:
> these restrictions, it can accelerate the change:
> >>> i18n =3D I18N()
> >>> ec.insert(i18n)
> >>> master=3Dec.faultForRawRow(masterSnapshot)
> >>> if master.isFault() and
> >>> ec.parentObjectStore().__class__=3D=3DEditingContext:
>
> ...    i18n.setMaster(master)
> ... else:
> ...    i18n.addObjectToBothSidesOfRelationshipWithKey(master, 'master')

Why does my EditingContext's parentObjectStore() is not an EditingContext ?

  >>> print ec.parentObjectStore().__class__
  Modeling.ObjectStoreCoordinator.ObjectStoreCoordinator

it's created with :

  ec =3D EditingContext()

What have I done wrong ?

 : \

=2D --=20
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HVtCrhy5Fqn/MRARAl97AJ9pZ1UkMziHr581ADE99GI2oJ4/UACgi5Ns
aEgkj0VHn2ndcLhjofgV1oY=3D
=3DH2l3
=2D----END PGP SIGNATURE-----

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Replying to my own post, I'd like to share some additional informations:

I wrote:
> Now back to your specific problem: based on your code, I assume you have
> the following model:
>=20
>     master <--->> i18n  (-> is to-one, ->> is to-many)
>=20
> So you'll do something like this, assuming that the master entity's name
> is 'Master' and the relationship' name i18n--->>master is: 'master'
> and the inverse is 'i18ns':
>=20
> >>> i18n =3D I18N()
> >>> ec.insert(i18n)
> >>> master=3Dec.faultForRawRow(masterSnapshot, 'Master')
> >>> i18n.addObjectToBothSidesOfRelationshipWithKey(master, 'master')
> >>> ec.saveChanges()
>=20
> or its alternate equivalent:
>=20
> >>> i18n =3D I18N()
> >>> ec.insert(i18n)
> >>> master=3Dec.faultForRawRow(masterSnapshot)
> >>> master.addToI18ns(i18n)
> >>> i18n.setMaster(master)
> >>> ec.saveChanges()
>
>   Naturally, this implies that the 'master' object is fetched and
>   initialized within the EditingContext.

Each of these approaches triggers in fact two faults:

  1. the fault for object 'master', which is fetched as soon as
     addToI18ns is called (triggered by self.willRead()),

  2. the fault for master.i18ns (array), which is fetched as soon as
     addToI18ns() appends the new object to the array.

In this particular situation, there is a third way of doing this. It
should be considered as a 'hack' to be used in controlled situations
(for example, in batches where you exactly know what has happened
before), because of the limitations discussed below. However and given
these restrictions, it can accelerate the change:

>>> i18n =3D I18N()
>>> ec.insert(i18n)
>>> master=3Dec.faultForRawRow(masterSnapshot)
>>> if master.isFault() and ec.parentObjectStore().__class__=3D=3DEditingCo=
ntext:
...    i18n.setMaster(master)
... else:
...    i18n.addObjectToBothSidesOfRelationshipWithKey(master, 'master')
>>> ec.saveChanges()

What happens here? The difference here is that when most of your masters
are faults, then you'll avoid to initialize both the masters and their
i18ns.

  It works here because of two reasons: this is a toMany relationship,
  and we know from the model that the information used for both
  relationships 'master' and 'i18ns' is ultimately stored in the foreign
  key of table I18N, nothing more. When the master object is a fault,
  it's not initialized yet, and knowing that its relations are about to
  change does not add anything compare to: just wait until it's needed,
  where it will get its related objects, based on the information stored
  in the I18N's FK.

  (I hope I succeeded in making this clear)

  However, I strongly suggest not to remove the test 'if
master.isFault()...', because if for some reasons the master object was
already fetched (believe me, it can happen and surprise you in your own
code ;) this would lead to an inconsistency in the graph of objects
(i.e. master object's i18ns array not being updated where it should be)

    The second test checks that the ec is not nested (see
    http://modeling.sf.net/UserGuide/nested-ec-dev-hints.html). If it is
    nested, I suspect (even if I did not check it explicitely) that
    there are particular situations in which the parent's graph of
    objects could become inconsistent when the child save its changes.

  Last, it should *never* been used in a multi-threaded environment,
  because too much thing can happen between the moment where the changes
  are made and the time saveChanges() is called. All these precautions
  are what I previously called a "controlled environment". May I suggest
  that this gets somehow specifically documented in your code if you use
  this, so that anyone else working on your code does not get bitten by
  this in 2 years, 2 months or 2 days?-)

(I initially wondered whether this should be shared after all, because
 of all this, but I think it might come in handy for particular
 situations, and that it cay also highlights some particular aspects of
 the framework).


  You can then verify that you won't get into trouble. For example, if
you then trigger the two faults, master.geti18n() will contain the new
one, as expected.


-- S=E9bastien.

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 21, 2003 11:41 am, you wrote:
> That's an illustration of why PKs and FKs should be considered
> *read-only* when they are made class properties, as stated in the faq
> [http://modeling.sf.net/UserGuide/faq-change-class-location.html].

Since PKs are automatic, it would be really nice to be able to skip
some ids in the sequence.  Mostly when restoring a backup with deleted
records using the framework instead of a plain SQL dump (why I do this
is pure mystery, even me).

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HBHMrhy5Fqn/MRARAkHKAJkBjs/kdD053DFsk4yhOhwjxY095wCffs6q
47CoxI7s7cCNqnx5gw5QY7c=
=j5iq
-----END PGP SIGNATURE-----

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 21, 2003 11:41 am, Sebastien Bigaret wrote:
> > > Is there a way to do it without breaking anything ?
> >
> > Of course there is a way, just set the foreign key to be class property.
> >
> > Forgive me to post before proper re-caffeination !
>
> Oh oh, here you can get into very serious troubles. Demonstration, using
> test package AuthorBooks where I made Book's FK_Writer_Id a class

You're just too right !  re-caffeination is so slow after those lan parties...

[sniped problem demo]

>   What happened here? The FK is automatically set by the framework, by
>   examining the relationships. Since no object is related to book for
>   relation 'author', the FK gets overriden w/ the None value.
>
> That's an illustration of why PKs and FKs should be considered
> *read-only* when they are made class properties, as stated in the faq
> [http://modeling.sf.net/UserGuide/faq-change-class-location.html].

I might be nice to raise when trying to set a read-only property.

The solution with faultForRawRow() is easier to apply anyway.

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HAw0rhy5Fqn/MRARAtTVAKCA6OjNLjrqdaQpCCrC0l75hlaDqgCfUkvh
QDgF1+ruhFYiQEYTYkgeNAI=
=UGv/
-----END PGP SIGNATURE-----

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 21, 2003 10:59 am, Sebastien Bigaret wrote:
>   Naturally, this implies that the 'master' object is fetched and
>   initialized within the EditingContext.

Fair enough...

>   Does it fulfill your needs?

It's a bit slower than expected but it works perfectly !

Thanks !

 : )

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/HApBrhy5Fqn/MRARAgszAJ9KLrywSA5GLe76BVdH9Gt5CTj5dwCePsYi
36Q7YxUFvb2WvRvNQSqXQo8=
=hbGi
-----END PGP SIGNATURE-----

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

Yannick Gingras <yan...@sa...> wrote:
> On July 21, 2003 10:09 am, Yannick Gingras wrote:
> > Hi, I'd like to set my foreign keys by hand ex.:
> >
> >   i18n =3D I18N()
> >   ec.insert(i18n)
> >   i18n.setMasterId(masterSnapshot["id"])
> >   ec.saveChanges()
> >
> > Is there a way to do it without breaking anything ?
>=20
> Of course there is a way, just set the foreign key to be class property.
>=20
> Forgive me to post before proper re-caffeination !

Oh oh, here you can get into very serious troubles. Demonstration, using
test package AuthorBooks where I made Book's FK_Writer_Id a class
property:

>>> from AuthorBooks.Book import Book
>>> from Modeling.EditingContext import EditingContext
>>> ec=3DEditingContext()
>>> rabelais=3Dec.fetch('Writer', 'lastName=3D=3D"Rabelais"', rawRows=3D1)[=
0]
>>> b=3DBook()
>>> b.setTitle('test')
>>> ec.insert(b)
>>> b.setFK_Writer_Id(rabelais['id'])
>>> print b.getAuthor()
None
>>> b.getFK_Writer_Id()
2
>>> ec.saveChanges()
>>> print b.getAuthor() # still None
>>> b.getFK_Writer_Id() # but fk is set
2
>>> # fetching doesn't help
... print ec.fetch('Book', 'title=3D=3D"test"')[0].getAuthor()
None
>>> # and fetching the other objects does not help either
... # because the fk_writer_id field in database is NULL!!
... books=3Dec.fetch('Writer', 'lastName=3D=3D"Rabelais"')[0].getBooks()
>>> print [b.getTitle() for b in books]
['Gargantua']

Worse: if you look at your database, you'll see that the fk_writer_id is
NULL!!

  Any fetch w/ a different EC will give you a different view:

>>> ec2=3DEditingContext()
>>> print ec2.fetch('Book', 'title=3D=3D"test"')[0].getAuthor()
None
>>> print ec2.fetch('Book', 'title=3D=3D"test"')[0].getFK_Writer_Id()
None

  What happened here? The FK is automatically set by the framework, by
  examining the relationships. Since no object is related to book for
  relation 'author', the FK gets overriden w/ the None value.

That's an illustration of why PKs and FKs should be considered
*read-only* when they are made class properties, as stated in the faq
[http://modeling.sf.net/UserGuide/faq-change-class-location.html].

  Note: this is only one of the numerous problems you can get, including
  inconsistencies in the graph of objects, changes not being saved,
  etc., when doing this. The fact is that setting a PK or a FK is not
  supported, not even expected, by the framework, and should not be done
  under any circumstances.


-- S=E9bastien.


BTW: I noticed while "playing" with this that FKs that are class
     properties do not get their values updated after saveChanges().
     This is a bug that will be fixed.

Re: [Modeling-users] Foreign Keys

[Sebastien B. <sbi...@us...>]

        Hi,

Yannick Gingras <yan...@sa...> wrote:
> Hi, I'd like to set my foreign keys by hand ex.:
>=20
>   i18n =3D I18N()
>   ec.insert(i18n)
>   i18n.setMasterId(masterSnapshot["id"])
>   ec.saveChanges()
>=20
> Is there a way to do it without breaking anything ?
>=20
> Since I switched to raw fetches, I end-up with a dict that is unusable wi=
th=20
> addObjectToBothSidesOfRelationship().  In fact, having=20
> addObjectToBothSidesOfRelationship() accept snapshots would be nice.

First, a general answer to this.

  As said at http://modeling.sf.net/UserGuide/ec-fetch-raw-rows.html,
  when you use raw row fetching you forego most of the capabilities of
  the framework. This includes tracking changes (no objects are even
  created, so the framework cannot keep track of the changes), all the
  more so manipulating relationships and saving the changes.

  I should probably make it even clearer in the documentation.

  So what? Fortunately you're not stuck. In general, you have two
  solutions at hand:
=20=20
    - either fall back to raw sql. Probably not what you want...
=20=20
    - or revert the row you want to change into a real object, using
      EC.faultForRawRow(), then make your changes and save them.


Now back to your specific problem: based on your code, I assume you have
the following model:

    master <--->> i18n  (-> is to-one, ->> is to-many)

So you'll do something like this, assuming that the master entity's name
is 'Master' and the relationship' name i18n--->>master is: 'master'
and the inverse is 'i18ns':

>>> i18n =3D I18N()
>>> ec.insert(i18n)
>>> master=3Dec.faultForRawRow(masterSnapshot, 'Master')
>>> i18n.addObjectToBothSidesOfRelationshipWithKey(master, 'master')
>>> ec.saveChanges()

or its alternate equivalent:

>>> i18n =3D I18N()
>>> ec.insert(i18n)
>>> master=3Dec.faultForRawRow(masterSnapshot)
>>> master.addToI18ns(i18n)
>>> i18n.setMaster(master)
>>> ec.saveChanges()


  Naturally, this implies that the 'master' object is fetched and
  initialized within the EditingContext.

    Does it fulfill your needs?

-- S=E9bastien.

Re: [Modeling-users] Foreign Keys

[Yannick G. <yan...@sa...>]

=2D----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 21, 2003 10:09 am, Yannick Gingras wrote:
> Hi, I'd like to set my foreign keys by hand ex.:
>
>   i18n =3D I18N()
>   ec.insert(i18n)
>   i18n.setMasterId(masterSnapshot["id"])
>   ec.saveChanges()
>
> Is there a way to do it without breaking anything ?

Of course there is a way, just set the foreign key to be class property.

=46orgive me to post before proper re-caffeination !

 ; )

=2D --=20
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468

=2D----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/G/m1rhy5Fqn/MRARAgBaAJ9QZEVuQOrHf/SpRS0inNS39049egCfZefp
8bSfxHLrCZy6anfU3WZ5itw=3D
=3DwXOV
=2D----END PGP SIGNATURE-----