modeling-users — For users: questions, support, announcements, etc.

[Modeling-users] Re: Consistency among different processes?

[Sebastien B. <sbi...@us...>]

Federico Heinz <fh...@vi...> wrote:
> Modeling's manual seems to imply that it caches objects in memory
> indefinitely, and raises a few issues of cache consistency among
> EdititingContexts, as well as strategies to work around them.

To be precise, an object's row is cached as long as one instance for
this row is registered in one EC.

> The workarounds, and even the "to-be-implemented clean solution" of
> creating a mechanism for ECs to broadcast changes all seem to assume,
> however, that all ECs reside in the same address space (a Zope server,
> for instance). This need not be the case, though: Imagine a typical
> client-server application that runs in a single address space on the
> client machine. If a machine fetches some rows, and then another one
> changes them, the first machine will never see the changes? It may even
> break them? This doesn't seem like something that can be fixed by
> broadcasting...

I see the point. This is the current status. Say you have two instances
(so, two adress spaces) with two ECs, ec1 and ec2. Both query and update
an object obj1.

  - if ec1 save its changes, ec2 will not see the changes, because it
    has already fetched (and cached) the obj1. We'll need the 'refresh'
    parameter for fetch() to actually the changes,

  - if ec1 saves its changes, then ec2 saves its changes: ec2 will
    actually override the changes made by ec1, and won't even notice
    this.

(In fact, as the documentation says and as you noted, we currently also
 have this problem between two different ECs in the same address space
 --but this will be solved by delivering notifications to the
 appropriate objects)

Now imagine that optimistic locking is implemented. Optimistic locking
examines the data when they are about to be saved in the db. When the
data in the db differs from the cached one, an error is raised. This
general mechanism can be used to make different address spaces notice
the changes when they save changes (while the 'refresh' parameter of
fetch would enable an EC to discard its cache and its changes, and to
refetch the data).

  Of course, optimistic locking will allow you to supply your own
  delegate to handle such situations. For example, you might decide to
  override the changes for a given entity, or mix the changes observed
  in the db with the ones made on the object, etc.).

This would probably address most problems, I think. Now if you want two
different address spaces to be notified of changes made by the other
before any attempt to save the changes in the db, we would need a more
general notification mechanism which should be able to broadcast changes
through the network, but even then, I suspect this is a hard problem to
solve. Another cleaner solution (and maybe the only one that can be
guaranteed to be 100% safe) could be to explicitely lock the appropriate
row before any attempt to modify an object, and to release the lock only
after changes has been made --this is the so-called pessimistic locking
strategy.

-- S=E9bastien.

[Modeling-users] Re: How are transaction failures handled?

[Sebastien B. <sbi...@us...>]

Federico Heinz <fh...@vi...> wrote:
> The Modeling manual explains pretty well how the validation mechanisms
> work. There is no mention, though, about what happens when a transaction
> fails because the dbms rejects it, nor of how transaction boundaries are
> defined.
>=20
> Or did I miss it?

I did not check but you're probably right that this is not described in
the manual.

EditingContexts hold the transaction. That is to say, they work with the
underlying classes to control the underlying RDBMS transactions. ECs
handles:

- transactions at the object-level: A transaction at this level starts
  when the EC is created, or after a saveChanges(). All changes to the
  objects held by an EC are guaranteed to be saved as a whole, or not at
  all.=20

- Let's have a look at how saveChanges() works:

  1. it checks consistency at the object level --in particular, the differe=
nt
     validation mechanisms are triggered here. If anything goes wrong,
     it stops here: no transaction to the underlying db is opened.

  2. It then opens a transaction in the db, and sends the necessary SQL
     statements (INSERT/UPDATE/DELETE) in this transaction. If any error
     occurres when these sql statements are executed, the whole
     transaction is rolled back.

  3. It commits the transaction. Here again, if an error occurs at the
     DB-level, the transaction is rolled back.

(I supposed here that we are talking about EC that are not nested;
 nested ECs write the changes in their parent EC, not in the db)

In any case, if an error occurs, nothing is committed in the db.
Moreover, no changes are applied to the EC itself: e.g. the objects
marked as inserted remain the same before and after, etc. This gives you
the opportunity to correct the error (a validation error for example),
then to retry the saveChanges().

  Does this answer your question?

-- S=E9bastien.

[Modeling-users] Implementing inheritance through vertical mapping

[Federico H. <fh...@vi...>]

Modeling does not implement vertical mapping yet, a feature we might[1]
be willing to contribute. However, from the looks of it, the XML schema
definition does not provide the needed information for it: the way it's
structured, it doesn't contain any provisions for telling the system
that a given class takes its data from a series of tables and not just
from one.

This can, of course, be corrected, and instantiating an object from
several tables at once was one of those things EOF did beautifully
(maybe still does). But the XML schema definition must be changed,
hopefully in a backwards-compatible way. Are there any plans about how
these changes could be implemented? We could suggest ways for doing it,
but if the issue has been discussed, we'd rather add our salt to the
discussion than make complete fools of ourselves :-)

	Fede
=20
[1] depending on priorities. We have done similar work in other
contexts, but for now we could choose to use PostgreSQL's inheritance
mechanism to implement the database, until we can spare the time to
create the vertical mapping layer.

[Modeling-users] Consistency among different processes?

[Federico H. <fh...@vi...>]

Modeling's manual seems to imply that it caches objects in memory
indefinitely, and raises a few issues of cache consistency among
EdititingContexts, as well as strategies to work around them.

The workarounds, and even the "to-be-implemented clean solution" of
creating a mechanism for ECs to broadcast changes all seem to assume,
however, that all ECs reside in the same address space (a Zope server,
for instance). This need not be the case, though: Imagine a typical
client-server application that runs in a single address space on the
client machine. If a machine fetches some rows, and then another one
changes them, the first machine will never see the changes? It may even
break them? This doesn't seem like something that can be fixed by
broadcasting...

	Fede

[Modeling-users] How are transaction failures handled?

[Federico H. <fh...@vi...>]

The Modeling manual explains pretty well how the validation mechanisms
work. There is no mention, though, about what happens when a transaction
fails because the dbms rejects it, nor of how transaction boundaries are
defined.

Or did I miss it?

	Fede

[Modeling-users] Modeling vs. ZODB?

[Federico H. <fh...@vi...>]

How does ZODB compare exactly to Modeling? I've been trying to
understand ZODB from its documentation, but I can't quite figure out
whether it ZODB and Modeling are substitutes for one another, or whether
they solve different problems and can be used together. At times, the
ZODB docs seem to say that it's a O-O database engine in itself, at
other times that it's an O/R mapper... I can't quite figure it out.

Any experienced Zopers that can tell?

	Fede
=20

[Modeling-users] A few questions on Modeling: current status, and future directions

[Federico H. <fh...@vi...>]

Hi, Folks!

I've been looking into Modeling, evaluating it for our project. It looks
very nice, maps very well into our strategy (we want to be able to run
our app against a Zope appserver as well as stand-alone), and the
reference to EOF made my NeXTstep developer heart bleed with nostalgy
:-)

In any case, if I understood the manual right, Modeling looks very nice,
and fits the bill quite perfectly. I have a few doubts still. I've had a
preliminary contact with Sebastien, but I thought these questions are
relevant enough for the mailing list, so maybe they'll tickle some
discussion. This is just an introductory mail, I'll post the questions
separately, so we can build threads that don't mix all the issues.

	Fede

Re: [Modeling-users] Sqlite numeric types returnes as strings

[Ernesto R. <er...@si...>]

Ok. I still didn't try this. The problem is that I'm working under WinXP =
and have not installed MSVC. My co-worker is using MinGW, but I have to =
ask him how it works to reompile these.

Thanks anyway. I'm a little busy now till the end of next week, so I'll =
try it after this period.

Erny


----- Original Message -----=20
From: "Sebastien Bigaret" <sbi...@us...>
To: "Ernesto Revilla" <er...@si...>
Cc: "modeling-users" <mod...@li...>
Sent: Thursday, September 18, 2003 7:32 PM
Subject: Re: [Modeling-users] Sqlite numeric types returnes as strings


>=20
>   A very, very quick answer: try to modify _sqlite.c
> in pysqlite, around l.1169 you'll find:
>=20
>             else if (strstr(type_name, "FLOAT"))
>             {
>                 type_code =3D tc_FLOAT;
>             }
>=20
> Replace it with:
>=20
>             else if (strstr(type_name, "FLOAT")
>      || strstr(type_name, "NUMERIC"))
>             {
>                 type_code =3D tc_FLOAT;
>             }
>=20
>=20
>   and you'll get the expected behaviour for NUMERIC types. Other types =
can be
> added the same way. I'll request for the necessary additions probably =
late this
> evening.
>=20
>   Regards,
>=20
> -- S=E9bastien.
>=20
>=20
> > pysqlite 4.3.0
> >=20
> > Ok, I've checked it out
> > * good types: ['integer', 'int', 'int4', 'smallint', 'int2', =
'bigint',
> > 'int8', 'float4', 'float8']
> >=20
> > * bad types ['real', 'double precision', 'double', 'numeric',
> > 'numeric(15)', 'numeric(15,5)', 'decimal', 'decimal(15)',
> > 'decimal(15,5)']
> >=20
> > I got the types from postgres (I know, that not all are ANSI SQL 92, =
but
> >  a lot of them are.)
> > I've sent a bug report to pysqlite @ sf.
> >=20
> > If anyone ones the test procedure used, let me know.
> >=20
> > Best regards,
> > Erny
>

Re: [Modeling-users] Sqlite numeric types returnes as strings

[Sebastien B. <sbi...@us...>]

[...]
>   and you'll get the expected behaviour for NUMERIC types. Other types can
> be added the same way. I'll request for the necessary additions probably
> late this evening.

I meant I would check and post them. Your review of the types
working/not working was helpful for finding out the right portion of
code. A patch has been added to your bug request at pysqlite@sf
http://sf.net/tracker/index.php?func=3Ddetail&aid=3D808693&group_id=3D54058=
&atid=3D472543

  Hopefully this helps,


-- S=E9bastien.

Re: [Modeling-users] Sqlite numeric types returnes as strings

[Sebastien B. <sbi...@us...>]

  A very, very quick answer: try to modify _sqlite.c
in pysqlite, around l.1169 you'll find:

            else if (strstr(type_name, "FLOAT"))
            {
                type_code = tc_FLOAT;
            }

Replace it with:

            else if (strstr(type_name, "FLOAT")
		     || strstr(type_name, "NUMERIC"))
            {
                type_code = tc_FLOAT;
            }


  and you'll get the expected behaviour for NUMERIC types. Other types can be
added the same way. I'll request for the necessary additions probably late this
evening.

  Regards,

-- Sébastien.


> pysqlite 4.3.0
> 
> Ok, I've checked it out
> * good types: ['integer', 'int', 'int4', 'smallint', 'int2', 'bigint',
> 'int8', 'float4', 'float8']
> 
> * bad types ['real', 'double precision', 'double', 'numeric',
> 'numeric(15)', 'numeric(15,5)', 'decimal', 'decimal(15)',
> 'decimal(15,5)']
> 
> I got the types from postgres (I know, that not all are ANSI SQL 92, but
>  a lot of them are.)
> I've sent a bug report to pysqlite @ sf.
> 
> If anyone ones the test procedure used, let me know.
> 
> Best regards,
> Erny

[Modeling-users] Sqlite numeric types returnes as strings

[Ernesto R. <er...@si...>]

pysqlite 4.3.0

Ok, I've checked it out
* good types: ['integer', 'int', 'int4', 'smallint', 'int2', 'bigint', =
'int8', 'float4', 'float8']

* bad types ['real', 'double precision', 'double', 'numeric', =
'numeric(15)', 'numeric(15,5)', 'decimal', 'decimal(15)', =
'decimal(15,5)']

I got the types from postgres (I know, that not all are ANSI SQL 92, but =
 a lot of them are.)
I've sent a bug report to pysqlite @ sf.

If anyone ones the test procedure used, let me know.

Best regards,
Erny

[Modeling-users] Request for testers: dynamic creation of classes from a model

[Sebastien B. <sbi...@us...>]

        Hi all,

  As I wrote earlier, I'm working on dynamic creation of modules/classes
  from a (py)model, either classic- or new-style (metaclass). That would
  be an alternative to code generation that already exists.

  Would anyone here have some time experimenting with an alpha release
of these features? If you think it would be of some interest in your
project(s), I think I could finish the first draft for tomorrow evening
and then share it as a patch to the current release, so that it can be
tested in different environments than mine.

  And in case this matters, it seems that new-style classes offers the
  best for python2.2: I've already experimented a pre-alpha version of
  this with a gain of 1.7+ for e.g. addObjectToBothSides(), and a gain
  of 1.20+ for fetching real objects (5000 objects fetched in 4.28s,
  compared to 5.16s in 0.9pre15)   ;)

        Thanks,

-- S=E9bastien.

Re: [Modeling-users] Some more questions

[Sebastien B. <sbi...@us...>]

  Back on an unanswered questions of yours:

"Ernesto Revilla" <er...@si...> wrote:
> Hi again,
>=20
> Grand Totals, Counters, etc.
> Say I have a heavily accesed table and I want to have grand totals per
> month. What is the best way to accomplish this? (The problem is that
> if I do this within the domain class, it has to read and update the
> total with added/updated/deleted data, but during these operations
> other people may also hace to need to update the same field, causing
> collisions or overwriting. If there was a 'add' operation which would
> be translated to SQL (field=3Dfiel+increment), this problem would
> disappear, as it would be executed inside a transaction. Also, the
> grand total should perhaps be a volatile field and no cached, that is,
> whenever we access entity.getGrandTotal() it would be read from the
> database.) In databases with triggers, this is easily resolved inside
> the DB, but we still need volatile attributes.

Due to time constraint tonight I won't exhaust all possibilities, but
basically you get it right: if you know how to do this with triggers
(which I surely the best way to achieve this in an highly-stressed db),
then all that you need is a way to get the actual data in the db, not a
cached version of your data.

  Right, this is yet unsupported, however this is planned, something
  like a 'refresh=3Dtrue' parameter in method ec.fetch().

This makes me think that this feature and the other one we discussed
today, Optimistic Locking, requires that EditingContexts get notified of
updates saved by an other EC, and updates their data accordingly. This
is the very basis for both of these features. Now I need priority. As I
said earlier, I'm currently working on dynamically instanciating
modules/classes from a (py)model, either classic-style or new-style
(with metaclasses). I can change priorities of tasks at demand, given
that this demand is loudly expressed ;) As always, users' requests
usually take precedence on my own willingness to code this or that, I
just need to know your timeframe and when you think you'll need this.

        Regards,

-- S=E9bastien.

Re: [Modeling-users] Some more questions

[Sebastien B. <sbi...@us...>]

Jerome Kerdreux <Jer...@fi...> wrote:
> Ernesto Revilla wrote:
> > Would it be possible to create a Wiki? Wouldn't it be the right
> > place to put questions like these?
> >
> Sebastien try to keep the documentation as simple / explicit as he can, i
> think. and he usually do some great job about this. now it's clear that he
> seems to have a lot of work right now, and miss some mails .

That's right, I can miss some emails and I sometimes forget about
specific issues posted here, just because it runs out of my mind --just
like the issue w/ ZODB.Persistent we discussed earlier.

   As a general rule, it is a good idea, if you have some time for that,
to submit a bug report along with the post on the mailing-list (that's
what I did for the documentation issue you reported today), since bug
reports are persistent on sf.net even if they are not in my brain ;)
It's definitely a great help for me to remember what should be fixed.

  Now back on the wiki: as far I know sf.net does not offer such a
  possibility and I currently do not have the time (and the money) to
  add such things to the project's page. Frankly, even if I had a place,
  I'd need someone to manage the wiki to keep it organized and tidy, I
  can't see how I could find the time for this. If any of you have
  suggestions, or want to take the lead on this, please speak, I'm
  really open to that.


-- S=E9bastien.

Re: [Modeling-users] SQLite, problemas with Floats

[Sebastien B. <sbi...@us...>]

        Hi,

"Ernesto Revilla" <er...@si...> wrote:
> Hi again (sorry for the attack),
>=20
> I'm using Modeling 0.9-pre15 and sqlite 2.8.6
>=20
> When I do a fetch for objects with float attributes, these are string and=
 not float. Say:
> ....
> >>> accounts=3Dec.fetch('Account')
> >>> accounts[0].getAmount()
> '0.0'

Which version of pysqlite are you currently using? I just checked the
recommended version (cf. http://modeling.sf.net/installation.html)
i.e. pysqlite v0.4.3, and this version does cast the types as expected:

>>> import sqlite
>>> sqlite.version
'0.4.3'
>>> cnx=3Dsqlite.connect('db_AuthorBooks.db')
>>> cur=3Dcnx.cursor()
>>> cur.execute("SELECT t0.ID, t0.LAST_NAME, t0.FIRST_NAME, t0.AGE, t0.BIRT=
HDAY, t0.FK_WRITER_ID FROM WRITER t0 WHERE t0.id=3D1")
>>> cur.fetchall()
[(1, 'Cleese', 'John', 24, <DateTime object for '1939-10-27 08:31:15.00' at=
 81d7ea8>, None)]


I couldn't find any clue on when this was introduced, however. Could you
check your own version and report? Thanks.

-- S=E9bastien.

Re: [Modeling-users] Some more questions

[Sebastien B. <sbi...@us...>]

"Ernesto Revilla" <er...@si...> wrote:
> Hi again,
>=20
> Queries with large resultsets:
> I remember I read about this but I don't see where. What is returned
> for a fetch with a result set of, say, 10000 rows? A list? Or an
> iterator? Is there a concept like database cursors, which iterate over
> objects (wherever they are stored), instead of database rows?

A list, yes, either a list of objects or a list of dictionaries if you
fetch with 'rawRows=3D1' (which is probably what you need for such a large
number of objects).

  There are some pending works for supporting OFFSET and LIMIT, but this
  is still unfinished.

> Multi-user environment:
> I read the deployment instructions of the user doc. Under SQLite, when
> two separe editing context (either different applications or one
> application but different editing context) save changes to the same
> object, the latter one overwrites silently the earlier ones, which is
> not that nice. Does this behave the same way with Postgres? And with
> mySQL?

The current default database policy is: no locking, which basically
means that no checks are done, ever. I do not know what you would need
in such a case, but maybe optimistic locking could do the
trick. Optimistic locking means: if the data has changed between the
moment you fetched an object, and the moment you save it, than raise
(and then you get the opportunity to ask the user/refresh the
data/whatever).
The last option would be pessimistic locking, which means that as long
as an object lives in an EC, its db row remains locked.

  Pessimistic locking is a really strong requirement and I'm not going
  into this unless there is a real need. However optimistic locking is a
  natural expectation and this could be added sooner than (I!) expected
  if there is a real need for this (and even better, if you plan to
  experiment and test it in various ways).

Last, as Soaf said, there are currently solutions involving using nested
ECs, and/or locking special tables by hand for that purpose, but all
these tricks are just what they are: tricks. But beware with sqlite,
it's probably not mt-safe (gotta check however).

> Grand Totals, Counters, etc.
> Say I have a heavily accesed table and I want to have grand totals per
> month. What is the best way to accomplish this? (The problem is that
> if I do this within the domain class, it has to read and update the
> total with added/updated/deleted data, but during these operations
> other people may also hace to need to update the same field, causing
> collisions or overwriting. If there was a 'add' operation which would
> be translated to SQL (field=3Dfiel+increment), this problem would
> disappear, as it would be executed inside a transaction. Also, the
> grand total should perhaps be a volatile field and no cached, that is,
> whenever we access entity.getGrandTotal() it would be read from the
> database.) In databases with triggers, this is easily resolved inside
> the DB, but we still need volatile attributes.

That's a too long questions to answer now; I'll answer later, probably
this evening.

> Simple question:
> can an editingContext access (fetch and update) classes of different
> models at the same time?
> (I think this is true, but still haven't tried it. It can also treat
> heterogenous data sources, although there may be no relations between
> different models, right?)

Absolutely, as many models can be loaded at the same time, the only
conditions is that two entities in the models cannot have the same name.

  Ok, I've got to go back to work now, I'll be back this evening.

        Regards,

-- S=E9bastien.



> Would it be possible to create a Wiki? Wouldn't it be the right place
> to put questions like these?

=20=20

Re: [Modeling-users] SQLite, problemas with Floats

[Sebastien B. <sbi...@us...>]

"Ernesto Revilla" <er...@si...> wrote:
> Ok, I've tracked down this and see the following issue:
>=20
> In DatabaseContext.py, near line 1540, the following two lines appear:
>=20
>    if value and attr.type() =3D=3D 'DateTime':
>       value=3Dattr.convertStringToAttributeType(value)
>=20
> So we only do type conversion if attr.type() =3D=3D 'DateTime' and value =
is not empty ???
> Is this an optimization?
>=20
> Shouldn't we force a conversion to Python types whenver possible? Should =
this work be done by the database layer (perhaps we need custom conversions=
 depending on the database)?

Usually these type conversions are done by the python adaptors --but
right, everything in sqlite is string, but I can't remeber how pysqlite
handles this, and I can't test this here.
  I've got to check this, especially why this is not catched by the
unittests; I'll have a look at it this evening and will report with
further details then.

-- S=E9bastien.

> ----- Original Message -----=20
> From: "Ernesto Revilla" <er...@si...>
> To: "modeling-users" <mod...@li...>
> Sent: Wednesday, September 17, 2003 1:31 AM
> Subject: [Modeling-users] SQLite, problemas with Floats
>=20
>=20
> Hi again (sorry for the attack),
>=20
> I'm using Modeling 0.9-pre15 and sqlite 2.8.6
>=20
> When I do a fetch for objects with float attributes, these are string and=
 not float. Say:
> ....
> >>> accounts=3Dec.fetch('Account')
> >>> accounts[0].getAmount()
> '0.0'
>=20
> I know that sqlite stores everything as strings, but it should be convert=
ed to float in Python, and I don't know where to do this. The problem is (s=
uppose account has a name):
> >>> accounts[0].setName('sales')
> >>> ec.saveChanges()
> ....
> Modeling.Validation.ValidationException: Validation for key saldo failed:
> - Wrong type
> Validation for key OBJECT_WIDE_VALIDATION failed:
> - Validation of object <Conta.Cuenta.Cuenta instance at 0x00DC4FE8> as a =
whole failed
>=20
> because although it has been initialised with 0.0 now it is '0.0' (type s=
tring).
>=20
> Thanx in advance,
> Erny

Re: [Modeling-users] validateForSave question

[Sebastien B. <sbi...@us...>]

"Ernesto Revilla" <er...@si...> wrote:
> In the doc: http://modeling.sourceforge.net/UserGuide/validate-for-save.h=
tml
> appears the following text:
>=20
> <quote>
[...]
> <quote>
>=20
> 1. doing the import and trying a Validation.Exception() fails. I think it=
 should be, Validation.ValidationException(), shouldn't it?
> 2. On the other hand the instruction sequence:
>=20
>           error=3DValidation.Exception()
>           try:
>             CustomObject.validateForSave(self)
>           except Validation.Exception, error:
>             error.aggregateException(error)
>=20
>   gets me confused. Shouldn't it be as follow?:
>=20
>           error=3DValidation.Exception()
>           try:
>             CustomObject.validateForSave(self)
>           except Validation.Exception, exc:
>             error.aggregateException(exc)
>=20
>   Note that I catch the exception argument with exc and add it afterwards=
 to error with 'aggregateException'.

You're absolutely right:

1. ValidationException was indeed Validation a long time ago, and doc. was
   not updated.

2. the code snippet in the code is really buggy, it should read e.g.

    _error=3DValidationException()
    try:
      CustomObject.validateForSave(self)
    except ValidationException, error:
      _error.aggregateException(error)

   or just the way you do. BTW I tend to think that requiring the super's
   method to be called is somehow not a good idea. This might change in a n=
ear
   future if you think that it could be a better idea (so that it works just
   as validate<AttributeName>() does, without calling super).


Thanks for reporting, I'll change the doc. accordingly.

-- S=E9bastien.

Re: [Modeling-users] Problems importing Modeling with ZODB 3.1.1 installed

[Sebastien B. <sbi...@us...>]

"Jerome Kerdreux (soaf)" <so...@la...> wrote:
> On Wednesday 17 September 2003 00:17, Ernesto Revilla wrote:
> > Hi again,
> >
> > what is the consequence of having ZODB standalone installed when using
> > Modeling? Are there any important differences between
> > Persistence.Persistent and Persistent.Persistent?
> >
> > regards,
> > Erny
>=20
> I have never tested this kind of stuff. but i really think that doesn't=20
> matter. as I Persistence is only used by the ZModeler ( as it is=20
> a normal zope object ). Perhaps this can change something
> when you bind the editing context with Zope session. But i=20
> don't like think . (And i haven't read / test this code)=20

Soaf is right, there's not consequences on the runtime, it's only there to
make model, entities, etc. persistent in a zope/zmodeler instance.

  To convince yourself, look at Modeling.Persistent.Persistent ;))
        class Persistent:
          pass


-- S=E9bastien.

Re: [Modeling-users] Some more questions

[Jerome K. <Jer...@fi...>]

Ernesto Revilla wrote:

>Hi again,
>
>Queries with large resultsets:
>I remember I read about this but I don't see where. What is returned for a fetch with a result set of, say, 10000 rows? A list? Or an iterator? Is there a concept like database cursors, which iterate over objects (wherever they are stored), instead of database rows?
>  
>
You can use both . either fetch using Ec ( via relationships..) so you 
will get a list of object
or using fetchRow and you will get a dict per row ( which isn't exactly 
why the Modeling is done for )

>Multi-user environment:
>I read the deployment instructions of the user doc. Under SQLite, when two separe editing context (either different applications or one application but different editing context) save changes to the same object, the latter one overwrites silently the earlier ones, which is not that nice. Does this behave the same way with Postgres? And with mySQL?
>  
>
2 things :
- SQLite isn't a multiuser DB. you can't open the db twice. so in 
Modeling open / fetch / close is the circle i
   think . that's why you get this .
- MySQL / Postgres and others can handle this smoothly. but again take 
care that Modeling have some caching
  functions (on fetch) that can drive you in the wall if you doesn't 
take care about that.
  here a sample
  authors = ec.fetchWith(....)
  will retrieve the authors from the DB .. and build Authors object w/
   now do authors[0].setName('Jean Pascal')
   ec.saveChanges()
   This will change the right row .. etc . but if on another host (or 
even another program ) you do the
    same thing as the same time, you won't get the right result cause 
 authors is in the cache . so the second program won't see the changes 
until the first one call ec.saveChanges() . and worst case happen . since
authors is in cache so won't be retrieve at second call ..

This really can be a severe issue, you can avoid this by using Nested EC 
.. or doing some ec.dispose ( with all the drawback this generate )


>Grand Totals, Counters, etc.
>Say I have a heavily accesed table and I want to have grand totals per month. What is the best way to accomplish this? (The problem is that if I do this within the domain class, it has to read and update the total with added/updated/deleted data, but during these operations other people may also hace to need to update the same field, causing collisions or overwriting. If there was a 'add' operation which would be translated to SQL (field=fiel+increment), this problem would disappear, as it would be executed inside a transaction. Also, the grand total should perhaps be a volatile field and no cached, that is, whenever we access entity.getGrandTotal() it would be read from the database.) In databases with triggers, this is easily resolved inside the DB, but we still need volatile attributes.
>  
>
Last time i work w/ the Modeling i got this kind of trouble. We use a 
manual SQL lock on a db table to ensure that other EC ( running on other 
hosts) won't try to access the DB at the same time. Pay caution that you 
CAN'T lock a Modeling  table as you don't know when modeling will access 
this (when fetch occur) so we used a empty table for locking only. for 
example we work on a table STATS we got a STATS_LOCK table which is SQL 
manually lock/unlock between statistics update.

>Simple question:
>can an editingContext access (fetch and update) classes of different models at the same time?
>(I think this is true, but still haven't tried it. It can also treat heterogenous data sources, although there may be no relations between different models, right?)
>  
>
It should yes.. but haven't tested . Sebastien should know that.

>Would it be possible to create a Wiki? Wouldn't it be the right place to put questions like these?
>  
>
Sebastien try to keep the documentation as simple / explicit as he can, 
i think. and he usually do some great job about this. now it's clear 
that he seems to have a lot of work right now, and miss some mails .


Bye Bye ..

[Modeling-users] Some more questions

[Ernesto R. <er...@si...>]

Hi again,

Queries with large resultsets:
I remember I read about this but I don't see where. What is returned for =
a fetch with a result set of, say, 10000 rows? A list? Or an iterator? =
Is there a concept like database cursors, which iterate over objects =
(wherever they are stored), instead of database rows?

Multi-user environment:
I read the deployment instructions of the user doc. Under SQLite, when =
two separe editing context (either different applications or one =
application but different editing context) save changes to the same =
object, the latter one overwrites silently the earlier ones, which is =
not that nice. Does this behave the same way with Postgres? And with =
mySQL?

Grand Totals, Counters, etc.
Say I have a heavily accesed table and I want to have grand totals per =
month. What is the best way to accomplish this? (The problem is that if =
I do this within the domain class, it has to read and update the total =
with added/updated/deleted data, but during these operations other =
people may also hace to need to update the same field, causing =
collisions or overwriting. If there was a 'add' operation which would be =
translated to SQL (field=3Dfiel+increment), this problem would =
disappear, as it would be executed inside a transaction. Also, the grand =
total should perhaps be a volatile field and no cached, that is, =
whenever we access entity.getGrandTotal() it would be read from the =
database.) In databases with triggers, this is easily resolved inside =
the DB, but we still need volatile attributes.

Simple question:
can an editingContext access (fetch and update) classes of different =
models at the same time?
(I think this is true, but still haven't tried it. It can also treat =
heterogenous data sources, although there may be no relations between =
different models, right?)

Would it be possible to create a Wiki? Wouldn't it be the right place to =
put questions like these?

best regards,
Erny

Re: [Modeling-users] SQLite, problemas with Floats

[Ernesto R. <er...@si...>]

Ok, I've tracked down this and see the following issue:

In DatabaseContext.py, near line 1540, the following two lines appear:

   if value and attr.type() =3D=3D 'DateTime':
      value=3Dattr.convertStringToAttributeType(value)

So we only do type conversion if attr.type() =3D=3D 'DateTime' and value =
is not empty ???
Is this an optimization?

Shouldn't we force a conversion to Python types whenver possible? Should =
this work be done by the database layer (perhaps we need custom =
conversions depending on the database)?

Erny



----- Original Message -----=20
From: "Ernesto Revilla" <er...@si...>
To: "modeling-users" <mod...@li...>
Sent: Wednesday, September 17, 2003 1:31 AM
Subject: [Modeling-users] SQLite, problemas with Floats


Hi again (sorry for the attack),

I'm using Modeling 0.9-pre15 and sqlite 2.8.6

When I do a fetch for objects with float attributes, these are string =
and not float. Say:
....
>>> accounts=3Dec.fetch('Account')
>>> accounts[0].getAmount()
'0.0'

I know that sqlite stores everything as strings, but it should be =
converted to float in Python, and I don't know where to do this. The =
problem is (suppose account has a name):
>>> accounts[0].setName('sales')
>>> ec.saveChanges()
....
Modeling.Validation.ValidationException: Validation for key saldo =
failed:
- Wrong type
Validation for key OBJECT_WIDE_VALIDATION failed:
- Validation of object <Conta.Cuenta.Cuenta instance at 0x00DC4FE8> as a =
whole failed

because although it has been initialised with 0.0 now it is '0.0' (type =
string).

Thanx in advance,
Erny




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Modeling-users mailing list
Mod...@li...
https://lists.sourceforge.net/lists/listinfo/modeling-users

Re: [Modeling-users] Problems importing Modeling with ZODB 3.1.1 installed

[Ernesto R. <er...@si...>]

----- Original Message -----=20
From: "Jerome Kerdreux (soaf)" <so...@la...>
>=20
> Are you using only ZModeler ? or all Modeling inside Zope ?=20
Actually Modeling is outside of Zope. But the final scenario isn't still =
clear for me. We'll do XML-RPC access from client software (also written =
in Python), so Zope is an attractive solution, as it unifies Web access =
and other client access to the data.

Erny

[Modeling-users] SQLite, problemas with Floats

[Ernesto R. <er...@si...>]

Hi again (sorry for the attack),

I'm using Modeling 0.9-pre15 and sqlite 2.8.6

When I do a fetch for objects with float attributes, these are string =
and not float. Say:
....
>>> accounts=3Dec.fetch('Account')
>>> accounts[0].getAmount()
'0.0'

I know that sqlite stores everything as strings, but it should be =
converted to float in Python, and I don't know where to do this. The =
problem is (suppose account has a name):
>>> accounts[0].setName('sales')
>>> ec.saveChanges()
....
Modeling.Validation.ValidationException: Validation for key saldo =
failed:
- Wrong type
Validation for key OBJECT_WIDE_VALIDATION failed:
- Validation of object <Conta.Cuenta.Cuenta instance at 0x00DC4FE8> as a =
whole failed

because although it has been initialised with 0.0 now it is '0.0' (type =
string).

Thanx in advance,
Erny

[Modeling-users] validateForSave question

[Ernesto R. <er...@si...>]

In the doc: =
http://modeling.sourceforge.net/UserGuide/validate-for-save.html
appears the following text:

<quote>

You can also define your own global validation method, like:=20


        from Modeling import Validation
        def validateForSave(self):
          "Validate "
          error=3DValidation.Exception()
          try:
            CustomObject.validateForSave(self)
          except Validation.Exception, error:
            error.aggregateException(error)
          # Your custom bizness logic goes here
          if self.getFirstName()=3D=3D'John': # No John, except the One
            if self.getLastName()!=3D'Cleese':
              error.aggregateError(Validation.CUSTOM_OBJECT_VALIDATION,
                                   Validation.OBJECT_WIDE_KEY)
          error.finalize() # raises, if appropriate
...
<quote>

1. doing the import and trying a Validation.Exception() fails. I think =
it should be, Validation.ValidationException(), shouldn't it?
2. On the other hand the instruction sequence:

          error=3DValidation.Exception()
          try:
            CustomObject.validateForSave(self)
          except Validation.Exception, error:
            error.aggregateException(error)

  gets me confused. Shouldn't it be as follow?:

          error=3DValidation.Exception()
          try:
            CustomObject.validateForSave(self)
          except Validation.Exception, exc:
            error.aggregateException(exc)

  Note that I catch the exception argument with exc and add it =
afterwards to error with 'aggregateException'.

regards, Erny