Re: [Modeling-users] Problem with modeling efficency

[Sebastien B. <sbi...@us...>]

        Hi,

<luk...@po...> wrote:
> I have some questions about modeling efficency. Lets consider the followi=
ng
> example:
>=20
> Model:
> (mother lives in one province, in one province lives many mothers)
> [ snipped model: Mother <<------> Province ]
>
> and python code (my orginal code translated from Polish to English:-):
>=20
>         EC=3DEditingContext()
>         mothers=3DEC.fetch('Mothers')
>         for mother in mothers:
>             id=3Dmother.getMothersId()
>             date=3Dmother.getBirthDate()
>=20
>             # efficency problematic code
>             province=3Dmother.gettoProvince().getName()
>=20
> According to the manual everything is all right, and so it works. But... =
in
> database there is over 100 records. When I'm fetching the hole databse and
> get only Id and BirthDate it takes 0.0 seconds - good. But when I try to =
get
> the name of the province by toProvince realtion it takes about 0,5 sec for
> each row! So presenting this small database takes about minute.
>=20
> 1. Am I doing something wrong or this slow working is the feature of
> Modeling Framework?

You're not doing anything wrong. What happens here is that the province
is lazily fetched when needed, meaning that each iteration triggers a
fetch when the line=20
>             province=3Dmother.gettoProvince().getName()

is hit. You can verify this by activating the mdl db-log (env. variable
MDL_ENABLE_DATABASE_LOGGING).

> 2. Is there any explanation for this or another way to get this data but
> much faster?

  There are two ways of making this faster:

1. by default the framework opens a connection to the db, fetches, then
   closes the connection; if you do not want that, set the env. variable=20
   MDL_PERMANENT_DB_CONNECTION to "1" (see details at
   http://modeling.sf.net/UserGuide/env-vars-core.html).

   NB: the reason for this default behaviour originally was that there
       were problems with psycopg in a MT environment and that seemed to
       solve this --but the problem is now solved and there is no reason
       for this to remain the default, we should probably change that.

2. Doing this, the fetch will probably be significantly faster, however
   you still get one fetch per ietration, which is not that efficient
   since you know that you'll need all of them anyway, so you probably
   want to fetch them all.

  We've discussed this some time ago with Yannick (see
https://sf.net/mailarchive/forum.php?thread_id=3D2766300&forum_id=3D10674)
and at that time I made a patch for DBContext working w/ to-many
relationships only. Since in your case you deal w/ to-one relationships
I've updated the patch:

https://sf.net/tracker/index.php?func=3Ddetail&aid=3D771009&group_id=3D5893=
5&atid=3D489337

so you can use it w/ to-one relationship as well. Then you can try it like
that:

------------------------------------------------------------------------
  def databaseContext(entityName, ec):
    from Modeling.FetchSpecification import FetchSpecification
    fs=3DFetchSpecification(entityName)
    return ec.rootObjectStore().objectStoreForFetchSpecification(fs)
=20=20
  from Modeling.ModelSet import defaultModelSet
  dbContext=3DdatabaseContext('Company', ec)
  rel=3DdefaultModelSet().entityNamed('Mothers').relationshipNamed('toProvi=
nce')

  EC=3DEditingContext()
  mothers=3DEC.fetch('Mothers')
=20=20
  # Now fetch all provinces of mothers in a single fetch
  dbContext.batchFetchRelationship(rel, mothers, ec)
=20=20
  for mother in mothers:
      id=3Dmother.getMothersId()
      date=3Dmother.getBirthDate()
=20=20
      province=3Dmother.gettoProvince().getName()
------------------------------------------------------------------------


  This makes the whole fetch even faster (because all the fetching is
  done in a single roundtrip to the db).

BTW is there anyone here still using the original patch for toMany
relationships? (Yannick?)

  I should probably add this feature to the framework, it's been
  regularly asked. Do we want this in the next release?


-- S=E9bastien.


PS: as I said it previously when I sent the first version of the patch,
    this is not the definitive user-friendly interface ;) There will
    probably be a more convenient method at the EditingContext level,
    something like ec.batchFetch(mothers, 'toProvince') e.g.