Re: [Modeling-users] Lazy Initialization Part 2

[Yannick G. <yan...@sa...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On July 11, 2003 04:25 pm, you wrote:
> 1. Do these times refer to the 30s spent on the initial algorithm? Does
>    this mean half of the time is already saved by prefetching? (I'm not
>    suggesting this is sufficient, I'm just curious there)

The "39.780 CPU seconds" refers to the time given by the python
profiler.  "User time (seconds): 15.18" is taken from /usr/bin/time
with profiling disabled (hence the lower CPU time amount).  In both
case no pre-fetch was done.


> 2. I'm surprised the benefit of MDL_PERMANENT_DB_CONNECTION is so
>    low. That's a very minor point however.

Running with MDL_ENABLE_DATABASE_LOGGING show that DB connection IS
re-used but the lack of improvement is utter mystery for me too.

> 3. How can it be that qualifierWithQualifierFormat() is called *so many
>    times*?? I do not understand that either.

I called it exactly 3 times, the rest is internal to the framework :
   3    0.000    0.000   35.940   11.980
/home/ygingras/modules/ModelManager.py:45(fetch)


>   I guess it's time for you to forget about qualifiers as strings and to
>   learn how to built your own Qualifier instances by hand. It's not that
>   complicated, and you will then avoid all the time lost in parsing the
>   qualifier. Of course, only change the qualifiers that are highly
>   stressed by the fetch of to-many.
>
>     Example:
>
>        ec.fetch('Writer', 'books.id in %s'%pks)
>
>     can be rewritten using:
>
>        from Modeling import Qualifier
>        q=Qualifier.KeyValueQualifier('books.id',
>                                      Qualifier.QualifierOperatorIn,
>                                      pks)
>        ec.fetch('Writer', q)

I thought you said key-value coding was evil  ; )

> > Is there any hope or should I forget about the nice abstraction provided
> > by Modeling and craft my own SQL ?
>
>   *I* really think that there is a solution --at least one that works on
>   the paper, and that I need to test; believe me, I won't say this if I
>   hadn't the strong feeling that it can be improved. But as I already
>   said, I won't have time until tuesday (maybe on monday, but I can't
>   promise for now) to actually test it, so we'll only be sure how good
>   it is then, you'll understand that I can't obviously be absolutely
>   positive before. It will involve two massive queries and no additional
>   fetch afterwards, so we can expect a big improvement.
>
>   I hope you can live with these results until then, and we'll see how
>   good this solution is then.
>
>     Thanks a lot for the figures and the profile log. I'd be interested
>     in the same after you change your qualifiers as suggested above, so
>     we can compare.

Thanks for your quick answers, I'll see what I can do and keep you in touch.

For the sake of completeness, I use :
 - Python 2.2.2
 - Modeling 0.9-pre-9 with the "IN" path
 - MySQL 3.23.56
 - mysql-python 0.9.2

The benchmark was performed on a PIII 1 Ghz with 1 Gb of RAM

- --
Yannick Gingras
Byte Gardener, Savoir-faire Linux inc.
(514) 276-5468


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/Eq6xrhy5Fqn/MRARAiDIAKCLX55h8EEnpvB4vhk78877weex9ACcD0o4
Pca2A2dJOLXNKcKr0cotkvo=
=lK8Q
-----END PGP SIGNATURE-----