[Gramps-devel] Questions on PersonView...

[Jérôme <rom...@ya...>]

Hello,



I know that DisplayModels/PersonView is different (own treeview for 
surnames) of the others "Views" (they are using _BaseModel.py).

/////////////////////////////

On DisplayModels/_PeopleModel.py, pylint is unable to import modules 
called by "from ... import..." method.
Modules are loaded, but is there a rule for importing modules ?

from gen.lib import Name, EventRef, EventType, EventRoleType, MarkerType
from BasicUtils import name_displayer
from gen.utils.longop import LongOpStatus
from Filters import SearchFilter, ExactSearchFilter
from Lru import LRU

F: 58: Unable to import 'gen.lib.Name' (No module named gen)
F: 58: Unable to import 'gen.lib.EventRef' (No module named gen)
F: 58: Unable to import 'gen.lib.EventType' (No module named gen)
F: 58: Unable to import 'gen.lib.EventRoleType' (No module named gen)
F: 58: Unable to import 'gen.lib.MarkerType' (No module named gen)
F: 59: Unable to import 'BasicUtils.name_displayer' (No module named 
BasicUtils)
F: 64: Unable to import 'gen.utils.longop.LongOpStatus' (No module named 
gen)
F: 65: Unable to import 'Filters.SearchFilter' (No module named Filters)
F: 65: Unable to import 'Filters.ExactSearchFilter' (No module named 
Filters)
F: 66: Unable to import 'Lru.LRU' (No module named Lru)


///////////////////////////////

On DisplayModels/_PeopleModel.py :

_CACHE_SIZE = 250

     def calculate_data(self, dfilter=None, skip=[]):
         """
         Calculate the new path to node values for the model.
         """
         self.in_build  = True
         self.lru_data  = LRU(_CACHE_SIZE)
         self.lru_name  = LRU(_CACHE_SIZE)
         self.lru_bdate = LRU(_CACHE_SIZE)
         self.lru_ddate = LRU(_CACHE_SIZE)

     def clear_cache(self):
         self.lru_data  = LRU(_CACHE_SIZE)
         self.lru_name  = LRU(_CACHE_SIZE)
         self.lru_bdate = LRU(_CACHE_SIZE)
         self.lru_ddate = LRU(_CACHE_SIZE)

Lru = "Least recently used algorithm"


I found an archive on mailing list (Scrolling page-down in people view 
behaves really funny - november 2006) --- thanks Benny !

> The people view is different from other views in a few regards.
> First is that it is grouped by the surname -- other views are flat.
> 
> Second is that computing the name is very expensive, because
> it goes through the NameDisplay routine to account for the name
> preferences set. This is done for every person displayed.
> In contrast, other views usually display the naked fields of
> the primary records, with little computation needed.
> Also depending on the column settings, the people view
> may display things like the spouse name etc that need yet more
> computation.
> 
> Third is that, because of a huge overhead need for displaying
> the person, starting in 2.2.2 we used the cache. We cache 500
> last used entries, so that the small movements can avoid hitting
> the database. This way you can scroll a page up or down without
> any slowness. I am not sure whether this cacheing is the culprit.
> I suppose it could be, if every time now we need to read 500
> entries instead of less than 50 that we would need to in order
> to draw the screen, but I am not positive. Don is the person
> to enlighten us on this one. 
>
> Alex

> Most of what is happening is a result of GTK. The only thing we can
> really affect is the display of the cells. GTK makes a request of us to
> return the cell's contents. All we do is return the cell contents based
> of the position we are given.
> 
> In fact, until you mentioned it, I was not aware that PgDn worked :-)
> 
> The basic algorithm we have to provide the cell contents is:
> 
> 1) GTK gives us a Node value (row indicator) and a column. We have set
>    things up so that the Node value we are given is our database handle.
> 2) Based on this, we use the column to select the function we call, and
>    pass the database handle to this routine.
> 3) The routine calculates the return value based off the handle.
> 
> We don't have the space to hold all the text strings in memory, so we
> have generate them when they are requested. Normally, this isn't too
> bad. Some functions, in particular are expensive. The Person's name is
> one of these.
> 
> To aid in reducing the time, We have a LRU cache (least recently used)
> which stores the last 500 names that were calculated. So while the old
> algorithm was:
> 
>   def display_name_column( handle ):
>      person = database.find_person_from_handle(handle)
>      return build_name(person)
> 
> The new algorithm is:
> 
>   def display_name_column( handle ):
>      if LRU.contains(handle):
>         return LRU[handle]
>      else:
>         person = database.find_person_from_handle(handle)
>         name = build_name(person)
>         LRU[handle] = name
>         return name
> 
> So, if the person has been recently used, no database look up or name
> calculation is required.
> 
> This is important, because this function gets called all the time - not
> just for display, but for sorting, searching, and everything else. For a
> sort, this can eliminate 1000s of database accesses.
> 
> Based off of this, I can only think of two things which could be causing
> the problem.
> 
>  1) PgDn causes us to miss everything in the cache, and we have to build
>     everything. In this case, our performance should be quite close to
>     2.2.1 or before levels, since we will only pay the cost of a few
>     memory lookups and a few memory assignments. I doubt this is the
>     case, because this can also happen on a basic scroll.
> 
>  2) GTK is doing something strange on a PgUp or PgDn - probably not
>     something related to cell contents, but maybe something odd with
>     screen positioning.
> 
> If this is a significant problem, we can add some tracing calls to try
> to figure out what GTK is doing on a PgUp/PgDn. We had to do this for
> sorting and searching, since they used to be major performance
> bottlenecks (and led to our LRU scheme).
> 
> Don 

I think I know what the cache size is, but why "250" ? If so, can I try 
to use 500 or 1000 ?

I have around 500 surnames and sometimes performance issues (database 
hang, window freeze)

//////////////////////////////

Also pylint returns :
W:247:PeopleModel.__init__: Dangerous default value [] as argument
     def __init__(self, db, filter_info=None, skip=[]):
W:309:PeopleModel.rebuild_data: Dangerous default value [] as argument
     def rebuild_data(self, data_filter=None, skip=[]):
W:379:PeopleModel.calculate_data: Dangerous default value [] as argument
     def calculate_data(self, dfilter=None, skip=[]):

Is it safe ?

/////////////////////////////




Best regards,
Jérôme R.