Re: [Gramps-devel] Profiling the GtkTreeView implemenation

[Richard T. <rjt...@th...>]

Benny

I too have done lots of database coding :-) 

The primary issues with GtkTreeView is the way that it calculates how many 
items there are in the data model. I think that the only reason it needs to 
know this is so that it can size the scrollbar. The model has to implement a 
method called get_iter_next(iter), this takes a reference to a row and 
returns the reference to the next row, when there are no more rows it returns 
None. When you set a model on a GtkTreeView the view repeatably calls 
get_iter_next() until it gets None and it counts how many times it had to 
call it. It uses this as the size of the model. (I my opinion this is 
bonkers, there should be a way to tell the View how many rows there are in 
the model directly, but this is not, at least I have not been able to find 
it.)

Clearly this approach means that the get_iter_next will be called N times for 
a model with N rows (or an index with N entries). So the best that can ever 
be done with GtkTreeView is a algorithmic complexity of N. It also means that 
what goes on in get_iter_next is also crucial. In Models/_FastModel.py I 
developed an approach that means that the implementation of get_iter_next is:

    def on_iter_next(self, rowref):
        """
        Calculate the next iter at the same level in the tree.
        """

        # The length of the rowref (i.e. the number of elements in the path)
        # tells us the level in the tree.
        if len(rowref) == 1:

            # If we are at the top of the tree we just increment the
            # first element in the iter until we reach the total length.
            if rowref[0]+1 >= self._length:
                ret = None
            else:
                ret = [rowref[0]+1,]

        elif len(rowref) == 2:

            # If we are at the second level we first check to see if we
            # have the number of children of this row already in the cache
            if not self._num_children_cache.has_key(rowref[0]):

                # If not calculate the number of children and put it in the 
cac\
he.
                self._num_children_cache[rowref[0]] = 
self._cursor.get_n_childr\
en([rowref[0],])

            num_children = self._num_children_cache[rowref[0]]

            # Now increment the second element of the iter path until we
            # reach the number of children.
            if rowref[1]+1 < num_children:
                ret = [rowref[0],rowref[1]+1]
            else:
                ret = None
        else:
            # We only support two levels.
            ret = None

        return ret

len(rowref) == 1 when drawing the collapsed list and == 2 when expanding an 
element.

You can see from this that, at least for the collapsed case, you can't get 
much better.

The only way to improve on this performance is to implement a custom widget, 
there is one already that uses a ListView and a separate ScrollBar 
(http://www.earthenware-services.org/software/EarthenwareLibrary/Easy%20Grid) 
but it look rather complicated to make use of last time I looked.

An alternative approach might be improve GtkTreeView and submit a patch back 
to gtk, any budding C programmers fancy giving that a go?

Regards

Richard


On Tuesday 16 January 2007 09:10, ben...@ug... wrote:
> Some further ideas in case the present method still fails on large
> databases: In the 90ties I programmed quite some database access. The
> practice at the company I worked was that there was a custom widget
> available for view in a scrolled window (no scrollbar however, only
> pageup/down, or buttons for scrolling, +1,-1,...). Only 3times the amount
> that can be viewed was fetched from database. The rest was only fetched if
> people actually start scrolling, discarding previously fetched data
> This means the window is visible immediately.
> As a programmer you only had to implement 3 methods: something like
> page(nrentriesonpage, keytop), pageup(nrentriesonpage, keytop),
> pagedown(nrentriesonpage, keytop). This then fed the model that was used
> for viewing (not GTK off course).
> Of course, this means quite some changes (find must be implemented
> differently,
> no column sorting), but as an alternative view for large databases it
> should work just fine, and have no noticable time issues.