Re: [Basket-devel] Technical Discussion About 2.0: Database Storage

[<sl...@li...>]

Le lundi 26 mars 2007, Alex Gontmakher a =E9crit=A0:
> > Out of 12 seconds, the XML document creation of every baskets take 2
> > seconds. This could perhapse be divided by 4 by using Qt 4.3 QXmlReader
> > (a sort of SAX optimized reader and developer-friendly).
>
> OK, the time is down to 6 seconds, out of which the XML creation takes 2.
> What is happening during the other 4 seconds?

Sorry, the 2 seconds are out of _12_ seconds of unmodified code.
What I call "XML document creation" is (routhly):
	loadFromFile(fullPath() + ".basket");
	QDomElement docElem =3D doc->documentElement();
	loadProperties(properties);

Today I've done the lazy-load speed-optimizations I talked about yesterday =
(do=20
not load images, do not instanciate QSimpleRichText objects, do not=20
relayout).

It is in SVN.
Please test it.
I haven't found any bug.
But I prefer more eyes test it before releasing 1.0.2.

Now the results:
* It takes 7 seconds to filter all baskets instead of 12 seconds.
* I restarted the computer and filtered all.
  It takes 15.5 seconds instead of 20.

15.5 seconds.
That's still a lot of time.

The idea of using a database is to do one seek on the disk.
One of both reasons KDE 4 will have a single file cache for icons:

http://aseigo.blogspot.com/2007/03/efficient-scallable-icons.html

=AB the almost-as-obvious: by putting icons into a single file, application
  start up should get faster as we will only need to do the "seek to a mill=
ion
  locations on disk per icon" for that one-time rendering. ok, it's not a
  million locations, but it can still easily be a dozen or so. and since
  moving physical parts =3D=3D slow, this should result in a speed up. in f=
act,
  talking with someone some months ago who had experimented with caching ev=
en
  just the current png's into a single cache file that is loaded on app
  startup showed user noticeable start up improvements. note that this is
  different than suggesting an icon server. =BB

But as said earlier, we need some concrete tests to measure the performance=
=20
gain.

I will study better Berkley DB.
I think we could "virtually move the 2000 files inside one Berkley DB file".
We will benefit of "one big file, very few disk seeks" while still be able =
to=20
refer to the notes without changes: we could do=20
loadFromFile("basket1/note1.txt") and this parameter would be the key in th=
e=20
BDB. Instead of reading from a file, loadFromFile would ask the BDB.
It is cheap on the number of lines of code :-)
Will it perform well? Perhapse.

I will also try to use the KOffice DOM implementation.
It is the exact same API as QDom but it load on the fly, like SAX: it is=20
transparent for developers.

http://wiki.koffice.org/index.php?title=3DKoXmlReader
(see the two blog links at the bottom to see speed comparitions)

This would save time on the 2 seconds. Not a big deal: perhapse it can be=20
reduced to 0.5 second after the change...