Menu
▾
▴
Re: [Gramps-devel] Fwd: dbdjango
|
From: Doug B. <dou...@gm...> - 2009-08-28 14:22:52
|
On Fri, Aug 28, 2009 at 10:07 AM, Gerald Britton <ger...@gm...>wrote: > ---------- Forwarded message ---------- > From: Gerald Britton <ger...@gm...> > Date: Fri, 28 Aug 2009 09:55:59 -0400 > Subject: Re: [Gramps-devel] dbdjango > To: Benny Malengier <ben...@gm...> > > I just wanted to point out that there is more than one way to skin this > cat. > > I do believe that we need to hide the actual storage structure. > That's what prompted me to think about abstracting it as a Python > dictionary. Ultimately I'd like to see simple expressions like: > > x = person[handle] # get person by handle > person[handle] = x # update or add person > > (we need to come up with a consistent naming convention for "person". > Perhaps your idea of making it a method of the Person class is the > right way to go but it needs a bit more discussion I think.) FYI, the Django method of doing that is: x = Person.objects.get(handle=handle) which gets turned into some type of query (usually backend-specific SQL, but not necessarily). I really like the Dango data model infrastructure which allows you to define the schema, value constraints, validation rules, relationships, access widgets, query interface, etc. in one place. GTK forms *could* be linked to these models as well. -Doug > > > This will completely hide the underlying mechanics. Though there will > still be a need for transaction-oriented processing. Most databases > support the concept. Abstracting it is easy to do. > > Along the way, we'll want to make sure that other dictionary methods > work as expected: [iter]keys, [iter]items, [iter]values, get, > setdefault, etc. though maybe not clear (which would effectively > delete everything in the dictionary). > > I've already got a good start on this and hope to have something ready > to show in a few weeks. When complete, we will be able to change the > back end without changing anything anywhere else. Then we can use a > full database (e.g. MySQL) or a remote server or RAM or anything at > all. > > On Fri, Aug 28, 2009 at 9:32 AM, Benny > Malengier<ben...@gm...> wrote: > > 2009/8/28 Gerald Britton <ger...@gm...>: > >> fwiw you can always call > >> > >> .db.person_map[key] or .db.person_map.get(key) which goes directly to > >> the underlying database object. However, _then_ you need to know the > >> names ot the attributes person_map, place_map etc. though they're not > >> hard to remember. I'm also working on an abstraction layer for that > >> access so that the "underlying database" can be anything that supports > >> dictionary-like methods, including, well, a Python dictionary (!) for > >> a simple in-memory database (could be handy for debugging or doing > >> something without leaving a trace on disk.) > > > > I'm not sure you should put so much effort in this or that this is > > even a good thing. Certainly possible. > > > > It's a question of design: how do we want plugin writers to access our > > data? Using the gen.lib methods, or directly via the db methods? There > > is no need to offer two ways of doing things. > > We should encourage users to do > > > > person = db.get_person_from_handle > > gid = person.gramps_id > > > > and not > > > > persondb = db.get_person_map[key] > > gid = persondb[Person.POSGID] > > > > The last will work just fine, but means we propagate the bsddb layout > > scheme into all the plugins of GRAMPS. That is just wrong. We should > > discourage it. > > > > So, we don't want users to access the position in the database map > > outside of gen.db classes, as that is a maintenance nightmare. > > A lookup of Person(key=handle) is interesting in that present > > gen.db.base.py does not need to offer a method to do that, it can just > > provide get_person_from_handle, which dbdir implements as a datamap > > lookup which is pumped in a Person class, while dbdjango would > > implement it as an sql query returning a django model that is pumped > > into a Person class. > > > > At the moment we use in some cases directly db methods for performance > > reasons. However, this just makes it more difficult to change the db > > layout or to plug in another backend. > > > > So if you do dictionary based methods, make sure it is done only in > > implementation of the base methods on a backend that can provide this > > functionality, and don't add it to the 'abstract' api as an sql > > backend might not have the functionality to implement this. > > > > So to wrap up, I see gen.db as a way to obtain gen.lib objects. We > > should remove access to underlying data structures everywhere in > > GRAMPS except in the most performance critical places. If those excist > > (building person view tree model eg), switches depending on the > > backend would be needed if multiple backends are supported. Being able > > to avoid those by changing gen.lib in such a way that obtaining > > gen.lib objects from gen.db is no longer a CPU hogger by using delayed > > access would be a nice improvement. > > > > Benny > > > >> On Fri, Aug 28, 2009 at 6:10 AM, Benny > >> Malengier<ben...@gm...> wrote: > >>> 2009/8/28 Doug Blank <dou...@gm...>: > >>>> > >>>> Yes, that sounds like a viable plan. In fact, such a lazy query lookup > might > >>>> be useful in GRAMPS gtk. (For example, I suspect that deserializing an > >>>> entire Person data might be more than is needed, if you are just > interested > >>>> in one's name. Might be useful to delay parts of the deserialize > process.). > >>> > >>> Why did I not think of that myself? That sounds like a great way to > >>> obtain a free speed boost. On deserialize of the person I do in > >>> unserialize: > >>> self._media_list = datamap[POS_MEDIALIST] > >>> > >>> and only on access of self.media_list the data of self._media_list > >>> itself is deserialized. > >>> > >>> I have to write the default engine anyway, I can do it that way. It > >>> might even mean we can do away with the complicated building of models > >>> for eg the person view directly from the datamap to avoid all the > >>> deserializes (well, probably not on the person view, but in other > >>> places ... ). > >>> > >>> Once there is an engine accessible, one could even go one step further > >>> and not do > >>> person = db.get_person_from_handle(handle). > >>> > >>> But instead add a method to allow for: > >>> > >>> person = Person(key=handle), and the Person object queries the engine > >>> to obtain this person. > >>> Well, that would be a break from how things are done now and probably > >>> another design choice, but the fact a plugin writer needs to know the > >>> gen.db access methods has always struck me as an extra layer of > >>> complexity one has to learn. Anyway, this is not something I'll > >>> pursue, only the delayed access is what I want to add to gen.lib. > >>> > >>> Benny > >>> > >>> > ------------------------------------------------------------------------------ > >>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 > 30-Day > >>> trial. Simplify your report design, integration and deployment - and > focus on > >>> what you do best, core application coding. Discover what's new with > >>> Crystal Reports now. http://p.sf.net/sfu/bobj-july > >>> _______________________________________________ > >>> Gramps-devel mailing list > >>> Gra...@li... > >>> https://lists.sourceforge.net/lists/listinfo/gramps-devel > >>> > >> > >> > >> > >> -- > >> Gerald Britton > >> > > > > > > -- > Gerald Britton > > -- > Sent from my mobile device > > Gerald Britton > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day > trial. Simplify your report design, integration and deployment - and focus > on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > |