Re: [Gramps-devel] Comments on DB API Idea

[Jon C. <jo...@go...>]

I was basically approaching it from the leave gen.lib alone and 
implement a "fully blown" SimpleAccess-esque solution.

At the moment I basically have a 'DB' object which represents an open 
database. This at the moment is populated from a Gramps XML file. This 
is then basically stored as lxml.objectify objects. Internally a graph 
structure is built to represent the linking inside the database (so 
relationships and ref. integrity is made easier).

'DBItem' objects consist of the 'node' data, the basic save/delete 
etc... Deleting an event automatically removes all other references to 
it (which has caught me out previously).

class Person(DBItem):
    DBTYPE = 'person'

Basically registers an object that 'wraps' a basic DBItem, but 
containing useful attributes/methods. So for a person, we can write 
attributes such as .birth, .mother, .families etc... etc... It can also 
over-ride how it should be saved/retrieved etc...

I chose this approach because it keeps the process incremental. We can 
still access the 'raw' data in a DBItem for the stuff I'm not caring 
about at the moment, but someone can write a 'Place' class later for 
instance.

The DB itself is an xpath queryable object (adds a bit of flexibility 
for selections that don't have convenient attributes as of yet).

I'll see if I can get the code example out this week.

Anyway, does this seem a reasonable approach?

Cheers,

Jon.




Benny Malengier wrote:
> This deserves some more discussion, and attention from Gerald and Brian!
>
> The current gen.lib implementation is quite minimal and tied closely
> to the available fields, almost all getters and setters. There has
> been talk to moving to attributes and removing the getters and
> setters, but no time to implement this invasive change which has no
> direct benefit but is much work. The present way is a design decision
> from before my time with the project. Business logic was moved to the
> utility modules. I am not a big fan of this approach as gen.lib is the
> obvious choice to look for some functionality, not some utils.py file.
>
> So, for me too, a higher level functionality in gen.lib would be nice.
> I'm in favour if it can be done cleanly. The current higher level
> methods are in src/Simple/_SimpleAccess.py, and implemented as
> functions. I'd rather see that approach deprecated, and things done in
> gen.lib, and some formatting utilities in something like
> gen.lib.utils. The main roadblock is that gen.lib does not store the
> db one is working on, so one cannot do higher level logic.
> SimpleAccess gets around this because .... it is initialized with the
> db, so functions can use it.
> One of the things I started trying for GEP13 was delayedaccess (aiming
> for speed increase), which among others needs to keep track of an
> 'engine' which much be used for extra access not done during
> initialization. In short, a hidden __db var for objects was something
> I was planning for. I do not like that officially we cannot add db to
> the gen.lib functions, but at the same time the serialize function has
> only one design: reading from a bsddb database.
> I would rather see gen.lib being more "easy" and "inviting" instead of
> a two track approach. What Jon mentions is a way to do that. It
> certainly would be more pythonic, but it needs broad consensus on how
> to achieve this without destroying the simplicity inherent in how
> gen.lib works.
>
> A higher level database access is more tricky, reading
>   for person in db.person(birth=lambda L: 1920 <= L.date.year <= 1985,
> last='CLEMENTS'):
> it might be better to do that in a layer over the lowest database
> access, to not have any speed impacts. Well, it depends on the
> implementation and what you mean with db here. Perhaps something
> wrapping the general db api and extending it with higher level
> functionality. Without seeing code from Jon or thinking about it more,
> can't say how I think this is best done.
>
> Benny
>
> 2009/12/7 Jon Clements <jo...@go...>:
>   
>> Doug Blank wrote:
>>     
>>> On Sun, Dec 6, 2009 at 4:29 PM, Jon Clements <jo...@go...> wrote:
>>>
>>>       
>>>> Hi All,
>>>>
>>>> A colleague of mine has found Gramps extremely useful in recording his
>>>> family tree. He initially requested a few changes to the timeline chart
>>>> to show marriages and births -- which was relatively easy to do. We then
>>>> looked into removing duplicated events (which he'd achieved by importing
>>>> a database or some such thing...)
>>>>
>>>> However, after delving around the API, I found it somewhat
>>>> 'un-inviting'... specifically for purposes of reporting.
>>>>
>>>> I have therefore put in some effort on re-organising the DB API and the
>>>> way it functions. Specifically on the ease of querying and integrity.
>>>>
>>>> Now, to be quite honest, it was an adhoc job, but served our purposes.
>>>> The reason I'm posting here is whether or not it would either 1) be
>>>> useful to the gramps core, or 2) be a useful third party add-on. If
>>>> either of these, then I'll certainly work on the code to tidy it up, and
>>>> do some documentation of usage.
>>>>
>>>> A quick example usage:
>>>>
>>>> jon = db.person(last='CLEMENTS', first='Jon') # retrieve a person or persons
>>>> jon_grandmum = jon.mother.mother # as it says
>>>> jon.mother in jon.mother.mother.children # just to check
>>>> jon.mother.nchildren # only child, so returns 1
>>>>
>>>> It also maintains the referential integrity of the DB, so that deleting
>>>> an event, deletes all objects that refer to the event. It's also xpath
>>>> queryable. It also mimics a graph, so that shortest paths and
>>>> relationships can be worked out. It does have some rough edges though.
>>>>
>>>> It enables you to write something such as:
>>>>
>>>> # Simple query for timeline report
>>>> for person in db.person(birth=lambda L: 1920 <= L.date.year <= 1985,
>>>> last='CLEMENTS'):
>>>>    events = [ev.date.year for ev in
>>>> person.events(include=['birth','death','marriage','child'])
>>>>
>>>> # Check families
>>>> clements = db.family(last='CLEMENTS')
>>>> jon in clements # True
>>>> jon & clements # Returns families I'm a member of
>>>>
>>>> Hope this makes sense, and might be of some use, feedback appreciated.
>>>>
>>>> Cheers,
>>>>
>>>> Jon.
>>>>
>>>>         
>>> Hi, Jon! I'm just one of the contributors, only speaking for myself,
>>> but what you speak of might be interesting to look at. At the least,
>>> you (or someone) could develop it as a third-party library plugin
>>> (what we now call an "addon") which you could build your own reports
>>> on top of. This would allow your colleague to continue to use it, and
>>> reports based on it, as gramps continues to evolve.
>>>
>>> Your interface looks a little bit like Django's API, and we are
>>> actively developing under Django for a web-based version of gramps.
>>> So, that might make it more interesting. On the other hand, it would
>>> need to be able to integrate into what we have in order for us to
>>> commit to maintaining more code.
>>>
>>> If you could make it available someplace, I (for one) would like to
>>> see it (if you have a GPL-compatible license).
>>>
>>> Thanks!
>>>
>>> -Doug
>>>
>>>       
>> Hi Doug,
>>
>> Thanks for the reply and the encouragement, I'll clean up the code, doc.
>> it, and
>> release under a GPL compat. licence. Then you (or someone else) can
>> sanitise it for me.
>> (This may not be before xmas, as obviously I need to do work that pays
>> the bills :( )
>>
>>
>> And yes, I realise that it does look a lot like Django's API (as well as
>> BeautifulSoup's) as I'm an active user of the framework Although I play
>> around with TurboGears and Zope, Django just seems to make the most
>> amount of sense.
>>
>> If you'd like any help with the development of gramps under Django,
>> please let me know -- I'm sure I can put some time aside to help out.
>>
>> When I'm happy enough with it, I'll put my code in SVN with a trac web
>> interface.
>>
>> Cheers,
>>
>> Jon.
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Join us December 9, 2009 for the Red Hat Virtual Experience,
>> a free event focused on virtualization and cloud computing.
>> Attend in-depth sessions from your desk. Your couch. Anywhere.
>> http://p.sf.net/sfu/redhat-sfdev2dev
>> _______________________________________________
>> Gramps-devel mailing list
>> Gra...@li...
>> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>>
>>