You can subscribe to this list here.
2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(35) |
Jun
(11) |
Jul
(41) |
Aug
(96) |
Sep
(29) |
Oct
(44) |
Nov
(70) |
Dec
(61) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2002 |
Jan
(126) |
Feb
(31) |
Mar
(64) |
Apr
(84) |
May
(57) |
Jun
(56) |
Jul
(20) |
Aug
(23) |
Sep
(31) |
Oct
(19) |
Nov
(60) |
Dec
(34) |
2003 |
Jan
(30) |
Feb
(61) |
Mar
(32) |
Apr
(30) |
May
(78) |
Jun
(55) |
Jul
(111) |
Aug
(108) |
Sep
(104) |
Oct
(98) |
Nov
(92) |
Dec
(73) |
2004 |
Jan
(71) |
Feb
(70) |
Mar
(68) |
Apr
(40) |
May
(118) |
Jun
(51) |
Jul
(71) |
Aug
(251) |
Sep
(77) |
Oct
(155) |
Nov
(74) |
Dec
(57) |
2005 |
Jan
(183) |
Feb
(94) |
Mar
(170) |
Apr
(242) |
May
(233) |
Jun
(42) |
Jul
(69) |
Aug
(70) |
Sep
(99) |
Oct
(60) |
Nov
(65) |
Dec
(210) |
2006 |
Jan
(169) |
Feb
(99) |
Mar
(31) |
Apr
(179) |
May
(142) |
Jun
(72) |
Jul
(131) |
Aug
(98) |
Sep
(110) |
Oct
(189) |
Nov
(262) |
Dec
(207) |
2007 |
Jan
(376) |
Feb
(255) |
Mar
(168) |
Apr
(174) |
May
(167) |
Jun
(192) |
Jul
(197) |
Aug
(171) |
Sep
(307) |
Oct
(572) |
Nov
(454) |
Dec
(471) |
2008 |
Jan
(707) |
Feb
(365) |
Mar
(464) |
Apr
(410) |
May
(251) |
Jun
(142) |
Jul
(112) |
Aug
(143) |
Sep
(94) |
Oct
(280) |
Nov
(337) |
Dec
(232) |
2009 |
Jan
(508) |
Feb
(491) |
Mar
(363) |
Apr
(280) |
May
(186) |
Jun
(250) |
Jul
(231) |
Aug
(487) |
Sep
(189) |
Oct
(344) |
Nov
(456) |
Dec
(439) |
2010 |
Jan
(514) |
Feb
(725) |
Mar
(591) |
Apr
(256) |
May
(209) |
Jun
(75) |
Jul
(118) |
Aug
(248) |
Sep
(230) |
Oct
(393) |
Nov
(368) |
Dec
(276) |
2011 |
Jan
(457) |
Feb
(308) |
Mar
(358) |
Apr
(277) |
May
(333) |
Jun
(320) |
Jul
(198) |
Aug
(186) |
Sep
(87) |
Oct
(238) |
Nov
(123) |
Dec
(129) |
2012 |
Jan
(127) |
Feb
(182) |
Mar
(208) |
Apr
(134) |
May
(230) |
Jun
(138) |
Jul
(126) |
Aug
(176) |
Sep
(231) |
Oct
(235) |
Nov
(164) |
Dec
(219) |
2013 |
Jan
(316) |
Feb
(371) |
Mar
(393) |
Apr
(165) |
May
(321) |
Jun
(301) |
Jul
(124) |
Aug
(183) |
Sep
(191) |
Oct
(285) |
Nov
(172) |
Dec
(196) |
2014 |
Jan
(392) |
Feb
(284) |
Mar
(134) |
Apr
(206) |
May
(123) |
Jun
(115) |
Jul
(41) |
Aug
(157) |
Sep
(122) |
Oct
(124) |
Nov
(63) |
Dec
(56) |
2015 |
Jan
(249) |
Feb
(335) |
Mar
(277) |
Apr
(98) |
May
(274) |
Jun
(214) |
Jul
(142) |
Aug
(240) |
Sep
(57) |
Oct
(77) |
Nov
(18) |
Dec
(171) |
2016 |
Jan
(217) |
Feb
(167) |
Mar
(62) |
Apr
(180) |
May
(270) |
Jun
(86) |
Jul
(112) |
Aug
(50) |
Sep
(57) |
Oct
(91) |
Nov
(62) |
Dec
(101) |
2017 |
Jan
(116) |
Feb
(115) |
Mar
(85) |
Apr
(35) |
May
(54) |
Jun
(123) |
Jul
(65) |
Aug
(35) |
Sep
(103) |
Oct
(28) |
Nov
(21) |
Dec
(7) |
2018 |
Jan
(27) |
Feb
(64) |
Mar
(42) |
Apr
(72) |
May
(49) |
Jun
(24) |
Jul
(18) |
Aug
(4) |
Sep
(9) |
Oct
(39) |
Nov
(11) |
Dec
(45) |
2019 |
Jan
(35) |
Feb
(24) |
Mar
(28) |
Apr
(38) |
May
(16) |
Jun
(41) |
Jul
|
Aug
(49) |
Sep
(41) |
Oct
(22) |
Nov
(15) |
Dec
(32) |
2020 |
Jan
(61) |
Feb
(3) |
Mar
(31) |
Apr
(29) |
May
(38) |
Jun
(16) |
Jul
(26) |
Aug
(87) |
Sep
(12) |
Oct
(4) |
Nov
(14) |
Dec
(5) |
2021 |
Jan
(3) |
Feb
(4) |
Mar
(26) |
Apr
(2) |
May
(55) |
Jun
(64) |
Jul
(96) |
Aug
(71) |
Sep
(1) |
Oct
(2) |
Nov
(17) |
Dec
(5) |
2022 |
Jan
(23) |
Feb
(22) |
Mar
(9) |
Apr
(7) |
May
(8) |
Jun
(6) |
Jul
(2) |
Aug
|
Sep
(14) |
Oct
(10) |
Nov
(1) |
Dec
(10) |
2023 |
Jan
(3) |
Feb
|
Mar
|
Apr
(34) |
May
(14) |
Jun
(25) |
Jul
(10) |
Aug
(13) |
Sep
|
Oct
|
Nov
(2) |
Dec
(7) |
2024 |
Jan
(1) |
Feb
(23) |
Mar
(33) |
Apr
(17) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Richard T. <rjt...@th...> - 2007-01-19 11:02:42
|
On Friday 19 January 2007 10:56, ben...@ug... wrote: > Quoting Richard Taylor <rjt...@th...>: > [snip] > > > Benny > > > > Lets be clear, I can see the benefit of a High Level API for > > GrampsDB/RelLib. I think that it could reduce the barrier to people > > writting plugins and that would be a good thing. It might also reduce a > > certain amount of boiler plate code in the core GUI. > > > > I do not see the same benefit from an SQL database backend. Don has > > explained the reasons for this in great detail before and I agree with > > his reasoning. > > > > That being said, I think that a High Level API should concentrate on the > > best presentation of the Gramps data model to the programmer and should > > not concern itself with how it maps to SQL. > > > > So I think that when someone steps forward to implement and maintain > > this High > > Level API they should be thinking about the best API for the programmer, > > built on top of the existing database, and should ignore the SQL backend > > issues. > > > > Regards > > > > Richard > > Richard, > > I mean the interface like one of those api's. The high level api should be > transaction aware, database must be set, objects made, saved, ... > In short, it behaves like one of those api's and the same functionality > must be > present. > Then we are in violent agreement :-) Regards Richard |
From: Richard T. <rjt...@th...> - 2007-01-19 10:48:50
|
On Friday 19 January 2007 09:52, ben...@ug... wrote: > Quoting Richard Taylor <rjt...@th...>: > > If someone is going to embark on the task of constructing a nice Pythonic > > OO interface layer that combines the GrampsDb and RelLib modules then I > > think they should keep it as a layer on top of these and not seek to > > drastically change what we already have. I can see the potential for a > > module structure > > > > like this (switch to fixed width font): > > |-------------| |-------------| > > | Plugin | | Core GUI | > > |-------------| |-------------| > > > > ^ ^ ^ ^ > > > > | v v | > > | --------------- | High Level API > > | > > | |-------------| | > > | | New Model | | > > | |-------------| | > > | > > | ^ ^ | > > > > v v v v > > -------------------------------- Low Level API > > > > |-------------| |-------------| > > | GrampsDB | | RelLib | > > |-------------| |-------------| > > Great. > > I think one could use for the new model one of the interfaces of the > existing SQL object db api's. Eg Django, bazar. The new model would have to > convert the requests to Grampsdb/Rellib. An SQL backend would then mean > making a real django model of gramps with SQL tables, and convert the > requests to this model (so changing the entire low level api, which with > Django is essentially just some config files of how the tables look like, > and let it generate automatically). > > That would be easier than implementing another backend of GrampsDB as SQL, > as that is more lowlevel than an SQL interface is. > > I personally have no problem converting the BSDDB to a relational > database in my > head. > > So, anybody now/has experience with an SQL object model in python, which is > quite stable? > Benny Lets be clear, I can see the benefit of a High Level API for GrampsDB/RelLib. I think that it could reduce the barrier to people writting plugins and that would be a good thing. It might also reduce a certain amount of boiler plate code in the core GUI. I do not see the same benefit from an SQL database backend. Don has explained the reasons for this in great detail before and I agree with his reasoning. That being said, I think that a High Level API should concentrate on the best presentation of the Gramps data model to the programmer and should not concern itself with how it maps to SQL. So I think that when someone steps forward to implement and maintain this High Level API they should be thinking about the best API for the programmer, built on top of the existing database, and should ignore the SQL backend issues. Regards Richard |
From: <ben...@ug...> - 2007-01-19 10:41:39
|
Quoting Richard Taylor <rjt...@th...>: > If someone is going to embark on the task of constructing a nice Pythonic OO > interface layer that combines the GrampsDb and RelLib modules then I think > they should keep it as a layer on top of these and not seek to drastically > change what we already have. I can see the potential for a module structure > like this (switch to fixed width font): > > > |-------------| |-------------| > | Plugin | | Core GUI | > |-------------| |-------------| > ^ ^ ^ ^ > | | | | > | v v | > | --------------- | High Level API > | |-------------| | > | | New Model | | > | |-------------| | > | ^ ^ | > | | | | > v v v v > -------------------------------- Low Level API > |-------------| |-------------| > | GrampsDB | | RelLib | > |-------------| |-------------| Great. I think one could use for the new model one of the interfaces of the existing SQL object db api's. Eg Django, bazar. The new model would have to convert the requests to Grampsdb/Rellib. An SQL backend would then mean making a real django model of gramps with SQL tables, and convert the requests to this model (so changing the entire low level api, which with Django is essentially just some config files of how the tables look like, and let it generate automatically). That would be easier than implementing another backend of GrampsDB as SQL, as that is more lowlevel than an SQL interface is. I personally have no problem converting the BSDDB to a relational database in my head. So, anybody now/has experience with an SQL object model in python, which is quite stable? Benny > There may be a need to augment the Low Level API but the strict separation > should be maintained. This would ensure that we don't have to make large > changes to the existing code and we can still dip into the Low Level API when > optimisation requirements make require it. It would also mean that the New > Model could be targeted at other DB or even RelLib implementations if anyone > ever wanted to do that. > > Regards > > Richard > > > On Thursday 18 January 2007 22:35, ben...@ug... wrote: >> Hi, >> >> first, don't like the GRAMPS 3000 name. Like Python 3000, coming in a >> year we'll >> never see... >> >> second, I don't want to defragment the gramps effort. I'd like to finish my >> plugin (this year), perhaps some webstuff, ... . Nevertheless, foundations >> are important, and if we can add a module to gramps to make life easier, >> why not. >> >> third, I now too little of the GRAMPS database to form an opinion on the >> implementation. I can only argue about how I have to use stuff to write a >> plugin or tool. >> >> My opinion is very different from present GRAMPS implementation. I always >> learned the objects should hide the database. I have done some googling to >> python DB API's to look at examples. >> >> I like things like in django (a MVC model see >> http://en.wikipedia.org/wiki/Model-view-controller): >> See eg: >> http://www.djangoproject.com/documentation/tutorial1/#creating-models So: >> p= Apple(color='red') >> p.save() >> allapples = Apple.objects.all() >> >> Yes, the object hide database. Logic can be added to Apple through methods. >> >> GRAMPS works more like bazaar (found via >> http://wiki.python.org/moin/HigherLevelDatabaseProgramming): >> See eg:http://www.nongnu.org/bazaar/ (scroll down to application code) >> So: >> apple = Apple() >> apple.color = 'red' >> db.add(apple) >> >> There is one big difference however: db here does not itself know it >> contains apples, and not one method in db will have the word apple in it. >> Db can insert, >> del, lookup, .... All geneneral names: >> http://www.nongnu.org/bazaar/public/bazaar.core.Bazaar-class.html >> The advantage: you learn what db does (easy), and you work with the >> objects from >> then on, always in the same way. This corresponds to what a database >> actually is. >> >> Moreover, if you look at bazar (or >> http://modeling.sourceforge.net/UserGuide/), >> you see things that gramps misses, especially easy access to associations, >> like: >> family.father = papa >> del family.father #this breaks the association, does not delete father! >> db.delete(papa) #this can be set up to delete father and all references to >> it family.father = papa >> See also fetching: >> people=db.fetch('People', qualifier='lastName=="Hugo"') >> >> >> The above two models I like, with preference for the first (probably due to >> my programming history), and I understand that both are rooted in the >> relational database world. >> >> So, in short, db.get_main_parents(person), looks wrong for me. In terms of >> Django or Bazaar, I'd like >> Family.objects.get(pk=person.main_parent_family_handle) >> or >> db.get('Family', handle = person.main_parent_family_handle) (I know, this >> is possible in GRAMPS, however, all other methods exist too.) >> >> I don't know, it's just that as a programmer starting in GRAMPS you have >> the impression you need to know RelLib, and that you have to go over all >> grampsdbbase too. >> >> Just some food for thought. The database is probably forced like this due >> to BSDDB, and little can be changed about it. I will have no time to >> seriously implement (or it should be you can wait several years for the >> result ;-) ) >> >> Benny >> >> Quoting Alex Roitman <sh...@gr...>: >> > On Thu, 2007-01-18 at 18:54 +0000, Richard Taylor wrote: >> >> This is not really true, the UI is decoupled from the DB through the >> >> GrampsDB >> >> interface. The use of handles is also not something that gets in the >> >> way of a >> >> DB backed, the handles would just be an indexed column on the tables. >> >> The real issues with a database backend is the mapping of the RelLib >> >> Objects into >> >> a relational schema. The current model serialises the objects and puts >> >> them into the database as key/value pairs where the key is the handle >> >> and the value is the serialised python object. >> > >> > Actually, this is not quite true in 2.2. We serialize a tuple, not a >> > custom python object. Great pains were taken to only serialize tuples >> > of the builtin python objects, that is: numbers, strings, lists, tuples, >> > None, and True/False. That way the saved data is not tied to any >> > particular classes. >> > >> > But to the topic -- yes, these serialized tuples may have nested >> > tuples, and so on. Not a flat table structure. >> > >> >> To take advantage of an SQL database it would be necassary to design >> >> an object >> >> relational mapping for the RelLib objects and to implement the >> >> marshalling/unmarshalling of the objects into that mapping. >> > >> > Exactly. Once you know how to organize the tables, it should be >> > easy to extract data from the tuple. But it has to be on the fly >> > for every access/write. >> > >> >> The other really big job would be to make all the filter/search >> >> functions in gramps use an abstract interface so that they could be >> >> implemented in different ways for different databases. This can be done >> >> but it is a big job. >> >> >> >> I think that you have to be really, really sure that you want to >> >> take this on. >> > >> > Yeah. So it would be a kind of effort that goes into pygtk, which >> > becomes the platform in itself. If/when we're at that point, we >> > could ship python-gramps packages and let everybody use them in whatever >> > programs :-) >> > >> > Cool, but the downside is that it takes an enormous amount of work >> > and maintenance. >> > >> > -- >> > Alexander Roitman http://www.gramps-project.org >> >> ---------------------------------------------------------------- >> This message was sent using IMP, the Internet Messaging Program. >> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share >> your opinions on IT & business topics through brief surveys - and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Gramps-devel mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-devel > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
From: <ben...@ug...> - 2007-01-19 09:30:14
|
Ok, you want to store probably_alive with start-stop and confidence value. Technically, how do you propose to do this? It sounds good, but I do not understand yet how you could do it technically. Let's brainstorm a bit. Cases: 1/born 1-1-1900 2/born 1-1900 3/born 1900 4/about 1-1-1900 5/between 1-11-1899 and 1-4-1901 6/no date given (incurr value from relatives) Problem 1: only look for probably_alive on year? Or allow day/month? Problem 2: what is confidence of the above: 1/ (1900, 1) certain born in 1900 2/ (1900, 1) certain born in 1900 3/ (1900, 1) certain born in 1900 4/ (1899, 0.5) 50% chance alive in 1889 (go 6 months back and forth of day) or (1900, 1) --> if you ask probably alive in 1889 you will not find this person! or (1899-1900, 1) or (1899, 0.5, 1900, 1) --> 0.5 in 1900 so total of 1, so alive in 1900, 1901 5/ (1899, 2/17, 1900, 14/17, 1901, 17/17) 6/ ??? Eg child born in 1-1-1900, then parent without birth event becomes (1850-1870, .3, 1850-1880, .95, 1880-1885, 1) . You could make this different for males and females. I have the impression your probabilities must add up to one, or you start guessing what the missing probability means. To store a medium and deviation would reduce storage. Problem 3: you change a date with a person. What should we do: go over all relationships and update this stored field? Is this feasible? Problem 4: how to add the probabilities: you know somebody alive with probability from 4 events, eg death niece, birth grandchild, birth grandfather and marriage uncle. This will result in one probability calculated from these four events. Now you add a child with a birth date. How to update the info? One would think the child has preference, as it is less removed from the parent. In the same case, you have eg death niece, birth grandchild, birth grandfather and now add a marriage of an uncle. How do you update the existing probabilities. In short, how to do it technically correct and better than is done now, without too large overhead. Benny Quoting "Douglas S. Blank" <db...@cs...>: > Gramps Enhancement Proposal > > Topic: Make probably_alive a core concept > > Motivation > > In almost every aspect of genealogy, having an idea of when someone was > alive is necessary information. It would be very handy to have available > an estimated date for every event. This proposal would move probably_alive > to a field of Person, and add some interface support. > > Details > > Probably_alive could be stored as a date with a start and stop, and > associated confidence values for each start and stop points. These values > could be used with the growing logic in Utils.probably_alive as substitute > values when real values are not known. It would be very handy when sorting > by dates to see these estimations. Also, in the main person view could > show them to help distinguish between people with the same, or similar, > names. Some other points: > > 1. Filters based on probably_alive would be faster. No need to recompute > for a different year, just compare any year to the start/stop points in > person.probably_alive. > > 2. People without birth/death events could use these values in displays > and sortings. > > 3. A different color/symbol would be needed to show that these are gramps > estimations (as different from USER calculations). > > 4. The timeline reports would be more useful as everyone would show up in > their approximate places in time. > > 5. (Almost) every person in the database has some connection to someone > who has a date on an event. This might take a while to recursively find, > but would only need to be done every once in a while. A highly-recursive > update procedure would scour the data looking for evidence from various > directions, and mark probably_alive and the confidence on birth/death. The > user could initiate this process. Or by suggestion: "Your age estimates > need to be updated. This will take about 30 seconds. Can I do that now?" > > 6. A related function is "Age at year N". Often I am looking through > records, and I see that Person A was 29 years old in 1851. Is that one of > my ancestors? Which one? If there were a (virtual) field that showed the > age on a given year, that would make this very easy. > > 7. One downside: if every person has an estimated death date, you will see > an estimation of your own demise :) > > 8. The age span estimates should be configurable/learnable, and for a > particular person, maybe even overridable. For example, you may have some > reason to believe that Person B didn't live as long as Person C. You could > enter manually these estimations (which would still be different from CALC > or EST dates). > > I think that there would be other uses, but that's a good start. > Discussion > > -Doug > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
From: Richard T. <rjt...@th...> - 2007-01-19 09:21:39
|
Doug I am generally in favour of this approach. However, like everything else, it will need active maintainance if it is really to be useful. Someone needs to rationalize the RoadMap section on the wiki with the Feature requests on the bug tracker. Generally the wiki is a bit out of date and needs someone to love it and maintain it. If the GEP initiative is coordinated with a wiki tidy up and is maintained then I think that it could help. Richard On Thursday 18 January 2007 13:47, Douglas S. Blank wrote: > Developers, > > Here is an idea stolen and adapted from Python: GRAMPS Enhancement > Proposals, or GEPs. The idea would be to put far-reaching ideas in a place > where we can easily find, discuss, and weight them. There are many ideas, > feature requests, todos, etc, sprinkled through the mailing list, wiki, > bug tracker, and code. This puts them all in one place. It also provides a > place for new gramps developers to see what has been discussed and the > outcome. > > Here is the basic idea, with two proposals ready for discussion, if you > find the idea useful: > > http://developers.gramps-project.org/tiki-index.php?page=GEPS > > The first proposal, #0, is for the proposal for GEPS itself, to serve as a > template. The second will be the subject of my next message. > > -Doug > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: <ben...@ug...> - 2007-01-19 08:44:47
|
So, the idea is to discuss this on the mailing list, or with a feature, discuss with notes and mail between people, and use the GEP to summarize, yes? I think it is needed to have a place to summarize things, as the bug list goes with added comments (and everybody seems to read only beginning and end, like on a forum), and the threads in the mailing list can be overly long. So yes, I like the idea. However, perhaps better to follow the outline of the PEP's concerning the sections. Can the tiki-wiki contain an outline section?? (I'm not sure tiki wiki is as versatile as what PEP uses, eg code segments. ) Also, main page like http://www.python.org/dev/peps/pep-0000/ perhaps with some sections to divide GEPs in. Also, the extended feature requests I have put at the bottom here: http://developers.gramps-project.org/tiki-index.php?page=UsersWiki Should they be GEP? It really concerns new stuff. Also, should you make with every GEP a feature request in the bug tracker? The advantage there is one links to the other, people (also users) can subscribe for notification, .... Benny Quoting "Douglas S. Blank" <db...@cs...>: > Developers, > > Here is an idea stolen and adapted from Python: GRAMPS Enhancement > Proposals, or GEPs. The idea would be to put far-reaching ideas in a place > where we can easily find, discuss, and weight them. There are many ideas, > feature requests, todos, etc, sprinkled through the mailing list, wiki, > bug tracker, and code. This puts them all in one place. It also provides a > place for new gramps developers to see what has been discussed and the > outcome. > > Here is the basic idea, with two proposals ready for discussion, if you > find the idea useful: > > http://developers.gramps-project.org/tiki-index.php?page=GEPS > > The first proposal, #0, is for the proposal for GEPS itself, to serve as a > template. The second will be the subject of my next message. > > -Doug > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
From: Richard T. <rjt...@th...> - 2007-01-19 08:38:45
|
If someone is going to embark on the task of constructing a nice Pythonic OO interface layer that combines the GrampsDb and RelLib modules then I think they should keep it as a layer on top of these and not seek to drastically change what we already have. I can see the potential for a module structure like this (switch to fixed width font): |-------------| |-------------| | Plugin | | Core GUI | |-------------| |-------------| ^ ^ ^ ^ | | | | | v v | | --------------- | High Level API | |-------------| | | | New Model | | | |-------------| | | ^ ^ | | | | | v v v v -------------------------------- Low Level API |-------------| |-------------| | GrampsDB | | RelLib | |-------------| |-------------| There may be a need to augment the Low Level API but the strict separation should be maintained. This would ensure that we don't have to make large changes to the existing code and we can still dip into the Low Level API when optimisation requirements make require it. It would also mean that the New Model could be targeted at other DB or even RelLib implementations if anyone ever wanted to do that. Regards Richard On Thursday 18 January 2007 22:35, ben...@ug... wrote: > Hi, > > first, don't like the GRAMPS 3000 name. Like Python 3000, coming in a > year we'll > never see... > > second, I don't want to defragment the gramps effort. I'd like to finish my > plugin (this year), perhaps some webstuff, ... . Nevertheless, foundations > are important, and if we can add a module to gramps to make life easier, > why not. > > third, I now too little of the GRAMPS database to form an opinion on the > implementation. I can only argue about how I have to use stuff to write a > plugin or tool. > > My opinion is very different from present GRAMPS implementation. I always > learned the objects should hide the database. I have done some googling to > python DB API's to look at examples. > > I like things like in django (a MVC model see > http://en.wikipedia.org/wiki/Model-view-controller): > See eg: > http://www.djangoproject.com/documentation/tutorial1/#creating-models So: > p= Apple(color='red') > p.save() > allapples = Apple.objects.all() > > Yes, the object hide database. Logic can be added to Apple through methods. > > GRAMPS works more like bazaar (found via > http://wiki.python.org/moin/HigherLevelDatabaseProgramming): > See eg:http://www.nongnu.org/bazaar/ (scroll down to application code) > So: > apple = Apple() > apple.color = 'red' > db.add(apple) > > There is one big difference however: db here does not itself know it > contains apples, and not one method in db will have the word apple in it. > Db can insert, > del, lookup, .... All geneneral names: > http://www.nongnu.org/bazaar/public/bazaar.core.Bazaar-class.html > The advantage: you learn what db does (easy), and you work with the > objects from > then on, always in the same way. This corresponds to what a database > actually is. > > Moreover, if you look at bazar (or > http://modeling.sourceforge.net/UserGuide/), > you see things that gramps misses, especially easy access to associations, > like: > family.father = papa > del family.father #this breaks the association, does not delete father! > db.delete(papa) #this can be set up to delete father and all references to > it family.father = papa > See also fetching: > people=db.fetch('People', qualifier='lastName=="Hugo"') > > > The above two models I like, with preference for the first (probably due to > my programming history), and I understand that both are rooted in the > relational database world. > > So, in short, db.get_main_parents(person), looks wrong for me. In terms of > Django or Bazaar, I'd like > Family.objects.get(pk=person.main_parent_family_handle) > or > db.get('Family', handle = person.main_parent_family_handle) (I know, this > is possible in GRAMPS, however, all other methods exist too.) > > I don't know, it's just that as a programmer starting in GRAMPS you have > the impression you need to know RelLib, and that you have to go over all > grampsdbbase too. > > Just some food for thought. The database is probably forced like this due > to BSDDB, and little can be changed about it. I will have no time to > seriously implement (or it should be you can wait several years for the > result ;-) ) > > Benny > > Quoting Alex Roitman <sh...@gr...>: > > On Thu, 2007-01-18 at 18:54 +0000, Richard Taylor wrote: > >> This is not really true, the UI is decoupled from the DB through the > >> GrampsDB > >> interface. The use of handles is also not something that gets in the > >> way of a > >> DB backed, the handles would just be an indexed column on the tables. > >> The real issues with a database backend is the mapping of the RelLib > >> Objects into > >> a relational schema. The current model serialises the objects and puts > >> them into the database as key/value pairs where the key is the handle > >> and the value is the serialised python object. > > > > Actually, this is not quite true in 2.2. We serialize a tuple, not a > > custom python object. Great pains were taken to only serialize tuples > > of the builtin python objects, that is: numbers, strings, lists, tuples, > > None, and True/False. That way the saved data is not tied to any > > particular classes. > > > > But to the topic -- yes, these serialized tuples may have nested > > tuples, and so on. Not a flat table structure. > > > >> To take advantage of an SQL database it would be necassary to design > >> an object > >> relational mapping for the RelLib objects and to implement the > >> marshalling/unmarshalling of the objects into that mapping. > > > > Exactly. Once you know how to organize the tables, it should be > > easy to extract data from the tuple. But it has to be on the fly > > for every access/write. > > > >> The other really big job would be to make all the filter/search > >> functions in gramps use an abstract interface so that they could be > >> implemented in different ways for different databases. This can be done > >> but it is a big job. > >> > >> I think that you have to be really, really sure that you want to > >> take this on. > > > > Yeah. So it would be a kind of effort that goes into pygtk, which > > becomes the platform in itself. If/when we're at that point, we > > could ship python-gramps packages and let everybody use them in whatever > > programs :-) > > > > Cool, but the downside is that it takes an enormous amount of work > > and maintenance. > > > > -- > > Alexander Roitman http://www.gramps-project.org > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: <ben...@ug...> - 2007-01-18 22:36:22
|
Hi, first, don't like the GRAMPS 3000 name. Like Python 3000, coming in a year we'll never see... second, I don't want to defragment the gramps effort. I'd like to finish my plugin (this year), perhaps some webstuff, ... . Nevertheless, foundations are important, and if we can add a module to gramps to make life easier, why not. third, I now too little of the GRAMPS database to form an opinion on the implementation. I can only argue about how I have to use stuff to write a plugin or tool. My opinion is very different from present GRAMPS implementation. I always learned the objects should hide the database. I have done some googling to python DB API's to look at examples. I like things like in django (a MVC model see http://en.wikipedia.org/wiki/Model-view-controller): See eg: http://www.djangoproject.com/documentation/tutorial1/#creating-models So: p= Apple(color='red') p.save() allapples = Apple.objects.all() Yes, the object hide database. Logic can be added to Apple through methods. GRAMPS works more like bazaar (found via http://wiki.python.org/moin/HigherLevelDatabaseProgramming): See eg:http://www.nongnu.org/bazaar/ (scroll down to application code) So: apple = Apple() apple.color = 'red' db.add(apple) There is one big difference however: db here does not itself know it contains apples, and not one method in db will have the word apple in it. Db can insert, del, lookup, .... All geneneral names: http://www.nongnu.org/bazaar/public/bazaar.core.Bazaar-class.html The advantage: you learn what db does (easy), and you work with the objects from then on, always in the same way. This corresponds to what a database actually is. Moreover, if you look at bazar (or http://modeling.sourceforge.net/UserGuide/), you see things that gramps misses, especially easy access to associations, like: family.father = papa del family.father #this breaks the association, does not delete father! db.delete(papa) #this can be set up to delete father and all references to it family.father = papa See also fetching: people=db.fetch('People', qualifier='lastName=="Hugo"') The above two models I like, with preference for the first (probably due to my programming history), and I understand that both are rooted in the relational database world. So, in short, db.get_main_parents(person), looks wrong for me. In terms of Django or Bazaar, I'd like Family.objects.get(pk=person.main_parent_family_handle) or db.get('Family', handle = person.main_parent_family_handle) (I know, this is possible in GRAMPS, however, all other methods exist too.) I don't know, it's just that as a programmer starting in GRAMPS you have the impression you need to know RelLib, and that you have to go over all grampsdbbase too. Just some food for thought. The database is probably forced like this due to BSDDB, and little can be changed about it. I will have no time to seriously implement (or it should be you can wait several years for the result ;-) ) Benny Quoting Alex Roitman <sh...@gr...>: > On Thu, 2007-01-18 at 18:54 +0000, Richard Taylor wrote: >> This is not really true, the UI is decoupled from the DB through the >> GrampsDB >> interface. The use of handles is also not something that gets in the >> way of a >> DB backed, the handles would just be an indexed column on the tables. The >> real issues with a database backend is the mapping of the RelLib >> Objects into >> a relational schema. The current model serialises the objects and puts them >> into the database as key/value pairs where the key is the handle and the >> value is the serialised python object. > > Actually, this is not quite true in 2.2. We serialize a tuple, not a > custom python object. Great pains were taken to only serialize tuples > of the builtin python objects, that is: numbers, strings, lists, tuples, > None, and True/False. That way the saved data is not tied to any > particular classes. > > But to the topic -- yes, these serialized tuples may have nested > tuples, and so on. Not a flat table structure. > >> To take advantage of an SQL database it would be necassary to design >> an object >> relational mapping for the RelLib objects and to implement the >> marshalling/unmarshalling of the objects into that mapping. > > Exactly. Once you know how to organize the tables, it should be > easy to extract data from the tuple. But it has to be on the fly > for every access/write. > >> The other really big job would be to make all the filter/search functions in >> gramps use an abstract interface so that they could be implemented in >> different ways for different databases. This can be done but it is a >> big job. >> >> I think that you have to be really, really sure that you want to >> take this on. > > Yeah. So it would be a kind of effort that goes into pygtk, which > becomes the platform in itself. If/when we're at that point, we > could ship python-gramps packages and let everybody use them in whatever > programs :-) > > Cool, but the downside is that it takes an enormous amount of work > and maintenance. > > -- > Alexander Roitman http://www.gramps-project.org > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
From: Douglas S. B. <db...@cs...> - 2007-01-18 21:41:23
|
I wrote: [snip] > ... I fully > understand that the current implementation of gramps has a > tightly coupled UI+DB... On Thu, 2007-01-18 at 18:54 +0000, Richard Taylor responded: > This is not really true, the UI is decoupled from the DB through the > GrampsDB interface. The use of handles is also not something that gets > in the way of a DB backed, the handles would just be an indexed column > on the tables. > > The real issues with a database backend is the mapping of the > RelLib Objects into a relational schema. The current model > serialises the objects and puts them into the database as > key/value pairs where the key is the handle and the value is > the serialised python object. To take advantage of an SQL database > it would be necassary to design an object > relational mapping for the RelLib objects and to implement the > marshalling/unmarshalling of the objects into that mapping. Yes, I guess those are the kinds of issues I was thinking of that would make the connecting of such a database onto gramps difficult. > The other really big job would be to make all the filter/search > functions in > gramps use an abstract interface so that they could be implemented in > different ways for different databases. This can be done but it is a > big job. > I think that you have to be really, really sure that you want to > take this on. But this is exactly the part that i do NOT want to take on. All I am proposing at this time is an easy-to-use interface in the database class, and also an experimental SQL interface that also supports this interface. The part about tying the UI and the above issues will be left for a future exercise IF it turns out that people want to use the gtk interface on the SQL (and other) databases. However, I think an experimental SQL interface in this new abstract API could also be useful, even if we never hook the gtk UI to it. For example, one could have import/export from the SQL tables, with some processing in-between. That might be a handy method of manipulating the data for some. -Doug |
From: Alex R. <sh...@gr...> - 2007-01-18 20:10:46
|
On Thu, 2007-01-18 at 18:54 +0000, Richard Taylor wrote: > This is not really true, the UI is decoupled from the DB through the Gram= psDB=20 > interface. The use of handles is also not something that gets in the way = of a=20 > DB backed, the handles would just be an indexed column on the tables. The= =20 > real issues with a database backend is the mapping of the RelLib Objects = into=20 > a relational schema. The current model serialises the objects and puts th= em=20 > into the database as key/value pairs where the key is the handle and the=20 > value is the serialised python object.=20 Actually, this is not quite true in 2.2. We serialize a tuple, not a custom python object. Great pains were taken to only serialize tuples of the builtin python objects, that is: numbers, strings, lists, tuples, None, and True/False. That way the saved data is not tied to any particular classes. But to the topic -- yes, these serialized tuples may have nested tuples, and so on. Not a flat table structure. > To take advantage of an SQL database it would be necassary to design an o= bject=20 > relational mapping for the RelLib objects and to implement the=20 > marshalling/unmarshalling of the objects into that mapping. Exactly. Once you know how to organize the tables, it should be easy to extract data from the tuple. But it has to be on the fly for every access/write. > The other really big job would be to make all the filter/search functions= in=20 > gramps use an abstract interface so that they could be implemented in=20 > different ways for different databases. This can be done but it is a big = job. >=20 > I think that you have to be really, really sure that you want to take thi= s on. Yeah. So it would be a kind of effort that goes into pygtk, which becomes the platform in itself. If/when we're at that point, we could ship python-gramps packages and let everybody use them in whatever programs :-) Cool, but the downside is that it takes an enormous amount of work and maintenance. --=20 Alexander Roitman http://www.gramps-project.org |
From: Richard T. <rjt...@th...> - 2007-01-18 18:51:43
|
Doug On Thursday 18 January 2007 18:31, Douglas S. Blank wrote: > If I can talk Benny into it, with his help (maybe others) I'd like to > begin on a new GEP: codename GRAMPS 3000. This would develop a new > interface in the database class for accessing the data through very > abstract, Pythonic methods. Comments below: > > Alex Roitman wrote: > > On Thu, 2007-01-18 at 09:05 -0500, Douglas S. Blank wrote: > >> Benny said what I was trying to say, only much better. To summarize the > >> main issue, I think it is all about abstraction: > >> > >> Currently, gramps code requires the developer to know HOW the data is > >> organized. > > > > Yes, absolutely! > > There is a very steep learning curve to being able to write even a basic > report. This need not be the case. > I agree with this. > >> A more abstract interface would allow the developer to just get > >> WHAT he/she wants, without regard for the layout of the tables. > >> > >> Another example: > >> > >> data = self.db.getPersonData(key) > >> for family in data["families"]: > >> for p in family["children"]: > >> print p["primary name"] > >> for p in family["parents"]: > >> print p["primary name"] > > > > This looks sweet. > > Sounds like a good project for somebody looking to improve gramps :-) > > I'll never suggest something that I wouldn't be willing to try :) Here > we go... > > > Although the above misses a lot of data. E.g. family["children"] > > apparently gives you person instances. There's more to child ref > > than just the child handle. There's e.g. the child relation to father > > and mother. Do we need similar access for those? Before too long, > > we are going to have a copy of the existing RelLib done through > > the dictionaries (or another API). > > There is no reason why an abstract interface need to miss any > information. We just need to think this through. It may be that some of > the "properties" are method calls, so that the abstract interface isn't > too inefficient. > > >> (of course, you don't have to use dictionaries, but the point is that > >> you should be able to do a lot without ever knowing about all of the > >> handles.) > > > > Should you? We don't mean the user, we mean the programmer. How is > > the programmer supposed to find out about the data, be it RelLib > > or the above interface? Only by reading docs or looking at the code. > > So we won't be able to have the programmer come in and know things > > without knowing how the data is organized. > > The API will have methods to access, add, edit, and delete all of the > data. The first goal would be to make gramps easy to program. For > example, this: > > person["primary name"]["first"] = "Henry" > > might actually be a call to update the necessary tables, with the proper > transactions, and signals being propagated. If this works, the Python > I agree that an interface to the datamodel like this would make it much easier to write plugins that are not interested in performance. However, the implicit commit that is implied by this interface is not good when many updated need to be made at the same time. Like you say, the interface needs lots of thinking about. > prompt might be a very nice gramps interface. Imagine: > >>> from gramps import * > >>> db = GrampsDB.open("myfile.grdb") > >>> lastnames = [] # simplifying a bit; this is a special dict > >>> for person in db["people"]: > > for name in person["names"]: > lastnames[name["last"]] += 1 > > >>> for person in db["people"]: > > if name["last"] == "?" > name["last"] = "Unknown" > > >>> db.close() > >> > >> This would also allow some of the optimizations in other databases to be > >> used. Rather than iterate over the keys at the Python level, SQL (and > >> maybe others) can do that in the lowlevel backend. > > > > I agree that the presently forced key-based data access poses problems > > for porting to SQL. Realistically, I don't think we should get rid > > of that unless and until there's a serious effort to do the SQL > > backend. > > I think I about have a handle on the creation of the SQL tables. I've > been using the latest sqllib in Python, and it is quite lovely. > Everything is in one file, and supports many of the good bits of SQL. > > > If/when there's such an effort, we could see about first making > > a more abstract API and then about using it everywhere. > > I'll consider the SQL backend a part of the GRAMPS 3000 plan. However, > hooking up the GUI to this API will be a separate process. I fully > understand that the current implementation of gramps has a tightly > coupled UI+DB, and that might not have anything to do with creating a > abstract interface for programmers. This is just for ease of use for now. > This is not really true, the UI is decoupled from the DB through the GrampsDB interface. The use of handles is also not something that gets in the way of a DB backed, the handles would just be an indexed column on the tables. The real issues with a database backend is the mapping of the RelLib Objects into a relational schema. The current model serialises the objects and puts them into the database as key/value pairs where the key is the handle and the value is the serialised python object. To take advantage of an SQL database it would be necassary to design an object relational mapping for the RelLib objects and to implement the marshalling/unmarshalling of the objects into that mapping. The other really big job would be to make all the filter/search functions in gramps use an abstract interface so that they could be implemented in different ways for different databases. This can be done but it is a big job. I think that you have to be really, really sure that you want to take this on. Regards Richard |
From: Douglas S. B. <db...@cs...> - 2007-01-18 18:32:03
|
If I can talk Benny into it, with his help (maybe others) I'd like to begin on a new GEP: codename GRAMPS 3000. This would develop a new interface in the database class for accessing the data through very abstract, Pythonic methods. Comments below: Alex Roitman wrote: > On Thu, 2007-01-18 at 09:05 -0500, Douglas S. Blank wrote: >> Benny said what I was trying to say, only much better. To summarize the >> main issue, I think it is all about abstraction: >> >> Currently, gramps code requires the developer to know HOW the data is >> organized. > > Yes, absolutely! There is a very steep learning curve to being able to write even a basic report. This need not be the case. >> A more abstract interface would allow the developer to just get >> WHAT he/she wants, without regard for the layout of the tables. >> >> Another example: >> >> data = self.db.getPersonData(key) >> for family in data["families"]: >> for p in family["children"]: >> print p["primary name"] >> for p in family["parents"]: >> print p["primary name"] > > This looks sweet. > Sounds like a good project for somebody looking to improve gramps :-) I'll never suggest something that I wouldn't be willing to try :) Here we go... > Although the above misses a lot of data. E.g. family["children"] > apparently gives you person instances. There's more to child ref > than just the child handle. There's e.g. the child relation to father > and mother. Do we need similar access for those? Before too long, > we are going to have a copy of the existing RelLib done through > the dictionaries (or another API). There is no reason why an abstract interface need to miss any information. We just need to think this through. It may be that some of the "properties" are method calls, so that the abstract interface isn't too inefficient. >> (of course, you don't have to use dictionaries, but the point is that you >> should be able to do a lot without ever knowing about all of the handles.) > > Should you? We don't mean the user, we mean the programmer. How is > the programmer supposed to find out about the data, be it RelLib > or the above interface? Only by reading docs or looking at the code. > So we won't be able to have the programmer come in and know things > without knowing how the data is organized. The API will have methods to access, add, edit, and delete all of the data. The first goal would be to make gramps easy to program. For example, this: person["primary name"]["first"] = "Henry" might actually be a call to update the necessary tables, with the proper transactions, and signals being propagated. If this works, the Python prompt might be a very nice gramps interface. Imagine: >>> from gramps import * >>> db = GrampsDB.open("myfile.grdb") >>> lastnames = [] # simplifying a bit; this is a special dict >>> for person in db["people"]: for name in person["names"]: lastnames[name["last"]] += 1 >>> for person in db["people"]: if name["last"] == "?" name["last"] = "Unknown" >>> db.close() >> This would also allow some of the optimizations in other databases to be >> used. Rather than iterate over the keys at the Python level, SQL (and >> maybe others) can do that in the lowlevel backend. > > I agree that the presently forced key-based data access poses problems > for porting to SQL. Realistically, I don't think we should get rid > of that unless and until there's a serious effort to do the SQL > backend. I think I about have a handle on the creation of the SQL tables. I've been using the latest sqllib in Python, and it is quite lovely. Everything is in one file, and supports many of the good bits of SQL. > If/when there's such an effort, we could see about first making > a more abstract API and then about using it everywhere. I'll consider the SQL backend a part of the GRAMPS 3000 plan. However, hooking up the GUI to this API will be a separate process. I fully understand that the current implementation of gramps has a tightly coupled UI+DB, and that might not have anything to do with creating a abstract interface for programmers. This is just for ease of use for now. -Doug > Alex > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > > ------------------------------------------------------------------------ > > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: <rom...@ya...> - 2007-01-18 18:04:43
|
> we have managed to make a few improvements in out load times Did you improve the quit times too or just disable the automatic save option for testing !!! I lost the "automatic save on quit" or it is very very fast ... > Check out the current subversion copy and let us know what you think. difference is impressive :) ___________________________________________________________________________ Yahoo! Mail rvente le mail ! Duvrez le nouveau Yahoo! Mail et son interface rlutionnaire. http://fr.mail.yahoo.com |
From: Alex R. <sh...@gr...> - 2007-01-18 17:49:37
|
On Thu, 2007-01-18 at 09:05 -0500, Douglas S. Blank wrote: > Benny said what I was trying to say, only much better. To summarize the > main issue, I think it is all about abstraction: >=20 > Currently, gramps code requires the developer to know HOW the data is > organized. Yes, absolutely! > A more abstract interface would allow the developer to just get > WHAT he/she wants, without regard for the layout of the tables. >=20 > Another example: >=20 > data =3D self.db.getPersonData(key) > for family in data["families"]: > for p in family["children"]: > print p["primary name"] > for p in family["parents"]: > print p["primary name"] This looks sweet. Sounds like a good project for somebody looking to improve gramps :-) Although the above misses a lot of data. E.g. family["children"] apparently gives you person instances. There's more to child ref than just the child handle. There's e.g. the child relation to father and mother. Do we need similar access for those? Before too long, we are going to have a copy of the existing RelLib done through the dictionaries (or another API). > (of course, you don't have to use dictionaries, but the point is that you > should be able to do a lot without ever knowing about all of the handles.= ) Should you? We don't mean the user, we mean the programmer. How is the programmer supposed to find out about the data, be it RelLib or the above interface? Only by reading docs or looking at the code. So we won't be able to have the programmer come in and know things without knowing how the data is organized. > This would also allow some of the optimizations in other databases to be > used. Rather than iterate over the keys at the Python level, SQL (and > maybe others) can do that in the lowlevel backend. I agree that the presently forced key-based data access poses problems for porting to SQL. Realistically, I don't think we should get rid of that unless and until there's a serious effort to do the SQL backend. If/when there's such an effort, we could see about first making a more abstract API and then about using it everywhere. Alex --=20 Alexander Roitman http://www.gramps-project.org |
From: Alex R. <sh...@gr...> - 2007-01-18 17:34:36
|
On Thu, 2007-01-18 at 14:47 +0100, ben...@ug... wrote: > Quoting Alex Roitman <sh...@gr...>: >=20 > [snip] > > One can use GrampsDb and RelLib packages of gramps > > as standalone python packages that are ui-independent, > > and then write another front end. Today :-) >=20 > True, however, the same is not (yet) true for the data/object-model. > For example: to delete a person: this code is part of _PersonView.py,=20 > namely in > method delete_person_response. So the entire logic of deleting a person=20 > is part > of GUI code. >=20 > This is something I struggle with with the code: larger database logic is > repeated everywhere, RelLib only contains basic methods, mimicking the=20 > database > very closely. Eg > family_handle =3D person.get_main_parents_family_handle() > if family_handle: > family =3D self.db.get_family_from_handle(family_handle) > ... do something >=20 > Why not a get_main_parents() and get_all_parents() method ? These methods could not belong to RelLib, because they need database. One could put these methods into the db itself: self.db.get_main_parents(person) Which is not a bad idea. I guess this is the reason that it has not been done: the 2-3 lines it takes to do this has not bothered anybody enough yet. At the same time, there's many slightly different scenarios in various places that are hard to foresee. So we use 2-3 lines here and there, as necessary. > Also, why all the references to db in GUI code. Handle should be just a=20 > key, but > for people new to the code is looks like a special thing. I mean: > p =3D self.db.get_person_from_handle(handle) in stead of p =3D Person(han= dle) > The methods now allow for more optimized access, but in many cases (and f= or > plugin writers specifically) this optimization is not really needed. Not just that: we currently keep objects completely db-agnostic. So Person(handle) is impossible because to find person by the handle one needs the database. The Person object may exist regardless of the database. Honestly, I don't see much difference between p =3D self.db.get_person_from_handle(handle) and p =3D Person(handle) as well as p =3D P(h) > It probably is just style, and I might be missing something... > I can copy a lot of code for the plugins I wrote from other plugins, but = never > as one single method, always as a group containing some logic. I agree with Richard: there would be some merit in having larger chunks written as db methods, or maybe separate utility functions. Maybe in gramps 2.4? > > But the sober reality is: most of gramps code is > > about interface and not data access. So writing > > another frontend is almost like writing another > > application. >=20 > It is about interface, and the logic on how to access the data. In the Ed= it > dialogs, you need to save, like: > trans =3D self.db.transaction_begin() > self.db.commit_media_object(self.obj,trans) > self.db.transaction_commit(trans,_("Edit Media Object")) >=20 > To use in another GUI, obj.save() would be easier. The obj.save() will need db. More like: db.save_media_object(obj,transaction_message) which can be easily done. Alex P.S. The original question was about alternative UI. You have to admit that the 2-3 lines here and there is a very minor part of the UI code. If you manage to write an interface, the few lines to get family from a person or to commit an object is really nothing :-) --=20 Alexander Roitman http://www.gramps-project.org |
From: Alex R. <sh...@gr...> - 2007-01-18 17:19:01
|
Doug, On Thu, 2007-01-18 at 09:41 -0500, Douglas S. Blank wrote: > I've updated from svn, and I do see a very dramatic speedup, but not as > extreme as you are reporting, and I have a faster CPU. >=20 > I get right about 60 seconds (sometimes a little more, sometimes a little > less) with a Pentium 4, 2.66 desktop with 512MB. How long does it take to open same db with 2.4.4 for you? > I wonder what can account for the differences. Most likely, the IO speed. A faster hard drive with larger hd cache helps a lot. Also the CPU cache size may matter, not just the clock rate. Alex --=20 Alexander Roitman http://www.gramps-project.org |
From: Richard T. <rjt...@th...> - 2007-01-18 14:52:38
|
On Thursday 18 January 2007 14:41, Douglas S. Blank wrote: > On Wed, January 17, 2007 11:38 pm, Don Allingham wrote: > > After quite a few discussions, and quite a bit of profiling, we have > > managed to make a few improvements in out load times. Thanks to Doug > > Blank, we got a good test case with 100K people. > > > > All tests are performed 1.7GHz laptop with 512MB of memory. > > > > The 2.2.4 version takes 4 min, 17 seconds (257 seconds) to load the > > database. > > > > The upcoming 2.2.5 version takes 12 seconds. > > Wow! That's stunning! > > I've updated from svn, and I do see a very dramatic speedup, but not as > extreme as you are reporting, and I have a faster CPU. > > I get right about 60 seconds (sometimes a little more, sometimes a little > less) with a Pentium 4, 2.66 desktop with 512MB. > > I wonder what can account for the differences. You're just selecting the > database from the Open Recent list? > You might take a look at tuning your disks with hdparam if you have not already done so. Loading the db requires loading in the surname index and although this is not huge it will be dependant on the access speed of you disks. Richard |
From: Douglas S. B. <db...@cs...> - 2007-01-18 14:41:35
|
On Wed, January 17, 2007 11:38 pm, Don Allingham wrote: > After quite a few discussions, and quite a bit of profiling, we have > managed to make a few improvements in out load times. Thanks to Doug > Blank, we got a good test case with 100K people. > > All tests are performed 1.7GHz laptop with 512MB of memory. > > The 2.2.4 version takes 4 min, 17 seconds (257 seconds) to load the > database. > > The upcoming 2.2.5 version takes 12 seconds. Wow! That's stunning! I've updated from svn, and I do see a very dramatic speedup, but not as extreme as you are reporting, and I have a faster CPU. I get right about 60 seconds (sometimes a little more, sometimes a little less) with a Pentium 4, 2.66 desktop with 512MB. I wonder what can account for the differences. You're just selecting the database from the Open Recent list? But even from 4 minutes to 1 minute is stunning; nice work! Thanks! -Doug > Check out the current subversion copy and let us know what you think. > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV_______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > -- Douglas S. Blank Associate Professor, Bryn Mawr College http://cs.brynmawr.edu/~dblank/ Office: 610 526 601 |
From: Richard T. <rjt...@th...> - 2007-01-18 14:07:25
|
Agreed, I think that these changes are going to take quite a while to sort out. I will start to look at the detail but I won't check anything in. Richard On Thursday 18 January 2007 13:52, Don Allingham wrote: > Richard, > > I like the idea of moving the rendering to the View instead of the > Model. The current scheme was originally done to make the move from 1.0 > to 2.0 easier, since it followed the TreeModel implementation used in > 1.0 (before we used our own custom GenericTreeModel). To be honest, the > idea of moving the rendering into the TreeView never occurred to me :-) > > I think proceeding with this would be a good idea, but lets hold off > checking it in until after 2.2.5. This big of a change needs more than a > week or so for verification. > > While originally we were looking at 2.2.5 this weekend, it looks like it > will probably hold off until next weekend. I want to give the current > round of optimizations a bit of time to be tested and I want to knock > off a few more bugs. > > Don > > On Thu, 2007-01-18 at 10:37 +0000, Richard Taylor wrote: > > Don > > > > I think that it is time that we merged the two model implementations. > > > > I have been looking through the DisplayModels and seeing what should be > > done. > > > > I'll start with a view of the main differences between the approaches. > > > > 1. yours builds an index list when the Model is initialized in > > rebuild_data. Whereas mine does not build an index but keeps a database > > cursor open to act as the index. The advantage of my approach is that it > > never has to iterate through the whole index, the disavantage is that it > > will only work in a modal dialog because the database backend can't deal > > with keeping the cursor open. The advantage of your approach is that it > > works in non-modal dialogs and it allows for ordering of the index other > > than the database order. The ordering is important for non default > > locals. > > > > 2. both models use a similar object cache mechanism. > > > > 3. both models have just about reached the limit of optimisation for > > iter/path mappings. > > > > 4. your model encodes the columns into the model, my model only has one > > column that contains the primary object. In your model the column > > rendering is done using methods in the model whereas in mine the column > > rendering methods are in the View. I am not sure what the advantage of > > your approach is but the advantage of my approach is that you can have > > different View classes that use the same model but display different > > columns. Future enhancements like column reordering, user selectable > > columns etc. are easier to deal with if the column rendering is in the > > view. > > > > 5. because your model has a complete copy of the index it can support > > both filtered and non-filtered modes. My model requires and different > > implementation for the filtered mode. > > > > So to summarize: your model is better because it allows locale based > > sorting, filtering and non-model dialogs. My model is better because it > > separates the job of rendering columns into the View (and it is a bit > > quicker). > > > > So I propose that we refactor your implementation to remove the column > > rendering and put it in the views. This would mean that we can junk the > > stuff in Models/ and I can use your model in the ObjectSelector. (an > > example of the column rendering code can be found in: > > TreeViews/_PersonTreeView.py) > > > > What are your views on this? > > > > Regards > > > > Richard > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share > > your opinions on IT & business topics through brief surveys - and earn > > cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > Gramps-devel mailing list > > Gra...@li... > > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Richard T. <rjt...@th...> - 2007-01-18 14:05:58
|
On Thursday 18 January 2007 13:47, ben...@ug... wrote: > Quoting Alex Roitman <sh...@gr...>: > > [snip] > > > One can use GrampsDb and RelLib packages of gramps > > as standalone python packages that are ui-independent, > > and then write another front end. Today :-) > > True, however, the same is not (yet) true for the data/object-model. > For example: to delete a person: this code is part of _PersonView.py, > namely in > method delete_person_response. So the entire logic of deleting a person > is part > of GUI code. > > This is something I struggle with with the code: larger database logic is > repeated everywhere, RelLib only contains basic methods, mimicking the > database > very closely. Eg > family_handle = person.get_main_parents_family_handle() > if family_handle: > family = self.db.get_family_from_handle(family_handle) > ... do something > > Why not a get_main_parents() and get_all_parents() method ? > > Also, why all the references to db in GUI code. Handle should be just a > key, but > for people new to the code is looks like a special thing. I mean: > p = self.db.get_person_from_handle(handle) in stead of p = Person(handle) > The methods now allow for more optimized access, but in many cases (and for > plugin writers specifically) this optimization is not really needed. > I believe that part of the motivation for this was so that RelLib is independant of the GrampsDB database. So the symantics of the GrampsDB model should not be wrapped up inside of the RelLib. > It probably is just style, and I might be missing something... > I can copy a lot of code for the plugins I wrote from other plugins, but > never as one single method, always as a group containing some logic. > > > But the sober reality is: most of gramps code is > > about interface and not data access. So writing > > another frontend is almost like writing another > > application. > > It is about interface, and the logic on how to access the data. In the Edit > dialogs, you need to save, like: > trans = self.db.transaction_begin() > self.db.commit_media_object(self.obj,trans) > self.db.transaction_commit(trans,_("Edit Media Object")) > > To use in another GUI, obj.save() would be easier. > There may be some milage in considering a general Gramp Data Model abstraction that is built on top of GrampsDB and RelLib and provides all of the expect data model methods on all of the objects. This might help to simplify some of the GUI code and to prevent a certain amount of code replition. I think that any such DataModel would need to have a very carefully designed API so that we don't loose in performance. Although I guess that the performance intensive parts could bypass the model and use the low level API where needed. Richard |
From: Douglas S. B. <db...@cs...> - 2007-01-18 14:05:29
|
Benny said what I was trying to say, only much better. To summarize the main issue, I think it is all about abstraction: Currently, gramps code requires the developer to know HOW the data is organized. A more abstract interface would allow the developer to just get WHAT he/she wants, without regard for the layout of the tables. Another example: data = self.db.getPersonData(key) for family in data["families"]: for p in family["children"]: print p["primary name"] for p in family["parents"]: print p["primary name"] (of course, you don't have to use dictionaries, but the point is that you should be able to do a lot without ever knowing about all of the handles.) This would also allow some of the optimizations in other databases to be used. Rather than iterate over the keys at the Python level, SQL (and maybe others) can do that in the lowlevel backend. -Doug On Thu, January 18, 2007 8:47 am, ben...@ug... wrote: > Quoting Alex Roitman <sh...@gr...>: > > [snip] >> One can use GrampsDb and RelLib packages of gramps >> as standalone python packages that are ui-independent, >> and then write another front end. Today :-) > > True, however, the same is not (yet) true for the data/object-model. > For example: to delete a person: this code is part of _PersonView.py, > namely in > method delete_person_response. So the entire logic of deleting a person > is part > of GUI code. > > This is something I struggle with with the code: larger database logic is > repeated everywhere, RelLib only contains basic methods, mimicking the > database > very closely. Eg > family_handle = person.get_main_parents_family_handle() > if family_handle: > family = self.db.get_family_from_handle(family_handle) > ... do something > > Why not a get_main_parents() and get_all_parents() method ? > > Also, why all the references to db in GUI code. Handle should be just a > key, but > for people new to the code is looks like a special thing. I mean: > p = self.db.get_person_from_handle(handle) in stead of p = Person(handle) > The methods now allow for more optimized access, but in many cases (and > for > plugin writers specifically) this optimization is not really needed. > > It probably is just style, and I might be missing something... > I can copy a lot of code for the plugins I wrote from other plugins, but > never > as one single method, always as a group containing some logic. > >> But the sober reality is: most of gramps code is >> about interface and not data access. So writing >> another frontend is almost like writing another >> application. > > It is about interface, and the logic on how to access the data. In the > Edit > dialogs, you need to save, like: > trans = self.db.transaction_begin() > self.db.commit_media_object(self.obj,trans) > self.db.transaction_commit(trans,_("Edit Media Object")) > > To use in another GUI, obj.save() would be easier. > > Well, just my 2 cents > Benny > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > -- Douglas S. Blank Associate Professor, Bryn Mawr College http://cs.brynmawr.edu/~dblank/ Office: 610 526 601 |
From: Don A. <don...@co...> - 2007-01-18 13:53:50
|
Richard, I like the idea of moving the rendering to the View instead of the Model. The current scheme was originally done to make the move from 1.0 to 2.0 easier, since it followed the TreeModel implementation used in 1.0 (before we used our own custom GenericTreeModel). To be honest, the idea of moving the rendering into the TreeView never occurred to me :-) I think proceeding with this would be a good idea, but lets hold off checking it in until after 2.2.5. This big of a change needs more than a week or so for verification. While originally we were looking at 2.2.5 this weekend, it looks like it will probably hold off until next weekend. I want to give the current round of optimizations a bit of time to be tested and I want to knock off a few more bugs. Don On Thu, 2007-01-18 at 10:37 +0000, Richard Taylor wrote: > Don >=20 > I think that it is time that we merged the two model implementations.=20 >=20 > I have been looking through the DisplayModels and seeing what should be d= one.=20 >=20 > I'll start with a view of the main differences between the approaches. >=20 > 1. yours builds an index list when the Model is initialized in rebuild_da= ta.=20 > Whereas mine does not build an index but keeps a database cursor open to = act=20 > as the index. The advantage of my approach is that it never has to iterat= e=20 > through the whole index, the disavantage is that it will only work in a m= odal=20 > dialog because the database backend can't deal with keeping the cursor op= en.=20 > The advantage of your approach is that it works in non-modal dialogs and = it=20 > allows for ordering of the index other than the database order. The order= ing=20 > is important for non default locals. >=20 > 2. both models use a similar object cache mechanism. >=20 > 3. both models have just about reached the limit of optimisation for iter= /path=20 > mappings. >=20 > 4. your model encodes the columns into the model, my model only has one c= olumn=20 > that contains the primary object. In your model the column rendering is d= one=20 > using methods in the model whereas in mine the column rendering methods a= re=20 > in the View. I am not sure what the advantage of your approach is but the= =20 > advantage of my approach is that you can have different View classes that= use=20 > the same model but display different columns. Future enhancements like co= lumn=20 > reordering, user selectable columns etc. are easier to deal with if the=20 > column rendering is in the view. >=20 > 5. because your model has a complete copy of the index it can support bot= h=20 > filtered and non-filtered modes. My model requires and different=20 > implementation for the filtered mode. >=20 > So to summarize: your model is better because it allows locale based sort= ing,=20 > filtering and non-model dialogs. My model is better because it separates = the=20 > job of rendering columns into the View (and it is a bit quicker). >=20 > So I propose that we refactor your implementation to remove the column=20 > rendering and put it in the views. This would mean that we can junk the s= tuff=20 > in Models/ and I can use your model in the ObjectSelector. (an example of= the=20 > column rendering code can be found in: TreeViews/_PersonTreeView.py) >=20 > What are your views on this? >=20 > Regards >=20 > Richard >=20 > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share y= our > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDEV > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Douglas S. B. <db...@cs...> - 2007-01-18 13:51:05
|
Gramps Enhancement Proposal Topic: Make probably_alive a core concept Motivation In almost every aspect of genealogy, having an idea of when someone was alive is necessary information. It would be very handy to have available an estimated date for every event. This proposal would move probably_alive to a field of Person, and add some interface support. Details Probably_alive could be stored as a date with a start and stop, and associated confidence values for each start and stop points. These values could be used with the growing logic in Utils.probably_alive as substitute values when real values are not known. It would be very handy when sorting by dates to see these estimations. Also, in the main person view could show them to help distinguish between people with the same, or similar, names. Some other points: 1. Filters based on probably_alive would be faster. No need to recompute for a different year, just compare any year to the start/stop points in person.probably_alive. 2. People without birth/death events could use these values in displays and sortings. 3. A different color/symbol would be needed to show that these are gramps estimations (as different from USER calculations). 4. The timeline reports would be more useful as everyone would show up in their approximate places in time. 5. (Almost) every person in the database has some connection to someone who has a date on an event. This might take a while to recursively find, but would only need to be done every once in a while. A highly-recursive update procedure would scour the data looking for evidence from various directions, and mark probably_alive and the confidence on birth/death. The user could initiate this process. Or by suggestion: "Your age estimates need to be updated. This will take about 30 seconds. Can I do that now?" 6. A related function is "Age at year N". Often I am looking through records, and I see that Person A was 29 years old in 1851. Is that one of my ancestors? Which one? If there were a (virtual) field that showed the age on a given year, that would make this very easy. 7. One downside: if every person has an estimated death date, you will see an estimation of your own demise :) 8. The age span estimates should be configurable/learnable, and for a particular person, maybe even overridable. For example, you may have some reason to believe that Person B didn't live as long as Person C. You could enter manually these estimations (which would still be different from CALC or EST dates). I think that there would be other uses, but that's a good start. Discussion -Doug |
From: <ben...@ug...> - 2007-01-18 13:48:26
|
Quoting Alex Roitman <sh...@gr...>: [snip] > One can use GrampsDb and RelLib packages of gramps > as standalone python packages that are ui-independent, > and then write another front end. Today :-) True, however, the same is not (yet) true for the data/object-model. For example: to delete a person: this code is part of _PersonView.py, namely in method delete_person_response. So the entire logic of deleting a person is part of GUI code. This is something I struggle with with the code: larger database logic is repeated everywhere, RelLib only contains basic methods, mimicking the database very closely. Eg family_handle = person.get_main_parents_family_handle() if family_handle: family = self.db.get_family_from_handle(family_handle) ... do something Why not a get_main_parents() and get_all_parents() method ? Also, why all the references to db in GUI code. Handle should be just a key, but for people new to the code is looks like a special thing. I mean: p = self.db.get_person_from_handle(handle) in stead of p = Person(handle) The methods now allow for more optimized access, but in many cases (and for plugin writers specifically) this optimization is not really needed. It probably is just style, and I might be missing something... I can copy a lot of code for the plugins I wrote from other plugins, but never as one single method, always as a group containing some logic. > But the sober reality is: most of gramps code is > about interface and not data access. So writing > another frontend is almost like writing another > application. It is about interface, and the logic on how to access the data. In the Edit dialogs, you need to save, like: trans = self.db.transaction_begin() self.db.commit_media_object(self.obj,trans) self.db.transaction_commit(trans,_("Edit Media Object")) To use in another GUI, obj.save() would be easier. Well, just my 2 cents Benny ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
From: Douglas S. B. <db...@cs...> - 2007-01-18 13:47:09
|
Developers, Here is an idea stolen and adapted from Python: GRAMPS Enhancement Proposals, or GEPs. The idea would be to put far-reaching ideas in a place where we can easily find, discuss, and weight them. There are many ideas, feature requests, todos, etc, sprinkled through the mailing list, wiki, bug tracker, and code. This puts them all in one place. It also provides a place for new gramps developers to see what has been discussed and the outcome. Here is the basic idea, with two proposals ready for discussion, if you find the idea useful: http://developers.gramps-project.org/tiki-index.php?page=GEPS The first proposal, #0, is for the proposal for GEPS itself, to serve as a template. The second will be the subject of my next message. -Doug |