From: Doug B. <dou...@gm...> - 2012-07-26 14:01:26
|
As you may know, a web-based version of Gramps has been underway for three years. It is now nearing completion! You can always see the latest version running at: http://gramps-connect.org/ Try it! You can view the site in three modes: * not logged in - should see data of people dead and public. If either a person is dead, or any data is marked private, you should not be able to see it. Living people are marked with "[Living]" (as determined in Preferences -> Text). * logged in as regular user, ID: admin1, password: gramps. You can see all data, run reports, export data, but cannot change the database. * logged in as super user, ID: admin, password: gramps. You can edit, delete, add, import, and access Admin pages There are still weeks of programming left, but the majority of functionality is now functional. Highlights: * Uses standard Gramps Database API, so all reports, tools, and plugins will work * Database backend can utilize professional databases (SQL-based) for collaboration and performance on Very Large Trees * Web design is based on CSS; you can change the way it looks under User -> Profile (uses Narrative Web CSS styles) * All file formats (GEDCOM, Gramps XML, etc) can be imported/exported on the website (import files must currently be on the web) * All data is viewable; most is also editable * Notes have markup which can be edited in a WYSIWYG editor * Editing a person requires a "Reason for change"... should that be added to all changes? * All reports can be run from the website (currently PDF output only) * History tab shows all past activity on an item; logging being added throughout Caveats: * Lots of bugs left; still checking integrity * Deleting an item may cascade and delete other related items * Referenced item viewing/editing not implemented * Reordering and deleting list items not complete * Can't upload media yet * No representation of pedigree yet (other than reports) You can also run your own version (testing only!). Details at: http://www.gramps-project.org/wiki/index.php?title=Gramps-Connect Let me know what you like, don't like, and would most like to see. -Doug |
From: Matt K. <mat...@gm...> - 2012-07-26 15:07:10
|
Doug, Looks good, certainly looking forward to this when complete. Will certainly make Gramps available on pretty much any device that has a browser, like my android tablet. Good work. Matt On 26 July 2012 15:01, Doug Blank <dou...@gm...> wrote: > As you may know, a web-based version of Gramps has been underway for > three years. It is now nearing completion! You can always see the > latest version running at: > > http://gramps-connect.org/ > > Try it! You can view the site in three modes: > > * not logged in - should see data of people dead and public. If either > a person is dead, or any data is marked private, you should not be > able to see it. Living people are marked with "[Living]" (as > determined in Preferences -> Text). > > * logged in as regular user, ID: admin1, password: gramps. You can see > all data, run reports, export data, but cannot change the database. > > * logged in as super user, ID: admin, password: gramps. You can edit, > delete, add, import, and access Admin pages > > There are still weeks of programming left, but the majority of > functionality is now functional. > > Highlights: > > * Uses standard Gramps Database API, so all reports, tools, and > plugins will work > * Database backend can utilize professional databases (SQL-based) for > collaboration and performance on Very Large Trees > * Web design is based on CSS; you can change the way it looks under > User -> Profile (uses Narrative Web CSS styles) > * All file formats (GEDCOM, Gramps XML, etc) can be imported/exported > on the website (import files must currently be on the web) > * All data is viewable; most is also editable > * Notes have markup which can be edited in a WYSIWYG editor > * Editing a person requires a "Reason for change"... should that be > added to all changes? > * All reports can be run from the website (currently PDF output only) > * History tab shows all past activity on an item; logging being added throughout > > Caveats: > > * Lots of bugs left; still checking integrity > * Deleting an item may cascade and delete other related items > * Referenced item viewing/editing not implemented > * Reordering and deleting list items not complete > * Can't upload media yet > * No representation of pedigree yet (other than reports) > > You can also run your own version (testing only!). Details at: > > http://www.gramps-project.org/wiki/index.php?title=Gramps-Connect > > Let me know what you like, don't like, and would most like to see. > > -Doug > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel -- HH Golf Society www.hh-gs.com IFLAF www.iflaf.com |
From: Doug B. <dou...@gm...> - 2012-07-26 16:03:02
|
On Thu, Jul 26, 2012 at 11:06 AM, Matt Keenan <mat...@gm...> wrote: > Doug, > > Looks good, certainly looking forward to this when complete. Will > certainly make Gramps available on pretty much any device that has a > browser, like my android tablet. > > Good work. Thanks! Yes, mobile use is one of the excellent side-effects. People have talked about a special app targeting a specific mobile device. But this is better, I think. If we come up with a mobile CSS for smaller phones, even more so. We can probably also do some custom integration between the desktop Gramps version and the server. And think about what Genealogy As A Service functionality we could add on the server... fun days ahead! -Doug > Matt > > On 26 July 2012 15:01, Doug Blank <dou...@gm...> wrote: >> As you may know, a web-based version of Gramps has been underway for >> three years. It is now nearing completion! You can always see the >> latest version running at: >> >> http://gramps-connect.org/ >> >> Try it! You can view the site in three modes: >> >> * not logged in - should see data of people dead and public. If either >> a person is dead, or any data is marked private, you should not be >> able to see it. Living people are marked with "[Living]" (as >> determined in Preferences -> Text). >> >> * logged in as regular user, ID: admin1, password: gramps. You can see >> all data, run reports, export data, but cannot change the database. >> >> * logged in as super user, ID: admin, password: gramps. You can edit, >> delete, add, import, and access Admin pages >> >> There are still weeks of programming left, but the majority of >> functionality is now functional. >> >> Highlights: >> >> * Uses standard Gramps Database API, so all reports, tools, and >> plugins will work >> * Database backend can utilize professional databases (SQL-based) for >> collaboration and performance on Very Large Trees >> * Web design is based on CSS; you can change the way it looks under >> User -> Profile (uses Narrative Web CSS styles) >> * All file formats (GEDCOM, Gramps XML, etc) can be imported/exported >> on the website (import files must currently be on the web) >> * All data is viewable; most is also editable >> * Notes have markup which can be edited in a WYSIWYG editor >> * Editing a person requires a "Reason for change"... should that be >> added to all changes? >> * All reports can be run from the website (currently PDF output only) >> * History tab shows all past activity on an item; logging being added throughout >> >> Caveats: >> >> * Lots of bugs left; still checking integrity >> * Deleting an item may cascade and delete other related items >> * Referenced item viewing/editing not implemented >> * Reordering and deleting list items not complete >> * Can't upload media yet >> * No representation of pedigree yet (other than reports) >> >> You can also run your own version (testing only!). Details at: >> >> http://www.gramps-project.org/wiki/index.php?title=Gramps-Connect >> >> Let me know what you like, don't like, and would most like to see. >> >> -Doug >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Gramps-devel mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-devel > > > > -- > HH Golf Society > www.hh-gs.com > > IFLAF > www.iflaf.com |
From: Doug B. <dou...@gm...> - 2012-07-30 15:39:29
|
On Thu, Jul 26, 2012 at 1:29 PM, Vladimir Perić <vla...@gm...> wrote: > On 7/26/12, Doug Blank <dou...@gm...> wrote: >> As you may know, a web-based version of Gramps has been underway for >> three years. It is now nearing completion! You can always see the >> latest version running at: >> >> http://gramps-connect.org/ >> >> Try it! You can view the site in three modes: >> >> * not logged in - should see data of people dead and public. If either >> a person is dead, or any data is marked private, you should not be >> able to see it. Living people are marked with "[Living]" (as >> determined in Preferences -> Text). >> >> * logged in as regular user, ID: admin1, password: gramps. You can see >> all data, run reports, export data, but cannot change the database. >> >> * logged in as super user, ID: admin, password: gramps. You can edit, >> delete, add, import, and access Admin pages >> >> There are still weeks of programming left, but the majority of >> functionality is now functional. >> >> Highlights: >> >> * Uses standard Gramps Database API, so all reports, tools, and >> plugins will work >> * Database backend can utilize professional databases (SQL-based) for >> collaboration and performance on Very Large Trees >> * Web design is based on CSS; you can change the way it looks under >> User -> Profile (uses Narrative Web CSS styles) >> * All file formats (GEDCOM, Gramps XML, etc) can be imported/exported >> on the website (import files must currently be on the web) >> * All data is viewable; most is also editable >> * Notes have markup which can be edited in a WYSIWYG editor >> * Editing a person requires a "Reason for change"... should that be >> added to all changes? >> * All reports can be run from the website (currently PDF output only) >> * History tab shows all past activity on an item; logging being added >> throughout >> >> Caveats: >> >> * Lots of bugs left; still checking integrity >> * Deleting an item may cascade and delete other related items >> * Referenced item viewing/editing not implemented >> * Reordering and deleting list items not complete >> * Can't upload media yet >> * No representation of pedigree yet (other than reports) >> >> You can also run your own version (testing only!). Details at: >> >> http://www.gramps-project.org/wiki/index.php?title=Gramps-Connect >> >> Let me know what you like, don't like, and would most like to see. > > Hi, this all looks great! I'm really looking forward to it! > > Is there anything we could do to help? Report bugs, develop X or Y, > whatever? I personally won't be free until the end of the month, but > after that I'd love to devote some time to this. Although there are still parts not completed, there are also those that can be tested via the website. In general, if it involves one of the major object types (Person, Family, Media, Source, Citation, Place, Note, Event, Repository, or Tag) then edit/delete/add should all be working. If it is property of one of those (Data, Associations, Alt Locations, LDS, etc) then those are works in progress and may not work and I am aware of those. Things to watch for and report: 1) deleting one item deletes too much. For example, deleting a person should not delete Family, Children, etc., nor vice versa. 2) adding/editing does keep correct results. For example, edit an object, then go to the page again anew, and see if it reflects your edits. 3) I've just started to test some functionality to try to make Gramps-Connect act more like its Gtk counterpart. I'm working with CSS, Ajax, and Json to make look, and function nicely. For example, I've just added combo boxes to select mother and father on a family, and to select a place on an event. These now have auto-complete and search. BTW, this uses a Json/Ajax interface, so that it is conceivable that other tools could take advantage. If you are logged into the site, you can: http://gramps-connect.org/json/?field=mother&q=Elizabeth to find all females who have "Elizabeth" in their names. This is used in the combo box on the Family detail page. You'll also notice the beginnings of some mouse-over menus (on Browse in main menu) and some other features. 4) I want to quickly add some functions for better collaboration: log messages on all addition/edits/deletes, messages, comments for logged in users who can't edit, email other users, full browsing of Log entries, etc. But I have to get the basics done first. Bugs can go to: http://www.gramps-project.org/bugs/main_page.php If anyone has additional skills, please join gramps-devel mailing list and we can discuss further. Thanks! -Doug >> >> -Doug >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Gramps-users mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-users >> > > > -- > Vladimir Perić |
From: Jiri K. <cz1...@gm...> - 2012-08-02 08:45:12
|
On Mon, 30 Jul 2012 11:39:18 -0400, Doug Blank wrote: > If anyone has additional skills, please join gramps-devel mailing list > and we can discuss further. what about using selenium{,-ide} for testing? ev. python-selenium scripts as gramps is pythonic? ########################## EXAMPLE STARTS ############################### from selenium import webdriver driver = webdriver.Chrome() driver.get("http://www.gramps-connect.org/") driver.find_element_by_link_text("Login").click() driver.find_element_by_id("id_username").click() driver.find_element_by_id("id_username").clear() driver.find_element_by_id("id_username").send_keys("admin") driver.find_element_by_id("id_password").clear() driver.find_element_by_id("id_password").send_keys("gramps") driver.find_element_by_css_selector("input[type=\"submit\"]").click() driver.find_element_by_css_selector("tr.odd > td > a").click() driver.find_element_by_link_text("ACHINCLOSS, Hugh").click() driver.find_element_by_xpath("//input[@value='Edit Person']").click() driver.find_element_by_link_text("Logout").click() ########################## EXAMPLE ENDS ################################## > Thanks! > > -Doug > > > |
From: Doug B. <dou...@gm...> - 2012-08-02 13:06:32
|
On Thu, Aug 2, 2012 at 4:33 AM, Jiri Kastner <cz1...@gm...> wrote: > On Mon, 30 Jul 2012 11:39:18 -0400, Doug Blank wrote: > >> If anyone has additional skills, please join gramps-devel mailing list >> and we can discuss further. > > what about using selenium{,-ide} for testing? ev. python-selenium scripts > as gramps is pythonic? > > ########################## EXAMPLE STARTS ############################### > > from selenium import webdriver > driver = webdriver.Chrome() > driver.get("http://www.gramps-connect.org/") > driver.find_element_by_link_text("Login").click() > driver.find_element_by_id("id_username").click() > driver.find_element_by_id("id_username").clear() > driver.find_element_by_id("id_username").send_keys("admin") > driver.find_element_by_id("id_password").clear() > driver.find_element_by_id("id_password").send_keys("gramps") > driver.find_element_by_css_selector("input[type=\"submit\"]").click() > driver.find_element_by_css_selector("tr.odd > td > a").click() > driver.find_element_by_link_text("ACHINCLOSS, Hugh").click() > driver.find_element_by_xpath("//input[@value='Edit Person']").click() > driver.find_element_by_link_text("Logout").click() > > ########################## EXAMPLE ENDS ################################## That looks really useful! That would also help test the web interface (which I wasn't sure could be done) along with the functionality. I'm also adding the ability to have a DictionaryDb copy of the database, and be able to run a diff between them after issuing specific commands. That should help make sure extra items are not deleted. That could be combined with the above. Thanks! -Doug >> Thanks! >> >> -Doug >> >> >> > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Gramps-users mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-users |
From: Doug B. <dou...@gm...> - 2012-08-14 01:03:28
|
On Thu, Aug 2, 2012 at 9:06 AM, Doug Blank <dou...@gm...> wrote: > On Thu, Aug 2, 2012 at 4:33 AM, Jiri Kastner <cz1...@gm...> wrote: >> On Mon, 30 Jul 2012 11:39:18 -0400, Doug Blank wrote: >> >>> If anyone has additional skills, please join gramps-devel mailing list >>> and we can discuss further. >> >> what about using selenium{,-ide} for testing? ev. python-selenium scripts >> as gramps is pythonic? >> >> ########################## EXAMPLE STARTS ############################### >> >> from selenium import webdriver >> driver = webdriver.Chrome() >> driver.get("http://www.gramps-connect.org/") >> driver.find_element_by_link_text("Login").click() >> driver.find_element_by_id("id_username").click() >> driver.find_element_by_id("id_username").clear() >> driver.find_element_by_id("id_username").send_keys("admin") >> driver.find_element_by_id("id_password").clear() >> driver.find_element_by_id("id_password").send_keys("gramps") >> driver.find_element_by_css_selector("input[type=\"submit\"]").click() >> driver.find_element_by_css_selector("tr.odd > td > a").click() >> driver.find_element_by_link_text("ACHINCLOSS, Hugh").click() >> driver.find_element_by_xpath("//input[@value='Edit Person']").click() >> driver.find_element_by_link_text("Logout").click() >> >> ########################## EXAMPLE ENDS ################################## > > That looks really useful! That would also help test the web interface > (which I wasn't sure could be done) along with the functionality. > > I'm also adding the ability to have a DictionaryDb copy of the > database, and be able to run a diff between them after issuing > specific commands. That should help make sure extra items are not > deleted. That could be combined with the above. Just a brief update on some related changes that I have just committed to trunk and gramps35. Each gen.lib object has an additional method called "to_struct". Calling this will return what we might have called a "record" structure 20 years ago, but maybe json now: it returns either a dictionary, list, or value depending on the object. Primary objects look something like: >>> p = db.get_person_from_handle(u'c3adac142e52d52d5a48840601d') >>> p.to_struct() {'parent_family_list': [], 'person_ref_list': [], 'gramps_id': u'I0001', 'handle': 'c3adac142e52d52d5a48840601d', 'media_list': [], 'death_ref_index': -1, 'birth_ref_index': 0, 'family_list': [], 'gender': 1, 'tag_list': (), 'alternate_names': [], 'attribute_list': [], 'lds_ord_list': [], 'private': False, 'primary_name': {'group_as': u'', 'suffix': u'', 'private': False, 'famnick': u'', 'date': None, 'note_list': [], 'first_name': u'Douglas', 'title': u'', 'type': {'string': u'', 'value': 2}, 'display_as': 0, 'nick': u'', 'call': u'', 'surname_list': [{'connector': u'', 'prefix': u'', 'surname': u'Blank', 'primary': True, 'origin_type': {'string': u'', 'value': 1}}], 'citation_list': [], 'sort_as': 0}, 'change': 1344691773, 'urls': [], 'citation_list': [], 'event_ref_list': [{'attribute': [], 'role': {'string': u'', 'value': 1}, 'ref': 'c3b0b20674052c2913ccbcab330', 'note_list': [], 'private': False}], 'note_list': [], 'address_list': []} The keys of the toplevel dictionary also match exactly the associated attribute names. What can you do with this? One thing you can do is dump an object out and see all of the values and related "paths". For example, with just a few lines of code you can take the above struct and produce something like: Person obj.parent_family_list[0] = 'c3b24d3e3352334df33df41a4d3' obj.gramps_id = 'I0811' obj.handle = 'c3b24d46f385580b32ae6202416' obj.death_ref_index = 1 obj.birth_ref_index = 0 obj.family_list[0] = 'c3b24d411bc6d4886d4692ac287' obj.gender = 1 obj.private = False obj.primary_name.group_as = '' obj.primary_name.suffix = '' obj.primary_name.private = False obj.primary_name.famnick = u'' obj.primary_name.date.sortval = 0 obj.primary_name.date.text = u'' obj.primary_name.date.newyear = 0 obj.primary_name.date.calendar = 0 obj.primary_name.date.modifier = 0 obj.primary_name.date.quality = 0 obj.primary_name.date.dateval[0] = 0 obj.primary_name.date.dateval[1] = 0 obj.primary_name.date.dateval[2] = 0 obj.primary_name.date.dateval[3] = False obj.primary_name.first_name = u'William' obj.primary_name.title = '' obj.primary_name.type.string = 'Birth Name' obj.primary_name.type.value = 2 obj.primary_name.display_as = 0 obj.primary_name.nick = u'' obj.primary_name.call = u'' obj.primary_name.surname_list[0].connector = '' obj.primary_name.surname_list[0].origintype.string = '' obj.primary_name.surname_list[0].origintype.value = 1 obj.primary_name.surname_list[0].prefix = '' obj.primary_name.surname_list[0].surname = u'Boucher' obj.primary_name.surname_list[0].primary = True obj.primary_name.sort_as = 0 obj.change = 1185438865 obj.citation_list[0] = 'c3b24d46f424004ca03e649f2d0' obj.event_ref_list[0].role.string = 'Primary' obj.event_ref_list[0].role.value = 1 obj.event_ref_list[0].ref = 'c3b24d38f625277c6ec78c01623' obj.event_ref_list[0].private = False obj.event_ref_list[1].role.string = 'Primary' obj.event_ref_list[1].role.value = 1 obj.event_ref_list[1].ref = 'c3b24d38f724e7cdd6f52151e3' obj.event_ref_list[1].private = False obj.event_ref_list[2].role.string = 'Primary' obj.event_ref_list[2].role.value = 1 obj.event_ref_list[2].ref = 'c3b24d38f8134e638f090438796' obj.event_ref_list[2].private = False (The "obj.attr.attr..." path is constructed on the fly. BTW, if you eval(path) you get the same value as what's in the struct, on the right hand side of the equal sign.) Now, what this is truly useful for is doing a diff and merge. I have a addon report that produces a list of every difference between the current databases and a exported version (GEDCOM, Gramps XML, etc). It reads the exported file into a DictionaryDB. Currently, a diffing with a GEDCOM produces 100% differences because the UID/handles are different and those are the unique IDs (could use gramps_ids, alternatively, FYI). But diffing with a Gramps XML file gives a useful report (see attached [1]). The report is useful in testing Gramps-Connect because Django's SQL ORM defaults to cascading deletes, which might make a Family disappear if you delete a child if you don't have it setup correctly. This report will be more useful when it becomes an interactive Merge UI. In any event, the to_struct interface might make it much more easy to write some code where one might like to programmatically query objects and their values, perhaps for testing, a table view, or SQL-like query interface etc. (see src/gen/merge/diff.py for other examples). -Doug [1] - Too big. See: http://www.gramps-project.org/wiki/images/6/6e/Database-diff-report.png > Thanks! > > -Doug > >>> Thanks! >>> >>> -Doug >>> >>> >>> >> >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Gramps-users mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-users |
From: Benny M. <ben...@gm...> - 2012-08-14 08:05:18
|
Doug, I see in the code, you wrote doc """ Convert the object to a serialized struct of data. """ and then in person.py no longer a doc string. For gen.lib, it is very important there are good doc strings, so please update that. Make the docstring such that it is very clear what this method is for, and what it should produce, so you don't have to read the code to understand what to do when there is a database change. Make it such that it reads nicely in http://gramps-project.org/docs/api.html#module-gen In person.py, what is the FIELDS for? Again no doc string to understand, and I don't see it used in person.py If FIELDS is used somewhere, do we really want them like that with the underscores? To not type FIELDS, you could do: Person().to_struct().iterkeys() which would avoid typing FIELDS everywhere and making an error. About implementation. I would expect it to be a dictionary of dictionaries, and on lowest level strings or integer, but sometimes it is list, because it has to be ordered. Well, I don't like that "citation_list": CitationBase.to_struct(self) is a list, but other to_structs are dicts (wrong doc of the method there by the way). I think these objects better have no to_struct method, and you just write the attribute you need. to_struct being list is counterintuitive. To keep the struction of dictionary of dictionary, you could use ordered dict http://docs.python.org/library/collections.html#collections.OrderedDict which is in python 2.7 which trunk depends on. But then you need a key of some sort, which we don't have. I don't understand how from_struct works self.citation_list = [CitationBase.from_struct(cref) for cref in to_struct["citation_list"]] There is no from_struct class method in CitationBase that I find. Anyway, both are a list anyway, no, so self.citation_list = to_struct["citation_list"] no from_struct needed, just as to_struct was not needed. Benny 2012/8/14 Doug Blank <dou...@gm...> > On Thu, Aug 2, 2012 at 9:06 AM, Doug Blank <dou...@gm...> wrote: > > On Thu, Aug 2, 2012 at 4:33 AM, Jiri Kastner <cz1...@gm...> wrote: > >> On Mon, 30 Jul 2012 11:39:18 -0400, Doug Blank wrote: > >> > >>> If anyone has additional skills, please join gramps-devel mailing list > >>> and we can discuss further. > >> > >> what about using selenium{,-ide} for testing? ev. python-selenium > scripts > >> as gramps is pythonic? > >> > >> ########################## EXAMPLE STARTS > ############################### > >> > >> from selenium import webdriver > >> driver = webdriver.Chrome() > >> driver.get("http://www.gramps-connect.org/") > >> driver.find_element_by_link_text("Login").click() > >> driver.find_element_by_id("id_username").click() > >> driver.find_element_by_id("id_username").clear() > >> driver.find_element_by_id("id_username").send_keys("admin") > >> driver.find_element_by_id("id_password").clear() > >> driver.find_element_by_id("id_password").send_keys("gramps") > >> driver.find_element_by_css_selector("input[type=\"submit\"]").click() > >> driver.find_element_by_css_selector("tr.odd > td > a").click() > >> driver.find_element_by_link_text("ACHINCLOSS, Hugh").click() > >> driver.find_element_by_xpath("//input[@value='Edit Person']").click() > >> driver.find_element_by_link_text("Logout").click() > >> > >> ########################## EXAMPLE ENDS > ################################## > > > > That looks really useful! That would also help test the web interface > > (which I wasn't sure could be done) along with the functionality. > > > > I'm also adding the ability to have a DictionaryDb copy of the > > database, and be able to run a diff between them after issuing > > specific commands. That should help make sure extra items are not > > deleted. That could be combined with the above. > > Just a brief update on some related changes that I have just committed > to trunk and gramps35. Each gen.lib object has an additional method > called "to_struct". Calling this will return what we might have called > a "record" structure 20 years ago, but maybe json now: it returns > either a dictionary, list, or value depending on the object. Primary > objects look something like: > > >>> p = db.get_person_from_handle(u'c3adac142e52d52d5a48840601d') > >>> p.to_struct() > {'parent_family_list': [], 'person_ref_list': [], 'gramps_id': > u'I0001', 'handle': 'c3adac142e52d52d5a48840601d', 'media_list': [], > 'death_ref_index': -1, 'birth_ref_index': 0, 'family_list': [], > 'gender': 1, 'tag_list': (), 'alternate_names': [], 'attribute_list': > [], 'lds_ord_list': [], 'private': False, 'primary_name': {'group_as': > u'', 'suffix': u'', 'private': False, 'famnick': u'', 'date': None, > 'note_list': [], 'first_name': u'Douglas', 'title': u'', 'type': > {'string': u'', 'value': 2}, 'display_as': 0, 'nick': u'', 'call': > u'', 'surname_list': [{'connector': u'', 'prefix': u'', 'surname': > u'Blank', 'primary': True, 'origin_type': {'string': u'', 'value': > 1}}], 'citation_list': [], 'sort_as': 0}, 'change': 1344691773, > 'urls': [], 'citation_list': [], 'event_ref_list': [{'attribute': [], > 'role': {'string': u'', 'value': 1}, 'ref': > 'c3b0b20674052c2913ccbcab330', 'note_list': [], 'private': False}], > 'note_list': [], 'address_list': []} > > The keys of the toplevel dictionary also match exactly the associated > attribute names. What can you do with this? One thing you can do is > dump an object out and see all of the values and related "paths". For > example, with just a few lines of code you can take the above struct > and produce something like: > > Person > obj.parent_family_list[0] = 'c3b24d3e3352334df33df41a4d3' > obj.gramps_id = 'I0811' > obj.handle = 'c3b24d46f385580b32ae6202416' > obj.death_ref_index = 1 > obj.birth_ref_index = 0 > obj.family_list[0] = 'c3b24d411bc6d4886d4692ac287' > obj.gender = 1 > obj.private = False > obj.primary_name.group_as = '' > obj.primary_name.suffix = '' > obj.primary_name.private = False > obj.primary_name.famnick = u'' > obj.primary_name.date.sortval = 0 > obj.primary_name.date.text = u'' > obj.primary_name.date.newyear = 0 > obj.primary_name.date.calendar = 0 > obj.primary_name.date.modifier = 0 > obj.primary_name.date.quality = 0 > obj.primary_name.date.dateval[0] = 0 > obj.primary_name.date.dateval[1] = 0 > obj.primary_name.date.dateval[2] = 0 > obj.primary_name.date.dateval[3] = False > obj.primary_name.first_name = u'William' > obj.primary_name.title = '' > obj.primary_name.type.string = 'Birth Name' > obj.primary_name.type.value = 2 > obj.primary_name.display_as = 0 > obj.primary_name.nick = u'' > obj.primary_name.call = u'' > obj.primary_name.surname_list[0].connector = '' > obj.primary_name.surname_list[0].origintype.string = '' > obj.primary_name.surname_list[0].origintype.value = 1 > obj.primary_name.surname_list[0].prefix = '' > obj.primary_name.surname_list[0].surname = u'Boucher' > obj.primary_name.surname_list[0].primary = True > obj.primary_name.sort_as = 0 > obj.change = 1185438865 > obj.citation_list[0] = 'c3b24d46f424004ca03e649f2d0' > obj.event_ref_list[0].role.string = 'Primary' > obj.event_ref_list[0].role.value = 1 > obj.event_ref_list[0].ref = 'c3b24d38f625277c6ec78c01623' > obj.event_ref_list[0].private = False > obj.event_ref_list[1].role.string = 'Primary' > obj.event_ref_list[1].role.value = 1 > obj.event_ref_list[1].ref = 'c3b24d38f724e7cdd6f52151e3' > obj.event_ref_list[1].private = False > obj.event_ref_list[2].role.string = 'Primary' > obj.event_ref_list[2].role.value = 1 > obj.event_ref_list[2].ref = 'c3b24d38f8134e638f090438796' > obj.event_ref_list[2].private = False > > (The "obj.attr.attr..." path is constructed on the fly. BTW, if you > eval(path) you get the same value as what's in the struct, on the > right hand side of the equal sign.) > > Now, what this is truly useful for is doing a diff and merge. I have a > addon report that produces a list of every difference between the > current databases and a exported version (GEDCOM, Gramps XML, etc). It > reads the exported file into a DictionaryDB. Currently, a diffing with > a GEDCOM produces 100% differences because the UID/handles are > different and those are the unique IDs (could use gramps_ids, > alternatively, FYI). But diffing with a Gramps XML file gives a useful > report (see attached [1]). The report is useful in testing Gramps-Connect > because Django's SQL ORM defaults to cascading deletes, which might > make a Family disappear if you delete a child if you don't have it > setup correctly. This report will be more useful when it becomes an > interactive Merge UI. > > In any event, the to_struct interface might make it much more easy to > write some code where one might like to programmatically query objects > and their values, perhaps for testing, a table view, or SQL-like query > interface etc. (see src/gen/merge/diff.py for other examples). > > -Doug > > [1] - Too big. See: > http://www.gramps-project.org/wiki/images/6/6e/Database-diff-report.png > > > Thanks! > > > > -Doug > > > >>> Thanks! > >>> > >>> -Doug > >>> > >>> > >>> > >> > >> > >> > >> > ------------------------------------------------------------------------------ > >> Live Security Virtual Conference > >> Exclusive live event will cover all the ways today's security and > >> threat landscape has changed and how IT managers can respond. > Discussions > >> will include endpoint security, mobile security and the latest in > malware > >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > >> _______________________________________________ > >> Gramps-users mailing list > >> Gra...@li... > >> https://lists.sourceforge.net/lists/listinfo/gramps-users > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > |
From: Doug B. <dou...@gm...> - 2012-08-14 11:42:08
|
On Tue, Aug 14, 2012 at 4:05 AM, Benny Malengier <ben...@gm...> wrote: > Doug, Thanks for the feedback! Comments below. > I see in the code, you wrote doc > > """ > Convert the object to a serialized struct of data. > """ > > and then in person.py no longer a doc string. > > For gen.lib, it is very important there are good doc strings, so please > update that. Make the docstring such that it is very clear what this method > is for, and what it should produce, so you don't have to read the code to > understand what to do when there is a database change. Make it such that it > reads nicely in > http://gramps-project.org/docs/api.html#module-gen Will do. > In person.py, what is the FIELDS for? Again no doc string to understand, and > I don't see it used in person.py > If FIELDS is used somewhere, do we really want them like that with the > underscores? > To not type FIELDS, you could do: > > Person().to_struct().iterkeys() > > which would avoid typing FIELDS everywhere and making an error. I was trying to decide whether a separate FIELDS would be worth it (to_struct gathers all of the associated data too, which can be expensive). But, I think until we find it too expensive, your method, as you say, will ensure consistency. I'll remove FIELDS. > About implementation. I would expect it to be a dictionary of dictionaries, > and on lowest level strings or integer, but sometimes it is list, because it > has to be ordered. > Well, I don't like that "citation_list": CitationBase.to_struct(self) is a > list, but other to_structs are dicts (wrong doc of the method there by the > way). I think these objects better have no to_struct method, and you just > write the attribute you need. > > to_struct being list is counterintuitive. > > To keep the struction of dictionary of dictionary, you could use ordered > dict http://docs.python.org/library/collections.html#collections.OrderedDict > which is in python 2.7 which trunk depends on. But then you need a key of > some sort, which we don't have. I considered moving the functionality to Python 2.7, but gramps35 still runs happily on older Pythons. I called it "to_struct" rather than "to_dict" because it does give structures. I'll look for an implementation of ordereddict, and try to unify to all dicts. > I don't understand how from_struct works > self.citation_list = [CitationBase.from_struct(cref) > for cref in to_struct["citation_list"]] > > There is no from_struct class method in CitationBase that I find. Anyway, > both are a list anyway, no, so > self.citation_list = to_struct["citation_list"] > no from_struct needed, just as to_struct was not needed. That wouldn't work because to_struct["citation_list"] is a list of structs. But from_struct should not have been included yet. I've been working on this for a long time, and have gone through a few iterations trying to get it just right. It is almost there. Thanks! -Doug > Benny > 2012/8/14 Doug Blank <dou...@gm...> >> >> On Thu, Aug 2, 2012 at 9:06 AM, Doug Blank <dou...@gm...> wrote: >> > On Thu, Aug 2, 2012 at 4:33 AM, Jiri Kastner <cz1...@gm...> wrote: >> >> On Mon, 30 Jul 2012 11:39:18 -0400, Doug Blank wrote: >> >> >> >>> If anyone has additional skills, please join gramps-devel mailing list >> >>> and we can discuss further. >> >> >> >> what about using selenium{,-ide} for testing? ev. python-selenium >> >> scripts >> >> as gramps is pythonic? >> >> >> >> ########################## EXAMPLE STARTS >> >> ############################### >> >> >> >> from selenium import webdriver >> >> driver = webdriver.Chrome() >> >> driver.get("http://www.gramps-connect.org/") >> >> driver.find_element_by_link_text("Login").click() >> >> driver.find_element_by_id("id_username").click() >> >> driver.find_element_by_id("id_username").clear() >> >> driver.find_element_by_id("id_username").send_keys("admin") >> >> driver.find_element_by_id("id_password").clear() >> >> driver.find_element_by_id("id_password").send_keys("gramps") >> >> driver.find_element_by_css_selector("input[type=\"submit\"]").click() >> >> driver.find_element_by_css_selector("tr.odd > td > a").click() >> >> driver.find_element_by_link_text("ACHINCLOSS, Hugh").click() >> >> driver.find_element_by_xpath("//input[@value='Edit Person']").click() >> >> driver.find_element_by_link_text("Logout").click() >> >> >> >> ########################## EXAMPLE ENDS >> >> ################################## >> > >> > That looks really useful! That would also help test the web interface >> > (which I wasn't sure could be done) along with the functionality. >> > >> > I'm also adding the ability to have a DictionaryDb copy of the >> > database, and be able to run a diff between them after issuing >> > specific commands. That should help make sure extra items are not >> > deleted. That could be combined with the above. >> >> Just a brief update on some related changes that I have just committed >> to trunk and gramps35. Each gen.lib object has an additional method >> called "to_struct". Calling this will return what we might have called >> a "record" structure 20 years ago, but maybe json now: it returns >> either a dictionary, list, or value depending on the object. Primary >> objects look something like: >> >> >>> p = db.get_person_from_handle(u'c3adac142e52d52d5a48840601d') >> >>> p.to_struct() >> {'parent_family_list': [], 'person_ref_list': [], 'gramps_id': >> u'I0001', 'handle': 'c3adac142e52d52d5a48840601d', 'media_list': [], >> 'death_ref_index': -1, 'birth_ref_index': 0, 'family_list': [], >> 'gender': 1, 'tag_list': (), 'alternate_names': [], 'attribute_list': >> [], 'lds_ord_list': [], 'private': False, 'primary_name': {'group_as': >> u'', 'suffix': u'', 'private': False, 'famnick': u'', 'date': None, >> 'note_list': [], 'first_name': u'Douglas', 'title': u'', 'type': >> {'string': u'', 'value': 2}, 'display_as': 0, 'nick': u'', 'call': >> u'', 'surname_list': [{'connector': u'', 'prefix': u'', 'surname': >> u'Blank', 'primary': True, 'origin_type': {'string': u'', 'value': >> 1}}], 'citation_list': [], 'sort_as': 0}, 'change': 1344691773, >> 'urls': [], 'citation_list': [], 'event_ref_list': [{'attribute': [], >> 'role': {'string': u'', 'value': 1}, 'ref': >> 'c3b0b20674052c2913ccbcab330', 'note_list': [], 'private': False}], >> 'note_list': [], 'address_list': []} >> >> The keys of the toplevel dictionary also match exactly the associated >> attribute names. What can you do with this? One thing you can do is >> dump an object out and see all of the values and related "paths". For >> example, with just a few lines of code you can take the above struct >> and produce something like: >> >> Person >> obj.parent_family_list[0] = 'c3b24d3e3352334df33df41a4d3' >> obj.gramps_id = 'I0811' >> obj.handle = 'c3b24d46f385580b32ae6202416' >> obj.death_ref_index = 1 >> obj.birth_ref_index = 0 >> obj.family_list[0] = 'c3b24d411bc6d4886d4692ac287' >> obj.gender = 1 >> obj.private = False >> obj.primary_name.group_as = '' >> obj.primary_name.suffix = '' >> obj.primary_name.private = False >> obj.primary_name.famnick = u'' >> obj.primary_name.date.sortval = 0 >> obj.primary_name.date.text = u'' >> obj.primary_name.date.newyear = 0 >> obj.primary_name.date.calendar = 0 >> obj.primary_name.date.modifier = 0 >> obj.primary_name.date.quality = 0 >> obj.primary_name.date.dateval[0] = 0 >> obj.primary_name.date.dateval[1] = 0 >> obj.primary_name.date.dateval[2] = 0 >> obj.primary_name.date.dateval[3] = False >> obj.primary_name.first_name = u'William' >> obj.primary_name.title = '' >> obj.primary_name.type.string = 'Birth Name' >> obj.primary_name.type.value = 2 >> obj.primary_name.display_as = 0 >> obj.primary_name.nick = u'' >> obj.primary_name.call = u'' >> obj.primary_name.surname_list[0].connector = '' >> obj.primary_name.surname_list[0].origintype.string = '' >> obj.primary_name.surname_list[0].origintype.value = 1 >> obj.primary_name.surname_list[0].prefix = '' >> obj.primary_name.surname_list[0].surname = u'Boucher' >> obj.primary_name.surname_list[0].primary = True >> obj.primary_name.sort_as = 0 >> obj.change = 1185438865 >> obj.citation_list[0] = 'c3b24d46f424004ca03e649f2d0' >> obj.event_ref_list[0].role.string = 'Primary' >> obj.event_ref_list[0].role.value = 1 >> obj.event_ref_list[0].ref = 'c3b24d38f625277c6ec78c01623' >> obj.event_ref_list[0].private = False >> obj.event_ref_list[1].role.string = 'Primary' >> obj.event_ref_list[1].role.value = 1 >> obj.event_ref_list[1].ref = 'c3b24d38f724e7cdd6f52151e3' >> obj.event_ref_list[1].private = False >> obj.event_ref_list[2].role.string = 'Primary' >> obj.event_ref_list[2].role.value = 1 >> obj.event_ref_list[2].ref = 'c3b24d38f8134e638f090438796' >> obj.event_ref_list[2].private = False >> >> (The "obj.attr.attr..." path is constructed on the fly. BTW, if you >> eval(path) you get the same value as what's in the struct, on the >> right hand side of the equal sign.) >> >> Now, what this is truly useful for is doing a diff and merge. I have a >> addon report that produces a list of every difference between the >> current databases and a exported version (GEDCOM, Gramps XML, etc). It >> reads the exported file into a DictionaryDB. Currently, a diffing with >> a GEDCOM produces 100% differences because the UID/handles are >> different and those are the unique IDs (could use gramps_ids, >> alternatively, FYI). But diffing with a Gramps XML file gives a useful >> report (see attached [1]). The report is useful in testing Gramps-Connect >> because Django's SQL ORM defaults to cascading deletes, which might >> make a Family disappear if you delete a child if you don't have it >> setup correctly. This report will be more useful when it becomes an >> interactive Merge UI. >> >> In any event, the to_struct interface might make it much more easy to >> write some code where one might like to programmatically query objects >> and their values, perhaps for testing, a table view, or SQL-like query >> interface etc. (see src/gen/merge/diff.py for other examples). >> >> -Doug >> >> [1] - Too big. See: >> http://www.gramps-project.org/wiki/images/6/6e/Database-diff-report.png >> >> > Thanks! >> > >> > -Doug >> > >> >>> Thanks! >> >>> >> >>> -Doug >> >>> >> >>> >> >>> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Live Security Virtual Conference >> >> Exclusive live event will cover all the ways today's security and >> >> threat landscape has changed and how IT managers can respond. >> >> Discussions >> >> will include endpoint security, mobile security and the latest in >> >> malware >> >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> >> _______________________________________________ >> >> Gramps-users mailing list >> >> Gra...@li... >> >> https://lists.sourceforge.net/lists/listinfo/gramps-users >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Gramps-devel mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-devel > > |
From: Nick H. <nic...@ho...> - 2012-08-14 14:12:22
|
On 14/08/12 02:03, Doug Blank wrote: > Now, what this is truly useful for is doing a diff and merge. I have a > addon report that produces a list of every difference between the > current databases and a exported version (GEDCOM, Gramps XML, etc). It > reads the exported file into a DictionaryDB. Currently, a diffing with > a GEDCOM produces 100% differences because the UID/handles are > different and those are the unique IDs (could use gramps_ids, > alternatively, FYI). But diffing with a Gramps XML file gives a useful > report (see attached [1]). Good work! This will be useful. It is better to do a diff this way rather than on the Gramps XML files. Have you put your report into gramps-addons yet? About your unique handle problem - is it now time to introduce a UUID into Gramps objects? Nick. |
From: Doug B. <dou...@gm...> - 2012-08-14 14:33:52
|
On Tue, Aug 14, 2012 at 10:12 AM, Nick Hall <nic...@ho...> wrote: > On 14/08/12 02:03, Doug Blank wrote: >> Now, what this is truly useful for is doing a diff and merge. I have a >> addon report that produces a list of every difference between the >> current databases and a exported version (GEDCOM, Gramps XML, etc). It >> reads the exported file into a DictionaryDB. Currently, a diffing with >> a GEDCOM produces 100% differences because the UID/handles are >> different and those are the unique IDs (could use gramps_ids, >> alternatively, FYI). But diffing with a Gramps XML file gives a useful >> report (see attached [1]). > > Good work! This will be useful. It is better to do a diff this way > rather than on the Gramps XML files. Thanks! Yes, this is much more convenient in Python. > Have you put your report into gramps-addons yet? No, not yet. I wasn't sure if the infrastructure for reports is Gtk3 ready, and I have only tested it in gramps35. But I'll add it, and mark it with gramps_target=3.5. There are some serious things to be figured out in diff/merge: reordering of sequential items, dependency on primary objects, etc. > About your unique handle problem - is it now time to introduce a UUID > into Gramps objects? I think so. If someone wanted to look at that, that would be great. I think the last that we talked about we decided: 1) to add a special attribute _UID on export. 2) use that _UID as the handle on import 3) allow the possibility that an object might have multiple _UIDs, for keeping track of merges, and that one would be the definitive UID. We had some discussions listed here: http://www.gramps-project.org/wiki/index.php?title=GEPS_009:_Import_Export_Merge#UID.2C_GUID_and_UID.2C_what_is_needed_in_GRAMPS.3F -Doug > Nick. > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Doug B. <dou...@gm...> - 2012-08-15 02:03:14
|
>> Have you put your report into gramps-addons yet? > > No, not yet. I wasn't sure if the infrastructure for reports is Gtk3 > ready, and I have only tested it in gramps35. But I'll add it, and > mark it with gramps_target=3.5. There are some serious things to be > figured out in diff/merge: reordering of sequential items, dependency > on primary objects, etc. Now in gramps-addons/trunk marked for Gramps 3.5. There is an "ignore fields" string option which is currently not used. Just select a Gramps XML exported version of your current, opened database, and select add, deleted, and/or edited records to see. -Doug >> About your unique handle problem - is it now time to introduce a UUID >> into Gramps objects? > > I think so. If someone wanted to look at that, that would be great. I > think the last that we talked about we decided: > > 1) to add a special attribute _UID on export. > 2) use that _UID as the handle on import > 3) allow the possibility that an object might have multiple _UIDs, for > keeping track of merges, and that one would be the definitive UID. > > We had some discussions listed here: > http://www.gramps-project.org/wiki/index.php?title=GEPS_009:_Import_Export_Merge#UID.2C_GUID_and_UID.2C_what_is_needed_in_GRAMPS.3F > > -Doug > >> Nick. >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Gramps-devel mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-devel |
From: Doug B. <dou...@gm...> - 2012-08-15 11:47:50
|
On Tue, Aug 14, 2012 at 7:42 AM, Doug Blank <dou...@gm...> wrote: [snip] >> About implementation. I would expect it to be a dictionary of dictionaries, >> and on lowest level strings or integer, but sometimes it is list, because it >> has to be ordered. >> Well, I don't like that "citation_list": CitationBase.to_struct(self) is a >> list, but other to_structs are dicts (wrong doc of the method there by the >> way). I think these objects better have no to_struct method, and you just >> write the attribute you need. >> >> to_struct being list is counterintuitive. I looked into making the output always be the same type (considered dicts, orderdicts, and namedtuples) but I think it is better to see this analogous to Object.serialize(). Object.serialize() will give you a variety of types depending on the Object (list, tuple, int, bool, and even dict). It would be handy to mark some of these items with additional metadata. For example, marking a handle as such indicates that it is a dependency. But I'll look into other ways to mark it... -Doug |
From: Benny M. <ben...@gm...> - 2012-08-15 13:44:52
|
2012/8/15 Doug Blank <dou...@gm...> > On Tue, Aug 14, 2012 at 7:42 AM, Doug Blank <dou...@gm...> wrote: > > [snip] > > >> About implementation. I would expect it to be a dictionary of > dictionaries, > >> and on lowest level strings or integer, but sometimes it is list, > because it > >> has to be ordered. > >> Well, I don't like that "citation_list": CitationBase.to_struct(self) > is a > >> list, but other to_structs are dicts (wrong doc of the method there by > the > >> way). I think these objects better have no to_struct method, and you > just > >> write the attribute you need. > >> > >> to_struct being list is counterintuitive. > > I looked into making the output always be the same type (considered > dicts, orderdicts, and namedtuples) but I think it is better to see > this analogous to Object.serialize(). Object.serialize() will give you > a variety of types depending on the Object (list, tuple, int, bool, > and even dict). > > It would be handy to mark some of these items with additional > metadata. For example, marking a handle as such indicates that it is a > dependency. But I'll look into other ways to mark it... > But why do these Base objects require a to_struct? Just obtain the single attribute you require. The fact that it is a list, indicates to me you don't need to_struct. Benny > > -Doug > |
From: Doug B. <dou...@gm...> - 2012-08-15 13:56:16
|
On Wed, Aug 15, 2012 at 9:44 AM, Benny Malengier <ben...@gm...> wrote: > > > 2012/8/15 Doug Blank <dou...@gm...> >> >> On Tue, Aug 14, 2012 at 7:42 AM, Doug Blank <dou...@gm...> wrote: >> >> [snip] >> >> >> About implementation. I would expect it to be a dictionary of >> >> dictionaries, >> >> and on lowest level strings or integer, but sometimes it is list, >> >> because it >> >> has to be ordered. >> >> Well, I don't like that "citation_list": CitationBase.to_struct(self) >> >> is a >> >> list, but other to_structs are dicts (wrong doc of the method there by >> >> the >> >> way). I think these objects better have no to_struct method, and you >> >> just >> >> write the attribute you need. >> >> >> >> to_struct being list is counterintuitive. >> >> I looked into making the output always be the same type (considered >> dicts, orderdicts, and namedtuples) but I think it is better to see >> this analogous to Object.serialize(). Object.serialize() will give you >> a variety of types depending on the Object (list, tuple, int, bool, >> and even dict). >> >> It would be handy to mark some of these items with additional >> metadata. For example, marking a handle as such indicates that it is a >> dependency. But I'll look into other ways to mark it... > > > But why do these Base objects require a to_struct? Just obtain the single > attribute you require. > The fact that it is a list, indicates to me you don't need to_struct. I guess for the same reason that Base objects have a serialize(): each object takes care of its own representation. I think that this will make it easier to maintain and develop. -Doug > Benny > > >> >> >> -Doug > > |
From: jerome <rom...@ya...> - 2012-08-16 05:55:18
|
It sounds good! Note, can we imagine a related feature: ability to import (merge) only one table (type of primary object)? Having a top level filter (factory, hierarchical level, etc ...), then like 'Export screen...' feature from menu into list views, to be able to import/dump a specific table without relation with others primary objects! Why to do that? *Most part of time this could be very useful when one has planned a migration from a closed file format to something more flexible like gramps[1]. *There is a lot of simple tables (places[2], events[3], sources/citations[4], persons, families, notes/transcriptions) dedicated to genealogy (3rd party indexes, publications, sql tables, etc ...). To have a quick way to handle content without new seizure might be productive (and consistent). As you said, this may be more difficult to merge without an unique ID. Anyway, I guess this is also one limitation on current Gramps CSV import[5]. Users know that and I guess that they have already tested CSV import. If we go further, with your code, it should be also possible to test consistency of the imported records, right? ie. syntax, type of imported data according to Gramps DB model. [1] http://www.cohsoft.com.au/tmg2gramps/ [2] http://www.gramps-project.org/wiki/index.php?title=Place_database [3] http://www.gramps-project.org/wiki/index.php?title=Events_manager [4] http://www.gramps-project.org/wiki/index.php?title=Tellico [5] http://www.gramps-project.org/wiki/index.php?title=Gramps_3.4_Wiki_Manual_-_Manage_Family_Trees:_CSV_Import_and_Export Thanks! Jérôme --- En date de : Mer 15.8.12, Doug Blank <dou...@gm...> a écrit : > De: Doug Blank <dou...@gm...> > Objet: Re: [Gramps-devel] [Gramps-users] Status report on Gramps-Connect > À: "Benny Malengier" <ben...@gm...> > Cc: gra...@li..., "Jiri Kastner" <cz1...@gm...> > Date: Mercredi 15 août 2012, 15h56 > On Wed, Aug 15, 2012 at 9:44 AM, > Benny Malengier > <ben...@gm...> > wrote: > > > > > > 2012/8/15 Doug Blank <dou...@gm...> > >> > >> On Tue, Aug 14, 2012 at 7:42 AM, Doug Blank <dou...@gm...> > wrote: > >> > >> [snip] > >> > >> >> About implementation. I would expect it to > be a dictionary of > >> >> dictionaries, > >> >> and on lowest level strings or integer, > but sometimes it is list, > >> >> because it > >> >> has to be ordered. > >> >> Well, I don't like that "citation_list": > CitationBase.to_struct(self) > >> >> is a > >> >> list, but other to_structs are dicts > (wrong doc of the method there by > >> >> the > >> >> way). I think these objects better have no > to_struct method, and you > >> >> just > >> >> write the attribute you need. > >> >> > >> >> to_struct being list is counterintuitive. > >> > >> I looked into making the output always be the same > type (considered > >> dicts, orderdicts, and namedtuples) but I think it > is better to see > >> this analogous to Object.serialize(). > Object.serialize() will give you > >> a variety of types depending on the Object (list, > tuple, int, bool, > >> and even dict). > >> > >> It would be handy to mark some of these items with > additional > >> metadata. For example, marking a handle as such > indicates that it is a > >> dependency. But I'll look into other ways to mark > it... > > > > > > But why do these Base objects require a to_struct? Just > obtain the single > > attribute you require. > > The fact that it is a list, indicates to me you don't > need to_struct. > > I guess for the same reason that Base objects have a > serialize(): each > object takes care of its own representation. I think that > this will > make it easier to maintain and develop. > > -Doug > > > Benny > > > > > >> > >> > >> -Doug > > > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's > security and > threat landscape has changed and how IT managers can > respond. Discussions > will include endpoint security, mobile security and the > latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Gramps-devel mailing list > Gra...@li... > https://lists.sourceforge.net/lists/listinfo/gramps-devel > |
From: Doug B. <dou...@gm...> - 2012-08-16 12:57:18
|
On Thu, Aug 16, 2012 at 1:55 AM, jerome <rom...@ya...> wrote: > It sounds good! > > Note, can we imagine a related feature: ability to import (merge) only one table (type of primary object)? I think that would be trivial; something like: from gen.merge.diff import import_as_dict from gen.db import DbTxn def import_types(db, filename, *types): newdb = import_as_dict(filename) db.disable_signals() with DbTxn(_("Import"), db, batch=True) as trans: for item in types: handles = newdb._tables[item]["handles_func"]() for handle in handles: obj = newdb._tables[item]["handle_func"](handle) if item == "Person": add_func = db.add_person commit_func = db.commit_person #... add_func(obj, trans) commit_func(obj, trans) db.enable_signals() db.request_rebuild() import_types(db, "data.gramps", "Person") Currently, we don't enforce a proper manner to import into the database, and some importers assume the BSDDB internals. We'll need to clean that up once we start having different internals (like DictionaryDb, DjangoDb, etc). So, not all file import formats will work going into DictionaryDb. > Having a top level filter (factory, hierarchical level, etc ...), then like 'Export screen...' feature from menu into list views, to be able to import/dump a specific table without relation with others primary objects! > > Why to do that? > *Most part of time this could be very useful when one has planned a migration from a closed file format to something more flexible like gramps[1]. > *There is a lot of simple tables (places[2], events[3], sources/citations[4], persons, families, notes/transcriptions) dedicated to genealogy (3rd party indexes, publications, sql tables, etc ...). To have a quick way to handle content without new seizure might be productive (and consistent). > > As you said, this may be more difficult to merge without an unique ID. > Anyway, I guess this is also one limitation on current Gramps CSV import[5]. Users know that and I guess that they have already tested CSV import. > > If we go further, with your code, it should be also possible to test consistency of the imported records, right? ie. syntax, type of imported data according to Gramps DB model. Sure. DictionaryDb (or a filebased TempDb/SqliteDb) can be a staging area for checking and selecting before adding/merging into the real database. -Doug > [1] http://www.cohsoft.com.au/tmg2gramps/ > [2] http://www.gramps-project.org/wiki/index.php?title=Place_database > [3] http://www.gramps-project.org/wiki/index.php?title=Events_manager > [4] http://www.gramps-project.org/wiki/index.php?title=Tellico > [5] http://www.gramps-project.org/wiki/index.php?title=Gramps_3.4_Wiki_Manual_-_Manage_Family_Trees:_CSV_Import_and_Export > > > Thanks! > Jérôme > > --- En date de : Mer 15.8.12, Doug Blank <dou...@gm...> a écrit : > >> De: Doug Blank <dou...@gm...> >> Objet: Re: [Gramps-devel] [Gramps-users] Status report on Gramps-Connect >> À: "Benny Malengier" <ben...@gm...> >> Cc: gra...@li..., "Jiri Kastner" <cz1...@gm...> >> Date: Mercredi 15 août 2012, 15h56 >> On Wed, Aug 15, 2012 at 9:44 AM, >> Benny Malengier >> <ben...@gm...> >> wrote: >> > >> > >> > 2012/8/15 Doug Blank <dou...@gm...> >> >> >> >> On Tue, Aug 14, 2012 at 7:42 AM, Doug Blank <dou...@gm...> >> wrote: >> >> >> >> [snip] >> >> >> >> >> About implementation. I would expect it to >> be a dictionary of >> >> >> dictionaries, >> >> >> and on lowest level strings or integer, >> but sometimes it is list, >> >> >> because it >> >> >> has to be ordered. >> >> >> Well, I don't like that "citation_list": >> CitationBase.to_struct(self) >> >> >> is a >> >> >> list, but other to_structs are dicts >> (wrong doc of the method there by >> >> >> the >> >> >> way). I think these objects better have no >> to_struct method, and you >> >> >> just >> >> >> write the attribute you need. >> >> >> >> >> >> to_struct being list is counterintuitive. >> >> >> >> I looked into making the output always be the same >> type (considered >> >> dicts, orderdicts, and namedtuples) but I think it >> is better to see >> >> this analogous to Object.serialize(). >> Object.serialize() will give you >> >> a variety of types depending on the Object (list, >> tuple, int, bool, >> >> and even dict). >> >> >> >> It would be handy to mark some of these items with >> additional >> >> metadata. For example, marking a handle as such >> indicates that it is a >> >> dependency. But I'll look into other ways to mark >> it... >> > >> > >> > But why do these Base objects require a to_struct? Just >> obtain the single >> > attribute you require. >> > The fact that it is a list, indicates to me you don't >> need to_struct. >> >> I guess for the same reason that Base objects have a >> serialize(): each >> object takes care of its own representation. I think that >> this will >> make it easier to maintain and develop. >> >> -Doug >> >> > Benny >> > >> > >> >> >> >> >> >> -Doug >> > >> > >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's >> security and >> threat landscape has changed and how IT managers can >> respond. Discussions >> will include endpoint security, mobile security and the >> latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Gramps-devel mailing list >> Gra...@li... >> https://lists.sourceforge.net/lists/listinfo/gramps-devel >> |