On Tue, Aug 14, 2012 at 4:05 AM, Benny Malengier
<benny.malengier@...> wrote:
> Doug,
Thanks for the feedback! Comments below.
> I see in the code, you wrote doc
>
> """
> Convert the object to a serialized struct of data.
> """
>
> and then in person.py no longer a doc string.
>
> For gen.lib, it is very important there are good doc strings, so please
> update that. Make the docstring such that it is very clear what this method
> is for, and what it should produce, so you don't have to read the code to
> understand what to do when there is a database change. Make it such that it
> reads nicely in
> http://gramps-project.org/docs/api.html#module-gen
Will do.
> In person.py, what is the FIELDS for? Again no doc string to understand, and
> I don't see it used in person.py
> If FIELDS is used somewhere, do we really want them like that with the
> underscores?
> To not type FIELDS, you could do:
>
> Person().to_struct().iterkeys()
>
> which would avoid typing FIELDS everywhere and making an error.
I was trying to decide whether a separate FIELDS would be worth it
(to_struct gathers all of the associated data too, which can be
expensive). But, I think until we find it too expensive, your method,
as you say, will ensure consistency. I'll remove FIELDS.
> About implementation. I would expect it to be a dictionary of dictionaries,
> and on lowest level strings or integer, but sometimes it is list, because it
> has to be ordered.
> Well, I don't like that "citation_list": CitationBase.to_struct(self) is a
> list, but other to_structs are dicts (wrong doc of the method there by the
> way). I think these objects better have no to_struct method, and you just
> write the attribute you need.
>
> to_struct being list is counterintuitive.
>
> To keep the struction of dictionary of dictionary, you could use ordered
> dict http://docs.python.org/library/collections.html#collections.OrderedDict
> which is in python 2.7 which trunk depends on. But then you need a key of
> some sort, which we don't have.
I considered moving the functionality to Python 2.7, but gramps35
still runs happily on older Pythons. I called it "to_struct" rather
than "to_dict" because it does give structures. I'll look for an
implementation of ordereddict, and try to unify to all dicts.
> I don't understand how from_struct works
> self.citation_list = [CitationBase.from_struct(cref)
> for cref in to_struct["citation_list"]]
>
> There is no from_struct class method in CitationBase that I find. Anyway,
> both are a list anyway, no, so
> self.citation_list = to_struct["citation_list"]
> no from_struct needed, just as to_struct was not needed.
That wouldn't work because to_struct["citation_list"] is a list of
structs. But from_struct should not have been included yet.
I've been working on this for a long time, and have gone through a few
iterations trying to get it just right. It is almost there. Thanks!
-Doug
> Benny
> 2012/8/14 Doug Blank <doug.blank@...>
>>
>> On Thu, Aug 2, 2012 at 9:06 AM, Doug Blank <doug.blank@...> wrote:
>> > On Thu, Aug 2, 2012 at 4:33 AM, Jiri Kastner <cz172638@...> wrote:
>> >> On Mon, 30 Jul 2012 11:39:18 -0400, Doug Blank wrote:
>> >>
>> >>> If anyone has additional skills, please join gramps-devel mailing list
>> >>> and we can discuss further.
>> >>
>> >> what about using selenium{,-ide} for testing? ev. python-selenium
>> >> scripts
>> >> as gramps is pythonic?
>> >>
>> >> ########################## EXAMPLE STARTS
>> >> ###############################
>> >>
>> >> from selenium import webdriver
>> >> driver = webdriver.Chrome()
>> >> driver.get("http://www.gramps-connect.org/")
>> >> driver.find_element_by_link_text("Login").click()
>> >> driver.find_element_by_id("id_username").click()
>> >> driver.find_element_by_id("id_username").clear()
>> >> driver.find_element_by_id("id_username").send_keys("admin")
>> >> driver.find_element_by_id("id_password").clear()
>> >> driver.find_element_by_id("id_password").send_keys("gramps")
>> >> driver.find_element_by_css_selector("input[type=\"submit\"]").click()
>> >> driver.find_element_by_css_selector("tr.odd > td > a").click()
>> >> driver.find_element_by_link_text("ACHINCLOSS, Hugh").click()
>> >> driver.find_element_by_xpath("//input[@value='Edit Person']").click()
>> >> driver.find_element_by_link_text("Logout").click()
>> >>
>> >> ########################## EXAMPLE ENDS
>> >> ##################################
>> >
>> > That looks really useful! That would also help test the web interface
>> > (which I wasn't sure could be done) along with the functionality.
>> >
>> > I'm also adding the ability to have a DictionaryDb copy of the
>> > database, and be able to run a diff between them after issuing
>> > specific commands. That should help make sure extra items are not
>> > deleted. That could be combined with the above.
>>
>> Just a brief update on some related changes that I have just committed
>> to trunk and gramps35. Each gen.lib object has an additional method
>> called "to_struct". Calling this will return what we might have called
>> a "record" structure 20 years ago, but maybe json now: it returns
>> either a dictionary, list, or value depending on the object. Primary
>> objects look something like:
>>
>> >>> p = db.get_person_from_handle(u'c3adac142e52d52d5a48840601d')
>> >>> p.to_struct()
>> {'parent_family_list': [], 'person_ref_list': [], 'gramps_id':
>> u'I0001', 'handle': 'c3adac142e52d52d5a48840601d', 'media_list': [],
>> 'death_ref_index': -1, 'birth_ref_index': 0, 'family_list': [],
>> 'gender': 1, 'tag_list': (), 'alternate_names': [], 'attribute_list':
>> [], 'lds_ord_list': [], 'private': False, 'primary_name': {'group_as':
>> u'', 'suffix': u'', 'private': False, 'famnick': u'', 'date': None,
>> 'note_list': [], 'first_name': u'Douglas', 'title': u'', 'type':
>> {'string': u'', 'value': 2}, 'display_as': 0, 'nick': u'', 'call':
>> u'', 'surname_list': [{'connector': u'', 'prefix': u'', 'surname':
>> u'Blank', 'primary': True, 'origin_type': {'string': u'', 'value':
>> 1}}], 'citation_list': [], 'sort_as': 0}, 'change': 1344691773,
>> 'urls': [], 'citation_list': [], 'event_ref_list': [{'attribute': [],
>> 'role': {'string': u'', 'value': 1}, 'ref':
>> 'c3b0b20674052c2913ccbcab330', 'note_list': [], 'private': False}],
>> 'note_list': [], 'address_list': []}
>>
>> The keys of the toplevel dictionary also match exactly the associated
>> attribute names. What can you do with this? One thing you can do is
>> dump an object out and see all of the values and related "paths". For
>> example, with just a few lines of code you can take the above struct
>> and produce something like:
>>
>> Person
>> obj.parent_family_list[0] = 'c3b24d3e3352334df33df41a4d3'
>> obj.gramps_id = 'I0811'
>> obj.handle = 'c3b24d46f385580b32ae6202416'
>> obj.death_ref_index = 1
>> obj.birth_ref_index = 0
>> obj.family_list[0] = 'c3b24d411bc6d4886d4692ac287'
>> obj.gender = 1
>> obj.private = False
>> obj.primary_name.group_as = ''
>> obj.primary_name.suffix = ''
>> obj.primary_name.private = False
>> obj.primary_name.famnick = u''
>> obj.primary_name.date.sortval = 0
>> obj.primary_name.date.text = u''
>> obj.primary_name.date.newyear = 0
>> obj.primary_name.date.calendar = 0
>> obj.primary_name.date.modifier = 0
>> obj.primary_name.date.quality = 0
>> obj.primary_name.date.dateval[0] = 0
>> obj.primary_name.date.dateval[1] = 0
>> obj.primary_name.date.dateval[2] = 0
>> obj.primary_name.date.dateval[3] = False
>> obj.primary_name.first_name = u'William'
>> obj.primary_name.title = ''
>> obj.primary_name.type.string = 'Birth Name'
>> obj.primary_name.type.value = 2
>> obj.primary_name.display_as = 0
>> obj.primary_name.nick = u''
>> obj.primary_name.call = u''
>> obj.primary_name.surname_list[0].connector = ''
>> obj.primary_name.surname_list[0].origintype.string = ''
>> obj.primary_name.surname_list[0].origintype.value = 1
>> obj.primary_name.surname_list[0].prefix = ''
>> obj.primary_name.surname_list[0].surname = u'Boucher'
>> obj.primary_name.surname_list[0].primary = True
>> obj.primary_name.sort_as = 0
>> obj.change = 1185438865
>> obj.citation_list[0] = 'c3b24d46f424004ca03e649f2d0'
>> obj.event_ref_list[0].role.string = 'Primary'
>> obj.event_ref_list[0].role.value = 1
>> obj.event_ref_list[0].ref = 'c3b24d38f625277c6ec78c01623'
>> obj.event_ref_list[0].private = False
>> obj.event_ref_list[1].role.string = 'Primary'
>> obj.event_ref_list[1].role.value = 1
>> obj.event_ref_list[1].ref = 'c3b24d38f724e7cdd6f52151e3'
>> obj.event_ref_list[1].private = False
>> obj.event_ref_list[2].role.string = 'Primary'
>> obj.event_ref_list[2].role.value = 1
>> obj.event_ref_list[2].ref = 'c3b24d38f8134e638f090438796'
>> obj.event_ref_list[2].private = False
>>
>> (The "obj.attr.attr..." path is constructed on the fly. BTW, if you
>> eval(path) you get the same value as what's in the struct, on the
>> right hand side of the equal sign.)
>>
>> Now, what this is truly useful for is doing a diff and merge. I have a
>> addon report that produces a list of every difference between the
>> current databases and a exported version (GEDCOM, Gramps XML, etc). It
>> reads the exported file into a DictionaryDB. Currently, a diffing with
>> a GEDCOM produces 100% differences because the UID/handles are
>> different and those are the unique IDs (could use gramps_ids,
>> alternatively, FYI). But diffing with a Gramps XML file gives a useful
>> report (see attached [1]). The report is useful in testing Gramps-Connect
>> because Django's SQL ORM defaults to cascading deletes, which might
>> make a Family disappear if you delete a child if you don't have it
>> setup correctly. This report will be more useful when it becomes an
>> interactive Merge UI.
>>
>> In any event, the to_struct interface might make it much more easy to
>> write some code where one might like to programmatically query objects
>> and their values, perhaps for testing, a table view, or SQL-like query
>> interface etc. (see src/gen/merge/diff.py for other examples).
>>
>> -Doug
>>
>> [1] - Too big. See:
>> http://www.gramps-project.org/wiki/images/6/6e/Database-diff-report.png
>>
>> > Thanks!
>> >
>> > -Doug
>> >
>> >>> Thanks!
>> >>>
>> >>> -Doug
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Live Security Virtual Conference
>> >> Exclusive live event will cover all the ways today's security and
>> >> threat landscape has changed and how IT managers can respond.
>> >> Discussions
>> >> will include endpoint security, mobile security and the latest in
>> >> malware
>> >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> >> _______________________________________________
>> >> Gramps-users mailing list
>> >> Gramps-users@...
>> >> https://lists.sourceforge.net/lists/listinfo/gramps-users
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Gramps-devel mailing list
>> Gramps-devel@...
>> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>
>
|