I don't have much time at the moment.

Perhaps you can start with a mail to the user list indicating that the Cause of Death event will be removed in the future and automatically changed into a Death event with attribute.
With the question then if everybody is ok with that, or if some users sees a problem.

That would allow us to decide if we can just do it automatically with a database upgrade, or not.


2008/3/27, James G. Sack (jim) <jgsack@san.rr.com>:
[excuse the amount of quoting, but I think the conversation still needs
the context. Perhaps the next round could be cut]

Benny Malengier wrote:
> James,
> Difficult stuff.
> 2008/3/27, James G. Sack (jim) <jgsack@san.rr.com>:
>> OK, in r10405 I've gotten all the get-ready work done for bug #1680,
>> which will get rid of the deprecated CAUSE_DEATH (Cause Of Death) event
>> type from eventtype.py. This will use a new "blacklist" functionality in
>> GrampsType which hides obsolete/retired type values without actually
>> removing the old data.
> Good idea. Try to make it general, so we can use the mechanism more
> generally, like separate family events from person events, ...

That was my intent. I believe it is general, and should work for any
class derived from GrampsType. The "blacklist" idea originated from some
earlier pioneer in GrampsType use -- I forget his name. O:-) The change
to grampstype.py was minimal, and the usage is somewhat illustrated in
the small unittest module src/gen/lib/test/grampstype_test.py. If you
have a chance, do have a quick look at the use of _BLACKLIST in
grampstype.py and the short tests in grampstype_test.py.

> I have not yet removed CAUSE_DEATH because that would break databases
>> that have existing events of that type.
>> ==> So I am looking for suggestions on how to proceed from here.
>> Here's the first scenario that comes to mind (but see [note0]):
>> 1) On reading a database, convert CAUSE_DEATH events (with description)
>> to new DEATH events[note1], putting the description into the attribute
>> CAUSE ("Cause"). Log output and UI feedback should be generated and it
>> would be nice to show a list of individuals affected, maybe.
> We could also convert to a CUSTOM TYPE, so keep the data of the user, but
> now as custom type instead of predefined type.

In general that would seem reasonable, but for an event now labeled
"Cause Of Death", I was assuming that no one would object to having it
converted to an attribute "Cause" in an event "Death" (leaving any
notes, sources, other attributes, etc unchanged -- ie, without any loss
of user data). I expect that the majority of events of this non-standard
type would have been created in the Gramps UI, just because the user
would have noticed that Cause Of Death in the event drop-down list
before noticing the attribute list-item Cause inside a Death event. In
other words,  I think the conversion achieves the most likely intention
of the user (my assumption, of course).

  Aside: we might need to be prepared to answer questions like:
  "Where did the 'Cause Of Death' event go? -- I use that all the time."

> Automatic conversions are not really liked by advanced users, be it as
> custom type or to death event.
> We could add options, but it is annoying to bother users on import with an
> option 'Change CAUSE_DEATH event to:' if he does not have that info in the
> file.

Noted, and agreed. But OTOH, something infrequent might be less
objectionable, especially when it is tied to a software version change.
"Oh, your firefox version has changed -- looking for extension
updates..". "Oh this database was generated by a previous version of
Gramps -- looking for data classification/mapping changes..".

> As another option, do we need to remove this type from GRAMPS? We could

> blacklist it and not show it anymore ..

Yes, the net effect of the blacklist is that blacklisted items no longer
show on any of the lists, so (eg) they do not get into the UI select
widgets. The original data remains, so that tokens and index values do
not get reused, and so that migrations of old data remain always
possible -- a requirement that was nicely explained to me by someone
named Benny. :-) I haven't yet blacklisted CAUSE_DEATH in eventtype.py,
but it only takes one line, "_BLACKLIST=[CAUSE_DEATH]" to do.

>..(I think NoteType does things like

> that), but just keep it around.

I wish you had mentioned this when I was asking about CAUSE_DEATH before
creating bug #1680. Or maybe not: the blacklist mechanism may perhaps
replace what's done in NoteType, and end up simplifying the code. It
_could be_ that this is the "best of all possible worlds"(tm), eh? :-)

I now see that
  src/gen/lib/notetype.py has
  _DATAMAPIGNORE (only use of this name; similar to my _BLACKLIST)
  get_ignore_list() (uses above, called from src/Editors/_EditNote.py)

  src/Editors/_EditNote.py in method EditNote._setup_fields
  sets ignore_values in a MonitoredDataType instantiation

  src/GrampsWidgets.py in _MonitoredDataType._init__
  seems to use ignore_values to remove items from the map returned by
  get_map (_I2SMAP).

- this particular ignore mechanism seems only used in NoteType. I don't
know whether there might be other such mechanisms that have escaped my
- get_ignore_list has a provision ("exception") for overrriding the
ignore list. Evidently these overrides come from an "extratype" list
passed into the EditNote's __init__.
- the extratype data is also used in the EditNote.empty_object member
- I can not find any instantiation of EditNote that passes anything in
for the extratype list, so perhaps that was never found needed?

Benny, perhaps you can tickle your memory from a year ago (circa r8441)
to see what I may have missed in my review above. I do think that the
blacklist mechanism turned out pretty clean. It would be nice if it
simplified the recipe for handling obsolete/retired type values in
NoteType as well.

> Then only GEDCOM export must be changed and handle old CAUSE_DEATH events as
> custom event or as attribute to DEATH...

Well, yes, removing the deprecated event type prevents further creations
of such events, and export code could be kludged to do on-the-fly
conversions of old database data, but that's what I think might be
messy/ugly in the long run. In the meantime UI display views of such
deprecated items would show as "unknown"s which may be considered
somewhat user-unfriendly.

To me, it seem that the database itself (and imports) should be the
focus of change -- not exports.

> 2) On importing[note2] from various formats (I guess that means: xml,
>> grdb, gedcom), it should do the exact same stuff as step (1), no?
> Yes. For import of xml I added an info object, and a messagedialog after
> import with the info text.
> The idea is to move to a gtk.Assistant in the future, and use the info text
> on the final assistant page.

OK, I'll have to keep an open eye on the info object and gtk.Assistant,
I guess. Hmmm, starting to run out of eyes. :-(

> 3) Import and export modules (including eg, _GedcomParse.py and
>> _WriteGedcom.py) and maybe some db handling code can remove leftover
>> unreachable code not needed for step (2).
> Yes. If we just leave CAUSE_DEATH but blacklist it, only gedcom export must
> change,

I suspect it is probably true that only gedcom export has leftover
obsolete code, because of the hard-coded strings, unlike the xml export.

>..and the rest remains the same. We could then provide a tool: convert

> events (it exists already), and let users run this tool to change
> database/xml info. We could adapt the tool to have a specific CAUSE_DEATH
> change option that would add the CAUS attribute to the death event if that
> is present.

Ahh, looking for this, I found the Tools > Analysis.. > Compare Events
dialog which seems like it should be quite useful for lots of things.
I'll have to look into that some more. But I assume  the convert tool
you mention is the Tools > Database Processing > Rename Event Types
(whose dialog title and embedded heading BTW is "Change Event Types").
My guess is that such an adaptation would be fine for a standalone tool,
but in mainline code, would probably not survive too long if it
underwent many such adaptations. Maybe there will never be any more
type-value retirements, but that seems wishful. I'm imagining that
future changes could require more complex changes than just
reclassifying a type, or moving an event-description field into an
attribute-description field.

> 4) Don't forget [note0].
>> - - -
>> [note0] Looking forward to a time when there are many obsolete/retired
>> types, it seems like it may NOT be a good idea to carry such conversion
>> code along forever. Maybe just one point-version's worth, or _some_
>> version-based cutoff. Or maybe it shouldn't be part of mainline code at
>> all? Perhaps opening a database or importing a file should simply report
>>   blacklisted types and tell the user to find and run a specific
>> standalone conversion utility. There might be a sequence of these:
>> fix3.1 fix3.2, fix4.0 (4.0 also rolls-up all the 3.x since 3.0?).
>> Even with only a few  obsolete type values to blacklist, I can imagine
>> that the fixup code would tend to clutter mainline code and constrain
>> the freedom of future design changes.
>> [note1] Multiple DEATH events is not forbidden. A utility operation to
>> find/display such multiples might be a useful diagnosis/repair tool. In
>> fact, generalizing this utility might produce a nice convenience tool.
>> There is no reason to prohibit multiples, but it might still be nice for
>> the user to be able to have a look on demand.
> Every database change means tools and upgrade paths. It is important to not
> only fix issues in GRAMPS, but also offer the required tools to the user.
> Eg, the check and repair tool could have code to remove the CAUS_DEATH event
> and replace it. Question is, does the user want to have that done
> automatically, or should it be another tool, or part of the change eventtype
> tool?

Yup. We have now just a simple problem to solve, but it is part of an
ongoing need that justifies some brainstorming. I'm thinking that every
case will likely differ in various details, and an external tool might
be worth considering -- as opposed to an open-ended tree of conversions
in the built-in change tool. The external tools could be part of the
distribution, and maybe even executed from Gramps, I suppose. Maybe that
makes them equivalent to built-in tools? Perhaps it's just necessary to
have multiple tools that track these changes according to various
version-jumps. "Oh, you have a db that was created by Gramps v2.6, and
you are now running v3.4. It is recommended to run major_fix_v3 and then
minor_fix_v3.4 to ensure your data can be handled by the current version
of Gramps".

> [note2] Exporting should not have to do any of this fixup/conversion,
>> since the database will have been already purged of the obsolete type
>> values. But see step (3).
>> - - -
>> It seems I have argued myself into preferring a freestanding tool to do
>> such migration chores. Are there some examples of that kind of code for
>> me to look at?
>> Who's got some past-practice information and/or ideas?

..thanks for your patience through my wordy expositions.


Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
Gramps-devel mailing list