gramps

Gerald said:


> Indeed, I noticed two sets of functions (find_... and check_...) that
> are pretty much only used by ImportXML.  I'm wondering if those
> functions shouldn't be in ImportXML instead.
>

What about other importers? Should there be a subclass for all of these? Or
is ImportXML different in a fundamental way?

-Doug

ImportXML is different since it is importing an internal format.
Also, it is the only "offender" in the sense of using special methods
added to base.py just for itself.

On Tue, Jul 21, 2009 at 9:40 AM, Doug Blank<doug.blank@...> wrote:
> Gerald said:
>
>>
>> Indeed, I noticed two sets of functions (find_... and check_...) that
>> are pretty much only used by ImportXML.  I'm wondering if those
>> functions shouldn't be in ImportXML instead.
>
> What about other importers? Should there be a subclass for all of these? Or
> is ImportXML different in a fundamental way?
>
> -Doug
>



-- 
Gerald Britton

>>> Indeed, I noticed two sets of functions (find_...
> and check_...) that
>>> are pretty much only used by ImportXML.  I'm
> wondering if those
>>> functions shouldn't be in ImportXML instead.


> > What about other importers? Should there be a subclass
> for all of these? Or
> > is ImportXML different in a fundamental way?

> ImportXML is different since it is
> importing an internal format.
> Also, it is the only "offender" in the sense of using
> special methods
> added to base.py just for itself.
> 

There are two importers that have tight coupling to the native database implementation: Grdb and XML. The reason for this is historical: GRDB and XML used to be native file formats for Gramps. When we switched to the current directory based database, and changed Grdb and Xml into importers, we didn't completely decouple them from the base implementation.

Anything we can do to get ImportGrdb, ImportXml and ExportXml to stand on their own and decouple them from gen.db, the better the code will be.

~Brian

Like Benny I am wondering about the signals. What do you think?
Certainly if the db is opened read only they should do nothing. Is
there any harm in not connecting them?



On 7/21/09, Brian Matherly <brian@...> wrote:
>
>>>> Indeed, I noticed two sets of functions (find_...
>> and check_...) that
>>>> are pretty much only used by ImportXML.  I'm
>> wondering if those
>>>> functions shouldn't be in ImportXML instead.
>
>
>> > What about other importers? Should there be a subclass
>> for all of these? Or
>> > is ImportXML different in a fundamental way?
>
>> ImportXML is different since it is
>> importing an internal format.
>> Also, it is the only "offender" in the sense of using
>> special methods
>> added to base.py just for itself.
>>
>
> There are two importers that have tight coupling to the native database
> implementation: Grdb and XML. The reason for this is historical: GRDB and
> XML used to be native file formats for Gramps. When we switched to the
> current directory based database, and changed Grdb and Xml into importers,
> we didn't completely decouple them from the base implementation.
>
> Anything we can do to get ImportGrdb, ImportXml and ExportXml to stand on
> their own and decouple them from gen.db, the better the code will be.
>
> ~Brian
>
> ------------------------------------------------------------------------------
> Enter the BlackBerry Developer Challenge
> This is your chance to win up to $100,000 in prizes! For a limited time,
> vendors submitting new applications to BlackBerry App World(TM) will have
> the opportunity to enter the BlackBerry Developer Challenge. See full prize
> details at: http://p.sf.net/sfu/Challenge
> _______________________________________________
> Gramps-devel mailing list
> Gramps-devel@...
> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>

-- 
Sent from my mobile device

Gerald Britton

> Like Benny I am wondering about the
> signals. What do you think?
> Certainly if the db is opened read only they should do
> nothing. Is
> there any harm in not connecting them?

Not that I know of. The only scenario that I can imagine is if some aspect of the user interface relies on certain signals being sent during initialization. If that scenario exists, there are ways to easily work around that.

~Brian

This may seem trivial, but I thought I'd raise it anyway.  I'd like to
arrange the methods by the order in which the objects appear in the
sidebar, that is:

People
Families
Events
Sources
Places
Media
Repositories
Notes

Now, I could do it like this:

    # methods to read objects

    def get_person
    ....
    def get_family
    ...

   # methods to write objects

    def put_person
    ....
    def put_family
    ...

etc. Or like this:

    # Methods for people objects:

    def get_person
    ...
    def put_person
    ...

    # Methods for family objects:

    def get_family
    ...
    def put_family
    ...

etc.

So, what is the preference? Group by function or group by object?
Which would be easier to read for the next person?  One advantage of
the first approach is that similar functions are grouped and
differences are easy to spot.  One advantage of the second approach is
that all functions for a given object appear together, giving an
at-a-glance view of how to access that object.

Personally, I'm leaning ever so slightly to organizing by object (just
seems more OOPy to me).

On Tue, Jul 21, 2009 at 6:23 PM, Brian Matherly<brian@...> wrote:
>
>> Like Benny I am wondering about the
>> signals. What do you think?
>> Certainly if the db is opened read only they should do
>> nothing. Is
>> there any harm in not connecting them?
>
> Not that I know of. The only scenario that I can imagine is if some aspect of the user interface relies on certain signals being sent during initialization. If that scenario exists, there are ways to easily work around that.
>
> ~Brian
>



-- 
Gerald Britton

For the new read.py module with the GrampsDbRead class, is there a
need for transactions?  I'm thinking that transactions are only for
providing control points for update operations.

On Wed, Jul 22, 2009 at 11:32 AM, Gerald
Britton<gerald.britton@...> wrote:
> This may seem trivial, but I thought I'd raise it anyway.  I'd like to
> arrange the methods by the order in which the objects appear in the
> sidebar, that is:
>
> People
> Families
> Events
> Sources
> Places
> Media
> Repositories
> Notes
>
> Now, I could do it like this:
>
>    # methods to read objects
>
>    def get_person
>    ....
>    def get_family
>    ...
>
>   # methods to write objects
>
>    def put_person
>    ....
>    def put_family
>    ...
>
> etc. Or like this:
>
>    # Methods for people objects:
>
>    def get_person
>    ...
>    def put_person
>    ...
>
>    # Methods for family objects:
>
>    def get_family
>    ...
>    def put_family
>    ...
>
> etc.
>
> So, what is the preference? Group by function or group by object?
> Which would be easier to read for the next person?  One advantage of
> the first approach is that similar functions are grouped and
> differences are easy to spot.  One advantage of the second approach is
> that all functions for a given object appear together, giving an
> at-a-glance view of how to access that object.
>
> Personally, I'm leaning ever so slightly to organizing by object (just
> seems more OOPy to me).
>
> On Tue, Jul 21, 2009 at 6:23 PM, Brian Matherly<brian@...> wrote:
>>
>>> Like Benny I am wondering about the
>>> signals. What do you think?
>>> Certainly if the db is opened read only they should do
>>> nothing. Is
>>> there any harm in not connecting them?
>>
>> Not that I know of. The only scenario that I can imagine is if some aspect of the user interface relies on certain signals being sent during initialization. If that scenario exists, there are ways to easily work around that.
>>
>> ~Brian
>>
>
>
>
> --
> Gerald Britton
>



-- 
Gerald Britton

When going through code, I like to have the methods nicely grouped so
paths followed are easy to navigate, and changing a specific thing is
normally also grouped in the file.
OOP is all very good, but as long as I cannot talk to my computer, I
need to click a sidebar to move to another function if that is not
close by in the file itself.

About the transactions, I don't know what technically is needed, but
it simplifies things if one only has one way to do things. We have
removed the fact that one can work without transactions in 3.0 I
believe, perhaps not all code reflects this.

Benny

2009/7/22 Gerald Britton <gerald.britton@...>:
> For the new read.py module with the GrampsDbRead class, is there a
> need for transactions?  I'm thinking that transactions are only for
> providing control points for update operations.
>
> On Wed, Jul 22, 2009 at 11:32 AM, Gerald
> Britton<gerald.britton@...> wrote:
>> This may seem trivial, but I thought I'd raise it anyway.  I'd like to
>> arrange the methods by the order in which the objects appear in the
>> sidebar, that is:
>>
>> People
>> Families
>> Events
>> Sources
>> Places
>> Media
>> Repositories
>> Notes
>>
>> Now, I could do it like this:
>>
>>    # methods to read objects
>>
>>    def get_person
>>    ....
>>    def get_family
>>    ...
>>
>>   # methods to write objects
>>
>>    def put_person
>>    ....
>>    def put_family
>>    ...
>>
>> etc. Or like this:
>>
>>    # Methods for people objects:
>>
>>    def get_person
>>    ...
>>    def put_person
>>    ...
>>
>>    # Methods for family objects:
>>
>>    def get_family
>>    ...
>>    def put_family
>>    ...
>>
>> etc.
>>
>> So, what is the preference? Group by function or group by object?
>> Which would be easier to read for the next person?  One advantage of
>> the first approach is that similar functions are grouped and
>> differences are easy to spot.  One advantage of the second approach is
>> that all functions for a given object appear together, giving an
>> at-a-glance view of how to access that object.
>>
>> Personally, I'm leaning ever so slightly to organizing by object (just
>> seems more OOPy to me).
>>
>> On Tue, Jul 21, 2009 at 6:23 PM, Brian Matherly<brian@...> wrote:
>>>
>>>> Like Benny I am wondering about the
>>>> signals. What do you think?
>>>> Certainly if the db is opened read only they should do
>>>> nothing. Is
>>>> there any harm in not connecting them?
>>>
>>> Not that I know of. The only scenario that I can imagine is if some aspect of the user interface relies on certain signals being sent during initialization. If that scenario exists, there are ways to easily work around that.
>>>
>>> ~Brian
>>>
>>
>>
>>
>> --
>> Gerald Britton
>>
>
>
>
> --
> Gerald Britton
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Gramps-devel mailing list
> Gramps-devel@...
> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>

Ok, but for me, "nicely grouped" could be by object type (my
preference by a little) or by function type.   I'm not sure which
approach you favor.

On Thu, Jul 23, 2009 at 11:00 AM, Benny
Malengier<benny.malengier@...> wrote:
> When going through code, I like to have the methods nicely grouped so
> paths followed are easy to navigate, and changing a specific thing is
> normally also grouped in the file.
> OOP is all very good, but as long as I cannot talk to my computer, I
> need to click a sidebar to move to another function if that is not
> close by in the file itself.
>
> About the transactions, I don't know what technically is needed, but
> it simplifies things if one only has one way to do things. We have
> removed the fact that one can work without transactions in 3.0 I
> believe, perhaps not all code reflects this.
>
> Benny
>
> 2009/7/22 Gerald Britton <gerald.britton@...>:
>> For the new read.py module with the GrampsDbRead class, is there a
>> need for transactions?  I'm thinking that transactions are only for
>> providing control points for update operations.
>>
>> On Wed, Jul 22, 2009 at 11:32 AM, Gerald
>> Britton<gerald.britton@...> wrote:
>>> This may seem trivial, but I thought I'd raise it anyway.  I'd like to
>>> arrange the methods by the order in which the objects appear in the
>>> sidebar, that is:
>>>
>>> People
>>> Families
>>> Events
>>> Sources
>>> Places
>>> Media
>>> Repositories
>>> Notes
>>>
>>> Now, I could do it like this:
>>>
>>>    # methods to read objects
>>>
>>>    def get_person
>>>    ....
>>>    def get_family
>>>    ...
>>>
>>>   # methods to write objects
>>>
>>>    def put_person
>>>    ....
>>>    def put_family
>>>    ...
>>>
>>> etc. Or like this:
>>>
>>>    # Methods for people objects:
>>>
>>>    def get_person
>>>    ...
>>>    def put_person
>>>    ...
>>>
>>>    # Methods for family objects:
>>>
>>>    def get_family
>>>    ...
>>>    def put_family
>>>    ...
>>>
>>> etc.
>>>
>>> So, what is the preference? Group by function or group by object?
>>> Which would be easier to read for the next person?  One advantage of
>>> the first approach is that similar functions are grouped and
>>> differences are easy to spot.  One advantage of the second approach is
>>> that all functions for a given object appear together, giving an
>>> at-a-glance view of how to access that object.
>>>
>>> Personally, I'm leaning ever so slightly to organizing by object (just
>>> seems more OOPy to me).
>>>
>>> On Tue, Jul 21, 2009 at 6:23 PM, Brian Matherly<brian@...> wrote:
>>>>
>>>>> Like Benny I am wondering about the
>>>>> signals. What do you think?
>>>>> Certainly if the db is opened read only they should do
>>>>> nothing. Is
>>>>> there any harm in not connecting them?
>>>>
>>>> Not that I know of. The only scenario that I can imagine is if some aspect of the user interface relies on certain signals being sent during initialization. If that scenario exists, there are ways to easily work around that.
>>>>
>>>> ~Brian
>>>>
>>>
>>>
>>>
>>> --
>>> Gerald Britton
>>>
>>
>>
>>
>> --
>> Gerald Britton
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Gramps-devel mailing list
>> Gramps-devel@...
>> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>>
>



-- 
Gerald Britton

> This may seem trivial, but I thought
> I'd raise it anyway.  I'd like to
> arrange the methods by the order in which the objects
> appear in the
> sidebar

<snip>

> So, what is the preference? Group by function or group by
> object?
> Which would be easier to read for the next person? 
> One advantage of
> the first approach is that similar functions are grouped
> and
> differences are easy to spot.  One advantage of the
> second approach is
> that all functions for a given object appear together,
> giving an
> at-a-glance view of how to access that object.
> 
> Personally, I'm leaning ever so slightly to organizing by
> object (just
> seems more OOPy to me).

I ever so slightly agree. I tend to group my getters and setters together.

~Brian

2009/7/23 Gerald Britton <gerald.britton@...>:
> Ok, but for me, "nicely grouped" could be by object type (my
> preference by a little) or by function type.   I'm not sure which
> approach you favor.

When hunting for a bug, grouped functionality is what I prefer. But
things keep working when grouped by object, as long as the functions
list has some logic so you do not need to search for functions in a
sidebar. Eg, first the repeated stuff is done for all objects, and
then the specific stuff, that is, things are grouped per object but
this per block, where a block can be get methods or iter methods or
private functions. In the case of the db, if you change it, then I
would prefer that you do something like

root functions
  get_object_by_handle
  get object_by_gid
  ....

person functions
  same as above + those basic repeat functions without a root

family functions
  same as above

specific person functions
  eg get_surname_list, get_researcher, which might be considered
'Person' functions, but break the repeat structure between the objects

private functions
  __check_by_handle

Benny

OK.  I'll probably leave the general ordering as is for now.  Maybe
come back to it later.

Here's something else I need input on:

Currently, both base.py and dbdir.py have "commit_base" functions.
base.py also has specific commit_* functions for the various objects.
Since I'm making base.py purely abstract, its functional code is going
into a new module, read.py.  However, to be true to its name. read.py
doesn't need to implement update methods (add, change, delete, etc.)
These should be implemented in write.py, I think.

In looking at the current base.py and dbdir.py, I can see that the
commit_base functions differ somewhat.  Since "real" database access
is currently through dbdir.py, I'm thinking that the commit_base
method in base.py is not used and can go away. Meanwhile, I'll move
the other update methods including commit_* to the new write.py
module, which will replace dbdir.py when it is done.

Make sense?


On Fri, Jul 24, 2009 at 2:29 AM, Benny
Malengier<benny.malengier@...> wrote:
> 2009/7/23 Gerald Britton <gerald.britton@...>:
>> Ok, but for me, "nicely grouped" could be by object type (my
>> preference by a little) or by function type.   I'm not sure which
>> approach you favor.
>
> When hunting for a bug, grouped functionality is what I prefer. But
> things keep working when grouped by object, as long as the functions
> list has some logic so you do not need to search for functions in a
> sidebar. Eg, first the repeated stuff is done for all objects, and
> then the specific stuff, that is, things are grouped per object but
> this per block, where a block can be get methods or iter methods or
> private functions. In the case of the db, if you change it, then I
> would prefer that you do something like
>
> root functions
>  get_object_by_handle
>  get object_by_gid
>  ....
>
> person functions
>  same as above + those basic repeat functions without a root
>
> family functions
>  same as above
>
> specific person functions
>  eg get_surname_list, get_researcher, which might be considered
> 'Person' functions, but break the repeat structure between the objects
>
> private functions
>  __check_by_handle
>
> Benny
>



-- 
Gerald Britton

> OK.  I'll probably leave the
> general ordering as is for now.  Maybe
> come back to it later.
> 
> Here's something else I need input on:
> 
> Currently, both base.py and dbdir.py have "commit_base"
> functions.
> base.py also has specific commit_* functions for the
> various objects.
> Since I'm making base.py purely abstract, its functional
> code is going
> into a new module, read.py.  However, to be true to
> its name. read.py
> doesn't need to implement update methods (add, change,
> delete, etc.)
> These should be implemented in write.py, I think.
> 
> In looking at the current base.py and dbdir.py, I can see
> that the
> commit_base functions differ somewhat.  Since "real"
> database access
> is currently through dbdir.py, I'm thinking that the
> commit_base
> method in base.py is not used and can go away. Meanwhile,
> I'll move
> the other update methods including commit_* to the new
> write.py
> module, which will replace dbdir.py when it is done.
> 
> Make sense?

Makes sense to me. I predict that you will run into a lot of left-over-unused-stuff during this process.

~BM

Indeed.  I wonder if you can help me understand another thing:

Lately, I've been experimenting with build a context manager for BSDDB
transactions.  That is, instead of writing:

    txn = self.env.txn_begin()

    ...do stuff

    txn.commit()

you could write:

    with BSDT() as txn:  # use new context manager
     ...do stuff

You don't have to remember to do a txn.commit().  In fact, it's better
than that, since the commit will happen automagically at the end of
the block controlled by "with" unless you do a txn.abort() or
txn.commit() before leaving the block.

Anyway, that's just the "context" to my question.  base.py has a
generic class "Transaction" that dbdir.py extends in BdbTransaction.
However, neither of these seem to be concerned with DBTxn objects.
I'd like to understand a little more about what Transaction and its
children do.  First off, I want to be sure that the Transaction class
is in the right module (possibly its own).  Second, I want to be sure
that, if I use the context manager I'm playing with, it plays nice
with Transaction.

On Fri, Jul 24, 2009 at 2:44 PM, Brian Matherly<brian@...> wrote:
>
>> OK.  I'll probably leave the
>> general ordering as is for now.  Maybe
>> come back to it later.
>>
>> Here's something else I need input on:
>>
>> Currently, both base.py and dbdir.py have "commit_base"
>> functions.
>> base.py also has specific commit_* functions for the
>> various objects.
>> Since I'm making base.py purely abstract, its functional
>> code is going
>> into a new module, read.py.  However, to be true to
>> its name. read.py
>> doesn't need to implement update methods (add, change,
>> delete, etc.)
>> These should be implemented in write.py, I think.
>>
>> In looking at the current base.py and dbdir.py, I can see
>> that the
>> commit_base functions differ somewhat.  Since "real"
>> database access
>> is currently through dbdir.py, I'm thinking that the
>> commit_base
>> method in base.py is not used and can go away. Meanwhile,
>> I'll move
>> the other update methods including commit_* to the new
>> write.py
>> module, which will replace dbdir.py when it is done.
>>
>> Make sense?
>
> Makes sense to me. I predict that you will run into a lot of left-over-unused-stuff during this process.
>
> ~BM
>



-- 
Gerald Britton

Gerald,

> Indeed.  I wonder if you can
> help me understand another thing:
> 
> Lately, I've been experimenting with build a context
> manager for BSDDB
> transactions.  That is, instead of writing:
> 
>     txn = self.env.txn_begin()
> 
>     ...do stuff
> 
>     txn.commit()
> 
> you could write:
> 
>     with BSDT() as txn:  # use new context
> manager
>      ...do stuff
> 
> You don't have to remember to do a txn.commit().  In
> fact, it's better
> than that, since the commit will happen automagically at
> the end of
> the block controlled by "with" unless you do a txn.abort()
> or
> txn.commit() before leaving the block.
> 
> Anyway, that's just the "context" to my question. 
> base.py has a
> generic class "Transaction" that dbdir.py extends in
> BdbTransaction.
> However, neither of these seem to be concerned with DBTxn
> objects.
> I'd like to understand a little more about what Transaction
> and its
> children do.  First off, I want to be sure that the
> Transaction class
> is in the right module (possibly its own).  Second, I
> want to be sure
> that, if I use the context manager I'm playing with, it
> plays nice
> with Transaction.

These are great questions. You have completely surpassed anything and everything I know about how the database works. I have never done any database programming for Gramps - I always stayed away from the BSDDB stuff and I only use the read-only functions in the database interface. I hope somebody else can take a shot at helping you.

~BM

Thanks Brian,  I'll get to the bottom of it all somehow!

On Fri, Jul 24, 2009 at 5:35 PM, Brian Matherly<brian@...> wrote:
>
> Gerald,
>
>> Indeed.  I wonder if you can
>> help me understand another thing:
>>
>> Lately, I've been experimenting with build a context
>> manager for BSDDB
>> transactions.  That is, instead of writing:
>>
>>     txn = self.env.txn_begin()
>>
>>     ...do stuff
>>
>>     txn.commit()
>>
>> you could write:
>>
>>     with BSDT() as txn:  # use new context
>> manager
>>      ...do stuff
>>
>> You don't have to remember to do a txn.commit().  In
>> fact, it's better
>> than that, since the commit will happen automagically at
>> the end of
>> the block controlled by "with" unless you do a txn.abort()
>> or
>> txn.commit() before leaving the block.
>>
>> Anyway, that's just the "context" to my question.
>> base.py has a
>> generic class "Transaction" that dbdir.py extends in
>> BdbTransaction.
>> However, neither of these seem to be concerned with DBTxn
>> objects.
>> I'd like to understand a little more about what Transaction
>> and its
>> children do.  First off, I want to be sure that the
>> Transaction class
>> is in the right module (possibly its own).  Second, I
>> want to be sure
>> that, if I use the context manager I'm playing with, it
>> plays nice
>> with Transaction.
>
> These are great questions. You have completely surpassed anything and everything I know about how the database works. I have never done any database programming for Gramps - I always stayed away from the BSDDB stuff and I only use the read-only functions in the database interface. I hope somebody else can take a shot at helping you.
>
> ~BM
>



-- 
Gerald Britton

Just to let you know that I'm making good progress on this but don't
want to commit the new code for a while.

On Fri, Jul 24, 2009 at 7:24 PM, Gerald Britton<gerald.britton@...> wrote:
> Thanks Brian,  I'll get to the bottom of it all somehow!
>
> On Fri, Jul 24, 2009 at 5:35 PM, Brian Matherly<brian@...> wrote:
>>
>> Gerald,
>>
>>> Indeed.  I wonder if you can
>>> help me understand another thing:
>>>
>>> Lately, I've been experimenting with build a context
>>> manager for BSDDB
>>> transactions.  That is, instead of writing:
>>>
>>>     txn = self.env.txn_begin()
>>>
>>>     ...do stuff
>>>
>>>     txn.commit()
>>>
>>> you could write:
>>>
>>>     with BSDT() as txn:  # use new context
>>> manager
>>>      ...do stuff
>>>
>>> You don't have to remember to do a txn.commit().  In
>>> fact, it's better
>>> than that, since the commit will happen automagically at
>>> the end of
>>> the block controlled by "with" unless you do a txn.abort()
>>> or
>>> txn.commit() before leaving the block.
>>>
>>> Anyway, that's just the "context" to my question.
>>> base.py has a
>>> generic class "Transaction" that dbdir.py extends in
>>> BdbTransaction.
>>> However, neither of these seem to be concerned with DBTxn
>>> objects.
>>> I'd like to understand a little more about what Transaction
>>> and its
>>> children do.  First off, I want to be sure that the
>>> Transaction class
>>> is in the right module (possibly its own).  Second, I
>>> want to be sure
>>> that, if I use the context manager I'm playing with, it
>>> plays nice
>>> with Transaction.
>>
>> These are great questions. You have completely surpassed anything and everything I know about how the database works. I have never done any database programming for Gramps - I always stayed away from the BSDDB stuff and I only use the read-only functions in the database interface. I hope somebody else can take a shot at helping you.
>>
>> ~BM
>>
>
>
>
> --
> Gerald Britton
>



-- 
Gerald Britton

I'm working on the Transaction class and the BdbTransaction class
derived from it.  Basically, this is the setup as far as I can see:

1. At a basic level, we use BSDDB transactions to protect our commits
to the database.  That is, if there is a failure before calling the
commit method, BSDDB will roll back the changes for us or at least
that's how I understand it.  Of course, these are pure record-level
(key/data pair) updates with no intelligence about related
transactions.

2. A a "meta" level, we often group transactions together for purposes
of undo/redo.  This is what the Transaction/BdbTransaction classes are
for.  The groupings are intelligent and usually accompanied by a
human-readable message like "Add Person" which may comprise several
database updates.

The code for #2 is split between base.py and dbdir.py.  I don't think
that this is good.  At the very least, it makes it hard to follow
(took me quite a while to figure it out).  I propose creating a new
module gen.db.metatrans, that covers the functions of #2.  Also the
Transaction class tries to be generic but since we only support BSDDB
at this point I'm not sure that I see the value in it. In any case,
the BdbTransaction class extends Transaction in some ways that we'd
have to duplicate if we went to a second back-end.  For the time being
though, I'd like to move Transaction and BdbTransaction into their own
module, and probably rename them MetaTrans and BdbMetaTrans or
something like that.

Also, key methods to accessing these classes are split between base.py
and dbdir.py but are  NOT methods of Transaction or BdbTransaction.  I
think that this is not the best design.  Among those methods are key
functions like "begin" and "commit."  I propose moving those methods
into the class(es) proper but leave a stub in the GrampsDbDir class so
that we don't have to ripple this change through the codebase.  This
approach would make these classes mirror the general arrangement of
the lower-level transactions in BSDDB which will hopefully help future
readers.

While I'm at it, I'll add context-management methods to the revised
classes, so that modules like EditPerson would be able to do something
like:

with MetaTrans(...) as transaction:
   do stuff

The context manager would automagically commit the metatransaction
when control flows off the end of the  "with" statement unless an
error occurred.

BTW, it appears that the meta commit routines in Transaction do NOT
use BSDDB transactions to protect the db-level changes.  I think that
that is risky.  I'd like to rectify that.

So, unless I've missed something fundamental, I'll get started on this
in the next day or two.

On Tue, Jul 28, 2009 at 2:22 PM, Gerald Britton<gerald.britton@...> wrote:
> Just to let you know that I'm making good progress on this but don't
> want to commit the new code for a while.
>
> On Fri, Jul 24, 2009 at 7:24 PM, Gerald Britton<gerald.britton@...> wrote:
>> Thanks Brian,  I'll get to the bottom of it all somehow!
>>
>> On Fri, Jul 24, 2009 at 5:35 PM, Brian Matherly<brian@...> wrote:
>>>
>>> Gerald,
>>>
>>>> Indeed.  I wonder if you can
>>>> help me understand another thing:
>>>>
>>>> Lately, I've been experimenting with build a context
>>>> manager for BSDDB
>>>> transactions.  That is, instead of writing:
>>>>
>>>>     txn = self.env.txn_begin()
>>>>
>>>>     ...do stuff
>>>>
>>>>     txn.commit()
>>>>
>>>> you could write:
>>>>
>>>>     with BSDT() as txn:  # use new context
>>>> manager
>>>>      ...do stuff
>>>>
>>>> You don't have to remember to do a txn.commit().  In
>>>> fact, it's better
>>>> than that, since the commit will happen automagically at
>>>> the end of
>>>> the block controlled by "with" unless you do a txn.abort()
>>>> or
>>>> txn.commit() before leaving the block.
>>>>
>>>> Anyway, that's just the "context" to my question.
>>>> base.py has a
>>>> generic class "Transaction" that dbdir.py extends in
>>>> BdbTransaction.
>>>> However, neither of these seem to be concerned with DBTxn
>>>> objects.
>>>> I'd like to understand a little more about what Transaction
>>>> and its
>>>> children do.  First off, I want to be sure that the
>>>> Transaction class
>>>> is in the right module (possibly its own).  Second, I
>>>> want to be sure
>>>> that, if I use the context manager I'm playing with, it
>>>> plays nice
>>>> with Transaction.
>>>
>>> These are great questions. You have completely surpassed anything and everything I know about how the database works. I have never done any database programming for Gramps - I always stayed away from the BSDDB stuff and I only use the read-only functions in the database interface. I hope somebody else can take a shot at helping you.
>>>
>>> ~BM
>>>
>>
>>
>>
>> --
>> Gerald Britton
>>
>
>
>
> --
> Gerald Britton
>



-- 
Gerald Britton

Sounds all very good to me.

2009/7/28, Gerald Britton <gerald.britton@...>:
> I'm working on the Transaction class and the BdbTransaction class
> derived from it.  Basically, this is the setup as far as I can see:
>
> 1. At a basic level, we use BSDDB transactions to protect our commits
> to the database.  That is, if there is a failure before calling the
> commit method, BSDDB will roll back the changes for us or at least
> that's how I understand it.  Of course, these are pure record-level
> (key/data pair) updates with no intelligence about related
> transactions.
>
> 2. A a "meta" level, we often group transactions together for purposes
> of undo/redo.  This is what the Transaction/BdbTransaction classes are
> for.  The groupings are intelligent and usually accompanied by a
> human-readable message like "Add Person" which may comprise several
> database updates.
>
> The code for #2 is split between base.py and dbdir.py.  I don't think
> that this is good.  At the very least, it makes it hard to follow
> (took me quite a while to figure it out).  I propose creating a new
> module gen.db.metatrans, that covers the functions of #2.  Also the
> Transaction class tries to be generic but since we only support BSDDB
> at this point I'm not sure that I see the value in it. In any case,
> the BdbTransaction class extends Transaction in some ways that we'd
> have to duplicate if we went to a second back-end.  For the time being
> though, I'd like to move Transaction and BdbTransaction into their own
> module, and probably rename them MetaTrans and BdbMetaTrans or
> something like that.
>
> Also, key methods to accessing these classes are split between base.py
> and dbdir.py but are  NOT methods of Transaction or BdbTransaction.  I
> think that this is not the best design.  Among those methods are key
> functions like "begin" and "commit."  I propose moving those methods
> into the class(es) proper but leave a stub in the GrampsDbDir class so
> that we don't have to ripple this change through the codebase.  This
> approach would make these classes mirror the general arrangement of
> the lower-level transactions in BSDDB which will hopefully help future
> readers.
>
> While I'm at it, I'll add context-management methods to the revised
> classes, so that modules like EditPerson would be able to do something
> like:
>
> with MetaTrans(...) as transaction:
>    do stuff
>
> The context manager would automagically commit the metatransaction
> when control flows off the end of the  "with" statement unless an
> error occurred.
>
> BTW, it appears that the meta commit routines in Transaction do NOT
> use BSDDB transactions to protect the db-level changes.  I think that
> that is risky.  I'd like to rectify that.
>
> So, unless I've missed something fundamental, I'll get started on this
> in the next day or two.
>
> On Tue, Jul 28, 2009 at 2:22 PM, Gerald Britton<gerald.britton@...>
> wrote:
>> Just to let you know that I'm making good progress on this but don't
>> want to commit the new code for a while.
>>
>> On Fri, Jul 24, 2009 at 7:24 PM, Gerald Britton<gerald.britton@...>
>> wrote:
>>> Thanks Brian,  I'll get to the bottom of it all somehow!
>>>
>>> On Fri, Jul 24, 2009 at 5:35 PM, Brian Matherly<brian@...>
>>> wrote:
>>>>
>>>> Gerald,
>>>>
>>>>> Indeed.  I wonder if you can
>>>>> help me understand another thing:
>>>>>
>>>>> Lately, I've been experimenting with build a context
>>>>> manager for BSDDB
>>>>> transactions.  That is, instead of writing:
>>>>>
>>>>>     txn = self.env.txn_begin()
>>>>>
>>>>>     ...do stuff
>>>>>
>>>>>     txn.commit()
>>>>>
>>>>> you could write:
>>>>>
>>>>>     with BSDT() as txn:  # use new context
>>>>> manager
>>>>>      ...do stuff
>>>>>
>>>>> You don't have to remember to do a txn.commit().  In
>>>>> fact, it's better
>>>>> than that, since the commit will happen automagically at
>>>>> the end of
>>>>> the block controlled by "with" unless you do a txn.abort()
>>>>> or
>>>>> txn.commit() before leaving the block.
>>>>>
>>>>> Anyway, that's just the "context" to my question.
>>>>> base.py has a
>>>>> generic class "Transaction" that dbdir.py extends in
>>>>> BdbTransaction.
>>>>> However, neither of these seem to be concerned with DBTxn
>>>>> objects.
>>>>> I'd like to understand a little more about what Transaction
>>>>> and its
>>>>> children do.  First off, I want to be sure that the
>>>>> Transaction class
>>>>> is in the right module (possibly its own).  Second, I
>>>>> want to be sure
>>>>> that, if I use the context manager I'm playing with, it
>>>>> plays nice
>>>>> with Transaction.
>>>>
>>>> These are great questions. You have completely surpassed anything and
>>>> everything I know about how the database works. I have never done any
>>>> database programming for Gramps - I always stayed away from the BSDDB
>>>> stuff and I only use the read-only functions in the database interface.
>>>> I hope somebody else can take a shot at helping you.
>>>>
>>>> ~BM
>>>>
>>>
>>>
>>>
>>> --
>>> Gerald Britton
>>>
>>
>>
>>
>> --
>> Gerald Britton
>>
>
>
>
> --
> Gerald Britton
>

Update:
Project is going well.  I've separated the Transaction class to its own
module and merged in the commit code from the old base.py and dbdir.py.  To
my eye, it looks more readable.  I also added context manager support. Now,
 I'm percolating some related changes through the various editors and trying
to use with...as constructs where possible.

I also moved the gramps 1.3 to 1.4 upgrade code out to it's own module that
is only imported when needed. Initially I did it to reduce the clutter but
now I see we are better positioned for future revisions of the database
structure.

I'm also taking a look at how the undo/redo operations are handled.  I'm
leaning towards splitting them out to their own module and creating a
GrampsDbUndo class to manage it.  However, along the way I got curious about
one thing:  Why is it an on-disk database?  Just creating the undo database
at every startup uses about 150k of memory, probably enough for 250-500
average undo transactions if they were kept in a list or dictionary in
memory.  The undo database is not persistent; it is created afresh upon each
startup.  Currently there is a hard-coded limit of 1000 undo operations.  If
kept in memory 1000 operations would probably be less than 500k and of
course much faster.

What I will do for now, is make it possible to switch between an on-disk
undo database and in-memory version built on a Python list.  Then we can
play with it both ways before deciding which way to go.

Note: I still haven't committed any of this work.  I've a ways to go and I
want to do pretty through unit testing before I do that.

Just keeping everyone in the loop!

On Mon, Aug 3, 2009 at 1:30 PM, Benny Malengier
<benny.malengier@...:

> Sounds all very good to me.
>
> 2009/7/28, Gerald Britton <gerald.britton@...>:
> > I'm working on the Transaction class and the BdbTransaction class
> > derived from it.  Basically, this is the setup as far as I can see:
> >
> > 1. At a basic level, we use BSDDB transactions to protect our commits
> > to the database.  That is, if there is a failure before calling the
> > commit method, BSDDB will roll back the changes for us or at least
> > that's how I understand it.  Of course, these are pure record-level
> > (key/data pair) updates with no intelligence about related
> > transactions.
> >
> > 2. A a "meta" level, we often group transactions together for purposes
> > of undo/redo.  This is what the Transaction/BdbTransaction classes are
> > for.  The groupings are intelligent and usually accompanied by a
> > human-readable message like "Add Person" which may comprise several
> > database updates.
> >
> > The code for #2 is split between base.py and dbdir.py.  I don't think
> > that this is good.  At the very least, it makes it hard to follow
> > (took me quite a while to figure it out).  I propose creating a new
> > module gen.db.metatrans, that covers the functions of #2.  Also the
> > Transaction class tries to be generic but since we only support BSDDB
> > at this point I'm not sure that I see the value in it. In any case,
> > the BdbTransaction class extends Transaction in some ways that we'd
> > have to duplicate if we went to a second back-end.  For the time being
> > though, I'd like to move Transaction and BdbTransaction into their own
> > module, and probably rename them MetaTrans and BdbMetaTrans or
> > something like that.
> >
> > Also, key methods to accessing these classes are split between base.py
> > and dbdir.py but are  NOT methods of Transaction or BdbTransaction.  I
> > think that this is not the best design.  Among those methods are key
> > functions like "begin" and "commit."  I propose moving those methods
> > into the class(es) proper but leave a stub in the GrampsDbDir class so
> > that we don't have to ripple this change through the codebase.  This
> > approach would make these classes mirror the general arrangement of
> > the lower-level transactions in BSDDB which will hopefully help future
> > readers.
> >
> > While I'm at it, I'll add context-management methods to the revised
> > classes, so that modules like EditPerson would be able to do something
> > like:
> >
> > with MetaTrans(...) as transaction:
> >    do stuff
> >
> > The context manager would automagically commit the metatransaction
> > when control flows off the end of the  "with" statement unless an
> > error occurred.
> >
> > BTW, it appears that the meta commit routines in Transaction do NOT
> > use BSDDB transactions to protect the db-level changes.  I think that
> > that is risky.  I'd like to rectify that.
> >
> > So, unless I've missed something fundamental, I'll get started on this
> > in the next day or two.
> >
> > On Tue, Jul 28, 2009 at 2:22 PM, Gerald Britton<gerald.britton@...
> >
> > wrote:
> >> Just to let you know that I'm making good progress on this but don't
> >> want to commit the new code for a while.
> >>
> >> On Fri, Jul 24, 2009 at 7:24 PM, Gerald Britton<
> gerald.britton@...>
> >> wrote:
> >>> Thanks Brian,  I'll get to the bottom of it all somehow!
> >>>
> >>> On Fri, Jul 24, 2009 at 5:35 PM, Brian Matherly<
> brian@...>
> >>> wrote:
> >>>>
> >>>> Gerald,
> >>>>
> >>>>> Indeed.  I wonder if you can
> >>>>> help me understand another thing:
> >>>>>
> >>>>> Lately, I've been experimenting with build a context
> >>>>> manager for BSDDB
> >>>>> transactions.  That is, instead of writing:
> >>>>>
> >>>>>     txn = self.env.txn_begin()
> >>>>>
> >>>>>     ...do stuff
> >>>>>
> >>>>>     txn.commit()
> >>>>>
> >>>>> you could write:
> >>>>>
> >>>>>     with BSDT() as txn:  # use new context
> >>>>> manager
> >>>>>      ...do stuff
> >>>>>
> >>>>> You don't have to remember to do a txn.commit().  In
> >>>>> fact, it's better
> >>>>> than that, since the commit will happen automagically at
> >>>>> the end of
> >>>>> the block controlled by "with" unless you do a txn.abort()
> >>>>> or
> >>>>> txn.commit() before leaving the block.
> >>>>>
> >>>>> Anyway, that's just the "context" to my question.
> >>>>> base.py has a
> >>>>> generic class "Transaction" that dbdir.py extends in
> >>>>> BdbTransaction.
> >>>>> However, neither of these seem to be concerned with DBTxn
> >>>>> objects.
> >>>>> I'd like to understand a little more about what Transaction
> >>>>> and its
> >>>>> children do.  First off, I want to be sure that the
> >>>>> Transaction class
> >>>>> is in the right module (possibly its own).  Second, I
> >>>>> want to be sure
> >>>>> that, if I use the context manager I'm playing with, it
> >>>>> plays nice
> >>>>> with Transaction.
> >>>>
> >>>> These are great questions. You have completely surpassed anything and
> >>>> everything I know about how the database works. I have never done any
> >>>> database programming for Gramps - I always stayed away from the BSDDB
> >>>> stuff and I only use the read-only functions in the database
> interface.
> >>>> I hope somebody else can take a shot at helping you.
> >>>>
> >>>> ~BM
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Gerald Britton
> >>>
> >>
> >>
> >>
> >> --
> >> Gerald Britton
> >>
> >
> >
> >
> > --
> > Gerald Britton
> >
>



-- 
Gerald Britton

I'm just about ready to start committing this work.  Out of an abundance of
caution, I propose to do it in phases:
Phase 1: Commit new modules only.  This will leave existing functionality
intact and not change anything in the build.

Phase 2: Commit build changes.  This will take care of translation and make
files to include the new work, but not activate the new code.

Phase 3: Commit final changes to activate the new code.  Turn it loose for
broader testing in the dev community

Phase 4: Remove any modules deprecated by the new structure.

As I go I will be throughly testing the new code against all gramps
functions from create to editing to reports to database repair and rebuild.
 I expect that each phase will include clean-up work as well.

I should be ready to begin the phased commits within the next week.  Please
let me know if you have any issues with the plan.  As I do the commits, I'll
be blogging about the work and the benefits I hope to be achieve.



On Wed, Aug 5, 2009 at 12:57 PM, Gerald Britton <gerald.britton@...:

> Update:
> Project is going well.  I've separated the Transaction class to its own
> module and merged in the commit code from the old base.py and dbdir.py.  To
> my eye, it looks more readable.  I also added context manager support. Now,
>  I'm percolating some related changes through the various editors and trying
> to use with...as constructs where possible.
>
> I also moved the gramps 1.3 to 1.4 upgrade code out to it's own module that
> is only imported when needed. Initially I did it to reduce the clutter but
> now I see we are better positioned for future revisions of the database
> structure.
>
> I'm also taking a look at how the undo/redo operations are handled.  I'm
> leaning towards splitting them out to their own module and creating a
> GrampsDbUndo class to manage it.  However, along the way I got curious about
> one thing:  Why is it an on-disk database?  Just creating the undo database
> at every startup uses about 150k of memory, probably enough for 250-500
> average undo transactions if they were kept in a list or dictionary in
> memory.  The undo database is not persistent; it is created afresh upon each
> startup.  Currently there is a hard-coded limit of 1000 undo operations.  If
> kept in memory 1000 operations would probably be less than 500k and of
> course much faster.
>
> What I will do for now, is make it possible to switch between an on-disk
> undo database and in-memory version built on a Python list.  Then we can
> play with it both ways before deciding which way to go.
>
> Note: I still haven't committed any of this work.  I've a ways to go and I
> want to do pretty through unit testing before I do that.
>
> Just keeping everyone in the loop!
>
>
> On Mon, Aug 3, 2009 at 1:30 PM, Benny Malengier <benny.malengier@...
> > wrote:
>
>> Sounds all very good to me.
>>
>> 2009/7/28, Gerald Britton <gerald.britton@...>:
>> > I'm working on the Transaction class and the BdbTransaction class
>> > derived from it.  Basically, this is the setup as far as I can see:
>> >
>> > 1. At a basic level, we use BSDDB transactions to protect our commits
>> > to the database.  That is, if there is a failure before calling the
>> > commit method, BSDDB will roll back the changes for us or at least
>> > that's how I understand it.  Of course, these are pure record-level
>> > (key/data pair) updates with no intelligence about related
>> > transactions.
>> >
>> > 2. A a "meta" level, we often group transactions together for purposes
>> > of undo/redo.  This is what the Transaction/BdbTransaction classes are
>> > for.  The groupings are intelligent and usually accompanied by a
>> > human-readable message like "Add Person" which may comprise several
>> > database updates.
>> >
>> > The code for #2 is split between base.py and dbdir.py.  I don't think
>> > that this is good.  At the very least, it makes it hard to follow
>> > (took me quite a while to figure it out).  I propose creating a new
>> > module gen.db.metatrans, that covers the functions of #2.  Also the
>> > Transaction class tries to be generic but since we only support BSDDB
>> > at this point I'm not sure that I see the value in it. In any case,
>> > the BdbTransaction class extends Transaction in some ways that we'd
>> > have to duplicate if we went to a second back-end.  For the time being
>> > though, I'd like to move Transaction and BdbTransaction into their own
>> > module, and probably rename them MetaTrans and BdbMetaTrans or
>> > something like that.
>> >
>> > Also, key methods to accessing these classes are split between base.py
>> > and dbdir.py but are  NOT methods of Transaction or BdbTransaction.  I
>> > think that this is not the best design.  Among those methods are key
>> > functions like "begin" and "commit."  I propose moving those methods
>> > into the class(es) proper but leave a stub in the GrampsDbDir class so
>> > that we don't have to ripple this change through the codebase.  This
>> > approach would make these classes mirror the general arrangement of
>> > the lower-level transactions in BSDDB which will hopefully help future
>> > readers.
>> >
>> > While I'm at it, I'll add context-management methods to the revised
>> > classes, so that modules like EditPerson would be able to do something
>> > like:
>> >
>> > with MetaTrans(...) as transaction:
>> >    do stuff
>> >
>> > The context manager would automagically commit the metatransaction
>> > when control flows off the end of the  "with" statement unless an
>> > error occurred.
>> >
>> > BTW, it appears that the meta commit routines in Transaction do NOT
>> > use BSDDB transactions to protect the db-level changes.  I think that
>> > that is risky.  I'd like to rectify that.
>> >
>> > So, unless I've missed something fundamental, I'll get started on this
>> > in the next day or two.
>> >
>> > On Tue, Jul 28, 2009 at 2:22 PM, Gerald Britton<
>> gerald.britton@...>
>> > wrote:
>> >> Just to let you know that I'm making good progress on this but don't
>> >> want to commit the new code for a while.
>> >>
>> >> On Fri, Jul 24, 2009 at 7:24 PM, Gerald Britton<
>> gerald.britton@...>
>> >> wrote:
>> >>> Thanks Brian,  I'll get to the bottom of it all somehow!
>> >>>
>> >>> On Fri, Jul 24, 2009 at 5:35 PM, Brian Matherly<
>> brian@...>
>> >>> wrote:
>> >>>>
>> >>>> Gerald,
>> >>>>
>> >>>>> Indeed.  I wonder if you can
>> >>>>> help me understand another thing:
>> >>>>>
>> >>>>> Lately, I've been experimenting with build a context
>> >>>>> manager for BSDDB
>> >>>>> transactions.  That is, instead of writing:
>> >>>>>
>> >>>>>     txn = self.env.txn_begin()
>> >>>>>
>> >>>>>     ...do stuff
>> >>>>>
>> >>>>>     txn.commit()
>> >>>>>
>> >>>>> you could write:
>> >>>>>
>> >>>>>     with BSDT() as txn:  # use new context
>> >>>>> manager
>> >>>>>      ...do stuff
>> >>>>>
>> >>>>> You don't have to remember to do a txn.commit().  In
>> >>>>> fact, it's better
>> >>>>> than that, since the commit will happen automagically at
>> >>>>> the end of
>> >>>>> the block controlled by "with" unless you do a txn.abort()
>> >>>>> or
>> >>>>> txn.commit() before leaving the block.
>> >>>>>
>> >>>>> Anyway, that's just the "context" to my question.
>> >>>>> base.py has a
>> >>>>> generic class "Transaction" that dbdir.py extends in
>> >>>>> BdbTransaction.
>> >>>>> However, neither of these seem to be concerned with DBTxn
>> >>>>> objects.
>> >>>>> I'd like to understand a little more about what Transaction
>> >>>>> and its
>> >>>>> children do.  First off, I want to be sure that the
>> >>>>> Transaction class
>> >>>>> is in the right module (possibly its own).  Second, I
>> >>>>> want to be sure
>> >>>>> that, if I use the context manager I'm playing with, it
>> >>>>> plays nice
>> >>>>> with Transaction.
>> >>>>
>> >>>> These are great questions. You have completely surpassed anything and
>> >>>> everything I know about how the database works. I have never done any
>> >>>> database programming for Gramps - I always stayed away from the BSDDB
>> >>>> stuff and I only use the read-only functions in the database
>> interface.
>> >>>> I hope somebody else can take a shot at helping you.
>> >>>>
>> >>>> ~BM
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Gerald Britton
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Gerald Britton
>> >>
>> >
>> >
>> >
>> > --
>> > Gerald Britton
>> >
>>
>
>
>
> --
> Gerald Britton
>



-- 
Gerald Britton

I've finally arrived at a stage where I feel confident enough to
commit this work. I am adding several new modules and am looking for
some input on meaningful names for them. We will have:

base20.py        old base.py, used by ImportGrdb to read old version
2.0 databases
base.py            existing module, updated to contain only abstract methods
bsddbtxn.py      new module wrapping bsddb transactions with python
context manager
cursor.py          existing module
dbconst.py       existing module, with new constants added
dbdir.py            existing module, but will be deprecated
exceptions.py   existing module, no changes
grampsdbtxn.py new module implementing GrampsDbTxn class
grampsup14.py  new module with functions to update a database at open
time (moved from
                        current dbdir.py)
_init__.py          existing module
iterator.py          existing module
mixin.py            mixin to add methods for use by ImportXml and
ImportGedcom (extracted
                        from current base.py
read.py             new module implementing GrampsDbRead class
undoredo.py      new module implementing GrampsDbUndo class
write.py            new module implementing GrampsDbWrite class

read.py, write,py and undoredo.py are probably OK names.  I'm
wondering about grampsdbtxn.py, mixin.py, grampsup14.py and
bsddbtxn.py.  I'm inclined to rename grampsup14.py to upgrade.py since
we might be adding more dynamic conversions in the future.
Grampsdbtxn.py could be just txn.py or transaction.py or trans.py, I
suppose. Mixin.py might be better called importmixin.py or some
variation on that.

Anyway, please let me know your thoughts.

Gerald Britton

On Tue, Aug 18, 2009 at 1:49 PM, Gerald Britton <gerald.britton@...:

> I've finally arrived at a stage where I feel confident enough to
> commit this work. I am adding several new modules and am looking for
> some input on meaningful names for them. We will have:
>
> base20.py        old base.py, used by ImportGrdb to read old version
> 2.0 databases
> base.py            existing module, updated to contain only abstract
> methods
> bsddbtxn.py      new module wrapping bsddb transactions with python
> context manager
> cursor.py          existing module
> dbconst.py       existing module, with new constants added
> dbdir.py            existing module, but will be deprecated
> exceptions.py   existing module, no changes
> grampsdbtxn.py new module implementing GrampsDbTxn class
> grampsup14.py  new module with functions to update a database at open
> time (moved from
>                        current dbdir.py)
> _init__.py          existing module
> iterator.py          existing module
> mixin.py            mixin to add methods for use by ImportXml and
> ImportGedcom (extracted
>                        from current base.py
> read.py             new module implementing GrampsDbRead class
> undoredo.py      new module implementing GrampsDbUndo class
> write.py            new module implementing GrampsDbWrite class
>
> read.py, write,py and undoredo.py are probably OK names.  I'm
> wondering about grampsdbtxn.py, mixin.py, grampsup14.py and
> bsddbtxn.py.  I'm inclined to rename grampsup14.py to upgrade.py since
> we might be adding more dynamic conversions in the future.
> Grampsdbtxn.py could be just txn.py or transaction.py or trans.py, I
> suppose. Mixin.py might be better called importmixin.py or some
> variation on that.
>
> Anyway, please let me know your thoughts.
>

Gerald,

Rarely does one need to have "gramps" in the name of anything below the root
(usually doesn't add any meaning) so I'd be in favor of something without
that in it. Also, if you can make it clear which ones have the low-level
code, and which ones are more abstract, that would be useful.

The first thing I will probably try to do is build a simple, in-memory
dictionary-based DB to make sure I understand all of the dependencies. From
what understand of the above is that I need only implement a transaction
file for this, correct? If that is true, what will be the naming scheme for
alternate db implementations of the transaction? trans_mem, trans_django?

Thanks again for all of this hard work!

-Doug


>
> Gerald Britton
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Gramps-devel mailing list
> Gramps-devel@...
> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>

Gerald,

> base20.py        old base.py, used by ImportGrdb to read old version 2.0 databases

If base20.py is only used by ImportGrdb.py (which would be totally awesome!), it can be moved into plugins/lib. A nice name for it would be "libgrdb.py".

~Brian