Re: [SQLObject] thoughts on structure and components

[Bud P. B. <bu...@si...>]

Hi Ian,

many thanks for your feedback.  I respond below..


On 08 May 2003 22:13:51 -0500
Ian Bicking <ia...@co...> wrote:

> I feel confident that foreign keys are hard, and I'm okay with that. 
> Which is to say -- the interface could use improvement (e.g., remove the
> double defining of the foreign key and the external join), but I don't
> think there's deep problems.

I'm experimenting a bit with my thoughts and yet have to figure out
whether my intuitions verify.  If I get some code to work, I'll send
it and we'll talk about s.th. more concrete.  What triggered my
thinking was a difficulty that didn't match _my_ intuition--so maybe
it reveals more about my thinking than it describes SQLObject..


> > Application Object:
> >   An Application Object is independent of backend storage
> 
> Independent of "backend storage", or independent of any particular
> backend storage that fits the rest of the model?
> 
> I.e., independent like SQLObject currently is (by changing _connection),
> or something more (like, doesn't inherit from SQLObject)...?

In my thought experiment, I thought it would be nice to write a class
without worries about persistance--just the business logic.  In a
second step (in my line of thought using inheritance), this is made
persistant.  This approach would make it possible to easily eachange
persistance layers... But it is really more of a devide and conquer
motivation...  Not sure whether that is possible--but think for
example of pre-existing classes that you want to make persistent..


> > Storable:
> >   A superclass that makes application objects storable.  
> >   It adds methods that interact with the Backend in order to
> >   - store
> >   - update
> >   - delete
> >   the object
> >   In addition, it provided methods to easily access related objects
> 
> I can see how this might be stylistically different from the SQLObject
> class, but how is it otherwise different?  (Well, there's SQLObject and
> MetaSQLObject, but those are tightly coupled, and I think justifiably
> so).

Maybe this is not at all different from SQLObject???  Can I write a
class and then make what I call a storable application class that
inherits from both the business-logic-only class and SQLObject?


> SQLObject is tightly coupled to DBConnection (i.e., they chat a lot,
> there's no well defined interface between the two, and information
> that's available between the two of them isn't necessarily otherwise
> public).  I can imagine them being decoupled, but only after SQLObject
> had a long time to mature.

I believe that my thoughts kind of split up DBConnection into several
different entities (connection, dbdriver, and maybe also backend?).
This way, I can for example define what I call mappings before opening
an actual connection to the dbms.  I see this as a sort of decoupling
of components..

> Now, SQLObject does place some restrictions on how you can create your
> classes, and some (though less) restrictions on the backend (the backend
> is less restricted because an RDBMS is already a much more restricted
> environment than Python).
> 
> But SQLObject is *not* an OODBMS (with RDBMS backends).  It's not ZODB
> or any of those... they use the RDBMS as a backend, where in SQLObject
> the Python class is more-or-less the slave to the RDBMS model and
> backend.  

I believe I think along the same lines.  I don't attempt to hide the
RDBMS issues as an oodb would nor have I (so far) thought of hiding
storage operations...  (Again, I describe my thoughts and don't
necessarily relate to SQLObject)

I believe compared to SQLObject (which is difficult at my current
vagueness of thought), I may have an additional abstraction layer
between objects and the db, represented by the Attr class that
implements my "higher-level" physical types.  

> There's a strong conceptual tie as well, which is a large
> benefit -- even if you let SQLObject create you schema, and don't touch
> the database except through SQLObject, you are using relational
> metaphors, which many people are comfortable with (and for more reasons
> than familiarity).
> 
> Inheritance falls into this, IMHO.  You can't support Python inheritance
> in an RDBMS.  

Actually, some RDBMS such as PostgreSQL support inheritance.   But I
haven't thought of implementing inheritance much.  I don't think along
the lines of an oodbms such as Zope DB...

> You can support something that vaguely approximates Python
> inheritance, and can solve some of the same problems, but it's not the
> same thing.

I guess you're right.  Haven't thought about this in detail.  

> But maybe you are really thinking of a super-superclass, i.e., something
> that SQLObject would inherit from, as well as other significantly
> different kinds of storage mechanisms.

I don't think so, but I don't understand the super-super concept
well...

> > Backend:
> >   An object that manages the interaction with the actual database.
> >   It has the following components:
> >   - mapping
> >   - dbDriver
> >   - dbConnection
> >   A first version can live with a single driver and connection--a later
> >   version may be able to work with multiple databases
> > 
> > Connection:
> >   An actual connection or connection pool that interacts with the dbms
> > 
> > DBdriver:
> >   A class that provides functionality that depends on the kind of connection
> >   used, typically depending on:
> >   - dbms used (e.g., postgres)
> >   - DBI module (and version)
> >   The DBdriver supports a subset of dbCols
> 
> BConnection is something of a misnomer -- DBDriver would probably be a
> better name (especially for the poorly named DBMConnection, which
> doesn't connect to anything).  Well, in the beginning DBConnection was
> really just a primitive form of connection pool, but that part is
> relatively small now.  There's nothing particularly important about the
> way they are joined.  If they weren't, though, the interface would look
> pretty much the same (since the type of connection you make implies
> which DBDriver you would use).  Separating DBConnection from DBDriver
> isn't a very significant change, since they are naturally coupled.

I separated the two since it makes it possible to define mappings
before having the connection in place.  Trying to devide and conquer,
I tried to separate the creation of SQL (strings) and actually
communicating with the dmbs..

> Now, there is the possibility of different kinds of connection pooling
> and general connection usage, but that's orthogonal to both connections
> themselves, and the drivers for those connections.
> 
> > Mapping:
> >   Description of the physical representation in a dbms-independent way.  
> >   The mapping is described by the following components:
> >   - Attributes of Storable Objects
> >   - Relationships of Storable Objects
> 
> Now, if I'm reading it right, this would be the most significant
> difference with SQLObject -- a mapping object is a separate from the
> class.  But why?  

Yes, I believe this is probably the most significant change--probably
at the core of what I call "weak coupling".  A mapping just
_describes_ the storage decisions made, doing nothing else at all.
While decoupling was my prime motivation for thinking this way, it may
also make it easier to cleanly separate representation decisions from
the rest of the code, making it easier to modify, tune, etc.

> Hmm... on second thought I see a reason that while you
> would potentially define fields in the class, you may wish to define the
> way those map to a database separately.  Then the class wouldn't be tied
> so much to SQLObject -- a very different kind of mapping could be used
> to take the same class and store it over SOAP.

Exactly

> I still think the current definitions would be easier and clearer for
> most uses.  At some point you may put it on a different backend, that
> may have different database names (for instance).

Just my intuition, that could be wrong--but I think whether to
separate or not has very little impact on the ease of defining the
representation decisions.  Something to verify..

My approach basically follows a pattern (maybe not in a formal sense)
that I have used in other projects of mine:  to represent just
decisions as objects and separate how to act on those decisions in
different classes..

> Right now you could hack something together to do this by using a Style
> subclass that fed off a mapping table (instead of using algorithms to
> map names).

This surprises me.  I see style and mapping as reasonably independent
things.  Style just defines some defaults (such as converting names
form case sensitive to case insensitive) and if I chose an appropriate
style, it saves me some typing.  

> But maybe I'm not clear on what you envision for mapping, since I would
> expect it would belong to the class 

I believe that here we think differently.  I explicitly want the
mapping to live independently of the class.  Maybe overly simplified,
an object asks the backend to store it, the backend matches a mapping
with the class of the object, asks the dbdriver to produce the
appropriate sql (passing it mapping and object) and asking the
connection to send it off...  But the essence of decoupling is that
the mapping is independent of the application class itself.  

Luke recently pointed to some Ambler articles and maybe my thoughts
are influenced by that.  The architecture Ambler describes actually
keeps the mappings in an rdbms and offers an administration interface
for mappings.  Obviously, I don't attempt to go that far..

> (with the class having N mappings
> for N backends).  I don't see how it would be part of the backend, so
> maybe you have something else in mind.

The mapping is mostly independent of the dmbs used.  So I don't see a
need to keep multiple mappings for a given class for different
backends.  Backend, in my understanding of the concept, is a class
that brings together some components (mapping, connection, dbdriver,
etc) and knows how to make them play together.  It provides the single
interface to storable objects.  A given object class will alway relate
to a single backend;  different classes may use different backends
since, for example, their data resides in different dmbs...

I suppose my quick writeup wasn't very clear... (bad, but difficult to
avoid...)

> > DbCol:  
> >   A low level physical type to represent a single field in the dbms. It is
> >   an abstraction of the actual SQL type since different DBdrivers can map it
> >   to different SQL expressions.  The types are named closely to standard
> >   SQL. Types such as TIMESTAMP could for example be translated to DateTime
> >   by a mySQL dbDriver.  
> 
> Pretty much Col...

Yes, exactly.  


> > Attribute:
> >   A high-level physical type to represent the higher-level physical rep. 
> >   This includes marshalling to and from a lower-level representation. 
> >   Marshalling can do things such as encoding structured objects (phone
> >   number with country and area code) to strings, pickling, mapping a single
> >   attribute (a point) to multiple columns (x, y or lat,long), etc.  
> 
> And something like this will come around at some point, it's just not
> developed yet.

Great!  So I hope my thoughts can help shape this...

I have a little newer version of my class diagram that I attach in
case you are interested in following the progress of my thoughts...

thanks for the interesting discussion

cheers

--b