An idea:
Some days ago, a co-worker, who has joined the group recently (dmartinez), proposed an idea to me, and we started to dedicate some time to improve it.
This is the idea: whenever you make an OO design, u think about objects, functionality and properties, not in tables and foreign keys. But when it comes to actual programming, it's inevitable: all data goes to an RDBMS.
But what happens when u have to extend the model? Simple: u inherit the classes and add functionality. But tables can't inherit another one, so u're stuck. This is where the idea goes. To put an OO layer on top of the DB, complete with a design tool for modeling objects and relations, and mapping them transparently to the DB tables/stored procedures/constraints, etc.
But this isn't a Visio design style. It should generate XML schemas and namespaces/classes for them, so u interact programatically with the objects, not the tables. U should be able to filter, sort, select, etc. directly to the objects, and this layer should eficiently execute SP's or anything against the DB.
Imagine:
  DBClient.FirstName.Filter("%John%");
  DBClient.Load();
Or
  DBInvoice.DeliverState = DBLayer.DBInvoice.DeliverStateType.Requested;
  DBClient.Sex = DBLayer.Global.Sex.Female;
Or adding validation formulas to the field, for auto validation in forms/webforms, the mentioned tagged values specification, and other properties.
All these should seamlessly integrate with Intellisense, and define it's metadata in XML, allowing easy migration from one RDBMS to another.
It could even be an option the use of stored procedures, and the connection to the actual DB should be made through the graphical design tool.
This way, when u add a new object, if it's inherited from another object (let's suppose it is the base object, in which case it will be a table), the new table created will contain only the additional properties defined, and a GUID field would join the two tables transparently. The programmer will work and see only one new object, when they are really two tables with additional metadata properties.
This could be a big productivity enhancer...
I'd like to hear your thoughts.

Daniel

Setting aside theoretical issues for the moment:

Will we have to replicate Microsoft's typed dataset functionality before we have the chance to implement any of these additional features?

It wouldn't be a replica, as typed datasets don't allow creation of tables, inheritance between tables for extensibility, nor enum values metaphora for field values (like the Sex case, or some DeliverState field for example). And you have to load them from a DatasetCommand anyway, and that means sending a SQL statement, or executing a stored procedure, not very OO indeed.
From an application point of view, you will still be interacting with a typed dataset, I guess... maybe the bridge should accept these datasets to update the underlying tables, but the idea is to isolate the DB design.
I think the main idea is metadata extensibility (or extensible attributes to fields, put another way) and inheritance at the DB level. If the RDBMS already had these things, there would be no need to think of it.
Think of what happens when you have a simple app, which has a simple Client table, which is accesed from a simple Client class, and you start a new project and need some additional data in the client table. Sure u can reuse the class and inherit it and whatever, but when this class loads itself, if will do so from a table, and u'll have to change this part. And you'll have to duplicate the structure of the Client table to add the fields without disturbing existing apps, or maybe add another table with the additional fields, and make the load method do a join. Or modify all stored procedures.
It sounds to me like we have all the OO power in our hands now, but the DB is becoming the new bottleneck.
The layer will be for sure an assembly, but one that should be built dynamically by the graphic designer, all with its classes, enums, and whatever, hiding the underlying DB tables, SP's and whatever.
It would make my programmer life easier, for sure...

How about these issues?

+ object identity?
Relational databases have no such concept.  OO mandates it.  You can fudge it by introducing object id keys into tables, but that ruins the purity of the relational model somewhat.

+ cache coherency?
Two applications accessing the same database won't have the same objects.  One application could be caching old data in its objects, while the other application may have updated the RDBMS.

+ querying?
It's all well if you could navigate the databases via implicit joins, but there is still the need to make queries, especially across one-to-many relationships, where indexing isn't what we always want.  I am really after something that is OO, but with the full expressiveness of SQL.

+ object identity?
The adding of ID keys to DB is inevitable, but that doesn't ruin anything, as u'll be interacting with the Bridge objects and the Bridge GUI, not the tables and IDs.
+ cache coherency?
This is the exact problem you will also have when using datasets, loading the results and modifying them... the platform may provide conflict resolution or anything else, but this is independent from the concept. Maybe the resolution method could be another functionality of the Bridge GUI and objects. The platform's objects would't contain data themselves, they would be just that, an OO bridge to the RDBMS, let's say static classes and methods maybe
+ querying?
What do u mean by "indexing"?. This is the scenario I view:
Bridge.Client.FirstName.Filter(FilterType.Equals, "John");
Bridge.Client.PurchaseOrders.DateOrdered.Filter(FilterType.GreaterThan, "01/01/2001");
Bridge.Client.FillDataSet(MyDataSet);

You would have selected the desired records, have the dataset loaded all with the relation between Client->Purchases set in his in-memory data, and all without a clue of which tables you were touching, nor SQL strings and the like. And the Bridge could have executed this through a stored procedure optimized for the type of DB selected.

The best functionality scenario of the Bridge would be:
1- you have a method in which u update the status of a purchase order to "C" which means in your DB "Canceled"
2- At some time, by design, you no longer admit "Canceled" values.
3- The app will continue to compile, but u would have to search all through it to find that status changed.
4- With the Bridge u would have deleted the "Canceled" status from the possible values of the object Purchase.
5- So the line where u changed the order' state now would be Bridge.PurchaseOrder.Status = PurchaseStatus.Canceled
6- After updating the Bridge and the undelying tables, your line of code won't compile again unless u change it, and u would have avoided to have a logic bug somewhere. Just recompiling would tell u if everything is still OK.

Daniel

> Bridge.Client.FirstName.Filter(FilterType.Equals, "John");

Which means?:

SELECT * FROM Client where Client.FirstName = 'John';

So that the return type is?:

Client_Set c = Bridge.Client.FirstName.Filter(FilterType.Equals, "John");

Or should that be?:

Client c = Bridge.Client.FirstName.Filter(FilterType.Equals, "John");

where we are treating the Client class as both the single record and a record set.

The second looks more appealing.

Well, I was thinking more in something like this:
Bridge.Client.FirstName.Filter(FilterType.Equals, "John");
Bridge.Client.FillDataSet(MyDataSet);

And this dataset could be a strong typed one generated by the same Bridge, what do u think?
Or it could return a collection of Client objects...
But I see the Bridge classes as interfaces and functions, not data themselves, which could me managed with datasets, I don't know. Maybe there should be Admin classes, to filter, etc., and Data classes with the actual data and collections.

use an XML snap shot to hold onto the metadata for any object, this abstraction is enough to keep the DB and the application services seperate in their schema. The mapping is trivial and can be setup when new object types are created.

Jason, plase read my other postings on the subject, 'cause it seems like u missed the point on the idea...
The functionality the Bridge should provide is just impossible with an XML snapshot of whatever.
If u follow this path, you're going with Dataset's XSDs, and that doesn't provide u anything else than a structure. You'll be querying tables, making joins, selects, etc. And y cant inherit anything with an XML snaphot.
Or maybe I didn't undestand your alternative

Daniel

Daniel,

Sorry, I didn't provide enough detail. What I am thinking about is using metadata also stored in a database to describe object structures. When you animate an object from a document stored in a DB you can use something like reflection (.NET has a namespace for this) to peek into it's structure. You can use this same metadata in modeling and derving new object types and their methods & props, using code generate (also in .NET) to handle producing database repositories, object code, and specifications documents. Some of this approach can be seen in CRM solutions like Pivotal, and in the OODB space with the Objectivity database. When you bring it all together in a framework with modeling you end up using logical functions of object orientation to drive physical needs. .NET appears to be built around this and already has tools built in to make the effort more productive.

Jason

Jason, I have the following questions regarding your idea:
1- What exactly do you mean by "animate an object"?
2- Using Reflection is right to get late binding, but it is surelly a pain at design time, specially if all your objects will be contructed and filled with properties and methods at runtime (I think that was the idea)
3- I've been watching Code generate namespace in .NET, but it has 0 documentation, but looks great!

My reflection: in the last paragraph, your idea is just identical to mine, but you started from the other extreme of the functionality. I see it the other way: you design the objects graphically in the modeling tool, u save this structure and relations as XML (for porting to other RDBMS), generate the tables, use code generation to create the classes, methods, properies, enums, etc., and compile it.
Therefore, at design time, you have full Intellisense and no reflection!
I think at last there's no difference between the two approaches, it's just a detail of implementation.

Daniel.

animate is a word I have and seen used to signify the creation of an object type and the population of default state from a data store. Rehydration is another term used by some industry folks. Since objects can have all kinds of properties and behaviors you can write systems that have object types where instances can be personified, thus the action words

I think you are absolutely right, we are both talking about the same type of solution, and I think where you are going with the addition of rich modeling and code generation in the framework is going to be a huge win for all of us

Jason

I'm working on a prototype document specification I'll make public ASAP for discussion. I can't wait to start serious coding, but we should agree on an arquitecture first.

Daniel

I have templates for a wide range of spec docs and a content guide available for review if the group is interested.

Jason

We sould use the Yahoo group for easy exchange of files.
If u can, send them to
http://groups.yahoo.com/group/dotnetopensource/files/SourceForge%20Files/
and I'll peek them from there.

Daniel

I put the source code zip and a general doc up in the Yahoo group file store

How about writing our own DBMS that includes OO and attributed programming features?

Maybe a RDBMS standard implementation we could rip off some database implementation text book, and then add support for inheritance and methods?

It would be much less of a hack than to build complex SQL strings behind the scenes with each access/update.

The reason I say this, is because I think we will exceed the limitations of an RDBMS pretty quickly (eg. performance of complex joins, complexities of naming conventions, lack of support for triggers, etc).

Another annoying thing about RDBMS, is I have to set up DSNs, and such.  Often, I just want to write an application with a database in a file/directory.  Such a project would start off as a sort of embedded database, and possibly later become a database server if there is demand.

Well, that surely is a huge task...
I'm not still convinved of full OO DB is fast enough (some time ago I read about Jazmine, I think)... but it could be...
I don't know if it's a task we can jump on right know... maybe the first bridge could be usefull to detect potential needs, functionality, and at a later time, when we all master the .NET runtime, we can go for the OODB.

That is fair enough.

John,

What you is 100% true, I've already tried it and the idea does suffer from many of the things you speak of. And then there is database independance to make everyone who wants to use the software happy. The solution seems to be a hybrid database that can work with data in a shared mechanism to provide the style of access necessary to solve a specific problem. Not sure what is out there though, searching has turned up some ideas.

What are a few simple challenges we all face building software and frameworks? If we all weigh in on that using simple descriptions we can choose some ideas to specify in detail, and keep others open for further brainstorming. As developers what are our user requirements?

My guess:
1- Data access layer components (with multithreading, concurrency conflict resolution)
2- Security
2- Centralized exception handling and logging

I have been thinking on a port of a database abstraction layer I released as free software in 1999. There is a VB version, and an updated Java version has been underway for a few months. A .NET version would be interesting. I can post the source code when we get into coding.

I say we go coding right now if it's ok with you. I would like to begin working ASAP.

Daniel

Daniel,

I'm not familiar with CVS and want to get the source of the data components I have into the public forum. Any ideas on what we do? Can we give a shout out to another project someone knows of for someone to post a quick start to CVS for us?

Jason