RE: [Rainbowportal-devel] Ideas about new DAL layer
Brought to you by:
danijel_kecman,
manudea
Menu
▾
▴
RE: [Rainbowportal-devel] Ideas about new DAL layer
|
From: Geert A. <Gee...@sy...> - 2003-08-25 17:36:19
|
I couldn't have written it better. The reason I started this discussion, is because I think there is far too much database access in our current solution. I did a small test and removed all the database calls for what Jeremy calls "framework related data". I replaced the data that originally came from the database with default data coming from a small serialized object model stored in the cache object. It speeded up my test site by 20x times. If you examine the framework code, you'll see that for every request a huge number of times the database is accessed. If we could eliminate these database requests, we also put less pressure on the database system. Greetz, Geert **************** Geert Audenaert Syntegra has moved! Please find hereunder our new contact details Address: Telecomlaan 9 at 1831 Diegem General Telephone Number: +32 (0)2 700 30 11 General Fax Number: +32 (0)2 700 30 12 E-mail: <mailto:in...@sy...> in...@sy... Internet: <http://www.syntegra.com> www.syntegra.com **************** -----Original Message----- From: rai...@li... [mailto:rai...@li...] On Behalf Of Jeremy Esland Sent: vrijdag 22 augustus 2003 20:26 To: cis...@ya...; 'Rainbow developer' Subject: RE: [Rainbowportal-devel] Ideas about new DAL layer Let me play the Devil's Advocate. I believe we should separate in our minds, in these discussions, and indeed in code, the data requirements of the Rainbow portal framework itself, and the data requirements of any particular module. They are different, unrelated, etc. - so let's treat them that way. Indeed, there are strong architectural reasons why they should be separated - but I'm going to assume that everyone will agree with me on that point and not waste your time reading my supporting argument for it. So. Rainbow's "framework data" - what is it? A collection of data describing portals/tabs/modules and users. User data is a separate issue, so let's actually divide Rainbow's total data requirements into three: 1. framework data 2. user data 3. module data Examining the characteristics of each of these data requirements, we find that framework data (portals/tabs/module settings, etc) is mostly hierarchical in nature, rather than relational; user data is more closely related to module data than it is to framework data; and module data is open-ended - sometimes flat, sometimes relational, sometimes hierarchical. In terms of access/edit frequency and performance requirements, framework data would be better off in memory at all times; user data access is "occasional" and not performance-sensitive; and module data should live in memory unless there's a very good reason to be reading it in real-time. Again, I'll spare you the long-winded argument behind these statements, but by all means disagree with them. So the challenge is to build three data access solutions, not one. Each solution should be optimised to its task and the demands that will be made of it. Framework data will be dealt with entirely within the Rainbow core - not particular need for fancy developer-friendly DAL stuff there - the core can offer a range of backend options, each of which may or may not support farming, but how it's implemented internally is of no interest outside the ring of core developers. User data is one of our big weaknesses at the moment - it needs to be flexible and extensible. In many cases it will also need to be shared with other applications. We need a desperately clever solution here but, again, it's a "one-off" exercise with no particular requirement for code-level friendliness. Module data. Well, forgive me if you disagree with me on this, but I really do think that the subject is too broad and too unknown for us to be discussing it in acronym-heavy specifics. Certainly we cannot assume that a relational store is always suitable. Certainly we cannot assume that a single store is always suitable. Certainly we cannot assume that Rainbow has any more than read-only access to the data. In fact, we shouldn't even assume that the data is available! To use an analogy, I believe we are discussing the dimensions and composition of bricks when we should still be talking to the architect. Jes1111 PS: I managed to write all of that without once mentioning XML! My psychiatrist will be so pleased! -----Original Message----- From: rai...@li... [mailto:rai...@li...] On Behalf Of Cory Isakson Sent: 19 August 2003 15:34 To: 'Rainbow developer' Subject: RE: [Rainbowportal-devel] Ideas about new DAL layer Geert, I beleive you are thinking similiar to me. The problem with not storing the object in the DB is that those of use using web server farms would have difficulty keeping everything in sync if it was stored on disk. We would need a DB or some other solution for storing the data in a central location. I beleive that scalability of web farms would be the reason that MS CMS uses SQL for storage. We all know they want to promote SQL server, but most any DB could be used to store serialized objects. One other problem with the serialized objects is that they cannot be searched or indexed as easilly as regular database items. I think there needs to be some balance between the central storage of data and web server data caching in memory/disk. Cory Geert Audenaert <Gee...@sy...> wrote: Hi Federico, Do you really need a database at all? When you create a nice object model, you can serialize it and just save it to disk. You can host the entry point to this object model as a singleton (via .NET Remoting) in rainbow itself. This way it only has to be instantiated once, when the portal website is started. It works great and its faster then database access. This is also the way MS CMS works, the only difference is, that they serialize it to sql server, just to self their dbms. I recently developed a small web application like this, and it works great! Just a thought, Greetz, Geert **************** Geert Audenaert Syntegra has moved! Please find hereunder our new contact details Address: Telecomlaan 9 at 1831 Diegem General Telephone Number: +32 (0)2 700 30 11 General Fax Number: +32 (0)2 700 30 12 E-mail: in...@sy... Internet: www.syntegra.com **************** -----Original Message----- From: rai...@li... [mailto:rai...@li...] On Behalf Of Federico Dal Maso Sent: dinsdag 19 augustus 2003 16:01 To: Rainbow developer Subject: [Rainbowportal-devel] Ideas about new DAL layer I want to write some ideas about a new structure for Rainbow Data Access LAyer and more. Perhaps it maybe adopted in Rainbow 2 I think that data access architecture in current Rainbow version has 3 big problems: 1. It is strongly linked to SQL Server 2. It is strongly linked to Module (Rainbow database structure is generated by module installation. Each module create its table, stored procedure and so on. For example if I want to create a new news module with some new features I have to create a new table for news and I cannot use and share my data with exist old news module. Because I must suppose that a new version of old module would change its db structure. On the other hand it's hard to create a new graphic interface for a exist news module) 3. It doesn't allow a simple OOP programming approach to Module programmer The best way to solve these problems would be a O/R mapper data access architecture like the java EJB or Hibernate. But the .NET portings of these mappers are currently in alpha stage. The NHibernate leader is missing (!) and project is stopping now. The OJB project is still in pre-alpha stage. So, I think we will not be able to play with a stable O/R mapper in the next months and we will not program in pure OOP in every layer till next year... To solve the 3 problems above I suggest to introduce in Rainbow system: A. a new module data layer B. a interface based (or abstract class based) data access layer The new architecture (i.e. for news) maybe connected like this graph UINewsModule1 UINewsModule2 UINewsModule3 | | | +-------+-------+ | | | FedeDataNewsModule ManuDataNewsModule | | +---------+------------+ | AbstractDALlayer | | Database A DataModule is a installable non-graphic module the expose a standard interface for UIModules and on the other side calls the AbstractDataAccessLayer. For example the FedeDataNewsModule could expose methods like: int AddNews(title, userId, text, expirationDate) void DeleteNews(int newsId) FedeNews[] GetThreeTopClickedNews(int newsCategoryId) ... ManuDataNewsModule could expose different methods. (Not IDataReader or IDataSet are passed to UIModule!) FedeDataNewsModule call AbstractDALLayer methods to perform actions in ITS db table ManuDataNewsModule call AbstractDALLayer methods to perform actions in ITS db table (usually different from FedeTables) The UINewsModuleX is standard rainbow graphics module that use methods expose by middle DataModule who it's linked to. They ignore the existance of database. The UINewsModule1 and UINewsModule2 is located in different pages and have a different graphics structure and different user interface and behavior, BUT they CAN modify the same set of news. Using this architecture we can build a dynamic easy to understand structure of site contents: For example: MySite | +-News (provided by FedeDataNewsModule) | | | +-Business news | | +- publ. in HomePage using UINewsModule1 | | +- publ. in ShoppingPage using UINewsModule2 | | | +-Customer news | +- publ. in WelcomeCustomer using UINewsModule2 | +-News (provided by ManuDataNewsModule) | | | +-Company news | +- publ. in InternalHomePage using UINewsModule3 | +-Forums (provided by ACMEDataForumModule) | | | +-Open discussion forum | | +- publ. in DiscussionPage using HyperBitIncForumsModule | | | . . . . . . Each graphic module is linked to a specific type of data source module. The installation process would be easy: if I install a new UImodule and his DataModules (maybe more than one) don't exist the they will be installed too. I think this approach wuold be simplify our work. UIModule will be more powerful, easier to create and db neutral. We could think to create some UIModule specialized in data managed for a specific DataModule, and so on... The AbstractDAL is responsible of db neutral access. The DataModule create SQL generic instruction and obtain IDataSet, IDataReader object. I know this is not good like a SQL-Server stored procedure, but the system can be extended to other database, and if in the future we will have a O/R mapper we will have to modify only the database interface of DataModule and nothing in UIModules. I think that DAAB is a good first step for AbstractDAL. Fede_ ------------------------------------------------------- This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET. http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01 /01 _______________________________________________ Rainbowportal-devel mailing list Rai...@li... https://lists.sourceforge.net/lists/listinfo/rainbowportal-devel ------------------------------------------------------- This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET. http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01 /01 _______________________________________________ Rainbowportal-devel mailing list Rai...@li... https://lists.sourceforge.net/lists/listinfo/rainbowportal-devel Cory Isakson Online Status: <http://www.lifelesspeople.com:4000/message/msn/cis...@ya...> Online Status Indicator <aim:goim?screenname=cwisakson> _____ 3.9 Cents/min. No Fees or Minimums! http://www.dial4life.com <http://ld.net/?dial4life> _____ _____ Do you Yahoo!? Yahoo! SiteBuilder <http://us.rd.yahoo.com/evt=10469/*http:/sitebuilder.yahoo.com> - Free, easy-to-use web site design software |