marauroa database related code changes

Developers
2009-07-13
2013-06-06
  • Hi,

    the new database layer in marauroa is there in CVS HEAD. jMaPacman and Stendhal have been ported. There are still some rough edges but the basics are there.

    Please have a look at http://stendhal.game-host.org/wiki/index.php/Refactoring_Database_Access_in_Marauroa for details.

    Hendrik

     
    • Hmmm, pretty similar to my Simple-Marauroa Work but I do like your approach more. I'll update mine when the code's final, at least released. I had to add JPA libraries so I'm curious how this was avoided...

       
    • Is there any documentation on the proposed changes? I would lie to propose some database changes for discussion. Mostly related to the character_stat table as it's not as dynamic as it should. I have something implemented that populates a dynamic version of the table from the RPClass definitions!

       
      • I took a look at you changes and I have to say that I love the DAO Register idea. I would suggest a n DAOObject interface since external frameworks would need to change the code deeply and/or extend and overwrite stuff. Again I'll suggest throwing generic Exceptions from those interfaces so the implementation will decide what's the suitable type.

        Having adapter for potentially each database is questionable. Why not use Hibernate, JPA or other persistence provider?

        I'll for sure follow the register DAO approach in Simple-Marauroa. If you agree we can share ideas.

         
        • What methods would this DAOObject interface have?

          We are currently throwing SQLException and IOException, that seems pretty generic to me.

          > Having adapter for potentially each database is questionable. Why not use Hibernate, JPA or other persistence provider?

          Oh. We use Hibernate at work. Which requires to use special extensions for both Informix and Postgres (which is 100% of the databases supported by us) to get Hibernate work with the latest releases. Hibernate does support older versions out of the box, but takes month to support new releases. Granted, we did not have had to write that ourself because some other nice people already did that and shared their results on the net.

          I am afraid, the usual points against hibernate are still valid:
          * Hibernate is slow. Especially on multi table queries.
          * Hibernate uses a proprietary query language
          * Hibernate is large (hibernate.jar: 2400KB, marauroa.jar+log4j.jar+mysql.jar: 864KB).  Three times the size of the complete software just for database abstraction?
          * Hibernate is complex and hard to debug
          * Hibernate tends to use lots of RAM.
          * Hibernate is designed for static object to table mappings, not something like RPObject serialized in a blob.
          * Hibernate is an additional dependency.

           
          • Markus
            Markus
            2009-07-20

            Hibernate is not the final word. It has some advantages, but i think the disadvantages hendrik listed here, are stronger.

            And I strongly like the "Metamodel" approach he suggested. It has many many advantages and is pretty advanced.

             
          • In my case it can be Hibernate/JPA Exceptions which kind of wrap an SQL exception but is not one.

            The interface can have create, delete and edit for example.

            * Hibernate is slow. Especially on multi table queries.
            I haven't seen many multi table queries in use. About performance: https://www.hibernate.org/15.html
            * Hibernate uses a proprietary query language
            EJB3-QL, I use JPA.
            * Hibernate is large (hibernate.jar: 2400KB, marauroa.jar+log4j.jar+mysql.jar: 864KB). Three times the size of the complete software just for database abstraction?
            Also you can use light version of those libraries...
            * Hibernate is complex and hard to debug
            Questionable.
            * Hibernate tends to use lots of RAM.
            See link above
            * Hibernate is designed for static object to table mappings, not something like RPObject serialized in a blob.
            Persistence support blobs. I already have Marauroa working on JPA without issues.
            * Hibernate is an additional dependency.
            No doubt.

             
            • >>* Hibernate is slow. Especially on multi table queries. 
              >I haven't seen many multi table queries in use. About performance: https://www.hibernate.org/15.html

              I trust my own experience a lot more that the things written by the company selling Hibernate related services.

              >>* Hibernate uses a proprietary query language 
              >EJB3-QL, I use JPA.

              Well, yes. That's what I mean.

              >>* Hibernate is complex and hard to debug 
              >Questionable.

              Well, we wasted way too much time on analyzing and fixing bugs in Hibernate at work. For example the issue of parameters being out of order in filters.

              >>* Hibernate tends to use lots of RAM. 
              >See link above

              >>* Hibernate is designed for static object to table mappings, not something like RPObject serialized in a blob. 
              >Persistence support blobs. I already have Marauroa working on JPA without issues.

              Sorry, my point is not that Hibernate can map byte arrays and streams to blobs.

               
      • The documentation is at http://stendhal.game-host.org/wiki/index.php/Refactoring_Database_Access_in_Marauroa

        This is about the API in Marauroa. It is not about the database table structure of Stendhal like character_stats

        A simple way to store generic object data in a relational database would be a meta model like the following. While it is nice nobody stepped up yet to implement it, which would be quite a bit of work with little gain.

        CREATE TABLE class (id SERIAL, name VARCHAR);

        CREATE TABLE inheritance (id SERIAL, sub_class_id INTEGER, super_class_id INTEGER);

        CREATE TABKE attribute_type (id SERIAL, name VARCHAR, class_id INTEGER, type CHAR(1));

        CREATE TABKE association_type (id SERIAL, name VARCHAR, class_id INTEGER, class_id INTEGER);

        CREATE TABLE object (id SERIAL, class_id INTEGER);

        CREATE TABLE value_char (id SERIAL, attribute_type_id INTEGER, object_id INTEGER, value VARCHAR);

        CREATE TABLE value_int (id SERIAL, attribute_type_id INTEGER, object_id INTEGER, value INTEGER);

        CREATE TABLE value_blob (id SERIAL, attribute_type_id INTEGER, object_id INTEGER, value BLOB);

        CREATE TABLE link (id SERIAL, association_type_id INTEGER, object_id INTEGER, target_object_id INTEGER);

        There is a type layer at the top, consisting of classes (e.g. Player), attributes (e. g. ATK) and associations (e. g. bank_slot). The Inheritance is the reference to the super object (Player, RPEntity)

        The second block is the instance layer. It consists of objects (4711 of type Player), values (43 for atk) and links (items in the bank_slot). Of cause there are some consistency conditions required. For example value.object.class <= value.attribute.class.

        This basic model can than be easily extended. For example in our case we want a sort order in links table. And because some databases are very slow at deleting rows, or just in case we may want a revision safe system there is an easy way to achieve that: Add a new table activity which basically has a timestamp. All the table on the instance layer get two additional columns: created_activity_id and deleted_activity_id. This is a lit bit redundant but required for performance reasons. Together with a second trick, the whole system is much faster as one would guess: This trick is to have two special activities: "beginning of time" (lowest possible date) and "end of time" (highest possible date). This allows to greatly simplify the queries, because deleted_activity now always has a valid value. Life objects are simply deleted at the end of time.

         
    • I'm not arguing the team's decisions. As a framework user I'm afraid my current work is not supported by the proposed approach. I'm trying to figure out how to implement this using persistence providers.  That's why I proposed using interfaces.

       
    • Can you provide a diagram of the above? It'll be easier to understand your proposal...

       
    • I noticed that Marauroa 3.0 is out there. I'll take a look at the code but meanwhile is there any signs of the interface approach?

      I think Marauroa stopped being as extendable as it used to be before the change. I'm trying t figure out a way to merge my work with current Marauroa and can't find an easy way.

      Any help/idea?

       
    • The approach doesn't provide for database changes. If for some reason a table needs to be modified, the DAO class won't match the existing one so there's no way of overwriting the original one. I suggest changing the register class from Class,Object to String, Object and having some final strings defined so they can be easily overwritten.

      I still disagree with the non-interface DAO classes. Eveything that is currently done can be handled with the interface I suggested and some pre defined queries.

      Again, Marauroa is less of a framework as time passes. At least with the jump to 3.0.

       
    • Take a look at cayenne it provides same functionality with a jar around 882 Kb.

      Looks promising. I'll take a look after I stabilize my game.

       


Anonymous


Cancel   Add attachments