Lazy Loading for Trees

2012-09-30
2013-03-08
  • Michael Judd
    Michael Judd
    2012-09-30

    Hi All,

    I have noticed some performance issues with loading trees as all nodes are loaded at initialisation.  As some of the trees I'm working with have over 10,000 nodes, I'm finding the load pretty slow.

    Has anyone looked at implementing lazy loading in the webui ?

    references:
    - http://www.zkoss.org/zkdemo/tree/load_on_demand
    - http://www.zkoss.org/forum/listComment/8604

    Mike

     
  • Jan Thielemann
    Jan Thielemann
    2012-11-16

    Hello Mike,
    since Adempiere has different kind of trees, could you tell me which kind of tree you are talking about? Lazy loading trees is one thing we would like to improve.

     
  • Michael Judd
    Michael Judd
    2013-02-11

    Hi All,

    I've tested this in a test environment and it works really well.  The JVM memory management also seems to benefit.  I'm going to trial it in an live (test) environment on Oracle later this week and I will feedback.

    I did commission another party to develop this for me and they provided a solution, but it doesn't work quite as well as your implementation.  I will follow up with more detailed testing including your use cases and feedback the results.

    Regards,

    Mike

     
  • jmpiloq
    jmpiloq
    2013-02-11

    will not work on Oracle, as SQL is currently targeted at PostgreSQL (uses LIMIT)

     
  • Colin Rooney
    Colin Rooney
    2013-02-11

    As a general comment, the norm (for historic reasons) is the embedded SQL is Oracle specific but then it is converted to Postgresql via the dbPort/Convert_PostgresSQL.java.

    You can implement the Postgres LIMIT using the Oracle ROWNUM… a quick glance there and it looks like someone has attempted the Rownum conversion but it's all commented so I'm not sure if it's simply deprecated to be implemented in another way or whoever was doing the work was unable to completed it.  I'm sure Mike could manage it ! :p
    :D

    Colin

     
  • Hi Jan!

    I reviewed the documentation http://www.adempiere.com/Lazy_loading_Trees and would be will great if improve can be implementing as general solution no only to posgresql, this way we can include into  our the development branch.

    Some  comments in my general revision, Note ( I have pending of test)

    1.- The code can see good, Is my recomendation the remove some comments in the code of other language that is not english ;-).
    2.- A rule to check the execution, I recommed the use of the Log class vs System.oi.println, I could  saw in some classes the use the System.out.print.
    3.- A rule to include a new improve or feature into development main, is necessary that it can be implement general way to all DBs, and not only for PostgreSQL. So my recomendation if you need use LIMIT or ROWNUM  use as altertative a  subselect using MAX or MIN to get a single record.  ie Field = (SELECT (MAX) FROM …..), in some of my testing the convertion of  ROWNUM to LIMIT is not is correct and SQL can faild, othe the other hand any ROMNUM in code was removed for make 100% compatible between oracle and postgresql, so it  force to development to implement a general solution.

    Kind regards
    Victor Perez
    www.e-evolution.com

     
  • jmpiloq
    jmpiloq
    2013-02-11

    Hi all,

    we know dbPort/Convert_PostgresSQL.java and dbPort/ConvertMap_PostgresSQL.java. They seem to be a bit outdated by now. We want to replace them by some functions in the database instead, so we do not need this kind of conversions any more.

    IIRC, the LIMIT is only used in WebUI. Swing version works without LIMIT. We will fix that.