Performance

Performance

It is very simple to get performace issues ;-). If you want to process big data you will get some performance relevant topics. In this document you will find some tipps for understanding the performance related behaviors.

Main memory

Adele processes the data in the main memory of your desktop computer. This means that if the memory completely used you will run into trouble. The java virtual machine will try different activities to solve your problems. But if the main memory is full the application will be much slower...

What can you do?

Check the max reserved memory for Adele at the right bottom corner. The currently used memory is available there too. If this is not enough for you but your computer has much more memory you can start Adele with some special option which is described here: Installation and configuration.

Import from desktop files

If you import data via the clipboard or from file the data has to be transfered from the foreign program to Adele.

Some facts for a file with 100000 rows and 20 columns, tab separated:
- import with LibreOffice 22 sec
- transfer from LibreOffice to Adele via clipboard 24 sec
- import from clipboard into Adele 50 sec
- import of the same direct from the file 50 sec

Comparing to the LibreOffice Adele needs factor 2 more time for importing the same data. This can be explained as follows:
1) While importing the raw data some calculations will be executed in parallel to optimize the time for later analyzing
2) The consumption of memory and performance is depending on the underlying java virtual machine. This can be differ by the java version and the operating system
3) The graphical interface is java swing based. Some operations are slower comparing with optimized other applications.

Import from remote databases

The performace of the import of data is depending on the performace of the connector for extracting the data for Adele. There are big differences between a poor MS Access driver and specialized database engine like SAP HANA.

If your driver is not fast enough try another for the same datasource if existing. Some time is needed by Adele to build an internal dataset for the extracted data.

Expert tipp: analyze the data before extracting and set the right filters to get a small dataset.

Transformations and other tasks inside of Adele

If the data is imported to Adele performance is depending on main memory and processor power. In some cases the transformation algorithm is not good enough. Contact the author and send your example then. Thank you.

Expert tipp:
You don't have to import data as dataset (table) in all cases. Sometimes it will be sufficent to import the data as OLAP cube or another data object. This option is planned for future versions.

Conclusion

Performance is main memory x processor power and a little bit optimzed algorithm, I try my best to improve the performance for different scenarios. You can help me with your tipps and examples.