Re: [Databionic-ESOM-User] Can ESOM scale to large data sets

[<fa...@my...>]

Sesan,

we have successfully used ESOM with datasets of this size and beyond. It
should in principle not make a difference
except for the training time. are you using the same normalization? an
error in preprocessing can lead to
meaningless results.

You may want to increase the map size to have a higher resolution and
not have a too 'crowded' map. Then you also
should increase the start radius proportionally.

If you can draw a small representative sample from your large dataset
you could try to cluster the sample, check
for meaningful results, and then project the full dataset onto the map
in classification mode. This will be much faster
than training with the full set. Of course this is problematic if the
sample is not representative and the full dataset
may contain additional clusters.

hope this helps
fabian

Sesan Adeyemo wrote:
> Hi
> I am using Databionics ESOM (along with other software) for my
> reseach. The data set that am working on can be really large (up to
> 10,000 rows of data in about 5 colums in an Excel spreadsheet.
>  
> When i tried ESOM using a smaller data set (about 600) i was able to
> get some meaningful results. However with a data set of 2000 data
> elements i couldn't get any meaningful clusters (other none graphical
> software did).
>  
> I really like the ESOM software because its comprehensive and
> flexible, however, it just won't give any meaning results. I varied
> the epochs (up to 500), varied the map size etc, but still won't work
> or can't visualize the clusters if they really are there!!!
>  
> How can i solve this problem or is it that the ESOM can't scale to
> large values.
>  
> Thanks
>  
> Sesan Adeyemo
>  
>
> ------------------------------------------------------------------------
> Finding fabulous fares is fun.
> Let Yahoo! FareChase search your favorite travel sites
> <http://farechase.yahoo.com/promo-generic-14795097;_ylc=X3oDMTFtNW45amVpBF9TAzk3NDA3NTg5BF9zAzI3MTk0ODEEcG9zAzEEc2VjA21haWx0YWdsaW5lBHNsawNxMS0wNw-->
> to find flight and hotel bargains.
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> ------------------------------------------------------------------------
>
> _______________________________________________
> Databionic-ESOM-User mailing list
> Dat...@li...
> https://lists.sourceforge.net/lists/listinfo/databionic-esom-user
>