Menu
▾
▴
Re: [Databionic-ESOM-User] eSOM Kfold, training error??
|
From: <fa...@my...> - 2007-06-02 00:56:04
|
Katerina, ESOM is an unsupervised method, so I think k-fold training to optimize certain parameters makes only very limited sense. In supervised problems all you care about is the accuracy (or a similar measure). Crucial parameters of a classifier (like C for SVM) can be optimized using k-fold crossvalidation to avoid overfitting. For unsupervised methods like ESOM or clustering there are several alternative measures that you might want to optimize. But if you would want to optimize map size to achieve a low quantization error you will most likely get smaller errors as the map grows, because a larger parameter space can better represent your data. A small quantization error does not necessarily mean you have the best map (e.g. the best visualization, the best untangling of clusters, ...). As for the speed there are people working on speeding up ESOM training and I hope these methods will be incorporated in a future version. fabian Katerina Mitrokotsa wrote: > Hi Mario, > thank you, but what about kfold???? > Is there a command I can use to perform kfold for the tuning of > parameters???? > > another problem I face is that the overhead is really high for > training datasets around 20000 records any suggestions for this problem? > > Best Regards, > Katerina > > Mario Noecker wrote: >> Hi Katerina, >> >> the only training error which is implemented yet, is the average >> quantization error. >> >> If you change the entry for log4j.category.databionics in the >> etc/esom.conf file from INFO to DEBUG the quantization error will be >> logged after every epoch of training. >> >> The parameters which can be tuned are (in my opinion) the size of the >> map, the values for radius and learning rate, the neighborhood function, >> the number of epochs and the initialization method. >> >> bye >> mario >> >> >> >> >> >> Katerina Mitrokotsa wrote: >> >>> Hi, >>> I would like to know if eSOM tool provides a way to perform kfold, if >>> there is way to see which is the training error when a perform a training >>> and which are the parameters that are more important to change in order to >>> perform the best tuning. >>> >>> thank you in advance, >>> Katerina >>> >>> >>> >>> ------------------------------------------------------------------------- >>> Take Surveys. Earn Cash. Influence the Future of IT >>> Join SourceForge.net's Techsay panel and you'll get the chance to share your >>> opinions on IT & business topics through brief surveys-and earn cash >>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >>> _______________________________________________ >>> Databionic-ESOM-User mailing list >>> Dat...@li... >>> https://lists.sourceforge.net/lists/listinfo/databionic-esom-user >>> >>> >> >> >> >> >> > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by DB2 Express > Download DB2 Express C - the FREE version of DB2 express and take > control of your XML. No limits. Just data. Click to get it now. > http://sourceforge.net/powerbar/db2/ > ------------------------------------------------------------------------ > > _______________________________________________ > Databionic-ESOM-User mailing list > Dat...@li... > https://lists.sourceforge.net/lists/listinfo/databionic-esom-user > |
