Re: [Databionic-ESOM-User] Can ESOM handle features of 9000 ?

[<fa...@in...>]

the error on map initialization is most likely the "out of memory"
error. i will try to make this error message appear more prominently.

it somewhat works with your file on my system if I change the java
memory setting in the start scripts:

  - open the file <ESOM folder>/bin/esomstart[.bat] with a text editor
  - change -Xmx512m to -Xms1800m (or higher for even larger files)

now I end up with the program running at 100% cpu, but no results after
a few hours. checking in the code I found a bottleneck, we need to
remove, before you should try again: the covariance matrix of the data
is calculated even if it is not needed for the map initialization (it is
only needed for the PCA initialization), with 9000 variables obviously a
demanding task. fixing this will take a while as we are all very busy
with other things, sorry.

bye
fabian

Yu Shi wrote:
> Thanks Fabian, this is the data file I used. I found the problem might
> not be the cls file or integer cause I tried your examples and there
> are same messages and just ignore them and start training. My problem
> is no action performs after "initilializing map", the CPU load is 0
> after that. I wonder whether this file works on your machine. Thanks
> for your time and help.
> 
> BR,
> 
> Shi
> 
> On 11/9/05, Fabian Mörchen <fa...@ma...> wrote:
> 
>>The software should be able to handle this many features, provided you
>>have enough memory. Recall, that even if you have only 18 examples, the
>>map contains a 9k vector for each neuron.
>>
>>I generated a file like you described and did not get the same error, I
>>only ran out of memory, because there are some other demanding tasks
>>currently running on my machine.
>>
>>Dou you have by any chance a "3" somewhere in the 3rd line of the *.lrn
>>file? That would be the deprecated code for a classification. Or did you
>>supply a *.cls file with the argument -cls? Otherwise I wouldn't know,
>>why the Java class ClsFile ist used at all. Note, that the --cls option
>>is not needed for unsupervised training, it is rather used for debugging
>>purposes. To find out why your file is not parsed, you could also sent
>>it to me (not the list!) if that is ok.
>>
>>bye
>>fabian
>>
>>Yu Shi wrote:
>>
>>>Hi. I just downloaded ESOM 0.9.4 today and try to build a map of 36
>>>genes with 9,175 features. The data file is correct because the main
>>>values have been tested in Genecluster2, Genesis and other clustering
>>>tools. I changed the required format as:
>>>
>>>% 36
>>>% 9176
>>>% 9           1            1        .... (9,172 times of '1')      1
>>>% Key       C1         C2           ............                     C9175
>>>1               0            0.2          ............                     0.03
>>>...
>>>36           ................
>>>
>>>
>>>however, this data file cannot been trained. Each time I start the
>>>training, the message says "missing input file for type: ClsFile
>>>         is not an integer"
>>>if I reduce the feature number to small one, it is ok. So I wonder
>>>whether the program can handle 9,000 features for clustering. If not,
>>>how many features can it mostly handle? Thanks.
>>>
>>>BR,
>>>
>>>Shi
>>>
>>>
>>>-------------------------------------------------------
>>>SF.Net email is sponsored by:
>>>Tame your development challenges with Apache's Geronimo App Server. Download
>>>it for free - -and be entered to win a 42" plasma tv or your very own
>>>Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
>>>_______________________________________________
>>>Databionic-ESOM-User mailing list
>>>Dat...@li...
>>>https://lists.sourceforge.net/lists/listinfo/databionic-esom-user
> 
>>