databionic-esom-user

[Databionic-ESOM-User] what if...

[davide c. <daw...@gm...>]

Hi againI'm trying to get something out of my ~4000 datapoints in 123
attributes. Some clustering techniques (i.e. EM clustering) and other data
analysis tools (i.e. PCA) tell me I can divide my dataset in 3 or 4 big
"groups".
I'm trying now ESOM tools and it looks like I have a mountain and two
orthogonal valleys (on a toroid map). On the mountain I have one group (are
those all outliers?) in the valleys I have all remaining data. It happens
that data that were in the same group in other analysis are still next to
each other but, since they are in the same valley (actually a plain under
mountains...), I should say that they cannot be clustered with ESOM
tools...
Do you have any hint to get out this?

Thanks

d

Re: [Databionic-ESOM-User] what if...

[Christian S. <st...@Ma...>]

Hi davide,

Those results might tell you that there are no separable groups in your
data. There is a few things you might try to clarify the situation.

First of all use the Clip slider in the view tab to make finer structures
visible in the existing map. It clips big heights in the U-Matrix, so the
color gradient will be mapped over lower heights. Structures in those
might be dominated by the big mountain right now and then become visible.

You should also try and train another map without the suspected outliers,
just to see what happens. You should also evaluate wether or not they are
outliers using other methods like scatter plots.

regards,
Christian



> Hi againI'm trying to get something out of my ~4000 datapoints in 123
> attributes. Some clustering techniques (i.e. EM clustering) and other data
> analysis tools (i.e. PCA) tell me I can divide my dataset in 3 or 4 big
> "groups".
> I'm trying now ESOM tools and it looks like I have a mountain and two
> orthogonal valleys (on a toroid map). On the mountain I have one group
> (are
> those all outliers?) in the valleys I have all remaining data. It happens
> that data that were in the same group in other analysis are still next to
> each other but, since they are in the same valley (actually a plain under
> mountains...), I should say that they cannot be clustered with ESOM
> tools...
> Do you have any hint to get out this?
>
> Thanks
>
> d
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________
> Databionic-ESOM-User mailing list
> Dat...@li...
> https://lists.sourceforge.net/lists/listinfo/databionic-esom-user
>

Re: [Databionic-ESOM-User] what if...

[<fa...@my...>]

Hi,

if you have two valleys separated by a mountain this corresponds to two
clusters on a coarse scale.

Please try to retrain without the points in the mountainous regions as
Christian suggested.

If you want more details than you can find with the sliders you can also
select all points in one valley, save the selection to a new (*.lrn)
file and train a new map only on this data. This will give you the most
detailed view into this region of the data space. Of course you can do
this for each valley.

Please note that normalizing the data and picking a good distance
function is crucial. If you want comparable results to PCA you need to
apply the same normalization, if any. Also note that EM creates clusters
that correspond to possibly overlapping Gaussian distributions in the
high dimensional space. ESOM tries to find clearly separable
non-overlapping clusters.

best
fabian

Christian Stamm wrote:
> Hi davide,
>
> Those results might tell you that there are no separable groups in your
> data. There is a few things you might try to clarify the situation.
>
> First of all use the Clip slider in the view tab to make finer structures
> visible in the existing map. It clips big heights in the U-Matrix, so the
> color gradient will be mapped over lower heights. Structures in those
> might be dominated by the big mountain right now and then become visible.
>
> You should also try and train another map without the suspected outliers,
> just to see what happens. You should also evaluate wether or not they are
> outliers using other methods like scatter plots.
>
> regards,
> Christian
>
>
>
>   
>> Hi againI'm trying to get something out of my ~4000 datapoints in 123
>> attributes. Some clustering techniques (i.e. EM clustering) and other data
>> analysis tools (i.e. PCA) tell me I can divide my dataset in 3 or 4 big
>> "groups".
>> I'm trying now ESOM tools and it looks like I have a mountain and two
>> orthogonal valleys (on a toroid map). On the mountain I have one group
>> (are
>> those all outliers?) in the valleys I have all remaining data. It happens
>> that data that were in the same group in other analysis are still next to
>> each other but, since they are in the same valley (actually a plain under
>> mountains...), I should say that they cannot be clustered with ESOM
>> tools...
>> Do you have any hint to get out this?
>>
>> Thanks
>>
>> d
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/_______________________________________________
>> Databionic-ESOM-User mailing list
>> Dat...@li...
>> https://lists.sourceforge.net/lists/listinfo/databionic-esom-user
>>
>>     
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Databionic-ESOM-User mailing list
> Dat...@li...
> https://lists.sourceforge.net/lists/listinfo/databionic-esom-user
>
>
>   

[Databionic-ESOM-User] colt 1.2.0.5

[<rde...@it...>]

Hello

First of all I'm grateful for your support and for you very good software.
I'm working with SOM class, in order to serialize it, if I modify it in
order to implements Serializable then I have one error in

"java.io.NotSerializableException: cern.jet.math.VectorFunctions"

When I try to look for the library "colt 1.2.0.5" in order to find the
source, however I can not find this library in internet and the last version
library found is 1.2 and this class doesn't exists. 

My Question, where can I find the source of this files or if it possible to
make serializable this class without colt modification.

Thanks in advance 

Best Regards Rafael

Re: [Databionic-ESOM-User] colt 1.2.0.5

[<fa...@my...>]

Rafael,

the 1.2.0.5 version is a modification made during the development
of the ESOM tools. The original authors were not interested in
integrating our changes. The complete modified colt can be
checked out from our sourceforge repository by using the module
name "colt" instead of "src" in the instructions on
http://databionic-esom.sourceforge.net/cvs-usage.html

the particular class VectorFunctions is really just a repository for static
functions and should not need to be serialized. even the SOM class
might not be the best to serialize. I have to admit that the code was
not designed with serialization in mind. you are certainly welcome
to suggest modifications. if you want to save a trained map, then
try to serialize the subclasses of the Grid class (but then you need
to serialize the bestmatches separately).

best
fabian

rde...@it... wrote:
> Hello
>
> First of all I'm grateful for your support and for you very good software.
> I'm working with SOM class, in order to serialize it, if I modify it in
> order to implements Serializable then I have one error in
>
> "java.io.NotSerializableException: cern.jet.math.VectorFunctions"
>
> When I try to look for the library "colt 1.2.0.5" in order to find the
> source, however I can not find this library in internet and the last version
> library found is 1.2 and this class doesn't exists. 
>
> My Question, where can I find the source of this files or if it possible to
> make serializable this class without colt modification.
>
> Thanks in advance 
>
> Best Regards Rafael
>
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Databionic-ESOM-User mailing list
> Dat...@li...
> https://lists.sourceforge.net/lists/listinfo/databionic-esom-user
>
>
>   

Re: [Databionic-ESOM-User] colt 1.2.0.5

[Niko E. <ne...@ne...>]

On Monday 21 January 2008, rde...@it... wrote:
> Hello
>
> I'm working with SOM class, in order to serialize it, if I
> modify it in order to implements Serializable then I have one error
> in

Hello,

as Fabian already said, serialazation was never taken into consideration 
in the design. Hoever, saving and loading maps/bestmatches/etc. is 
already in place, so it probably would be better to extend this 
funktionality, if there is something you are missing. Could you explane 
your use case a bit?

Niko