Re: [Java-ML-support] Question about AIC and BIC scores

[Sergio P. <spu...@gm...>]

Hi Tomas,

I found this:
http://books.nips.cc/papers/files/nips16/NIPS2003_AA36.pdf
http://www.autonlab.org/autonweb/14661/version/3/part/5/data/pelleg-xmeans.pdf?branch=main&language=en

They say that actually k (the number of free parameters) is bigger than the
number of clusters - 1 as was my initial thought. It also depends on
dimensionality, k should be:

The papers do not agree in what should be the numbers of free parameters (I
just looked at them very fast so I am probably wrong or not seeing
something obvious)

The first papers says: "p = k(d + 1) is the number of parameters in the
model"
double k = clusters.lenght * (clusters[0].get(0).noAttributes() + 1);

the second says: "the number of free parameters p is simply the sum of k -
1 class probabilities, M.K centroid coordinates, and one variance estimate"
double k = (clusters.lenght - 1) +
clusters.lenght*(clusters[0].get(0).noAttributes() + 1);

I think the second is the best:
1. It put additional penalties for the number of clusters but not greatly
(just an aditional clusters.lenght - 1). It count as a parameter the
existence of each cluster (?)
2. It have the -1... in the other one I do not see the bound parameter
anywhere.

The problem I see with this is that that is the calculation of free
parameters for a KMeans clustering, but not necessary for other types of
clustering algorithms with different assumptions, K-means assumes a
Gaussian circular distribution.

Anyhow, it should be better than just k = 1 :)

Cheers,
Sergio.






On Fri, Nov 22, 2013 at 3:18 PM, Thomas Abeel <th...@ab...> wrote:

>  Hi Sergio,
>
> This has been a while. This may be a bug. Can you point me to any
> literature reference that says the number of free parameters =
> clusters.length-1?
>
> cheers,
> Thomas
>
> On 11/12/2013 9:08 AM, Sergio Pulido wrote:
>
> Hello all,
>
>  I see that the implementation of AICScore and BICScore uses a k=1 as the
> number of free parameters
>
>  // number of free parameters K
> double k = 1;
>
>  Can you explain me why the number of free parameters is not the number
> of clusters -1?
>
>  // number of free parameters K
> double k = clusters.length - 1;
>
>  Thank you a lot,
> Sergio.
>
>
> ------------------------------------------------------------------------------
> Shape the Mobile Experience: Free Subscription
> Software experts and developers: Be at the forefront of tech innovation.
> Intel(R) Software Adrenaline delivers strategic insight and game-changing
> conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
>
>
>
> _______________________________________________
> Java-ml-support mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/java-ml-support
>
>
>
>
> ------------------------------------------------------------------------------
> Shape the Mobile Experience: Free Subscription
> Software experts and developers: Be at the forefront of tech innovation.
> Intel(R) Software Adrenaline delivers strategic insight and game-changing
> conversations that shape the rapidly evolving mobile landscape. Sign up
> now.
> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk
> _______________________________________________
> Java-ml-support mailing list
> Jav...@li...
> https://lists.sourceforge.net/lists/listinfo/java-ml-support
>
>