Recent changes to 1: matthews correlation coefficient (mcc) et al.http://sourceforge.net/p/pyml/feature-requests/1/2004-09-02T00:05:53Zmatthews correlation coefficient (mcc) et al.2004-09-02T00:05:53Z2004-09-02T00:05:53Zozihttp://sourceforge.net/u/jstelzman/http://sourceforge.netccff9a1c398eb588e9617aa0d1bf492b9a22fff8<div class="markdown_content"><p>matthews correlation coefficient (mcc) is a valuable<br />
tool for model selection.</p>
<p>Especially if classes are very unevenly distributed you<br />
can get acceptable values for ROC, ppv, balancedSuccess<br />
etc. although the classifier assigns all examples to<br />
the same class. mcc gives 0 in such cases. AFAIK the<br />
extension to more than two classes ist straightforward<br />
(i think the Baldi/Brunak book has a formula).</p>
<p>However pure mcc assigns 0 also to perfect classifiers,<br />
so one needs probably a measure which is mcc derived. <br />
The minimum and the standard deviation of the<br />
performance measures obtained by stratifiedCV are also<br />
important indicators and should be given together with<br />
the average.</p>
<p>In summary I would like to see a bit more flexibility<br />
in how the best model is selected from e.g. a ParamGrid.</p>
<p>Thank you.</p></div>