Re: [TMVA-users] Training a BDT with sweights

[Helge V. <Hel...@ce...>]

Hi Christophe,

yes.. that's a tiny bug I've fixed already some time ago, but it
didn't make it's
way yet into the official ROOT release. Its caused by the fact that the weights
are not taken into account when calculating if the mean value of
MVA_Signal > or < than mean MVA_Backgound... hence deciding if a cut
selects rather Signal or Background.

It wasn't spotted earlier, as typcially the event weight's don't
change the distributions
so much... but with s-weights, they do. Hence...  Training and
everything is correct,
it is just when doing the efficinecy plots/calculations that this
matters and goes wrong.



Now, unfortunatly, the current 'dev' version that you could check out
from svn is quite a
bit different fromt ROOT version. I you don't mind that.. please use
that. Otherwise I
would have to get you a patch.. and you'd have to 're-compile' your ROOT..

sorry..

Cheers,

Helge


2012/11/9 Christophe Cauet <chr...@tu...>:
> Dear experts,
>
> I'm experiencing some trouble training a BDT with TMVA.
>
> As far as I can decide the classifier distribution looks not too bad for the
> BDT
> * https://dl.dropbox.com/u/171315/tmva/classifier_distribution_BDT.png
> as well as for a Fisher discriminant which I use as a reference, since I was
> always getting quite good results with it:
> * https://dl.dropbox.com/u/171315/tmva/classifier_distribution_Fisher.png
>
> However the the cut efficiencies as well as the ROC curve look very strange:
> * https://dl.dropbox.com/u/171315/tmva/cut_efficiencies_BDT.png
> * https://dl.dropbox.com/u/171315/tmva/cut_efficiencies_Fisher.png
> * https://dl.dropbox.com/u/171315/tmva/ROC_curve.png
>
> I'm using a single TTree as input and sweights from an sSplot via:
>
> factory->SetWeightExpression("sweight_sig", "Signal");
> factory->SetWeightExpression("sweight_bkg", "Background");
>
> to discriminate between signal and background events for training and
> testing.
>
> There are about 130k signal and 20k background events in my sample.
>
> As I never experienced such problems before I guess it has something to do
> with the sweights.
>
> Do you ever tried that before or have a clue whats going wrong?
>
> Thanks
> Christophe
>
>
>
> PS:
> The TMVA output is:
> https://dl.dropbox.com/u/171315/tmva/tmva_output.txt
>
> The factory options are
> V:!Silent:Color:DrawProgressBar:Transformations=I;D;P;G,D:AnalysisType=Classification
>
> The split options are:
> SplitMode=Random:!V:SplitSeed=0:nTrain_Signal=0:nTest_Signal=0:nTrain_Background=0:nTest_Background=0
>
> The BDT options are:
> !V:NTrees=400:nEventsMin=400:MaxDepth=3:BoostType=AdaBoost:SeparationType=GiniIndex:nCuts=20:PruneMethod=NoPruning:VarTransform=Decorrelate
> I also tried adding IgnoreNegWeightsInTraining
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_nov
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users
>