tmva-users — Open mailing list for TMVA users

Re: [TMVA-users] TMVA understanding related

[Kim A. <kia...@ce...>]

Hi Subhasish!

And welcome to TMVA!

Also, please note that the best support for TMVA is now given in the 
ROOT fourm <http://root-forum.cern.ch> :)
> 1. How do I generate the histogram root file ?
> : I have events file which contain many things, with many branches and 
> leaves. Now is there a simple programme that can read the complicated 
> root file and generate now root files with only required info for TMVA.
TMVA can work with your root files directly, but this might have a 
performance impact when TMVA converts the data from the rootfile format 
into what it uses internally. Thus it is, as you imply, a good idea to 
preprocess the root file if it is large and only a smaller part is 
needed for learning.

In general, this preprocessing must be done manually, by reading in the 
root file, selecting what branches you want to use, and writing these 
out to a new file.

This is most easily done using the new root feature TDataFrame if you 
have access to root 6.10 or later. TDataFrame is currently under heavy 
development so the more recent version of root you can use, the better 
will your experience with it be. Examples and documentation can be found 
here <https://root.cern.ch/doc/master/group__tutorial__tdataframe.html> 
and here 
<https://root.cern.ch/doc/master/classROOT_1_1Experimental_1_1TDataFrame.html> 
respectively.

If you are using root 5, or for some reason the above does not work for 
you, the way to go is with TTreeReader. A tutorial can be found here 
<https://root.cern.ch/doc/master/group__tutorial__tree.html> (in 
particular hsimpleReader and tree1), and documentation here 
<https://root.cern.ch/doc/master/classTTreeReader.html>.
> 2.    How can I see the ROC for an analysis ?
Please see the TMVA tutorial found here 
<https://root.cern.ch/doc/master/group__tutorial__tmva.html> (in 
particular TMVAClassification.C). Basically, you open the TMVA Gui and 
generate the plots you want, there is a button for ROC curves. Make sure 
to run the tutorial without the `-b` option, e.g. `root -l 
TMVAClassification.C`.
> 3.    I am also not sure about what kind of linear combination of 
> variables the sample codes are taking and do I have a control on them 
> as well ?
I am not sure I follow you here, could you elaborate? If you want to 
understand the input data in the `TMVAClassification.C` example you can 
inspect the file `tmva_class_example.root` with the command `rootbrowse 
tmva_class_example.root` after running the example.

Please get back to me if you have any further questions.

Cheers,
   Kim

>
> Thanks,
> Subhasish
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] TMVA understanding related

[Subhasish B. <sub...@gm...>]

Hi,

I am a Research scholar in High Energy Particle Physics working on Collider
physics theory. Assume I am a beginner to data analysis with TMVA, though I
have used Root earlier for Data analysis with some codes etc,. The
following few questions I have in mind, before doing analysis with TMVA.

1.   How do I generate the histogram root file ?
                                          : I have events file which
contain many things, with many branches and leaves. Now is there a simple
programme that can read the complicated root file and generate now root
files with only required info for TMVA.

2.    How can I see the ROC for an analysis ?

3.    I am also not sure about what kind of linear combination of variables
the sample codes are taking and do I have a control on them as well ?



Thanks,
Subhasish

Re: [TMVA-users] BDT training results dependence on normalization

[Bobovnikov, I. <ily...@de...>]

Dear Kim,

Thank you for the answer.
That works for me.

Sorry all for the spamming, there was is a long delay between sending and reaching the mailing list.

Best regards, Ilya 

----- Original Message -----
From: "Kim Albertsson" <kia...@ce...>
To: "Ilya Bobovnikov" <ily...@de...>
Cc: "TMVA-users" <TMV...@li...>
Sent: Thursday, 22 February, 2018 12:20:08
Subject: Re: [TMVA-users] BDT training results dependence on normalization

Hi Ilya,

Sorry for the late reply. In the future please post to the root forum at 
https://root-forum.cern.ch, this is where the primary support for TMVA 
is located now (This mailing list still works but the response times are 
expected to be better with the forum :) ).

When using BDT's for classification (excluding BoostType=GradBoost) 
there is an internal rescaling done before training as this "should" be 
performed implicitly as part of the boosting procedure. This can be 
disabled either by adding "SkipNormalisation=True" to the method options.

There should be a textual output of this in the log just before the 
training starts where you can read a bit more.

Cheers,
   Kim

Bobovnikov, Ilya wrote:
> Dear experts,
>
> I am doing BDT training for classification in TMVA.
> My expectation was that BDT training results should depend on overall normalization of my signal and background, since node splitting function like significance (for example S/Sqrt(S+B)) depend on it. But I got the same results for the training with different normalization. Can it be that I am missing some default options or it should be like this?
>
>
> I am using ROOT 6.06
>
> And I am using the options
> factory->PrepareTrainingAndTestTree( mycuts, "NormMode=None" );
> factory->BookMethod( TMVA::Types::kBDT, "BDT"+BDTname,                           "!H:!V:NTrees=850:MinNodeSize=2.5%:MaxDepth=3:BoostType=AdaBoost:AdaBoostBeta=0.5:UseBaggedBoost:BaggedSampleFraction=0.5:SeparationType=SDivSqrtSPlusB:nCuts=20:NegWeightTreatment=Pray" );
>
>
> For example I took
>
> --- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 52014.9
> --- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 3.92221
>
> I got
>
> --- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi-
> --- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:
> --- Factory                  : --------------------------------------------------------------------------------
> --- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321
>
> And then scaled a bit
>
> --- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 183752
> --- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 13.8161
>
> And got the same (the BDT distributions are identical)
>
> --- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi-
> --- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:
> --- Factory                  : --------------------------------------------------------------------------------
> --- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321
>
>
> Best regards, Ilya
>
> P.S. code example and outputs for these two cases are attached.
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] BDT training results dependence on normalization

[Bobovnikov, I. <ily...@de...>]

Dear experts, 

I am doing BDT training for classification in TMVA. 
My expectation was that BDT training results should depend on overall normalization of my signal and background, since node splitting function like significance (for example S/Sqrt(S+B)) depend on it. But I got the same results for the training with different normalization. Can it be that I am missing some default options or it should be like this?  


I am using ROOT 6.06

And I am using the options
factory->PrepareTrainingAndTestTree( mycuts, "NormMode=None" );
factory->BookMethod( TMVA::Types::kBDT, "BDT"+BDTname,                           "!H:!V:NTrees=850:MinNodeSize=2.5%:MaxDepth=3:BoostType=AdaBoost:AdaBoostBeta=0.5:UseBaggedBoost:BaggedSampleFraction=0.5:SeparationType=SDivSqrtSPlusB:nCuts=20:NegWeightTreatment=Pray" );


For example I took

--- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 52014.9
--- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 3.92221

I got 

--- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi- 
--- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321

And then scaled a bit

--- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 183752
--- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 13.8161

And got the same (the BDT distributions are identical)

--- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi- 
--- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321


Best regards, Ilya

P.S. code example and outputs for these two cases are attached.

Re: [TMVA-users] BDT training results dependence on normalization

[Kim A. <kia...@ce...>]

Hi Ilya,

Sorry for the late reply. In the future please post to the root forum at 
https://root-forum.cern.ch, this is where the primary support for TMVA 
is located now (This mailing list still works but the response times are 
expected to be better with the forum :) ).

When using BDT's for classification (excluding BoostType=GradBoost) 
there is an internal rescaling done before training as this "should" be 
performed implicitly as part of the boosting procedure. This can be 
disabled either by adding "SkipNormalisation=True" to the method options.

There should be a textual output of this in the log just before the 
training starts where you can read a bit more.

Cheers,
   Kim

Bobovnikov, Ilya wrote:
> Dear experts,
>
> I am doing BDT training for classification in TMVA.
> My expectation was that BDT training results should depend on overall normalization of my signal and background, since node splitting function like significance (for example S/Sqrt(S+B)) depend on it. But I got the same results for the training with different normalization. Can it be that I am missing some default options or it should be like this?
>
>
> I am using ROOT 6.06
>
> And I am using the options
> factory->PrepareTrainingAndTestTree( mycuts, "NormMode=None" );
> factory->BookMethod( TMVA::Types::kBDT, "BDT"+BDTname,                           "!H:!V:NTrees=850:MinNodeSize=2.5%:MaxDepth=3:BoostType=AdaBoost:AdaBoostBeta=0.5:UseBaggedBoost:BaggedSampleFraction=0.5:SeparationType=SDivSqrtSPlusB:nCuts=20:NegWeightTreatment=Pray" );
>
>
> For example I took
>
> --- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 52014.9
> --- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 3.92221
>
> I got
>
> --- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi-
> --- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:
> --- Factory                  : --------------------------------------------------------------------------------
> --- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321
>
> And then scaled a bit
>
> --- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 183752
> --- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 13.8161
>
> And got the same (the BDT distributions are identical)
>
> --- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi-
> --- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:
> --- Factory                  : --------------------------------------------------------------------------------
> --- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321
>
>
> Best regards, Ilya
>
> P.S. code example and outputs for these two cases are attached.
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] BDT training results dependence on normalization

[Alexis K. <Ale...@ce...>]

Dear experts,
I am posting on behalf of a colleague who for some strange reason could not post direct into the forum

Regards

Alexis


----- Forwarded Message -----
From: "Ilya Bobovnikov" <ily...@de...>
To: "TMVA-users" <TMV...@li...>
Sent: Thursday, 15 February, 2018 11:51:17
Subject: BDT training results dependence on normalization

Dear experts, 

I am doing BDT training for classification in TMVA. 
My expectation was that BDT training results should depend on overall normalization of my signal and background, since node splitting function like significance (for example S/Sqrt(S+B)) depend on it. But I got the same results for the training with different normalization. Can it be that I am missing some default options or it should be like this?  


I am using ROOT 6.06

And I am using the options
factory->PrepareTrainingAndTestTree( mycuts, "NormMode=None" );
factory->BookMethod( TMVA::Types::kBDT, "BDT"+BDTname,                           "!H:!V:NTrees=850:MinNodeSize=2.5%:MaxDepth=3:BoostType=AdaBoost:AdaBoostBeta=0.5:UseBaggedBoost:BaggedSampleFraction=0.5:SeparationType=SDivSqrtSPlusB:nCuts=20" );


For example I took

--- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 52014.9
--- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 3.92221

I got 

--- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi- 
--- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321

And then scaled a bit

--- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 183752
--- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 13.8161

And got the same (the BDT distributions are identical)

--- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi- 
--- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321


Best regards, Ilya

P.S. code example and outputs for these two cases are attached.

------------------------------------------------
Dr. Alexis Kalogeropoulos

Princeton Univ., Dept of Physics 
CMS group
alk...@ce... <mailto:alk...@ce...>
Ale...@pr... <mailto:Ale...@pr...>
--------------------------------------------------

[TMVA-users] BDT training results dependence on normalization

[Bobovnikov, I. <ily...@de...>]

Dear experts, 

I am doing BDT training for classification in TMVA. 
My expectation was that BDT training results should depend on overall normalization of my signal and background, since node splitting function like significance (for example S/Sqrt(S+B)) depend on it. But I got the same results for the training with different normalization. Can it be that I am missing some default options or it should be like this?  


I am using ROOT 6.06

And I am using the options
factory->PrepareTrainingAndTestTree( mycuts, "NormMode=None" );
factory->BookMethod( TMVA::Types::kBDT, "BDT"+BDTname,                           "!H:!V:NTrees=850:MinNodeSize=2.5%:MaxDepth=3:BoostType=AdaBoost:AdaBoostBeta=0.5:UseBaggedBoost:BaggedSampleFraction=0.5:SeparationType=SDivSqrtSPlusB:nCuts=20:NegWeightTreatment=Pray" );


For example I took

--- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 52014.9
--- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 3.92221

I got 

--- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi- 
--- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321

And then scaled a bit

--- DataSetFactory           :     Background      -- number of events       : 86123  / sum of weights: 183752
--- DataSetFactory           :     Signal          -- number of events       : 9117   / sum of weights: 13.8161

And got the same (the BDT distributions are identical)

--- Factory                  : MVA              Signal efficiency at bkg eff.(error):       | Sepa-    Signifi- 
--- Factory                  : Method:          @B=0.01    @B=0.10    @B=0.30    ROC-integ. | ration:  cance:   
--- Factory                  : --------------------------------------------------------------------------------
--- Factory                  : BDTmutau       : 0.705(510)  0.938(269)  0.978(164)    0.974    | 0.769    2.321


Best regards, Ilya

P.S. code example and outputs for these two cases are attached.

Re: [TMVA-users] Further categorisation of "Background" events per input process

[Kim A. <kia...@ce...>]

Hi,

If you are using 6.10 you should also have the DNN implementation available.

Also from the example `TMVAMulticlass.C` it should be clear that you 
have the Multilayer Perceptron (MLP), Functional Discriminant (FDA_GA) 
and PDEFoam are available as well. Unfortunately, it is only these 
methods that support multiclass classification currently.

Cheers,
   Kim

Alexis Kalogeropoulos wrote:
> Hello
>
>
> Ok, so using 6.10 solves my problems ;)
>
> But still it seems that MultiClass is currently supporting only BDTG 
>  - are there any plans to include more methods ?
>
> Thanks for your time
>
> Alexis
>
>
>> On 17. Jan 2018, at 09:31, Wolf Behrenhoff <wol...@gm... 
>> <mailto:wol...@gm...>> wrote:
>>
>> Hi Alexis,
>>
>> the TMVA/DataLoader is available from ROOT >= 6.08.00.
>>
>> There were some breaking changes in TMVA between 6.06 and 6.08.
>>
>> The DataLoader.h #include in TMVAMulticlass.C was also introduced 
>> with 6.08. Therefore it seems you are mixing versions.
>>
>> Cheers,
>> Wolf
>>
>> Am Dienstag, 16. Januar 2018, 16:20:48 CET schrieb Alexis Kalogeropoulos:
>> > Hi Kim
>> >
>> > Thanks for your reply - Trying already to modify my scripts, I get 
>> already
>> >
>> > fatal error: 'TMVA/DataLoader.h' file not found
>> > #include "TMVA/DataLoader.h"
>> >
>> > I am under root 6.06 (cmssw_8_0_25 if that helps as well)
>> >
>> > Thanks
>> >
>> > -a
>> >
>> > > On 16. Jan 2018, at 15:59, Kim Albertsson <kia...@ce... 
>> <mailto:kia...@ce...>> wrote:
>> > >
>> > > Hi Alexis!
>> > >
>> > > It sounds as if you are looking for multi class classification. 
>> With multi
>> > > class classification you specify more than just background/signal.
>> > > Instead you specify e.g. TTJets, WJets, Signal, etc...
>> > >
>> > > You can check out the TMVAMulticlass.C example in TMVA to see for
>> > > yourself. Let me know if you have any further questions.
>> > >
>> > > Cheers,
>> > > Kim
>> > >
>> > > Alexis Kalogeropoulos wrote:
>> > >> Hello
>> > >>
>> > >> I was wondering if it would be possible to have some extra 
>> classification
>> > >> of background type events ie when assigning the TrainingEvents 
>> to have a
>> > >> extra string in order to store a specific type of background 
>> like if it
>> > >> is TTJets, WJets etc - That would be very helpful when trying to 
>> draw
>> > >> the input variables and see the contribution per process.
>> > >>
>> > >> Is this already existing ? I could not find it though...
>> > >>
>> > >> Thanks
>> > >>
>> > >> Alexis
>> > >>
>> > >>
>> > >>
>> > >> 
>> -------------------------------------------------------------------------
>> > >> ----- Check out the vibrant tech community on one of the world's 
>> most
>> > >> engaging tech sites, Slashdot.org <http://Slashdot.org>! 
>> http://sdm.link/slashdot
>> > >> _______________________________________________
>> > >> TMVA-users mailing list
>> > >> TMV...@li... 
>> <mailto:TMV...@li...>
>> > >> https://lists.sourceforge.net/lists/listinfo/tmva-users
>> >
>> > 
>> ----------------------------------------------------------------------------
>> > -- Check out the vibrant tech community on one of the world's most
>> > engaging tech sites, Slashdot.org <http://slashdot.org/>! 
>> http://sdm.link/slashdot
>> > _______________________________________________
>> > TMVA-users mailing list
>> > TMV...@li... 
>> <mailto:TMV...@li...>
>> > https://lists.sourceforge.net/lists/listinfo/tmva-users
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites,Slashdot.org 
>> <http://slashdot.org/>!http://sdm.link/slashdot_______________________________________________
>> TMVA-users mailing list
>> TMV...@li... 
>> <mailto:TMV...@li...>
>> https://lists.sourceforge.net/lists/listinfo/tmva-users
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

Re: [TMVA-users] Further categorisation of "Background" events per input process

[Alexis K. <Ale...@ce...>]

Dear Kim,

I ve tried the MultiClass to incorporate more bkg sources. Now, what I need to understand is how to read the results, ie for example for two bkg and one signal sources, I get three Classifier Output Distributions, labeled for instance "BDTG response for Bkg1", "BDTG response for Bkg2", "BDTG response for Signal" - So how are these should be treated/interpreted ? Ultimately, I want basically to have one canvas with training/testing considering Bkg  = Bkg1+BKg2 vs Signal, and as such I would like to have a plot with the Bkg1, Bk2 overlaid/stacked and compared to the signal trend. Isn't this possible ?


Thanks in advance


Alexis


PS - On a side node, the Multiclass I use (root 6.10) complains that only BTDG is supported so far, although one in the TMVAMulticlass.C can see that there are placeholders for other methods as well.




> On 16. Jan 2018, at 15:59, Kim Albertsson <kia...@ce...> wrote:
> 
> Hi Alexis!
> 
> It sounds as if you are looking for multi class classification. With multi class classification you specify more than just background/signal. Instead you specify e.g. TTJets, WJets, Signal, etc...
> 
> You can check out the TMVAMulticlass.C example in TMVA to see for yourself. Let me know if you have any further questions.
> 
> Cheers,
> Kim
> 
> Alexis Kalogeropoulos wrote:
>> 
>> Hello
>> 
>> I was wondering if it would be possible to have some extra classification of background type events ie when assigning the TrainingEvents to have a extra string in order to store a specific type of background like if it is TTJets, WJets etc - That would be very helpful when trying to draw the input variables and see the contribution per process.
>> 
>> Is this already existing ? I could not find it though...
>> 
>> Thanks
>> 
>> Alexis
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> TMVA-users mailing list
>> TMV...@li...
>> https://lists.sourceforge.net/lists/listinfo/tmva-users

Re: [TMVA-users] Further categorisation of "Background" events per input process

[Alexis K. <Ale...@ce...>]

Hello


Ok, so using 6.10 solves my problems ;)

But still it seems that MultiClass is currently supporting only BDTG  - are there any plans to include more methods ? 

Thanks for your time

Alexis


> On 17. Jan 2018, at 09:31, Wolf Behrenhoff <wol...@gm...> wrote:
> 
> Hi Alexis,
>  
> the TMVA/DataLoader is available from ROOT >= 6.08.00.
>  
> There were some breaking changes in TMVA between 6.06 and 6.08. 
>  
> The DataLoader.h #include in TMVAMulticlass.C was also introduced with 6.08. Therefore it seems you are mixing versions.
>  
> Cheers,
> Wolf
>  
> Am Dienstag, 16. Januar 2018, 16:20:48 CET schrieb Alexis Kalogeropoulos:
> > Hi Kim
> > 
> > Thanks for your reply - Trying already to modify my scripts, I get already
> > 
> > fatal error: 'TMVA/DataLoader.h' file not found
> > #include "TMVA/DataLoader.h"
> > 
> > I am under root 6.06 (cmssw_8_0_25 if that helps as well)
> > 
> > Thanks
> > 
> > -a
> > 
> > > On 16. Jan 2018, at 15:59, Kim Albertsson <kia...@ce...> wrote:
> > > 
> > > Hi Alexis!
> > > 
> > > It sounds as if you are looking for multi class classification. With multi
> > > class classification you specify more than just background/signal.
> > > Instead you specify e.g. TTJets, WJets, Signal, etc...
> > > 
> > > You can check out the TMVAMulticlass.C example in TMVA to see for
> > > yourself. Let me know if you have any further questions.
> > > 
> > > Cheers,
> > > Kim
> > > 
> > > Alexis Kalogeropoulos wrote:
> > >> Hello
> > >> 
> > >> I was wondering if it would be possible to have some extra classification
> > >> of background type events ie when assigning the TrainingEvents to have a
> > >> extra string in order to store a specific type of background like if it
> > >> is TTJets, WJets etc - That would be very helpful when trying to draw
> > >> the input variables and see the contribution per process.
> > >> 
> > >> Is this already existing ? I could not find it though...
> > >> 
> > >> Thanks
> > >> 
> > >> Alexis
> > >> 
> > >> 
> > >> 
> > >> -------------------------------------------------------------------------
> > >> ----- Check out the vibrant tech community on one of the world's most
> > >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> > >> _______________________________________________
> > >> TMVA-users mailing list
> > >> TMV...@li...
> > >> https://lists.sourceforge.net/lists/listinfo/tmva-users
> > 
> > ----------------------------------------------------------------------------
> > -- Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot <http://sdm.link/slashdot>
> > _______________________________________________
> > TMVA-users mailing list
> > TMV...@li... <mailto:TMV...@li...>
> > https://lists.sourceforge.net/lists/listinfo/tmva-users <https://lists.sourceforge.net/lists/listinfo/tmva-users>
>  
>  
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________>
> TMVA-users mailing list
> TMV...@li... <mailto:TMV...@li...>
> https://lists.sourceforge.net/lists/listinfo/tmva-users <https://lists.sourceforge.net/lists/listinfo/tmva-users>

[TMVA-users] BDT training vs testing discrepancies

[Alexis K. <Ale...@ce...>]

Hello

I am trying to understand while my training vs testing result for signal is so much different. It is true that in general I have limited statistics for the signal. Also, each event is weighted to a given Lumi scenario for what matters. I ve also noticed that for less input variables, the agreement becomes better, but still not what I would want to see. What would you propose to do as tests to get to the bottom of it ?

Thanks

Alex

Re: [TMVA-users] Further categorisation of "Background" events per input process

[Wolf B. <wol...@gm...>]

Hi Alexis,

the TMVA/DataLoader is available from ROOT >= 6.08.00.

There were some breaking changes in TMVA between 6.06 and 6.08. 

The DataLoader.h #include in TMVAMulticlass.C was also introduced with 6.08. 
Therefore it seems you are mixing versions.

Cheers,
Wolf

Am Dienstag, 16. Januar 2018, 16:20:48 CET schrieb Alexis Kalogeropoulos:
> Hi Kim
> 
> Thanks for your reply - Trying already to modify my scripts, I get already
> 
> fatal error: 'TMVA/DataLoader.h' file not found
> #include "TMVA/DataLoader.h"
> 
> I am under root 6.06 (cmssw_8_0_25 if that helps as well)
> 
> Thanks
> 
> -a
> 
> > On 16. Jan 2018, at 15:59, Kim Albertsson <kia...@ce...> wrote:
> > 
> > Hi Alexis!
> > 
> > It sounds as if you are looking for multi class classification. With multi
> > class classification you specify more than just background/signal.
> > Instead you specify e.g. TTJets, WJets, Signal, etc...
> > 
> > You can check out the TMVAMulticlass.C example in TMVA to see for
> > yourself. Let me know if you have any further questions.
> > 
> > Cheers,
> > Kim
> > 
> > Alexis Kalogeropoulos wrote:
> >> Hello
> >> 
> >> I was wondering if it would be possible to have some extra classification
> >> of background type events ie when assigning the TrainingEvents to have a
> >> extra string in order to store a specific type of background like if it
> >> is TTJets, WJets etc - That would be very helpful when trying to draw
> >> the input variables and see the contribution per process.
> >> 
> >> Is this already existing ? I could not find it though...
> >> 
> >> Thanks
> >> 
> >> Alexis
> >> 
> >> 
> >> 
> >> -------------------------------------------------------------------------
> >> ----- Check out the vibrant tech community on one of the world's most
> >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> >> _______________________________________________
> >> TMVA-users mailing list
> >> TMV...@li...
> >> https://lists.sourceforge.net/lists/listinfo/tmva-users
> 
> ----------------------------------------------------------------------------
> -- Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

Re: [TMVA-users] Further categorisation of "Background" events per input process

[Alexis K. <Ale...@ce...>]

Hi Kim

Thanks for your reply - Trying already to modify my scripts, I get already 

fatal error: 'TMVA/DataLoader.h' file not found
#include "TMVA/DataLoader.h"

I am under root 6.06 (cmssw_8_0_25 if that helps as well)

Thanks

-a


> On 16. Jan 2018, at 15:59, Kim Albertsson <kia...@ce...> wrote:
> 
> Hi Alexis!
> 
> It sounds as if you are looking for multi class classification. With multi class classification you specify more than just background/signal. Instead you specify e.g. TTJets, WJets, Signal, etc...
> 
> You can check out the TMVAMulticlass.C example in TMVA to see for yourself. Let me know if you have any further questions.
> 
> Cheers,
> Kim
> 
> Alexis Kalogeropoulos wrote:
>> 
>> Hello
>> 
>> I was wondering if it would be possible to have some extra classification of background type events ie when assigning the TrainingEvents to have a extra string in order to store a specific type of background like if it is TTJets, WJets etc - That would be very helpful when trying to draw the input variables and see the contribution per process.
>> 
>> Is this already existing ? I could not find it though...
>> 
>> Thanks
>> 
>> Alexis
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> TMVA-users mailing list
>> TMV...@li...
>> https://lists.sourceforge.net/lists/listinfo/tmva-users

Re: [TMVA-users] Further categorisation of "Background" events per input process

[Kim A. <kia...@ce...>]

Hi Alexis!

It sounds as if you are looking for multi class classification. With 
multi class classification you specify more than just background/signal. 
Instead you specify e.g. TTJets, WJets, Signal, etc...

You can check out the TMVAMulticlass.C example in TMVA to see for 
yourself. Let me know if you have any further questions.

Cheers,
Kim

Alexis Kalogeropoulos wrote:
>
> Hello
>
> I was wondering if it would be possible to have some extra 
> classification of background type events ie when assigning the 
> TrainingEvents to have a extra string in order to store a specific 
> type of background like if it is TTJets, WJets etc - That would be 
> very helpful when trying to draw the input variables and see the 
> contribution per process.
>
> Is this already existing ? I could not find it though...
>
> Thanks
>
> Alexis
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] Further categorisation of "Background" events per input process

[Alexis K. <Ale...@ce...>]

Hello

I was wondering if it would be possible to have some extra classification of background type events ie when assigning the TrainingEvents to have a extra string in order to store a specific type of background like if it is TTJets, WJets etc - That would be very helpful when trying to draw the input variables and see the contribution per process.

Is this already existing ? I could not find it though...

Thanks

Alexis

[TMVA-users] Drawing input variables for a certain BDT range

[Alexis K. <Ale...@ce...>]

Hello

In the .root file one gets from TMVA, there are the TrainTree and TestTree which also contain the input variables - So, using something like

TestTree->Draw("var1","BDT>x") 

one can easily draw the input variable for a specific range of the BDT method. The problem is that there are only the Bkg variables saved, so it is not possible to also plot the same variable for signal. Is there a solution to this ? I am using TMVA under CMSSW8025 (root 6.06)

Thanks

Alexis

[SPAM] R5Bbicn2GH4F3nF 永利直营3314 68点C0Ｍ邀您紸冊拿⑤8赢5⒏0提+K服Q31903390 8拿

[<370...@qq...>]

Re: [TMVA-users] Change name of TestTree and TrainTree in output root file

[Kim A. <kia...@ce...>]

Hi Julia,

No sorry, there is currently no way of renaming the "TrainTree" and 
"TestTree" trees. I'm afraid the only way currently is to save the 
generate and save the trees manually.

K-folds CV is currently being integrated into TMVA so it would be very 
useful to hear more about your usecase. If you are up for it, you could 
reply here or directly to me kim.albertsson [at] cern.ch and tell us 
about your workflow and what you think is important to have easily 
accessible.

Cheers,
   Kim

Julia Iturbe wrote:
> Dear all,
>
> Is there a way to change the name of the "TestTree" and "TrainTree" in 
> the output root file from training? I am using a k-fold splitting of 
> my data which results in k-BDTs so the output file has all the results 
> for the different BDTs but only the TestTree and TrainTree of the last 
> BDT, because they get overwritten.
>
> Cheers,
> Julia.
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] Change name of TestTree and TrainTree in output root file

[Julia I. <jul...@ce...>]

Dear all,

Is there a way to change the name of the "TestTree" and "TrainTree" in the
output root file from training? I am using a k-fold splitting of my data
which results in k-BDTs so the output file has all the results for the
different BDTs but only the TestTree and TrainTree of the last BDT, because
they get overwritten.

Cheers,
Julia.

[TMVA-users] Different behavior of the BDT as a function of number of training events

[Sang H. Ko <san...@ce...>]

Dear TMVA experts,

Hi, I'm a newcomer to TMVA and I have some questions about the behavior of BDT w.r.t. the number of training events. I'm running BDTs with relatively small amount of the signal events ( ~8k) and large amount of the bkg events (~7M). However, while optimizing hyper-parameters, I've found that the results of training are quite different when I put different number of bkg events in the training process.

However, in my point of view, the results of training should not be dependent on the number of training events except statistical fluctuations since the distribution of each training variable remains same. Also, BDT in TMVA automatically 'reweights' effective number of training events so that the number of signal & bkg training events are equal. Otherwise the definition of purity = S/(S+B) gets unphysical... I guess?

So now I'm quite confused why BDT shows such different behavior as a function of training events. Here I attach some plots with various number of bkg training events and settings that I used. Plots show that the results of training got highly overtrained in low nTrain_Background trials and not properly trained in high nTrain_Background trials. Also, lastly I tried 10000 Trees with 3.5M bkg training events then it turns out that it is well trained compared to 2000 Trees with 3.5M bkg training events. Does it mean that we need more trees to train larger amount of events?

dropbox link for the plots : https://www.dropbox.com/s/vo5m5742cvaz7fy/Plots.zip?dl=0

options for 'dataloader->PrepareTrainingAndTestTree'
    nTrain_Signal=0 (fixed to 4k, which is a half of 8k)
    nTrain_Background=n (varies)
    SplitMode=Random
    NormMode=EqualNumEvents

options for 'factory->BookMethod'
    TMVA::Types::kBDT
    "BDTG"
    NTrees=2000
    MaxDepth=5
    nCuts=500
    BoostType=Grad
    Shrinkage=0.1
    NegWeightTreatment=Pray
    UseBaggedBoost=False
    NodePurityLimit=0.5
    MinNodeSize=2%

Best regards,

Sanghyun Ko.

Re: [TMVA-users] Setting limits of classifier output histogram

[Kim A. <kia...@ce...>]

Hi Julia,

Not that I know of unfortunately. What you could do is generating the 
histograms manually with the TMVA::Reader applying the classification.

Cheers,
   Kim

Julia Iturbe wrote:
> Hi,
>
> Is there a way of setting the x-axis limits of the classifier output 
> (score) histograms to go from -1 to 1? I have a few BDTs and I would 
> like to combine the score histograms but they have slightly different 
> ranges (-0.99x, 0.99y).
>
> Thanks,
> Julia.
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] Setting limits of classifier output histogram

[Julia I. <jul...@ce...>]

Hi,

Is there a way of setting the x-axis limits of the classifier output
(score) histograms to go from -1 to 1? I have a few BDTs and I would like
to combine the score histograms but they have slightly different ranges
(-0.99x, 0.99y).

Thanks,
Julia.

Re: [TMVA-users] How to change the formula s/sqrt(b)

[Kim A. <kia...@ce...>]

Hi Saswati!

Is it the function that is optimised per decision tree leaf that you 
want to change? In that case you can select one of several cut 
optimisation functions with the `SeparationType` option. You can check 
Chapter 8.13 in the TMVA User's guide for more info 
(https://github.com/root-project/root/raw/master/documentation/tmva/UsersGuide/TMVAUsersGuide.pdf).

Example
```
factory->BookMethod(Types::kBDT, "BDT", "SeparationType:CrossEntropy");
```

Cheers,
   Kim

Saswati Nandan wrote:
> Hi,
>
>
>  In BDT, the optimized value of a variable is set where the value of 
> s/sqrt(b) is maximum. But this formula is applicable when s<<b, if 
> this formula doesn't hold for example if the statistics is very low, 
> one needs to use the asymptotic formula for the best separation of 
> signal & background. Is there any way where we can change the formula 
> s/sqrt(b) by the asymptotic formula & it will give the best value of 
> the variables where signal & background is separated by maximum?
>
> Thanks,
> Saswati
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> TMVA-users mailing list
> TMV...@li...
> https://lists.sourceforge.net/lists/listinfo/tmva-users

[TMVA-users] How to change the formula s/sqrt(b)

[Saswati N. <sas...@ce...>]

Hi,


 In BDT, the optimized value of a variable is set where the value of s/sqrt(b) is maximum. But this formula is applicable when s<<b, if this formula doesn't hold for example if the statistics is very low, one needs to use the asymptotic formula for the best separation of signal & background. Is there any way where we can change the formula s/sqrt(b) by the asymptotic formula & it will give the best value of the variables where signal & background is separated by maximum?

Thanks,
Saswati

Re: [TMVA-users] Problem running TMVA on CERN cluster

[Kim A. <kia...@ce...>]

Thank you for the feedback Konstantinos, we'll take it into consideration :)

Cheers,
Kim

Konstantinos Vellidis wrote:
>
> Dear Kim,
>
> Thank you for your advise. Indeed the Root6 distribution of TMVA on
> lxplus works fine and solves my problem.
>
> As a side remark (my apologies if it is not relevant to this forum),
> it is rather unfortunate that all Root applications on CVMFS, such as
> TMVA and RooFit, come under the directory $ROOTSYS/tutorials. One
> (like me) would expect to see documents and maybe some simple generic
> demos, not the whole application code there.
>
> A good idea might be to rename that directory to something like
> $ROOTSYS/applications, to guide people and prevent searching on the
> Web for possibly obsolete distributions.
>
> Thanks again for your kind help.
>
> Costas
>
>
>
>>
>> On 11 Nov 2017, at 9:27 PM, Kim Albertsson <kim...@ce...
>> <mailto:kim...@ce...>> wrote:
>>
>> Hi Konstantinos,
>>
>> Sorry for my brevity, resonding from my phone.
>>
>> TMVA is distributed as part of ROOT6 as well. The package from
>> sourceforge is deprecated (I know this information is unclear in some
>> places, we are working on cleaning this up.).
>>
>> You should be able to use it ”out-of-the-box” on lxplus, and you have
>> different versions available to you should you need it. Otherwise go
>> with the latest.
>>
>> Cheers,
>> Kim
>>
>> On 11 Nov 2017, at 18:31, Konstantinos Vellidis
>> <kon...@ce...
>> <mailto:kon...@ce...>> wrote:
>>
>>>
>>> Dear TMVA experts,
>>>
>>> I encounter a problem running TMVA 4.2.0 on lxplus.cern.ch
>>> <http://lxplus.cern.ch/>.
>>>
>>> I downloaded the package from SourceForge and built it on lxplus
>>> following the instructions from the User’s Guide, without noticing
>>> any problem.
>>>
>>> Running the baseline example with TMVAClassification.C in a Root
>>> session, I noticed an error message at the beginning of execution,
>>> saying that ‘TMVAGlob is not a namespace, class or enumeration.’
>>>
>>> At the end of the execution, when the GUI popped up, clicking on any
>>> button issued errors for undeclared methods belonging to that
>>> namespace. No plot could be made.
>>>
>>> The same situation occurred when I ended the Root session and tried
>>> to run TMVAGui.C on the output file TMVA.root from the classification.
>>>
>>> I checked the tmvaglob.C file, where the TMVAGlob namespace is
>>> defined, and found nothing obviously wrong. This file is properly
>>> included both in TMVAlogon.C, that sets up the environment for the
>>> GUI, and in test/variables.C, which treats the input variables from
>>> TMVA.root.
>>>
>>> Interestingly, the GUI runs correctly as part of the Root
>>> installation on my laptop (macOS Sierra 10.12.6). Same macros, no
>>> modification, and no complaint about namespaces.
>>>
>>> I could proceed by using TMVA exclusively on my laptop, but this is
>>> not the proper solution: it is much slower than the central service
>>> cluster, plus I want to launch an analysis with my students who will
>>> need to use files, applications, etc located on the cluster. It is
>>> impractical to transfer everything on laptops.
>>>
>>> I suspect that the problem on lxplus comes from its Root version
>>> (6.06.00), being not the same as on my laptop (5.34.23). I gave it
>>> some tries to fix the problem on lxplus myself, e.g. adding the
>>> namespace TMVAGlob:: in front of the method names, but nothing worked.
>>>
>>> Could you give me some help?
>>>
>>> Thank you in advance,
>>>
>>> Costas Vellidis
>>> ------------------------------------------------------------------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org <http://slashdot.org/>!
>>> http://sdm.link/slashdot
>>> _______________________________________________
>>> TMVA-users mailing list
>>> TMV...@li...
>>> <mailto:TMV...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/tmva-users
>>
>