Re: [Quantlib-dev] FW: SABR regression

[Luigi B. <lui...@gm...>]

It looks like we crossed answers :)

I'll check what happens using long instead of size_t.

Luigi


On Wed, Nov 19, 2014 at 10:42 AM, Luigi Ballabio <lui...@gm...>
wrote:

> No, I think long is 32 bits too.
>
> How about simply checking the hash against two different values depending
> on the size of std::size_t?
>
>
> On Wed, Nov 19, 2014 at 10:25 AM, Peter Caspers <pca...@gm...>
> wrote:
>
>> std::size_t could be the problem (noarbsabr.cpp L329), is that 32 Bit
>> under VC9 ? Should we use usigned long instead, is that always 64 Bit ?
>>
>> On 19 November 2014 09:43, Luigi Ballabio <lui...@gm...>
>> wrote:
>>
>>> It only fails on VC++9.  NoArbSabrTest::testAbsorptionMatrix fails to
>>> verify the hash value of the absorption matrix.
>>>
>>> On Wed, Nov 19, 2014 at 6:43 AM, Peter Caspers <pca...@gm...>
>>> wrote:
>>>
>>>> oops. For me the test suite in the current master both with and without
>>>> #170 runs fine. Only the new regression case included in 170 fails without
>>>> the patch. Let me know what error pops up on your side.
>>>> Peter
>>>>
>>>>
>>>>
>>>> On 18 November 2014 21:57, Ferdinando M. Ametrano <na...@am...>
>>>> wrote:
>>>>
>>>>> wow, this was fast!
>>>>> btw by "sabr test" I meant one test in the QuantlIb test suite
>>>>>
>>>>> On Tue, Nov 18, 2014 at 9:47 PM, Peter Caspers <pca...@gm...
>>>>> > wrote:
>>>>>
>>>>>> Alright, I added a test case for the regression you found and one for
>>>>>> the direct/inverse functions (a deterministic one, yes). I'll have a look
>>>>>> at your data and see if we can extract more interesting test cases from
>>>>>> that.
>>>>>>
>>>>>> And please send the new case when you are back.
>>>>>>
>>>>>> Peter
>>>>>>
>>>>>> On 18 November 2014 20:31, Ferdinando M. Ametrano <na...@am...
>>>>>> > wrote:
>>>>>>
>>>>>>> Yes Peter, I can provide a caplet snapshot for a test.
>>>>>>>
>>>>>>> As for a direct/inverse test: yes please, but avoid random numbers
>>>>>>> as that would make tests unpredictable.
>>>>>>>
>>>>>>> Please also note that one sabr test is currently failing: I don't
>>>>>>> know if because of yesterday's patch or it has been failing for a while.
>>>>>>> thurdays, back at my desk,  I can provide more details, but you might want
>>>>>>> to check it up on your setup in the meantime
>>>>>>> On Nov 18, 2014 9:33 AM, "Peter Caspers" <pca...@gm...>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> yes, please send an example (otherwise I could already take the one
>>>>>>>> from the excel you sent earlier ? or was that made up ?). In addtion I
>>>>>>>> think I should add a technical test like
>>>>>>>>
>>>>>>>> for random input x
>>>>>>>> - y = direct(x) generates admissable values
>>>>>>>> - direct( inverse( y ) ) = y
>>>>>>>>
>>>>>>>> for sabr, noarbsabr, zabr and svi. What do you think ?
>>>>>>>>
>>>>>>>> Peter
>>>>>>>>
>>>>>>>>
>>>>>>>> On 17 November 2014 20:07, Ferdinando M. Ametrano <
>>>>>>>> na...@am...> wrote:
>>>>>>>>
>>>>>>>>> Thank you Peter for the timely fix.
>>>>>>>>>
>>>>>>>>> One takeaway is to create a unit test to check for possible future
>>>>>>>>> regressions.
>>>>>>>>> Do you have a relevant dataset from some paper to suggest? If not,
>>>>>>>>> I could provide a snapshot of current euro caplets.
>>>>>>>>> On Nov 17, 2014 7:40 PM, "MAZZOCCHI PAOLO" <
>>>>>>>>> pao...@es...> wrote:
>>>>>>>>>
>>>>>>>>>>  Hi Peter,
>>>>>>>>>>
>>>>>>>>>> I tested your modifications and they work properly. The results
>>>>>>>>>> are in line with the ones obtained using QL 1.2.
>>>>>>>>>>
>>>>>>>>>> Anyway, the details (forward and time to expiry) are written in
>>>>>>>>>> the excel file attached in the previous email.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> time
>>>>>>>>>>
>>>>>>>>>> fwd
>>>>>>>>>>
>>>>>>>>>> 0.1342
>>>>>>>>>>
>>>>>>>>>> 1.1075%
>>>>>>>>>>
>>>>>>>>>> 0.3833
>>>>>>>>>>
>>>>>>>>>> 1.1025%
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you very much for your help.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Paolo
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Paolo Mazzocchi*
>>>>>>>>>>
>>>>>>>>>> *Deloitte Consulting Srl*
>>>>>>>>>>
>>>>>>>>>> consulente di
>>>>>>>>>>
>>>>>>>>>> *FINANCIAL ENGINEERING - Banca IMI*
>>>>>>>>>>
>>>>>>>>>> Tel: 02-72615029  Int: 35029
>>>>>>>>>>
>>>>>>>>>> *pao...@es...
>>>>>>>>>> <pao...@es...>*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *From:* Peter Caspers [mailto:pca...@gm...]
>>>>>>>>>> *Sent:* Monday, November 17, 2014 7:00 PM
>>>>>>>>>> *To:* Ferdinando M. Ametrano
>>>>>>>>>> *Cc:* maz...@li...; QuantLib Mailing Lists; maddalena zanzi
>>>>>>>>>> *Subject:* Re: [Quantlib-dev] FW: SABR regression
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi Ferdinando,
>>>>>>>>>>
>>>>>>>>>> I pushed a fix here (effectively it should yield the same results
>>>>>>>>>> as the file attached earlier)
>>>>>>>>>>
>>>>>>>>>> https://github.com/lballabio/quantlib/pull/170
>>>>>>>>>>
>>>>>>>>>> As far as I remember the reason for the change in the
>>>>>>>>>> transformation function (for 1.3 or 1.4 ?) were nan / inf values occuring
>>>>>>>>>> from time to time in the calibration. This is why I replaced the parabolas
>>>>>>>>>> by truncated parabolas with linear wings (such that the result is C^1).
>>>>>>>>>>
>>>>>>>>>> If you feel that your test case is still working better in 1.2,
>>>>>>>>>> could you please send some more details (forward, time to expiry), I'd be
>>>>>>>>>> happy to discuss again then.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Peter
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 17 November 2014 13:25, Peter Caspers <pca...@gm...>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Ferdinando,
>>>>>>>>>>
>>>>>>>>>> one problem is the transformation function direct(...) which is
>>>>>>>>>> buggy for alpha and nu. Can you try the attached version instead and see if
>>>>>>>>>> it works better ?
>>>>>>>>>>
>>>>>>>>>> I'd like to have a closer look and try on more cases before I
>>>>>>>>>> send a PR though.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> Peter
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 17 November 2014 12:06, Ferdinando M. Ametrano <
>>>>>>>>>> na...@am...> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Peter
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> SABR implementation on the current trunk has a regression
>>>>>>>>>> compared with QuantLib 1.2.1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> While trying to calibrate problematic smiles as:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> strike
>>>>>>>>>>
>>>>>>>>>> 1.00%
>>>>>>>>>>
>>>>>>>>>> 1.13%
>>>>>>>>>>
>>>>>>>>>> 1.25%
>>>>>>>>>>
>>>>>>>>>> 1.38%
>>>>>>>>>>
>>>>>>>>>> 1.50%
>>>>>>>>>>
>>>>>>>>>> vol1
>>>>>>>>>>
>>>>>>>>>> 23.20%
>>>>>>>>>>
>>>>>>>>>> 20.25%
>>>>>>>>>>
>>>>>>>>>> 31.21%
>>>>>>>>>>
>>>>>>>>>> 39.02%
>>>>>>>>>>
>>>>>>>>>> 50.45%
>>>>>>>>>>
>>>>>>>>>> vol2
>>>>>>>>>>
>>>>>>>>>> 16.67%
>>>>>>>>>>
>>>>>>>>>> 20.20%
>>>>>>>>>>
>>>>>>>>>> 27.85%
>>>>>>>>>>
>>>>>>>>>> 32.79%
>>>>>>>>>>
>>>>>>>>>> 37.27%
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1.2.1 was able to offer these (non-optimal but decent) solutions:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> max error
>>>>>>>>>>
>>>>>>>>>> rms error
>>>>>>>>>>
>>>>>>>>>> end criteria
>>>>>>>>>>
>>>>>>>>>> alpha
>>>>>>>>>>
>>>>>>>>>> beta
>>>>>>>>>>
>>>>>>>>>> nu
>>>>>>>>>>
>>>>>>>>>> rho
>>>>>>>>>>
>>>>>>>>>> 1.79%
>>>>>>>>>>
>>>>>>>>>> 0.94%
>>>>>>>>>>
>>>>>>>>>>  StationaryPoint
>>>>>>>>>>
>>>>>>>>>> 0.58%
>>>>>>>>>>
>>>>>>>>>> 25.00%
>>>>>>>>>>
>>>>>>>>>> 346.03%
>>>>>>>>>>
>>>>>>>>>> 34.19%
>>>>>>>>>>
>>>>>>>>>> 0.65%
>>>>>>>>>>
>>>>>>>>>> 0.43%
>>>>>>>>>>
>>>>>>>>>> StationaryPoint
>>>>>>>>>>
>>>>>>>>>> 0.62%
>>>>>>>>>>
>>>>>>>>>> 25.00%
>>>>>>>>>>
>>>>>>>>>> 193.97%
>>>>>>>>>>
>>>>>>>>>> 63.66%
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> the current trunk instead fails for the second smile with error
>>>>>>>>>> "qlSABRInterpolationAlpha - nu must be non negative: -124.897 not allowed".
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> If one tries to just use the four leftmost volatilities to
>>>>>>>>>> simplify the problem, both calibrations fail with the same error. This
>>>>>>>>>> leads me to believe the problem is about the solver looking for a solution
>>>>>>>>>> in the negative nu region. If this is the case it could be solved having a
>>>>>>>>>> positive constraint for nu. Do you agree? Could you please look into it?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please find attached a spreadsheet reproducing the error. We
>>>>>>>>>> might even want to add a non regression test to the test suite.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> thank you for your help
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> regards
>>>>>>>>>>
>>>>>>>>>> F
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Prima di stampare, pensa all'ambiente ** Think about the
>>>>>>>>>> environment before printing
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> Il presente messaggio, inclusi gli eventuali allegati, ha natura
>>>>>>>>>> aziendale e potrebbe contenere informazioni confidenziali e/o riservate.
>>>>>>>>>> Chiunque lo ricevesse per errore, è pregato di avvisare tempestivamente il
>>>>>>>>>> mittente e di cancellarlo.
>>>>>>>>>> E’ strettamente vietata qualsiasi forma di utilizzo, riproduzione
>>>>>>>>>> o diffusione non autorizzata del contenuto di questo messaggio o di parte
>>>>>>>>>> di esso.
>>>>>>>>>> Pur essendo state assunte le dovute precauzioni per ridurre al
>>>>>>>>>> minimo il rischio di trasmissione di virus, si suggerisce di effettuare gli
>>>>>>>>>> opportuni controlli sui documenti allegati al presente messaggio. Non si
>>>>>>>>>> assume alcuna responsabilità per eventuali danni o perdite derivanti dalla
>>>>>>>>>> presenza di virus.
>>>>>>>>>>
>>>>>>>>>> Per lo svolgimento delle attività di investimento nel Regno
>>>>>>>>>> Unito, la società è autorizzata da Banca d'Italia ed è soggetta alla
>>>>>>>>>> vigilanza limitata della Financial Conduct Authority ( FCA ) e della
>>>>>>>>>> Prudential Regulation Authority ( PRA ) . Maggiori informazioni in merito
>>>>>>>>>> ai poteri di vigilanza della Financial Conduct Authority ( FCA ) e della
>>>>>>>>>> Prudential Regulation Authority ( PRA ) sono a disposizione previa
>>>>>>>>>> richiesta.
>>>>>>>>>>
>>>>>>>>>> Nel Regno Unito Intesa Sanpaolo S.p.A. opera attraverso la
>>>>>>>>>> filiale di Londra, sita in 90 Queen Street, London EC4N 1SA, registrata in
>>>>>>>>>> Inghilterra & Galles sotto No.FC016201, Branch No.BR000036
>>>>>>>>>>
>>>>>>>>>> In osservanza dei requisito imposti dal Internal Revenue Service
>>>>>>>>>> (Agenzia delle Entrate degli Stati Uniti), qualunque discussione relativa a
>>>>>>>>>> temi di natura fiscale contenuta in questo messaggio o nei suoi allegati
>>>>>>>>>> non e’ intesa ne’ e’ stata scritta per essere utilizzata, ne’ puo’ essere
>>>>>>>>>> utilizata per (i) evitare l’imposizione di gravami fiscali secondo il
>>>>>>>>>> codice tributario vigente negli Stati Uniti o (ii) per promuovere,
>>>>>>>>>> sollecitare o raccomandare una operazione finanziaria o altra transazione
>>>>>>>>>> indirizzata ad un altro destinatario.
>>>>>>>>>>
>>>>>>>>>> Nella Repubblica d’Irlanda, Intesa Sanpaolo Bank Ireland plc è
>>>>>>>>>> regolamentata dalla Banca Centrale d’Irlanda ed è parte del Gruppo Bancario
>>>>>>>>>> Intesa Sanpaolo S.p.A. Registrata in Irlanda come società numero 125216 –
>>>>>>>>>> IVA Reg. IE4817418C IE, sita in, KBC House, 4 George Dock, IFSC, Dublino 1,
>>>>>>>>>> Irlanda.
>>>>>>>>>>
>>>>>>>>>> ***
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> This email (including any attachment) is a corporate message and
>>>>>>>>>> may contain confidential and/or privileged and/or proprietary information.
>>>>>>>>>> If you have received this email in error, please notify the sender
>>>>>>>>>> immediately, do not use or share it and destroy this email. Any
>>>>>>>>>> unauthorised use, copying or disclosure of the material in this email or of
>>>>>>>>>> parts hereof (including reliance thereon) is strictly forbidden.
>>>>>>>>>> We have taken precautions to minimize the risk of transmitting
>>>>>>>>>> software viruses but nevertheless advise you to carry out your own virus
>>>>>>>>>> checks on any attachment of this message. We accept no liability for loss
>>>>>>>>>> or damage caused by software viruses.
>>>>>>>>>>
>>>>>>>>>> For the conduct of investment business in the UK, the Company is
>>>>>>>>>> authorised by Banca d’Italia and subject to limited regulation in the UK by
>>>>>>>>>> the Financial Conduct Authority ( FCA ) and the Prudential Regulation
>>>>>>>>>> Authority ( PRA ). Details about the extent of our regulation by the
>>>>>>>>>> Financial Conduct Authority ( FCA ) and the Prudential Regulation Authority
>>>>>>>>>> ( PRA ) are available from us on request.
>>>>>>>>>>
>>>>>>>>>> In the UK Intesa Sanpaolo S.p.A. operates through its London
>>>>>>>>>> Branch, located at 90 Queen Street, London EC4N 1SA. Registered in England
>>>>>>>>>> & Wales under No.FC016201, Branch No.BR000036
>>>>>>>>>>
>>>>>>>>>> To comply with requirements imposed by the IRS, we inform you
>>>>>>>>>> that any discussion of U.S. federal tax issues contained herein (including
>>>>>>>>>> any attachments) was not intended or written to be used, and cannot be used
>>>>>>>>>> by you, for the purpose of (i) avoiding penalties under the Internal
>>>>>>>>>> Revenue Code or (ii) promoting, marketing or recommending any transaction
>>>>>>>>>> or matter addressed herein to another party.
>>>>>>>>>>
>>>>>>>>>> In the Republic of Ireland, Intesa Sanpaolo Bank Ireland plc is
>>>>>>>>>> regulated by the Central Bank of Ireland and is a member of the Intesa
>>>>>>>>>> Sanpaolo Group. It is registered in Ireland as company no.125216 – VAT Reg.
>>>>>>>>>> No. IE 4817418C and located at, 3rd Floor, KBC House, 4 George’s Dock,
>>>>>>>>>> IFSC, Dublin 1, Ireland.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>>> with Interactivity, Sharing, Native Excel Exports, App Integration &
>>>> more
>>>> Get technology previously reserved for billion-dollar corporations, FREE
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> QuantLib-dev mailing list
>>>> Qua...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/quantlib-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> <https://implementingquantlib.blogspot.com>
>>> <https://twitter.com/lballabio>
>>>
>>
>>
>
>
> --
> <https://implementingquantlib.blogspot.com>
> <https://twitter.com/lballabio>
>



-- 
<https://implementingquantlib.blogspot.com>
<https://twitter.com/lballabio>