Re: [SignServer-develop] HSM slow

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Thanks Markus, I think it will resolve my problems.

Today, the certificates are stored in the HSM but we could save them in 
database.

I'll make some tests to :
  - generate the key pair
  - call generatecertreq
  - save the certificate in database
  - in the worker, get the certificate in the database (with the alias 
sent in the request)
  - in the worker, get the private key in the HSM (with the alias sent 
in the request)

I delete the step "Call activation method of the cryptotoken".

I hope it will work !

Have a nice day !

Antoine

Le 29/04/2014 10:44, Markus Kilås a écrit :
> On 2014-04-28 16:26, Antoine Louiset wrote:
>>
>> Le 28/04/2014 14:05, Markus Kilås a écrit :
>>> On 2014-04-28 09:35, Antoine Louiset wrote:
>>>> Thanks for your answer. In my case, this is not really the configuration which changes but the private key changes for each request. So I need to activate the cryptotoken each time. It s still slow as in earlier versions, not slower.
>>> How does the private key change?
>> I send the alias of the certificate and the private key to use in the
>> request. So for each signing, a new alias is used.
>>> Are you generating a new key for every request? Even that shouldn't
>>> require to run activate again as long as it is SignServer who generated
>>> the key.
>> Not necessary. Sometimes, I generate a new key, sometimes, it is not
>> needed. This is not signserver which generate the keys, this is an
>> independant program. When signserver generates new keys, it could send
>> for signing the demand to EJBCA ? Is that case, the keystore is updated
>> at this moment ? Could it be done in a Java program ?
> The 'generatekey' command (CLI/EJB or AdminWS) can generate new
> key-pairs in the KeyStore used by the CryptoToken. Those keys are then
> immediately available. As far as I can see it won't require any extra
> activation.
>
> The 'generatecertreq' command can be used to get a PKCS#11 certificate
> signing request (CSR) which can be sent to the CA to get the certificate.
>
> The 'installcertificate' and 'installcertificatechain' commands updates
> the ceritificate worker properties with the new certificates.
> Note, though that this changes the configuration and thus require a new
> activate. An other alternative would be to have a command which stores
> the certificates in the HSM replacing the dummy certificate stored there
> just to point out the keys.
> Where do you store the certificates today?
>
> The RenewalWorker can be set up to do the above steps + request
> certificates from EJBCA over web services. Note though that it also
> updates the configuration with the aliases of the new keys and thus
> require activation. The RenewalWorker implementation could maybe be
> changed to optionally instead skip those steps.
>
>
> Cheers,
> Markus
>
>
>>>> This is not a problem of changing the configuration of the worker. I do not update the configuration of the worker, I send the alias directly in the request.
>>> Regards,
>>> Markus
>>>
>>>> Le 28 avr. 2014 09:18, =?ISO-8859-1?Q?Markus_Kil=E5s?= <ma...@pr...> a écrit :
>>>>> On 2014-04-27 19:22, Antoine Louiset wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I upgrade to signserver 3.5.0. The HSM is quite slow.
>>>>> You mean that activation is still slow as in earlier versions or slower now?
>>>>>
>>>>>> If I do not call PKCS11CryptoToken activate method, I can not get the
>>>>>> certificate (and the private key) in the keystore so I can not sign anymore.
>>>>> Yes, you need to call activate if the configuration changed. Normally
>>>>> this doesn't happen so often.
>>>>>
>>>>> Do you change configuration of the workers frequently?
>>>>>
>>>>>> If I call PKCS11CryptoToken activate method, it takes 13 seconds. So
>>>>>> it's quiet long, the getKeystore() method is now very fast.
>>>>>>
>>>>>> In PKCS11CryptoToken.java in Cesecore, we could see that in the activate
>>>>>> method, the keystore is created for each call :
>>>>>>
>>>>>> final KeyStore keyStore = createKeyStore(authCode);
>>>>>> setKeyStore(keyStore);
>>>>>>
>>>>>> Is there a way tu just update the keystore and not create it each time ?
>>>>>> Or another idea to accelerate the process ?
>>>>> I think the issue is that after a configuration change of the worker a
>>>>> new instance is created. This also means a new instance of the
>>>>> PKCS11CryptoToken (both the SignServer one and the CESeCore one it uses
>>>>> internally). This means that we could not cache any KeyStore instance there.
>>>>>
>>>>> We have some open tickets for separating the worker and crypto token
>>>>> which would mean that a worker could be changed and the crypto token
>>>>> stay active:
>>>>> https://jira.primekey.se/browse/DSS-716
>>>>>
>>>>>
>>>>> Regards,
>>>>> Markus
>>>>>
>>>>>> Thanks for your help,
>>>>>>
>>>>>>
>>>>>> Antoine
>>>>>>
>>>>>>
>>>>>> Le 19/03/2014 21:35, Tomas Gustavsson a écrit :
>>>>>>> Interesting, thanks for the info.
>>>>>>>
>>>>>>>
>>>>>>> On 19 mars 2014 16:45:23 CET, Luis Maia <lui...@gm...> wrote:
>>>>>>>> On 03/19/2014 09:04 AM, Tomas Gustavsson wrote:
>>>>>>>>> SunPKCS11 always keeps the session open and reuses it. Authentication
>>>>>>>> is
>>>>>>>>> needed in order to create new sessions right, so even if SunPKCS11
>>>>>>>> would
>>>>>>>>> be able to create new sessions, it would have to store the PKCS#11
>>>>>>>>> password (i.e. autologin), preventing use of smart cards for PKCS#11
>>>>>>>>> login etc.
>>>>>>>>>
>>>>>>>>> If the session is broken (network pulled) you usually need to restart
>>>>>>>>> Java in order for SunPKCS11 to create new sessions.
>>>>>>>> Actually if the card invalidates the session with the provider logout()
>>>>>>>>
>>>>>>>> method you do not have to restart JAVA.
>>>>>>>>
>>>>>>>> I've been developing a smartcard library (used in persistent applets
>>>>>>> >from the browser) using the SUNPKCS11 and taking care of issues like
>>>>>>>> terminal disconnection events, card removal, card insertion,etc.
>>>>>>>> I noticed that some middleware for smartcards do not invalidate
>>>>>>>> sessions
>>>>>>>> when the logout method is called, but apart from that (required a few
>>>>>>>> changes in the middleware source code) it works without restarting java
>>>>>>>>
>>>>>>>> for long-lived sessions interacting with the webapp (and multiple cards
>>>>>>>>
>>>>>>>> being removed and inserted, browser refreshed,etc).
>>>>>>>>
>>>>>>>> As i cache the keystore across multiple signatures, when a card is
>>>>>>>> removed if i call the logout method and reinsert a new card it works
>>>>>>>> fine, but i must catch the insertion events and force a logout
>>>>>>>> (SUNPKCS11 is not aware of cards being removed).
>>>>>>>>
>>>>>>>> To implement the card events we used Threads checking with the
>>>>>>>> smartcardio the card presence or absence from the terminal (you can
>>>>>>>> even
>>>>>>>> use blocking methods).
>>>>>>>> It may not be the nicest solution but it works with buggy middleware
>>>>>>>> and
>>>>>>>> since the session will only be reestablished when a card is absent it
>>>>>>>> is
>>>>>>>> fast.
>>>>>>>>
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Luís.
>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Tomas
>>>>>>>>>
>>>>>>>>> On 2014-03-19 09:33, Markus Kilås wrote:
>>>>>>>>>> On 2014-03-18 15:56, Luis Maia wrote:
>>>>>>>>>>> On Tue, Mar 18, 2014 at 2:35 PM, Markus Kilås <ma...@pr...
>>>>>>>>>>> <mailto:ma...@pr...>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>         On 2014-03-18 14:10, Luis Maia wrote:
>>>>>>>>>>>         >
>>>>>>>>>>>         > Em 18/03/2014 09:10, "Markus Kilås" <ma...@pr...
>>>>>>>>>>>         <mailto:ma...@pr...>
>>>>>>>>>>>         > <mailto:ma...@pr... <mailto:ma...@pr...>>>
>>>>>>>> escreveu:
>>>>>>>>>>>         >>
>>>>>>>>>>>         >> On 2014-03-18 09:32, Antoine Louiset wrote:
>>>>>>>>>>>         >> > call the getKeystore() method because the private key
>>>>>>>> changes
>>>>>>>>>>>         for every
>>>>>>>>>>>         >> > signing.
>>>>>>>>>>>         >
>>>>>>>>>>>         >> Yes, a quick look in the CESeCore code seems to show that
>>>>>>>> after
>>>>>>>>>>>         >> activation the keystore is cached. So I believe it is
>>>>>>>> likely that
>>>>>>>>>>>         >> upgrading to SignServer 3.5 would resolve this issue for
>>>>>>>> you.
>>>>>>>>>>>         >
>>>>>>>>>>>         > I am not so sure that caching is a solution, because the
>>>>>>>> keystore
>>>>>>>>>>>         would
>>>>>>>>>>>         > return the cached private key...
>>>>>>>>>>>
>>>>>>>>>>>         In SignServer 3.5 (or if it was 3.4) we have the option to
>>>>>>>> actually
>>>>>>>>>>>         cache the PrivateKey instance which gives a different
>>>>>>>> performance as
>>>>>>>>>>>         compared to the normal way the getPrivateKey() method obtains
>>>>>>>> the key
>>>>>>>>>>>         (from the keystore) so I don't think the PrivateKey is
>>>>>>>> completely cached
>>>>>>>>>>>         only because the KeyStore is.
>>>>>>>>>>>
>>>>>>>>>>>         Anyway, would it be a problem if the PrivateKey was cached?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I have no idea how the underlying implementation should work, but
>>>>>>>> i've
>>>>>>>>>>> seen some EID pkcs#11 devices behaving erratically if the private
>>>>>>>> key is
>>>>>>>>>>> cached.
>>>>>>>>>> I could imagine their would be problem if a cached PrivateKey
>>>>>>>> instance
>>>>>>>>>> tries to use some session not available anymore. Haven't experienced
>>>>>>>>>> this yet when testing with Utimaco and SoftHSM but for sure their
>>>>>>>> could
>>>>>>>>>> be some issues.
>>>>>>>>>>
>>>>>>>>>>> An explanation I've been told (feature not a bug) to throw
>>>>>>>> exception's
>>>>>>>>>>> on cached keys from their developers is due to the strict  non
>>>>>>>> caching
>>>>>>>>>>> policy in qualified signatures...
>>>>>>>>>>> This would also mean that a session would remain established and
>>>>>>>> the
>>>>>>>>>>> card would try to reuse the session of a qualified signature and
>>>>>>>> throw
>>>>>>>>>>> an exception.
>>>>>>>>>>>
>>>>>>>>>>> Also, in a library I've been implementing, the pin would be cached
>>>>>>>> for a
>>>>>>>>>>> qualified signature and an exception thrown immediately IF the
>>>>>>>> private
>>>>>>>>>>> key object was reused (which is kind of stupid) instead of
>>>>>>>> destroying
>>>>>>>>>>> the previous session...
>>>>>>>>>>>
>>>>>>>>>>> Notice that I've no idea what should be the "right" implementation,
>>>>>>>> but
>>>>>>>>>>> i've had problems before with maintaining sessions and had to make
>>>>>>>> some
>>>>>>>>>>> workarounds.
>>>>>>>>>> I think the SunPKCS11 implementation often re-uses old sessions, I
>>>>>>>> tried
>>>>>>>>>> some time to have it close all old session but it always seems to
>>>>>>>> have
>>>>>>>>>> at least one left open, but was some time ago.
>>>>>>>>>>
>>>>>>>>>>> Meanwhile using our HSM none of this problems have ever surfaced,
>>>>>>>> but
>>>>>>>>>>> thinking about what Antoine told :
>>>>>>>>>>>
>>>>>>>>>>> " the private key changes for every signing."
>>>>>>>>>>>
>>>>>>>>>>> I keep wondering if caching the private key will maintain the
>>>>>>>> session on
>>>>>>>>>>> the device and will work properly.
>>>>>>>>>> Yes, I was wondering about this statement too and I thought it means
>>>>>>>>>> that he selects a key from the keystore based on which user it is.
>>>>>>>> In
>>>>>>>>>> that case caching the PrivateKey instance would not help, however
>>>>>>>>>> caching the KeyStore (assuming it works with the HSM/PKCS#11
>>>>>>>>>> implementation) could give better performance as it might not have
>>>>>>>> to
>>>>>>>>>> ask the HSM to enumerate all keys every time.
>>>>>>>>>>
>>>>>>>>>>> I know i'm getting in middle of the discussion here, but i think we
>>>>>>>> will
>>>>>>>>>>> have the same problem soon when we will rotate our keys and it is
>>>>>>>> nice
>>>>>>>>>>> to have a discussion before we hit the problems.
>>>>>>>>>> I think this is a useful discussion. Your input is very welcome.
>>>>>>>>>>
>>>>>>>>>> We are also interested in the topic of how to make it useful for
>>>>>>>> signers
>>>>>>>>>> to have access to multiple key-pairs and certificates.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Markus
>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Luis.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Learn Graph Databases - Download FREE O'Reilly Book
>>>>>>>>> "Graph Databases" is the definitive new guide to graph databases and
>>>>>>>> their
>>>>>>>>> applications. Written by three acclaimed leaders in the field,
>>>>>>>>> this first edition is now available. Download your free book today!
>>>>>>>>> http://p.sf.net/sfu/13534_NeoTech
>>>>>>>>> _______________________________________________
>>>>>>>>> SignServer-develop mailing list
>>>>>>>>> Sig...@li...
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/signserver-develop
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Learn Graph Databases - Download FREE O'Reilly Book
>>>>>>>> "Graph Databases" is the definitive new guide to graph databases and
>>>>>>>> their
>>>>>>>> applications. Written by three acclaimed leaders in the field,
>>>>>>>> this first edition is now available. Download your free book today!
>>>>>>>> http://p.sf.net/sfu/13534_NeoTech
>>>>>>>> _______________________________________________
>>>>>>>> SignServer-develop mailing list
>>>>>>>> Sig...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/signserver-develop
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Learn Graph Databases - Download FREE O'Reilly Book
>>>>>>> "Graph Databases" is the definitive new guide to graph databases and their
>>>>>>> applications. Written by three acclaimed leaders in the field,
>>>>>>> this first edition is now available. Download your free book today!
>>>>>>> http://p.sf.net/sfu/13534_NeoTech
>>>>>>> _______________________________________________
>>>>>>> SignServer-develop mailing list
>>>>>>> Sig...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/signserver-develop
>>>>>>
>>>>>>
>>>>>> photo
>>>>>> *Antoine Louiset*
>>>>>> Co-founder Yousign
>>>>>> t: 33 6 76 66 80 34 | e: ant...@yo... | w: yousign.fr
>>>>>>
>>>>>> Facebook
>>>>>> <http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.facebook.com%2Fpages%2FYousign%2F172490519609309>
>>>>>> Twitter
>>>>>> <http://s.wisestamp.com/links?url=https%3A%2F%2Ftwitter.com%2FYousignfr>
>>>>>> LinkedIn
>>>>>> <http://s.wisestamp.com/links?url=http%3A%2F%2Ffr.linkedin.com%2Fpub%2Fyousign-sas%2F89%2F556%2F9a6>
>>>>>> YouTube
>>>>>> <http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.youtube.com%2Fchannel%2FUCwIeklozHF4J85SBsbwAT5Q>
>>>>>> Notre dernier article : Découvrez l’API Yousign
>>>>>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fblog.yousign.fr%2Fdecouvrez-lapi-yousign%2F%3Futm_source%3Drss%26utm_medium%3Drss%26utm_campaign%3Ddecouvrez-lapi-yousign>
>>>>>> Designed with WiseStamp -
>>>>>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fr1.wisestamp.com%2Fr%2Flanding%3Fu%3D72825a891bdebde7%26v%3D3.13.31%26t%3D1398604529412%26promo%3D10%26dest%3Dhttp%253A%252F%252Fwww.wisestamp.com%252Femail-install%253Futm_source%253Dextension%2526utm_medium%253Demail%2526utm_campaign%253Dpromo_10>Get
>>>>>> yours
>>>>>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fr1.wisestamp.com%2Fr%2Flanding%3Fu%3D72825a891bdebde7%26v%3D3.13.31%26t%3D1398604529412%26promo%3D10%26dest%3Dhttp%253A%252F%252Fwww.wisestamp.com%252Femail-install%253Futm_source%253Dextension%2526utm_medium%253Demail%2526utm_campaign%253Dpromo_10>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Start Your Social Network Today - Download eXo Platform
>>>>>> Build your Enterprise Intranet with eXo Platform Software
>>>>>> Java Based Open Source Intranet - Social, Extensible, Cloud Ready
>>>>>> Get Started Now And Turn Your Intranet Into A Collaboration Platform
>>>>>> http://p.sf.net/sfu/ExoPlatform
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> SignServer-develop mailing list
>>>>>> Sig...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/signserver-develop
>>>>>>
>>>>>
>>>>> -- 
>>>>> Kind regards,
>>>>> Markus Kilås
>>>>> PKI Specialist
>>>>>
>>>>> PrimeKey Solutions AB
>>>>>
>>>>> Anderstorpsv. 16
>>>>> 171 54 Solna
>>>>> Sweden
>>>>>
>>>>> Phone: +46 70 424 94 85
>>>>> Skype: markusatskype
>>>>> Email: mar...@pr...
>>>>>
>>>>> www.primekey.se
>>>>>
>>>>>
>>>
>>
>>
>>
>> photo 	
>> *Antoine Louiset*
>> Co-founder Yousign
>> t: 33 6 76 66 80 34 | e: ant...@yo... | w: yousign.fr
>>
>> Facebook
>> <http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.facebook.com%2Fpages%2FYousign%2F172490519609309>
>> Twitter
>> <http://s.wisestamp.com/links?url=https%3A%2F%2Ftwitter.com%2FYousignfr>
>> LinkedIn
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Ffr.linkedin.com%2Fpub%2Fyousign-sas%2F89%2F556%2F9a6>
>> YouTube
>> <http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.youtube.com%2Fchannel%2FUCwIeklozHF4J85SBsbwAT5Q>
>> Notre dernier article : Découvrez l’API Yousign
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fblog.yousign.fr%2Fdecouvrez-lapi-yousign%2F%3Futm_source%3Drss%26utm_medium%3Drss%26utm_campaign%3Ddecouvrez-lapi-yousign>
>> Designed with WiseStamp -
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fr1.wisestamp.com%2Fr%2Flanding%3Fu%3D72825a891bdebde7%26v%3D3.13.31%26t%3D1398690047952%26promo%3D10%26dest%3Dhttp%253A%252F%252Fwww.wisestamp.com%252Femail-install%253Futm_source%253Dextension%2526utm_medium%253Demail%2526utm_campaign%253Dpromo_10>Get
>> yours
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fr1.wisestamp.com%2Fr%2Flanding%3Fu%3D72825a891bdebde7%26v%3D3.13.31%26t%3D1398690047952%26promo%3D10%26dest%3Dhttp%253A%252F%252Fwww.wisestamp.com%252Femail-install%253Futm_source%253Dextension%2526utm_medium%253Demail%2526utm_campaign%253Dpromo_10>
>
>

photo 	
*Antoine Louiset*
Co-founder Yousign
t: 33 6 76 66 80 34 | e: ant...@yo... | w: yousign.fr

Facebook 
<http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.facebook.com%2Fpages%2FYousign%2F172490519609309> 
Twitter 
<http://s.wisestamp.com/links?url=https%3A%2F%2Ftwitter.com%2FYousignfr> 
LinkedIn 
<http://s.wisestamp.com/links?url=http%3A%2F%2Ffr.linkedin.com%2Fpub%2Fyousign-sas%2F89%2F556%2F9a6> 
YouTube 
<http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.youtube.com%2Fchannel%2FUCwIeklozHF4J85SBsbwAT5Q>
Notre dernier article : Découvrez l’API Yousign 
<http://s.wisestamp.com/links?url=http%3A%2F%2Fblog.yousign.fr%2Fdecouvrez-lapi-yousign%2F%3Futm_source%3Drss%26utm_medium%3Drss%26utm_campaign%3Ddecouvrez-lapi-yousign>
Designed with WiseStamp - 
<http://s.wisestamp.com/links?url=http%3A%2F%2Fr1.wisestamp.com%2Fr%2Flanding%3Fu%3D72825a891bdebde7%26v%3D3.13.31%26t%3D1398764363589%26promo%3D10%26dest%3Dhttp%253A%252F%252Fwww.wisestamp.com%252Femail-install%253Futm_source%253Dextension%2526utm_medium%253Demail%2526utm_campaign%253Dpromo_10>Get 
yours 
<http://s.wisestamp.com/links?url=http%3A%2F%2Fr1.wisestamp.com%2Fr%2Flanding%3Fu%3D72825a891bdebde7%26v%3D3.13.31%26t%3D1398764363589%26promo%3D10%26dest%3Dhttp%253A%252F%252Fwww.wisestamp.com%252Femail-install%253Futm_source%253Dextension%2526utm_medium%253Demail%2526utm_campaign%253Dpromo_10> 