Menu
▾
▴
Re: [Ejbca-develop] EJBCA OCSP Responder does not appear to be finding revoked certificates in large CRL?
|
From: Tomas G. <to...@pr...> - 2019-07-15 16:10:59
|
Most large users, some very very large, typically uses on line updates directly from the CA. Many CAs today don't accept only importing CRLs, as they need to keep track of issued certificates as well not only revoked (can't use "non existing is good").
Having said thst, it's interesting to be able to import large CRLs, so improving it is of interest.
Importing CRLs would actually not have to generate a dummy cert at all, since a certificate object is not needed on the responder, which should make it very fast.
If you are willing to test patches, we can work with you on improving.
Regards,
Tomas
On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
> Hi,
>I am curious whether you have other users that import large CRLs like
>this?
>
>Are we the only ones who would encounter these really long import
>times, because the other users who might have large CRLs use the EJBCA
>CA also, so they don't run into having the dummy certs being created
>for every entry in the CRL?
>Jim
>
>On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
>
>I see...
>
>I can see that a new key pair is being generated for each dummy cert.
>
>In CaImportCRLCommand.java:
>
>final KeyPair key_pair = KeyTools.genKeys("2048",
>AlgorithmConstants.KEYALGORITHM_RSA);
>
>Chaning that to 1024 or 512 would be an easy improvement. A bit more
>complex, but even larger speedup would be to use a "hard coded" dummy
>key, i.e. same key for all dummy certs.
>
>You can experiment by changing 2048 to 512 (not secure but it doesn't
>have to be for this dummy cert).
>
>Regards,
>Tomas
>
>On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
>> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
>> entries in it, and the import is only up to 8899 now, after about 40
>> minutes... which would take about 75 hours!!
>>
>> In our production systems, we are required to update some CRLs every
>> hour, so that would be a problem.
>>
>> What can I do to speed that up??
>>
>> Thanks,
>> Jim
>>
>>
>> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> AHHH... Ok, it is running now, and from checking the CertificateData
>in
>> the Mysql database, I can see that it is now populating the
>> CertificateData!!
>>
>> Thanks!
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> I wasn't sure if we should use ADAPTIVE, since we are not using the
>CA part.
>>
>>
>> I will try with "-o ADAPTIVE" now.
>>
>>
>>
>>
>> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
>> <to...@pr...> wrote:
>>
>>
>>
>> It should be ADAPTIVE.
>>
>> The documentation says:
>> bin/ejbca.sh ca importcrl --help
>> ---
>> ADAPTIVE means that missing certficates will be replaced by dummy
>> certificates to cater for proper CRLs for missing certificates.
>> ---
>>
>> Regards,
>> Tomas
>>
>> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>>> Hi,
>>>
>>> I've been looking at the code in CaImportCRLCommand, and I am
>wondering
>>> if, when the command is run in LENIENT mode and it doesn't find the
>cert
>>> in the database (and outputs the "missing in database" error
>message),
>>> what is supposed to happen?
>>>
>>> Mind you, the only error messages I am seeing are the "missing in
>>> database" error messages. I don't see any of the other error
>messages
>>> that are in/from CaImportCRLCommand.java.
>>>
>>> I am wondering if, when the command is run with "-o LENIENT", and
>when
>>> it outputs the "missing in database" error message, then the
>ejbca.sh is
>>> basically not doing anything (and not storing anything into the
>>> CertificateData table)?
>>>
>>> From the lack of any other error messages (vs. what I see in the
>>> CaImportCRLCommand.java code), that is what it looks like is
>happening.
>>>
>>> So given I am using EJBCA for OCSP responder only, is running the
>>> ejbca.sh in the "-o LENIENT" mode correct?
>>>
>>> Jim
>>>
>>>
>>>
>>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
>Ejbca-develop
>>> <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I've been doing a bunch of testing with trying to import the large
>CRL,
>>> and wanted to summarize:
>>>
>>> - The ejbca.sh runs fine - it completes without outputting an
>errors,
>>> except for outputting a bunch of the "cert xxxx missing in database"
>>> messages.
>>>
>>> - When the import completes, and I check the Mysql database, the
>crlData
>>> (the full PEM of the CRL) is in the database, but there is no rows
>in
>>> the CertificateData table for that CA at all.
>>>
>>> - I even figured out how to get the ejbca.sh to log using
>>> log4j.properties, and it output a bunch of logging, mainly the
>missing
>>> in database errors, but other than that I didn't see any obvious
>errors.
>>>
>>>
>>> So, bottom line is that even though the ejbca.sh seems to run
>without
>>> errors (other than the missing in database errors), the import is
>>> causing the CRL PEM to be stored in the database, but the
>>> CertificateData table is not being populated at all for that CA.
>>>
>>>
>>> Jim
>>>
>>>
>>>
>>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
>Ejbca-develop
>>> <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> Ok, I found Tomas' original message... it was in my Spam folder....
>>>
>>> I also tried re-running the "./ejbca.sh ca importcrl" command a
>bunch of
>>> ways. I think it is running without outputting any errors, but
>still,
>>> no rows are appearing with that issuerDN at all.
>>>
>>>
>>>
>>>
>>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>>> <to...@pr... <mailto:to...@pr...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> OCSP responses does not have anything to do with the CRL itself.
>Once
>>> the CRL has been imported each certificate status is in its own
>record
>>> in the CertificateData table. This is where status is read when
>reponses
>>> are created.
>>> If there are issues, it is in that case related to the import of the
>CRL.
>>>
>>> Verify that certificate records are created in CertificateData when
>>> importing the CRL.
>>> Also, a debug log when making OCSP request will tell exact details
>of
>>> what the responder finds in the database or not.
>>>
>>> Regards,
>>> Tomas
>>>
>>>
>>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>>> <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>>>
>>> [Re-sending, as this (handling of large CRLs) is still a problem
>and
>>> is still not working.]
>>>
>>>
>>>
>>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
>> <mailto:oh...@ya...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I think that I've found how to set/change the max_allowed_packet
>on
>>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>>> standalone/configuration/standalone.xml to:
>>>
>>>
>>
><connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>>
>>> However, when I test the OCSP Responder with a serial number (and
>CA
>>> cert) from the CA with the large (>71MB) CRL file, I am still
>>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING
>the
>>> cert serial number.
>>>
>>> NOTE, again, that I can test with certs (and CA cert) that have
>>> smaller CRL files, and those tests work.
>>>
>>> So it is only when testing with the large CRL file that is
>failing
>>> with the OCSP Responder.
>>>
>>> This seems like a potential BUG for EJBCA OCSP Responder?
>>>
>>> Please advise.
>>>
>>> Thanks,
>>> Jim
>>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
>> <mailto:oh...@ya...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> BTW, just to check, I have actually gone through the process of
>>> getting the crlData from the MySQL using mysql client (I had to
>add
>>> the --max_allowed-packet=100M to the mysql command) and I was
>able
>>> to actually get the PEM from the Mysql database that way.
>>>
>>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>>> when I run an "openssl ocsp" test against the EJBCA OCSP
>Responder,
>>> it says it cannot find the revocation.
>>>
>>> OCSP Request Data:
>>> Version: 1 (0x0)
>>> Requestor List:
>>> Certificate ID:
>>> Hash Algorithm: sha1
>>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>>> Serial Number: 2FFFF4
>>> Request Extensions:
>>> OCSP Nonce:
>>> 04103054A60A55E389B239A55228033D8F62
>>> Response verify OK
>>> 0x2FFFF4: unknown
>>> This Update: Jun 29 19:31:02 2019 GMT
>>>
>>>
>>> In the server log:
>>>
>>> 2019-06-29 15:31:02,150 DEBUG
>>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>>> (default task-1) Unable to find revocation information for
>>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>>
>>>
>>> Honestly, I am no 100% sure that the problem is that
>>> max_allowe_packet is too small on the mysql client side, but that
>is
>>> my best guess.
>>>
>>> What I AM sure is that the CRL is in mysql, but when I do a test
>>> with OCSP Responder, it cannot find the cert serial number in the
>CRL.
>>>
>>> Jim
>>>
>>>
>>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
>> <mailto:oh...@ya...>> wrote:
>>>
>>>
>>> Hi Tomas,
>>>
>>> I already had the max_allowed_packet configured on the Mysql
>>> server-side (in /etc/my.cnf).
>>>
>>> So as I said, when testing with mysql (client) I had to include
>>> "--max_allowed_packet=100M" on the mysql (client) command line,
>e.g.:
>>>
>>> mysql -u root -p --max_allowed_packet=100M
>>>
>>> If I did NOT include the "--max_allowed_packet=100M" on that
>command
>>> line, then when I did a "select" on the field base64Crl, I would
>get
>>> an error.
>>>
>>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>>> work with such large CRLs, there needs to be some way to set the
>>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>>
>>> How can I do that?
>>>
>>> If that can't be done, then I think that when EJBCA tries to
>>> retrieve the CRL from the Mysql, it (EJBCA) will see the same
>error.
>>>
>>> Thanks,
>>> Jim
>>>
>>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>>> <to...@pr... <mailto:to...@pr...>> wrote:
>>>
>>>
>>>
>>> You configure max allowed packet size in you MySQL configuration
>on the
>>> MySQL server.
>>>
>>> /Tomas
>>>
>>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>>> > I am not 100% sure, but I think I know what might be causing
>this
>>> problem.
>>> >
>>> > I did a test, where I just tried to log into mysql and do the
>>> following
>>> > select:
>>> >
>>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out'
>from
>>> CRLData
>>> > where cRLNumber=3167;
>>> >
>>> > and I got an error saying that the result was more
>>> than max_allowed_packet.
>>> >
>>> > So then I did the same thing, but for the mysql command I used:
>>> >
>>> > mysql -u root -p --max_allowed_packet=100M
>>> >
>>> > and then when I did that same select, I was able to output the
>entire
>>> > CRL as a PEM.
>>> >
>>> > So, I am theorizing that that is what is happening with EJBCA,
>i.e.,
>>> > when I run the "openssl ocsp" command, EJBCA is querying the
>> base64CRL
>>> > but the result is too large and so that is causing the query to
>fail.
>>> >
>>> > So the question now is: How can I cause EJBCA OCSP Responder
>to run
>>> > with a larger max_allowed_packet so that the query succeeds
>(and then
>>> > hopefully EJBCA OCSP Responder will be able to check the
>revocation
>>> > properly)?
>>> >
>>> > Does anyone know how that can be done?
>>> >
>>> > Thanks,
>>> > Jim
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
>> <oh...@ya... <mailto:oh...@ya...>
>>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>>> >
>>> >
>>> > Hi,
>>> >
>>> > As discussed in an earlier thread, it looks like I was able to
>>> import a
>>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so
>now,
>>> I am
>>> > doing testing, using "openssl ocsp" to send requests into the
>>> EJBCA OCSP
>>> > Responder:
>>> >
>>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>>> -req_text -serial
>>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>>> >
>>> > But it looks like no matter what serial number I use, whether
>the
>>> serial
>>> > number is in the CRL or not, returns a response like:
>>> >
>>> > OCSP Request Data:
>>> > Version: 1 (0x0)
>>> > Requestor List:
>>> > Certificate ID:
>>> > Hash Algorithm: sha1
>>> > Issuer Name Hash: 37...00
>>> > Issuer Key Hash: 39...E4
>>> > Serial Number: 2FFFF9
>>> > Request Extensions:
>>> > OCSP Nonce:
>>> > 041078B1311984A110B5E19C8DC4895F485B
>>> > Response verify OK
>>> > 0x2FFFF9: unknown
>>> > This Update: Jun 28 18:56:20 2019 GMT
>>> >
>>> > i.e., it looks like the EJBCA OCSP Responder is not able to
>actually
>>> > find any revoked certs in the CRL?
>>> >
>>> >
>>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>>> properly, in
>>> > general, because I can do the same test against a smaller CRL
>in the
>>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an
>actual
>>> > "Revoked" response:
>>> >
>>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
>> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>>> -issuer simpleca.crt
>>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9
>-req_text
>>> > -host 192.168.0.28:80
>>> > OCSP Request Data:
>>> > Version: 1 (0x0)
>>> > Requestor List:
>>> > Certificate ID:
>>> > Hash Algorithm: sha1
>>> > Issuer Name Hash:
>0C16107310427EA4ADB3C6436915CE44A15FFE55
>>> > Issuer Key Hash:
>E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>>> > Serial Number: 8486394C03E1F5D9
>>> > Request Extensions:
>>> > OCSP Nonce:
>>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>>> > Response verify OK
>>> > 0x008486394c03e1f5d9: revoked
>>> > This Update: Jun 28 19:12:48 2019 GMT
>>> > Reason: unspecified
>>> > Revocation Time: May 26 12:30:44 2019 GMT
>>> >
>>> > So it seems like the EJBCA OCSP Responder is either not
>> processing the
>>> > large CRL properly, or maybe the import didn't import the large
>>> CRL into
>>> > the OCSP Responder correctly.
>>> >
>>> > Can anyone tell me how do I diagnose this problem?
>>> >
>>> > Thanks,
>>> > Jim
>>>
>>> >
>>> >
>>> > _______________________________________________
>>> > Ejbca-develop mailing list
>>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>> >
>>>
>>>
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>>
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>>
>>>
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>>
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
>_______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
