Re: [Ejbca-develop] EJBCA OCSP Responder does not appear to be finding revoked certificates in larg

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

It should be ADAPTIVE.

The documentation says:
bin/ejbca.sh ca importcrl --help
---
ADAPTIVE means that missing certficates will be replaced by dummy
certificates to cater for proper CRLs for missing certificates.
---

Regards,
Tomas

On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> Hi,
> 
> I've been looking at the code in CaImportCRLCommand, and I am wondering
> if, when the command is run in LENIENT mode and it doesn't find the cert
> in the database (and outputs the "missing in database" error message),
> what is supposed to happen?
> 
> Mind you, the only error messages I am seeing are the "missing in
> database" error messages.  I don't see any of the other error messages
> that are in/from CaImportCRLCommand.java.
> 
> I am wondering if, when the command is run with "-o LENIENT", and when
> it outputs the "missing in database" error message, then the ejbca.sh is
> basically not doing anything (and not storing anything into the
> CertificateData table)?
> 
> From the lack of any other error messages (vs. what I see in the
> CaImportCRLCommand.java code), that is what it looks like is happening.
> 
> So given I am using EJBCA for OCSP responder only, is running the
> ejbca.sh in the "-o LENIENT" mode correct?
> 
> Jim
> 
> 
> 
> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
> 
> 
> Hi,
> 
> I've been doing a bunch of testing with trying to import the large CRL,
> and wanted to summarize:
> 
> - The ejbca.sh runs fine - it completes without outputting an errors,
> except for outputting a bunch of the "cert xxxx missing in database"
> messages.
> 
> - When the import completes, and I check the Mysql database, the crlData
> (the full PEM of the CRL) is in the database, but there is no rows in
> the CertificateData table for that CA at all.
> 
> - I even figured out how to get the ejbca.sh to log using
> log4j.properties, and it output a bunch of logging, mainly the missing
> in database errors, but other than that I didn't see any obvious errors.
> 
> 
> So, bottom line is that even though the ejbca.sh seems to run without
> errors (other than the missing in database errors), the import is
> causing the CRL PEM to be stored in the database, but the
> CertificateData table is not being populated at all for that CA.
> 
> 
> Jim
> 
> 
> 
> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
> 
> 
> Hi,
> 
> Ok, I found Tomas' original message... it was in my Spam folder....
> 
> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
> ways.  I think it is running without outputting any errors, but still,
> no rows are appearing with that issuerDN at all.
> 
> 
> 
> 
> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
> 
> 
> Hi,
> 
> OCSP responses does not have anything to do with the CRL itself. Once
> the CRL has been imported each certificate status is in its own record
> in the CertificateData table. This is where status is read when reponses
> are created.
> If there are issues, it is in that case related to the import of the CRL.
> 
> Verify that certificate records are created in CertificateData when
> importing the CRL.
> Also, a debug log when making OCSP request will tell exact details of
> what the responder finds in the database or not.
> 
> Regards,
> Tomas
> 
> 
> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
> 
>     [Re-sending, as this (handling of large CRLs) is still a problem and
>     is still not working.]
> 
> 
> 
>     On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote:
> 
> 
>     Hi,
> 
>     I think that I've found how to set/change the max_allowed_packet on
>     the client (i.e., EJBCA) side, by editing the Mysql URL in the
>     standalone/configuration/standalone.xml to:
> 
>     <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> 
>     However, when I test the OCSP Responder with a serial number (and CA
>     cert) from the CA with the large (>71MB) CRL file, I am still
>     getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
>     cert serial number.
> 
>     NOTE, again, that I can test with certs (and CA cert) that have
>     smaller CRL files, and those tests work.
> 
>     So it is only when testing with the large CRL file that is failing
>     with the OCSP Responder.
> 
>     This seems like a potential BUG for EJBCA OCSP Responder?
> 
>     Please advise.
> 
>     Thanks,
>     Jim
>     On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote:
> 
> 
>     Hi,
> 
>     BTW, just to check, I have actually gone through the process of
>     getting the crlData from the MySQL using mysql client (I had to add
>     the --max_allowed-packet=100M to the mysql command) and I was able
>     to actually get the PEM from the Mysql database that way.
> 
>     So, I KNOW that the entire CRL is in the Mysql database, and yet
>     when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
>     it says it cannot find the revocation.
> 
>     OCSP Request Data:
>     Version: 1 (0x0)
>     Requestor List:
>     Certificate ID:
>     Hash Algorithm: sha1
>     Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>     Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>     Serial Number: 2FFFF4
>     Request Extensions:
>     OCSP Nonce:
>     04103054A60A55E389B239A55228033D8F62
>     Response verify OK
>     0x2FFFF4: unknown
>     This Update: Jun 29 19:31:02 2019 GMT
> 
> 
>     In the server log:
> 
>     2019-06-29 15:31:02,150 DEBUG
>     [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>     (default task-1) Unable to find revocation information for
>     certificate with serial '2ffff4' from issuer 'CN=XXX....'
> 
> 
>     Honestly, I am no 100% sure that the problem is that
>     max_allowe_packet is too small on the mysql client side, but that is
>     my best guess.
> 
>     What I AM sure is that the CRL is in mysql, but when I do a test
>     with OCSP Responder, it cannot find the cert serial number in the CRL.
> 
>     Jim
> 
> 
>     On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote:
> 
> 
>     Hi Tomas,
> 
>     I already had the max_allowed_packet configured on the Mysql
>     server-side (in /etc/my.cnf).
> 
>     So as I said, when testing with mysql (client) I had to include
>     "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
> 
>     mysql -u root -p --max_allowed_packet=100M
> 
>     If I did NOT include the "--max_allowed_packet=100M" on that command
>     line, then when I did a "select" on the field base64Crl, I would get
>     an error.
> 
>     I suspect that EJBCA is acting as a Mysql client, so, in order to
>     work with such large CRLs, there needs to be some way to set the
>     equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
> 
>     How can I do that?
> 
>     If that can't be done, then I think that when EJBCA tries to
>     retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
> 
>     Thanks,
>     Jim
> 
>     On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>     <to...@pr...> wrote:
> 
> 
> 
>     You configure max allowed packet size in you MySQL configuration on the
>     MySQL server.
> 
>     /Tomas
> 
>     On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>     > I am not 100% sure, but I think I know what might be causing this
>     problem.
>     >
>     > I did a test, where I just tried to log into mysql and do the
>     following
>     > select:
>     >
>     > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
>     CRLData
>     > where cRLNumber=3167; 
>     >
>     > and I got an error saying that the result was more
>     than max_allowed_packet.
>     >
>     > So then I did the same thing, but for the mysql command I used:
>     >
>     > mysql -u root -p --max_allowed_packet=100M
>     >
>     > and then when I did that same select, I was able to output the entire
>     > CRL as a PEM.
>     >
>     > So, I am theorizing that that is what is happening with EJBCA, i.e.,
>     > when I run the "openssl ocsp" command, EJBCA is querying the base64CRL
>     > but the result is too large and so that is causing the query to fail.
>     >
>     > So the question now is:  How can I cause EJBCA OCSP Responder to run
>     > with a larger max_allowed_packet so that the query succeeds (and then
>     > hopefully EJBCA OCSP Responder will be able to check the revocation
>     > properly)?
>     >
>     > Does anyone know how that can be done?
>     >
>     > Thanks,
>     > Jim
>     >
>     >
>     >
>     >
>     >
>     >
>     >
>     > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya...
>     <mailto:oh...@ya...>> wrote:
>     >
>     >
>     > Hi,
>     >
>     > As discussed in an earlier thread, it looks like I was able to
>     import a
>     > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
>     I am
>     > doing testing, using "openssl ocsp" to send requests into the
>     EJBCA OCSP
>     > Responder:
>     >
>     > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>     -req_text -serial
>     > 0x<CERT_SERIAL> -host <HOST:PORT>
>     >
>     > But it looks like no matter what serial number I use, whether the
>     serial
>     > number is in the CRL or not, returns a response like:
>     >
>     > OCSP Request Data:
>     >     Version: 1 (0x0)
>     >     Requestor List:
>     >         Certificate ID:
>     >           Hash Algorithm: sha1
>     >           Issuer Name Hash: 37...00
>     >           Issuer Key Hash: 39...E4
>     >           Serial Number: 2FFFF9
>     >     Request Extensions:
>     >         OCSP Nonce:
>     >             041078B1311984A110B5E19C8DC4895F485B
>     > Response verify OK
>     > 0x2FFFF9: unknown
>     >         This Update: Jun 28 18:56:20 2019 GMT
>     >
>     > i.e., it looks like the EJBCA OCSP Responder is not able to actually
>     > find any revoked certs in the CRL?
>     >
>     >
>     > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>     properly, in
>     > general, because I can do the same test against a smaller CRL in the
>     > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
>     > "Revoked" response:
>     >
>     > [root@ejbca <mailto:root@ejbca> testocspresponder]# openssl ocsp
>     -issuer simpleca.crt
>     > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
>     > -host 192.168.0.28:80
>     > OCSP Request Data:
>     >     Version: 1 (0x0)
>     >     Requestor List:
>     >         Certificate ID:
>     >           Hash Algorithm: sha1
>     >           Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
>     >           Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>     >           Serial Number: 8486394C03E1F5D9
>     >     Request Extensions:
>     >         OCSP Nonce:
>     >             04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>     > Response verify OK
>     > 0x008486394c03e1f5d9: revoked
>     >         This Update: Jun 28 19:12:48 2019 GMT
>     >         Reason: unspecified
>     >         Revocation Time: May 26 12:30:44 2019 GMT
>     >
>     > So it seems like the EJBCA OCSP Responder is either not processing the
>     > large CRL properly, or maybe the import didn't import the large
>     CRL into
>     > the OCSP Responder correctly.
>     >
>     > Can anyone tell me how do I diagnose this problem?
>     >
>     > Thanks,
>     > Jim
> 
>     >
>     >
>     > _______________________________________________
>     > Ejbca-develop mailing list
>     > Ejb...@li...
>     <mailto:Ejb...@li...>
>     > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>     >
> 
> 
>     _______________________________________________
>     Ejbca-develop mailing list
>     Ejb...@li...
>     <mailto:Ejb...@li...>
>     https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> 
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> 
> 
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> 

Re: [Ejbca-develop] EJBCA OCSP Responder does not appear to be finding revoked certificates in larg

Re: [Ejbca-develop] EJBCA OCSP Responder does not appear to be finding revoked certificates in large CRL?