Menu
▾
▴
ejbca-develop — Development discussions
You can subscribe to this list here.
| 2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(1) |
Dec
(3) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2002 |
Jan
(3) |
Feb
(2) |
Mar
(8) |
Apr
(3) |
May
(6) |
Jun
(1) |
Jul
(15) |
Aug
(6) |
Sep
|
Oct
(10) |
Nov
(2) |
Dec
(4) |
| 2003 |
Jan
(1) |
Feb
(7) |
Mar
(3) |
Apr
(6) |
May
(7) |
Jun
(5) |
Jul
(5) |
Aug
(25) |
Sep
(14) |
Oct
(2) |
Nov
|
Dec
(2) |
| 2004 |
Jan
(7) |
Feb
(4) |
Mar
(12) |
Apr
(16) |
May
(43) |
Jun
(56) |
Jul
(43) |
Aug
(40) |
Sep
(66) |
Oct
(12) |
Nov
(26) |
Dec
(10) |
| 2005 |
Jan
(13) |
Feb
(33) |
Mar
(16) |
Apr
(7) |
May
(10) |
Jun
(34) |
Jul
(41) |
Aug
(8) |
Sep
(4) |
Oct
(32) |
Nov
(20) |
Dec
(25) |
| 2006 |
Jan
(30) |
Feb
(101) |
Mar
(5) |
Apr
(75) |
May
(74) |
Jun
(22) |
Jul
(6) |
Aug
(70) |
Sep
(19) |
Oct
(21) |
Nov
(31) |
Dec
(50) |
| 2007 |
Jan
(15) |
Feb
(20) |
Mar
(24) |
Apr
(33) |
May
(13) |
Jun
(18) |
Jul
(13) |
Aug
(7) |
Sep
(63) |
Oct
(68) |
Nov
(29) |
Dec
(68) |
| 2008 |
Jan
(30) |
Feb
(33) |
Mar
(30) |
Apr
(103) |
May
(78) |
Jun
(48) |
Jul
(72) |
Aug
(24) |
Sep
(62) |
Oct
(63) |
Nov
(70) |
Dec
(37) |
| 2009 |
Jan
(34) |
Feb
(35) |
Mar
(64) |
Apr
(34) |
May
(34) |
Jun
(58) |
Jul
(30) |
Aug
(30) |
Sep
(46) |
Oct
(52) |
Nov
(12) |
Dec
(23) |
| 2010 |
Jan
(121) |
Feb
(18) |
Mar
(53) |
Apr
(62) |
May
(62) |
Jun
(20) |
Jul
(33) |
Aug
(20) |
Sep
(36) |
Oct
(35) |
Nov
(44) |
Dec
(63) |
| 2011 |
Jan
(19) |
Feb
(32) |
Mar
(94) |
Apr
(41) |
May
(47) |
Jun
(25) |
Jul
(34) |
Aug
(20) |
Sep
(9) |
Oct
(41) |
Nov
(33) |
Dec
(24) |
| 2012 |
Jan
(12) |
Feb
(36) |
Mar
(48) |
Apr
(32) |
May
(20) |
Jun
(15) |
Jul
(32) |
Aug
(13) |
Sep
(33) |
Oct
(54) |
Nov
(25) |
Dec
(16) |
| 2013 |
Jan
(45) |
Feb
(39) |
Mar
(38) |
Apr
(50) |
May
(29) |
Jun
(30) |
Jul
(33) |
Aug
(12) |
Sep
(9) |
Oct
(25) |
Nov
(29) |
Dec
(20) |
| 2014 |
Jan
(25) |
Feb
(19) |
Mar
(16) |
Apr
(33) |
May
(27) |
Jun
(37) |
Jul
(29) |
Aug
(27) |
Sep
(37) |
Oct
(58) |
Nov
(109) |
Dec
(26) |
| 2015 |
Jan
(4) |
Feb
(35) |
Mar
(22) |
Apr
(35) |
May
(28) |
Jun
(20) |
Jul
(4) |
Aug
(16) |
Sep
(37) |
Oct
(13) |
Nov
(13) |
Dec
(14) |
| 2016 |
Jan
(22) |
Feb
(7) |
Mar
(23) |
Apr
(30) |
May
(10) |
Jun
(10) |
Jul
(15) |
Aug
(12) |
Sep
(22) |
Oct
(31) |
Nov
(5) |
Dec
(5) |
| 2017 |
Jan
(30) |
Feb
(25) |
Mar
(28) |
Apr
(4) |
May
(19) |
Jun
(13) |
Jul
(7) |
Aug
(1) |
Sep
(2) |
Oct
(5) |
Nov
(12) |
Dec
(2) |
| 2018 |
Jan
(7) |
Feb
|
Mar
(7) |
Apr
(2) |
May
(8) |
Jun
(18) |
Jul
(6) |
Aug
(3) |
Sep
(15) |
Oct
(33) |
Nov
(13) |
Dec
(7) |
| 2019 |
Jan
(5) |
Feb
(7) |
Mar
(30) |
Apr
(5) |
May
(4) |
Jun
(69) |
Jul
(86) |
Aug
(22) |
Sep
(6) |
Oct
(7) |
Nov
(5) |
Dec
(3) |
| 2020 |
Jan
(10) |
Feb
(12) |
Mar
(22) |
Apr
(5) |
May
(1) |
Jun
(4) |
Jul
(6) |
Aug
|
Sep
(9) |
Oct
|
Nov
|
Dec
(1) |
| 2021 |
Jan
(4) |
Feb
(11) |
Mar
(7) |
Apr
(7) |
May
|
Jun
(3) |
Jul
(10) |
Aug
(6) |
Sep
|
Oct
|
Nov
(18) |
Dec
(2) |
| 2022 |
Jan
(1) |
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
(4) |
Sep
|
Oct
|
Nov
|
Dec
|
| 2023 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
(1) |
Jun
|
Jul
|
Aug
(5) |
Sep
|
Oct
|
Nov
|
Dec
|
|
From: <oh...@ya...> - 2019-07-18 13:53:11
|
Hi,
I notice that there is:
- create-index-ejbca.sql
- optimize-ejbca-mysql.sql
Should I run BOTH of them, or just the "create-index-ejbca.sql "?
Thanks,Jim
On Thursday, July 18, 2019, 7:40:03 AM UTC, Tomas Gustavsson <to...@pr...> wrote:
Yes, i could see from your output from top that mysql takes all CPU. This indicates that you don't have any indexes in your database, which makes it horribly slow as numbers grow.
Apply database indexes from doc/sql-scripts.
Regards,
Tomas
On July 18, 2019 4:43:45 AM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
So far, processed only 185706 entries, and it looks like the processing is going to a crawl (really slow now).
Which kind of makes me wonder if maybe part of reason for the slowness is that it is having to search the certificate database for the serial number, and now, as there are more dummy certs in the database, that search is taking longer and longer?
Jim
On Wednesday, July 17, 2019, 6:03:37 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
This is what top is showing now (the import is still running):
top - 13:56:19 up 1 day, 1:09, 3 users, load average: 1.18, 1.11, 1.07
Tasks: 167 total, 2 running, 165 sleeping, 0 stopped, 0 zombie
%Cpu(s): 47.4 us, 2.0 sy, 0.0 ni, 50.3 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
KiB Mem : 4045880 total, 123192 free, 3053072 used, 869616 buff/cache
KiB Swap: 3905532 total, 3753980 free, 151552 used. 688136 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1489 mysql 20 0 1580320 129624 4396 S 92.7 3.2 1227:41 mysqld
3322 root 20 0 4031732 1.3g 7236 S 6.0 33.2 139:05.67 java
3617 root 20 0 3546412 1.1g 7256 S 1.3 29.3 176:26.68 java
22466 root 20 0 0 0 0 S 0.7 0.0 0:00.73 kworker/u4:1
That is running on 2-CPU machine with 4GB RAM.
Jim
On Wednesday, July 17, 2019, 2:33:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Yes, like top.
On July 17, 2019 3:52:53 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi Tomas,
I'm sorry that I missed the message below, but I am not clear what you are asking me to check/monitor? Do you mean like running something like "top" command?
Thanks,Jim
On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
To verify that the changes actually happen...
Do you monitor CPU usage to see what processes use CPU and memory?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
The import has been running for about an hour now and has imported 17836 entries.
Per what I had said previously, this CRL file has 1002255entries, that means it would take about 56+ hours to import this CRL.
So, this import processing is still way too slow.
I don't know what is involved, or even how much it would improve things, but I think that the next thing to try is to eliminated both the check for the serial number in the DB AND the dummy cert creation.
Jim
On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-18 07:40:12
|
Yes, i could see from your output from top that mysql takes all CPU. This indicates that you don't have any indexes in your database, which makes it horribly slow as numbers grow.
Apply database indexes from doc/sql-scripts.
Regards,
Tomas
On July 18, 2019 4:43:45 AM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
> Hi,
>So far, processed only 185706 entries, and it looks like the processing
>is going to a crawl (really slow now).
>Which kind of makes me wonder if maybe part of reason for the slowness
>is that it is having to search the certificate database for the serial
>number, and now, as there are more dummy certs in the database, that
>search is taking longer and longer?
>Jim
>
>On Wednesday, July 17, 2019, 6:03:37 PM UTC, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
>
> Hi,
>This is what top is showing now (the import is still running):
>top - 13:56:19 up 1 day, 1:09, 3 users, load average: 1.18, 1.11,
>1.07
>Tasks: 167 total, 2 running, 165 sleeping, 0 stopped, 0 zombie
>%Cpu(s): 47.4 us, 2.0 sy, 0.0 ni, 50.3 id, 0.0 wa, 0.0 hi, 0.3
>si, 0.0 st
>KiB Mem : 4045880 total, 123192 free, 3053072 used, 869616
>buff/cache
>KiB Swap: 3905532 total, 3753980 free, 151552 used. 688136 avail
>Mem
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
>COMMAND
> 1489 mysql 20 0 1580320 129624 4396 S 92.7 3.2 1227:41
>mysqld
> 3322 root 20 0 4031732 1.3g 7236 S 6.0 33.2 139:05.67
>java
> 3617 root 20 0 3546412 1.1g 7256 S 1.3 29.3 176:26.68
>java
>22466 root 20 0 0 0 0 S 0.7 0.0 0:00.73
>kworker/u4:1
>
>That is running on 2-CPU machine with 4GB RAM.
>Jim
>
>
>
>
>On Wednesday, July 17, 2019, 2:33:45 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
> Yes, like top.
>
>
>On July 17, 2019 3:52:53 PM GMT+02:00, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
> Hi Tomas,
>I'm sorry that I missed the message below, but I am not clear what you
>are asking me to check/monitor? Do you mean like running something
>like "top" command?
>Thanks,Jim
>
>On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
> To verify that the changes actually happen...
>
>Do you monitor CPU usage to see what processes use CPU and memory?
>
>
>On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
> Hi,
>The import has been running for about an hour now and has imported
>17836 entries.
>Per what I had said previously, this CRL file has 1002255entries, that
>means it would take about 56+ hours to import this CRL.
>So, this import processing is still way too slow.
>I don't know what is involved, or even how much it would improve
>things, but I think that the next thing to try is to eliminated both
>the check for the serial number in the DB AND the dummy cert creation.
>Jim
>
>On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
>
> Hi,
>Ok, I rebuilt the JAR using your Java class, and it is running now, and
>creating rows in the CertificateData, but it still SEEMS kind of slow.
>
>
>I am running now and will try to get a measurement of revoked
>entries/time and will post back.
>Jim
>
>
>
>On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya...
><oh...@ya...> wrote:
>
> Hi,
>So I just tried what I said (emptying the "if") and you are right, it
>doesn't create the rows in the CeritificateData table :(...
>Ok, I will try your version...
>Jim
>
>On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
>
>Hi,
>
>I don't think so, no quite actually. In order to use the API to
>populate
>the database you need the certificate object it seems. Something like
>the attached file should work, it still creates a dummy certificate,
>but
>uses a static hard coded key, so no key generation.
>
>The file assumes you are on 6.15.x.
>
>Regards,
>Tomas
>
>On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
>> For testing (only...), I am thinking that if I modify this If clause
>in
>> CaImportCrlCommand.java:
>>
>> if (adaptive)
>> {
>> Date time = new Date();
>> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>>
>> SubjectPublicKeyInfo pkinfo =
>> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
>> X500Name dnName = new X500Name("CN=Dummy Missing in
>> Imported CRL, serialNumber=" + serialHex);
>> Date notAfter = new Date(time.getTime() +
>315360000000L);
>> X509v3CertificateBuilder certbuilder = new
>> X509v3CertificateBuilder(X500Name.getInstance(cacert
>> .getSubjectX500Principal().getEncoded()), serialNr,
>> time, notAfter, dnName, pkinfo);
>> ContentSigner signer = new BufferingContentSigner(new
>>
>JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
>> .getPrivate()), 20480);
>> X509CertificateHolder certHolder =
>certbuilder.build(signer);
>> X509Certificate certificate =
>>
>(X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
>> X509Certificate.class);
>>
>> String fingerprint =
>> CertTools.getFingerprintAsString(certificate);
>>
>> EndEntityInformation missingUserEndEntityInformation =
>>
>((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
>> missing_user_name);
>> if (missingUserEndEntityInformation == null)
>> {
>> log.debug("Loading/updating user " +
>missing_user_name);
>>
>> EndEntityInformation userdataNew = new
>> EndEntityInformation(missing_user_name,
>> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null,
>10,
>> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0,
>null);
>>
>> userdataNew.setPassword("foo123");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
>> userdataNew, false);
>>
>> log.info("User '" + missing_user_name + "' has been
>> added.");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
>> missing_user_name, 50);
>>
>> log.info("User '" + missing_user_name + "' has been
>> updated.");
>> }
>>
>>
>((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>>
>> EJBTools.wrap(certificate), missing_user_name,
>> fingerprint, 20, 1, 1, 0, null, new Date()
>>
>> .getTime());
>> log.info("Dummy certificate '" + serialHex + "' has
>been
>> stored.");
>> }
>>
>> to (for example):
>>
>> if (adaptive)
>> {
>>
>> log.info("*** SKIPPING *** Dummy certificate '" +
>> serialHex + "' has been stored.");
>> }
>>
>>
>> Would THAT work??
>>
>> Thanks,
>> Jim
>>
>>
>>
>>
>> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> Hi Tomas (or anyone else),
>>
>> Sorry - I was typing my last msg on phone, and it wasn't as detailed
>or
>> clear as I wanted.
>>
>> Basically what I was saying is I was thinking that, as a quick sanity
>> test, if there was one or two places where the code in
>> the CaImportCRLCommand.java was maybe making a method call to create
>the
>> dummy cert and put it into the database, then maybe I could decompile
>> and modify the CaImportCRLCommand.java, and comment out those method
>> calls, then, re-do my large CRL import test and time it?
>>
>> If so, can you let me know specifically where those calls are, and if
>I
>> just comment them out, would that work?
>>
>> Ironically, that (not doing the dummy cert creation and writing) was
>> what I *thought* the "LENIENT" option was doing).
>>
>> Thanks,
>> Jim
>>
>> P. S. I have time today to try this if I figure out what needs to be
>done.
>>
>>
>>
>> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>>
>> Yes, we are definitely interested/willing to test patches. I
>actually
>> was wondering if it would be a quick test to mod the code somewhere
>to
>> avoid a call that creates the dummy cert, to get an idea if that
>would
>> reduce the import time enough??
>>
>>
>> Sent from Yahoo Mail on Android
>>
><https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>>
>> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
>> <to...@pr...> wrote:
>> Most large users, some very very large, typically uses on line
>> updates directly from the CA. Many CAs today don't accept only
>> importing CRLs, as they need to keep track of issued certificates
>as
>> well not only revoked (can't use "non existing is good").
>> Having said thst, it's interesting to be able to import large
>CRLs,
>> so improving it is of interest.
>>
>> Importing CRLs would actually not have to generate a dummy cert at
>> all, since a certificate object is not needed on the responder,
>> which should make it very fast.
>>
>> If you are willing to test patches, we can work with you on
>improving.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>> Hi,
>>
>> I am curious whether you have other users that import large
>CRLs
>> like this?
>>
>> Are we the only ones who would encounter these really long
>> import times, because the other users who might have large
>CRLs
>> use the EJBCA CA also, so they don't run into having the dummy
>> certs being created for every entry in the CRL?
>>
>> Jim
>>
>> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
>> <to...@pr...> wrote:
>>
>>
>>
>> I see...
>>
>> I can see that a new key pair is being generated for each
>dummy
>> cert.
>>
>> In CaImportCRLCommand.java:
>>
>> final KeyPair key_pair = KeyTools.genKeys("2048",
>> AlgorithmConstants.KEYALGORITHM_RSA);
>>
>> Chaning that to 1024 or 512 would be an easy improvement. A
>bit more
>> complex, but even larger speedup would be to use a "hard
>coded"
>> dummy
>> key, i.e. same key for all dummy certs.
>>
>> You can experiment by changing 2048 to 512 (not secure but it
>> doesn't
>> have to be for this dummy cert).
>>
>> Regards,
>> Tomas
>>
>> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
>> > FYI, the is running, but it is REALLY slow. The CRL file
>has
>> 1002255
>> > entries in it, and the import is only up to 8899 now, after
>> about 40
>> > minutes... which would take about 75 hours!!
>> >
>> > In our production systems, we are required to update some
>CRLs
>> every
>> > hour, so that would be a problem.
>> >
>> > What can I do to speed that up??
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > AHHH... Ok, it is running now, and from checking the
>> CertificateData in
>> > the Mysql database, I can see that it is now populating the
>> > CertificateData!!
>> >
>> > Thanks!
>> >
>> > Jim
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > I wasn't sure if we should use ADAPTIVE, since we are not
>> using the CA part.
>> >
>> >
>> > I will try with "-o ADAPTIVE" now.
>> >
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
>> > <to...@pr... <mailto:to...@pr...>> wrote:
>> >
>> >
>> >
>> > It should be ADAPTIVE.
>> >
>> > The documentation says:
>> > bin/ejbca.sh ca importcrl --help
>> > ---
>> > ADAPTIVE means that missing certficates will be replaced by
>dummy
>> > certificates to cater for proper CRLs for missing
>certificates.
>> > ---
>> >
>> > Regards,
>> > Tomas
>> >
>> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> >> Hi,
>> >>
>> >> I've been looking at the code in CaImportCRLCommand, and I
>am
>> wondering
>> >> if, when the command is run in LENIENT mode and it doesn't
>> find the cert
>> >> in the database (and outputs the "missing in database"
>error
>> message),
>> >> what is supposed to happen?
>> >>
>> >> Mind you, the only error messages I am seeing are the
>"missing in
>> >> database" error messages. I don't see any of the other
>error
>> messages
>> >> that are in/from CaImportCRLCommand.java.
>> >>
>> >> I am wondering if, when the command is run with "-o
>LENIENT",
>> and when
>> >> it outputs the "missing in database" error message, then
>the
>> ejbca.sh is
>> >> basically not doing anything (and not storing anything into
>the
>> >> CertificateData table)?
>> >>
>> >> From the lack of any other error messages (vs. what I see
>in the
>> >> CaImportCRLCommand.java code), that is what it looks like
>is
>> happening.
>> >>
>> >> So given I am using EJBCA for OCSP responder only, is
>running the
>> >> ejbca.sh in the "-o LENIENT" mode correct?
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I've been doing a bunch of testing with trying to import
>the
>> large CRL,
>> >> and wanted to summarize:
>> >>
>> >> - The ejbca.sh runs fine - it completes without outputting
>an
>> errors,
>> >> except for outputting a bunch of the "cert xxxx missing in
>> database"
>> >> messages.
>> >>
>> >> - When the import completes, and I check the Mysql
>database,
>> the crlData
>> >> (the full PEM of the CRL) is in the database, but there is
>no
>> rows in
>> >> the CertificateData table for that CA at all.
>> >>
>> >> - I even figured out how to get the ejbca.sh to log using
>> >> log4j.properties, and it output a bunch of logging, mainly
>> the missing
>> >> in database errors, but other than that I didn't see any
>> obvious errors.
>> >>
>> >>
>> >> So, bottom line is that even though the ejbca.sh seems to
>run
>> without
>> >> errors (other than the missing in database errors), the
>import is
>> >> causing the CRL PEM to be stored in the database, but the
>> >> CertificateData table is not being populated at all for
>that CA.
>> >>
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> Ok, I found Tomas' original message... it was in my Spam
>> folder....
>> >>
>> >> I also tried re-running the "./ejbca.sh ca importcrl"
>command
>> a bunch of
>> >> ways. I think it is running without outputting any errors,
>> but still,
>> >> no rows are appearing with that issuerDN at all.
>> >>
>> >>
>> >>
>> >>
>> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> OCSP responses does not have anything to do with the CRL
>> itself. Once
>> >> the CRL has been imported each certificate status is in its
>> own record
>> >> in the CertificateData table. This is where status is read
>> when reponses
>> >> are created.
>> >> If there are issues, it is in that case related to the
>import
>> of the CRL.
>> >>
>> >> Verify that certificate records are created in
>> CertificateData when
>> >> importing the CRL.
>> >> Also, a debug log when making OCSP request will tell exact
>> details of
>> >> what the responder finds in the database or not.
>> >>
>> >> Regards,
>> >> Tomas
>> >>
>> >>
>> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via
>Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >> [Re-sending, as this (handling of large CRLs) is still a
>> problem and
>> >> is still not working.]
>> >>
>> >>
>> >>
>> >> On Monday, July 1, 2019, 12:47:23 PM UTC,
><oh...@ya...
>> <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I think that I've found how to set/change the
>> max_allowed_packet on
>> >> the client (i.e., EJBCA) side, by editing the Mysql URL
>in the
>> >> standalone/configuration/standalone.xml to:
>> >>
>> >>
>> >
>>
><connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>> >>
>> >> However, when I test the OCSP Responder with a serial
>> number (and CA
>> >> cert) from the CA with the large (>71MB) CRL file, I am
>still
>> >> getting "unknown" and EJBCA OCSP Responder is still NOT
>> FINDING the
>> >> cert serial number.
>> >>
>> >> NOTE, again, that I can test with certs (and CA cert)
>that
>> have
>> >> smaller CRL files, and those tests work.
>> >>
>> >> So it is only when testing with the large CRL file that
>is
>> failing
>> >> with the OCSP Responder.
>> >>
>> >> This seems like a potential BUG for EJBCA OCSP
>Responder?
>> >>
>> >> Please advise.
>> >>
>> >> Thanks,
>> >> Jim
>> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> BTW, just to check, I have actually gone through the
>> process of
>> >> getting the crlData from the MySQL using mysql client (I
>> had to add
>> >> the --max_allowed-packet=100M to the mysql command) and
>I
>> was able
>> >> to actually get the PEM from the Mysql database that
>way.
>> >>
>> >> So, I KNOW that the entire CRL is in the Mysql database,
>> and yet
>> >> when I run an "openssl ocsp" test against the EJBCA OCSP
>> Responder,
>> >> it says it cannot find the revocation.
>> >>
>> >> OCSP Request Data:
>> >> Version: 1 (0x0)
>> >> Requestor List:
>> >> Certificate ID:
>> >> Hash Algorithm: sha1
>> >> Issuer Name Hash:
>37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> >> Issuer Key Hash:
>3929A80864E28F83AA816147119133F85A37A6E4
>> >> Serial Number: 2FFFF4
>> >> Request Extensions:
>> >> OCSP Nonce:
>> >> 04103054A60A55E389B239A55228033D8F62
>> >> Response verify OK
>> >> 0x2FFFF4: unknown
>> >> This Update: Jun 29 19:31:02 2019 GMT
>> >>
>> >>
>> >> In the server log:
>> >>
>> >> 2019-06-29 15:31:02,150 DEBUG
>> >>
>>
>[org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> >> (default task-1) Unable to find revocation information
>for
>> >> certificate with serial '2ffff4' from issuer
>'CN=XXX....'
>> >>
>> >>
>> >> Honestly, I am no 100% sure that the problem is that
>> >> max_allowe_packet is too small on the mysql client side,
>> but that is
>> >> my best guess.
>> >>
>> >> What I AM sure is that the CRL is in mysql, but when I
>do
>> a test
>> >> with OCSP Responder, it cannot find the cert serial
>number
>> in the CRL.
>> >>
>> >> Jim
>> >>
>> >>
>> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi Tomas,
>> >>
>> >> I already had the max_allowed_packet configured on the
>Mysql
>> >> server-side (in /etc/my.cnf).
>> >>
>> >> So as I said, when testing with mysql (client) I had to
>> include
>> >> "--max_allowed_packet=100M" on the mysql (client)
>command
>> line, e.g.:
>> >>
>> >> mysql -u root -p --max_allowed_packet=100M
>> >>
>> >> If I did NOT include the "--max_allowed_packet=100M" on
>> that command
>> >> line, then when I did a "select" on the field base64Crl,
>I
>> would get
>> >> an error.
>> >>
>> >> I suspect that EJBCA is acting as a Mysql client, so, in
>> order to
>> >> work with such large CRLs, there needs to be some way to
>> set the
>> >> equivalent of the "--max_allowed_packet=100M" on the
>EJBCA
>> side.
>> >>
>> >> How can I do that?
>> >>
>> >> If that can't be done, then I think that when EJBCA
>tries to
>> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
>> same error.
>> >>
>> >> Thanks,
>> >> Jim
>> >>
>> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas
>Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >>
>> >> You configure max allowed packet size in you MySQL
>> configuration on the
>> >> MySQL server.
>> >>
>> >> /Tomas
>> >>
>> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> >> > I am not 100% sure, but I think I know what might be
>> causing this
>> >> problem.
>> >> >
>> >> > I did a test, where I just tried to log into mysql and
>> do the
>> >> following
>> >> > select:
>> >> >
>> >> > select base64Crl INTO OUTFILE
>> '/var/lib/mysql-files/xx.out' from
>> >> CRLData
>> >> > where cRLNumber=3167;
>> >> >
>> >> > and I got an error saying that the result was more
>> >> than max_allowed_packet.
>> >> >
>> >> > So then I did the same thing, but for the mysql
>command
>> I used:
>> >> >
>> >> > mysql -u root -p --max_allowed_packet=100M
>> >> >
>> >> > and then when I did that same select, I was able to
>> output the entire
>> >> > CRL as a PEM.
>> >> >
>> >> > So, I am theorizing that that is what is happening
>with
>> EJBCA, i.e.,
>> >> > when I run the "openssl ocsp" command, EJBCA is
>querying the
>> > base64CRL
>> >> > but the result is too large and so that is causing the
>> query to fail.
>> >> >
>> >> > So the question now is: How can I cause EJBCA OCSP
>> Responder to run
>> >> > with a larger max_allowed_packet so that the query
>> succeeds (and then
>> >> > hopefully EJBCA OCSP Responder will be able to check
>the
>> revocation
>> >> > properly)?
>> >> >
>> >> > Does anyone know how that can be done?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
>> > <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>
>> >> <mailto:oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
>> >> >
>> >> >
>> >> > Hi,
>> >> >
>> >> > As discussed in an earlier thread, it looks like I was
>> able to
>> >> import a
>> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
>> Responder), so now,
>> >> I am
>> >> > doing testing, using "openssl ocsp" to send requests
>> into the
>> >> EJBCA OCSP
>> >> > Responder:
>> >> >
>> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> >> -req_text -serial
>> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >> >
>> >> > But it looks like no matter what serial number I use,
>> whether the
>> >> serial
>> >> > number is in the CRL or not, returns a response like:
>> >> >
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash: 37...00
>> >> > Issuer Key Hash: 39...E4
>> >> > Serial Number: 2FFFF9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 041078B1311984A110B5E19C8DC4895F485B
>> >> > Response verify OK
>> >> > 0x2FFFF9: unknown
>> >> > This Update: Jun 28 18:56:20 2019 GMT
>> >> >
>> >> > i.e., it looks like the EJBCA OCSP Responder is not
>able
>> to actually
>> >> > find any revoked certs in the CRL?
>> >> >
>> >> >
>> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is
>functioning
>> >> properly, in
>> >> > general, because I can do the same test against a
>> smaller CRL in the
>> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
>> get an actual
>> >> > "Revoked" response:
>> >> >
>> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
>> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
>> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
>> openssl ocsp
>> >> -issuer simpleca.crt
>> >> > -VAfile simplecabinding2.crt -serial
>> 0x008486394c03e1f5d9 -req_text
>> >> > -host 192.168.0.28:80
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash:
>> 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> >> > Issuer Key Hash:
>> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> >> > Serial Number: 8486394C03E1F5D9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> >> > Response verify OK
>> >> > 0x008486394c03e1f5d9: revoked
>> >> > This Update: Jun 28 19:12:48 2019 GMT
>> >> > Reason: unspecified
>> >> > Revocation Time: May 26 12:30:44 2019 GMT
>> >> >
>> >> > So it seems like the EJBCA OCSP Responder is either
>not
>> > processing the
>> >> > large CRL properly, or maybe the import didn't import
>> the large
>> >> CRL into
>> >> > the OCSP Responder correctly.
>> >> >
>> >> > Can anyone tell me how do I diagnose this problem?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Ejbca-develop mailing list
>> >> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> >
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> >
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >>
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>_______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
|
From: <oh...@ya...> - 2019-07-18 02:44:03
|
Hi,
So far, processed only 185706 entries, and it looks like the processing is going to a crawl (really slow now).
Which kind of makes me wonder if maybe part of reason for the slowness is that it is having to search the certificate database for the serial number, and now, as there are more dummy certs in the database, that search is taking longer and longer?
Jim
On Wednesday, July 17, 2019, 6:03:37 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
This is what top is showing now (the import is still running):
top - 13:56:19 up 1 day, 1:09, 3 users, load average: 1.18, 1.11, 1.07
Tasks: 167 total, 2 running, 165 sleeping, 0 stopped, 0 zombie
%Cpu(s): 47.4 us, 2.0 sy, 0.0 ni, 50.3 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
KiB Mem : 4045880 total, 123192 free, 3053072 used, 869616 buff/cache
KiB Swap: 3905532 total, 3753980 free, 151552 used. 688136 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1489 mysql 20 0 1580320 129624 4396 S 92.7 3.2 1227:41 mysqld
3322 root 20 0 4031732 1.3g 7236 S 6.0 33.2 139:05.67 java
3617 root 20 0 3546412 1.1g 7256 S 1.3 29.3 176:26.68 java
22466 root 20 0 0 0 0 S 0.7 0.0 0:00.73 kworker/u4:1
That is running on 2-CPU machine with 4GB RAM.
Jim
On Wednesday, July 17, 2019, 2:33:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Yes, like top.
On July 17, 2019 3:52:53 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi Tomas,
I'm sorry that I missed the message below, but I am not clear what you are asking me to check/monitor? Do you mean like running something like "top" command?
Thanks,Jim
On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
To verify that the changes actually happen...
Do you monitor CPU usage to see what processes use CPU and memory?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
The import has been running for about an hour now and has imported 17836 entries.
Per what I had said previously, this CRL file has 1002255entries, that means it would take about 56+ hours to import this CRL.
So, this import processing is still way too slow.
I don't know what is involved, or even how much it would improve things, but I think that the next thing to try is to eliminated both the check for the serial number in the DB AND the dummy cert creation.
Jim
On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-17 18:03:22
|
Hi,
This is what top is showing now (the import is still running):
top - 13:56:19 up 1 day, 1:09, 3 users, load average: 1.18, 1.11, 1.07
Tasks: 167 total, 2 running, 165 sleeping, 0 stopped, 0 zombie
%Cpu(s): 47.4 us, 2.0 sy, 0.0 ni, 50.3 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st
KiB Mem : 4045880 total, 123192 free, 3053072 used, 869616 buff/cache
KiB Swap: 3905532 total, 3753980 free, 151552 used. 688136 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1489 mysql 20 0 1580320 129624 4396 S 92.7 3.2 1227:41 mysqld
3322 root 20 0 4031732 1.3g 7236 S 6.0 33.2 139:05.67 java
3617 root 20 0 3546412 1.1g 7256 S 1.3 29.3 176:26.68 java
22466 root 20 0 0 0 0 S 0.7 0.0 0:00.73 kworker/u4:1
That is running on 2-CPU machine with 4GB RAM.
Jim
On Wednesday, July 17, 2019, 2:33:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Yes, like top.
On July 17, 2019 3:52:53 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi Tomas,
I'm sorry that I missed the message below, but I am not clear what you are asking me to check/monitor? Do you mean like running something like "top" command?
Thanks,Jim
On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
To verify that the changes actually happen...
Do you monitor CPU usage to see what processes use CPU and memory?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
The import has been running for about an hour now and has imported 17836 entries.
Per what I had said previously, this CRL file has 1002255entries, that means it would take about 56+ hours to import this CRL.
So, this import processing is still way too slow.
I don't know what is involved, or even how much it would improve things, but I think that the next thing to try is to eliminated both the check for the serial number in the DB AND the dummy cert creation.
Jim
On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-17 14:33:49
|
Yes, like top.
On July 17, 2019 3:52:53 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
> Hi Tomas,
>I'm sorry that I missed the message below, but I am not clear what you
>are asking me to check/monitor? Do you mean like running something
>like "top" command?
>Thanks,Jim
>
>On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
> To verify that the changes actually happen...
>
>Do you monitor CPU usage to see what processes use CPU and memory?
>
>
>On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
> Hi,
>The import has been running for about an hour now and has imported
>17836 entries.
>Per what I had said previously, this CRL file has 1002255entries, that
>means it would take about 56+ hours to import this CRL.
>So, this import processing is still way too slow.
>I don't know what is involved, or even how much it would improve
>things, but I think that the next thing to try is to eliminated both
>the check for the serial number in the DB AND the dummy cert creation.
>Jim
>
>On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
>
> Hi,
>Ok, I rebuilt the JAR using your Java class, and it is running now, and
>creating rows in the CertificateData, but it still SEEMS kind of slow.
>
>
>I am running now and will try to get a measurement of revoked
>entries/time and will post back.
>Jim
>
>
>
>On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya...
><oh...@ya...> wrote:
>
> Hi,
>So I just tried what I said (emptying the "if") and you are right, it
>doesn't create the rows in the CeritificateData table :(...
>Ok, I will try your version...
>Jim
>
>On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
>
>Hi,
>
>I don't think so, no quite actually. In order to use the API to
>populate
>the database you need the certificate object it seems. Something like
>the attached file should work, it still creates a dummy certificate,
>but
>uses a static hard coded key, so no key generation.
>
>The file assumes you are on 6.15.x.
>
>Regards,
>Tomas
>
>On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
>> For testing (only...), I am thinking that if I modify this If clause
>in
>> CaImportCrlCommand.java:
>>
>> if (adaptive)
>> {
>> Date time = new Date();
>> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>>
>> SubjectPublicKeyInfo pkinfo =
>> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
>> X500Name dnName = new X500Name("CN=Dummy Missing in
>> Imported CRL, serialNumber=" + serialHex);
>> Date notAfter = new Date(time.getTime() +
>315360000000L);
>> X509v3CertificateBuilder certbuilder = new
>> X509v3CertificateBuilder(X500Name.getInstance(cacert
>> .getSubjectX500Principal().getEncoded()), serialNr,
>> time, notAfter, dnName, pkinfo);
>> ContentSigner signer = new BufferingContentSigner(new
>>
>JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
>> .getPrivate()), 20480);
>> X509CertificateHolder certHolder =
>certbuilder.build(signer);
>> X509Certificate certificate =
>>
>(X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
>> X509Certificate.class);
>>
>> String fingerprint =
>> CertTools.getFingerprintAsString(certificate);
>>
>> EndEntityInformation missingUserEndEntityInformation =
>>
>((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
>> missing_user_name);
>> if (missingUserEndEntityInformation == null)
>> {
>> log.debug("Loading/updating user " +
>missing_user_name);
>>
>> EndEntityInformation userdataNew = new
>> EndEntityInformation(missing_user_name,
>> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null,
>10,
>> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0,
>null);
>>
>> userdataNew.setPassword("foo123");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
>> userdataNew, false);
>>
>> log.info("User '" + missing_user_name + "' has been
>> added.");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
>> missing_user_name, 50);
>>
>> log.info("User '" + missing_user_name + "' has been
>> updated.");
>> }
>>
>>
>((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>>
>> EJBTools.wrap(certificate), missing_user_name,
>> fingerprint, 20, 1, 1, 0, null, new Date()
>>
>> .getTime());
>> log.info("Dummy certificate '" + serialHex + "' has
>been
>> stored.");
>> }
>>
>> to (for example):
>>
>> if (adaptive)
>> {
>>
>> log.info("*** SKIPPING *** Dummy certificate '" +
>> serialHex + "' has been stored.");
>> }
>>
>>
>> Would THAT work??
>>
>> Thanks,
>> Jim
>>
>>
>>
>>
>> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> Hi Tomas (or anyone else),
>>
>> Sorry - I was typing my last msg on phone, and it wasn't as detailed
>or
>> clear as I wanted.
>>
>> Basically what I was saying is I was thinking that, as a quick sanity
>> test, if there was one or two places where the code in
>> the CaImportCRLCommand.java was maybe making a method call to create
>the
>> dummy cert and put it into the database, then maybe I could decompile
>> and modify the CaImportCRLCommand.java, and comment out those method
>> calls, then, re-do my large CRL import test and time it?
>>
>> If so, can you let me know specifically where those calls are, and if
>I
>> just comment them out, would that work?
>>
>> Ironically, that (not doing the dummy cert creation and writing) was
>> what I *thought* the "LENIENT" option was doing).
>>
>> Thanks,
>> Jim
>>
>> P. S. I have time today to try this if I figure out what needs to be
>done.
>>
>>
>>
>> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>>
>> Yes, we are definitely interested/willing to test patches. I
>actually
>> was wondering if it would be a quick test to mod the code somewhere
>to
>> avoid a call that creates the dummy cert, to get an idea if that
>would
>> reduce the import time enough??
>>
>>
>> Sent from Yahoo Mail on Android
>>
><https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>>
>> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
>> <to...@pr...> wrote:
>> Most large users, some very very large, typically uses on line
>> updates directly from the CA. Many CAs today don't accept only
>> importing CRLs, as they need to keep track of issued certificates
>as
>> well not only revoked (can't use "non existing is good").
>> Having said thst, it's interesting to be able to import large
>CRLs,
>> so improving it is of interest.
>>
>> Importing CRLs would actually not have to generate a dummy cert at
>> all, since a certificate object is not needed on the responder,
>> which should make it very fast.
>>
>> If you are willing to test patches, we can work with you on
>improving.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>> Hi,
>>
>> I am curious whether you have other users that import large
>CRLs
>> like this?
>>
>> Are we the only ones who would encounter these really long
>> import times, because the other users who might have large
>CRLs
>> use the EJBCA CA also, so they don't run into having the dummy
>> certs being created for every entry in the CRL?
>>
>> Jim
>>
>> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
>> <to...@pr...> wrote:
>>
>>
>>
>> I see...
>>
>> I can see that a new key pair is being generated for each
>dummy
>> cert.
>>
>> In CaImportCRLCommand.java:
>>
>> final KeyPair key_pair = KeyTools.genKeys("2048",
>> AlgorithmConstants.KEYALGORITHM_RSA);
>>
>> Chaning that to 1024 or 512 would be an easy improvement. A
>bit more
>> complex, but even larger speedup would be to use a "hard
>coded"
>> dummy
>> key, i.e. same key for all dummy certs.
>>
>> You can experiment by changing 2048 to 512 (not secure but it
>> doesn't
>> have to be for this dummy cert).
>>
>> Regards,
>> Tomas
>>
>> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
>> > FYI, the is running, but it is REALLY slow. The CRL file
>has
>> 1002255
>> > entries in it, and the import is only up to 8899 now, after
>> about 40
>> > minutes... which would take about 75 hours!!
>> >
>> > In our production systems, we are required to update some
>CRLs
>> every
>> > hour, so that would be a problem.
>> >
>> > What can I do to speed that up??
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > AHHH... Ok, it is running now, and from checking the
>> CertificateData in
>> > the Mysql database, I can see that it is now populating the
>> > CertificateData!!
>> >
>> > Thanks!
>> >
>> > Jim
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > I wasn't sure if we should use ADAPTIVE, since we are not
>> using the CA part.
>> >
>> >
>> > I will try with "-o ADAPTIVE" now.
>> >
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
>> > <to...@pr... <mailto:to...@pr...>> wrote:
>> >
>> >
>> >
>> > It should be ADAPTIVE.
>> >
>> > The documentation says:
>> > bin/ejbca.sh ca importcrl --help
>> > ---
>> > ADAPTIVE means that missing certficates will be replaced by
>dummy
>> > certificates to cater for proper CRLs for missing
>certificates.
>> > ---
>> >
>> > Regards,
>> > Tomas
>> >
>> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> >> Hi,
>> >>
>> >> I've been looking at the code in CaImportCRLCommand, and I
>am
>> wondering
>> >> if, when the command is run in LENIENT mode and it doesn't
>> find the cert
>> >> in the database (and outputs the "missing in database"
>error
>> message),
>> >> what is supposed to happen?
>> >>
>> >> Mind you, the only error messages I am seeing are the
>"missing in
>> >> database" error messages. I don't see any of the other
>error
>> messages
>> >> that are in/from CaImportCRLCommand.java.
>> >>
>> >> I am wondering if, when the command is run with "-o
>LENIENT",
>> and when
>> >> it outputs the "missing in database" error message, then
>the
>> ejbca.sh is
>> >> basically not doing anything (and not storing anything into
>the
>> >> CertificateData table)?
>> >>
>> >> From the lack of any other error messages (vs. what I see
>in the
>> >> CaImportCRLCommand.java code), that is what it looks like
>is
>> happening.
>> >>
>> >> So given I am using EJBCA for OCSP responder only, is
>running the
>> >> ejbca.sh in the "-o LENIENT" mode correct?
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I've been doing a bunch of testing with trying to import
>the
>> large CRL,
>> >> and wanted to summarize:
>> >>
>> >> - The ejbca.sh runs fine - it completes without outputting
>an
>> errors,
>> >> except for outputting a bunch of the "cert xxxx missing in
>> database"
>> >> messages.
>> >>
>> >> - When the import completes, and I check the Mysql
>database,
>> the crlData
>> >> (the full PEM of the CRL) is in the database, but there is
>no
>> rows in
>> >> the CertificateData table for that CA at all.
>> >>
>> >> - I even figured out how to get the ejbca.sh to log using
>> >> log4j.properties, and it output a bunch of logging, mainly
>> the missing
>> >> in database errors, but other than that I didn't see any
>> obvious errors.
>> >>
>> >>
>> >> So, bottom line is that even though the ejbca.sh seems to
>run
>> without
>> >> errors (other than the missing in database errors), the
>import is
>> >> causing the CRL PEM to be stored in the database, but the
>> >> CertificateData table is not being populated at all for
>that CA.
>> >>
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> Ok, I found Tomas' original message... it was in my Spam
>> folder....
>> >>
>> >> I also tried re-running the "./ejbca.sh ca importcrl"
>command
>> a bunch of
>> >> ways. I think it is running without outputting any errors,
>> but still,
>> >> no rows are appearing with that issuerDN at all.
>> >>
>> >>
>> >>
>> >>
>> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> OCSP responses does not have anything to do with the CRL
>> itself. Once
>> >> the CRL has been imported each certificate status is in its
>> own record
>> >> in the CertificateData table. This is where status is read
>> when reponses
>> >> are created.
>> >> If there are issues, it is in that case related to the
>import
>> of the CRL.
>> >>
>> >> Verify that certificate records are created in
>> CertificateData when
>> >> importing the CRL.
>> >> Also, a debug log when making OCSP request will tell exact
>> details of
>> >> what the responder finds in the database or not.
>> >>
>> >> Regards,
>> >> Tomas
>> >>
>> >>
>> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via
>Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >> [Re-sending, as this (handling of large CRLs) is still a
>> problem and
>> >> is still not working.]
>> >>
>> >>
>> >>
>> >> On Monday, July 1, 2019, 12:47:23 PM UTC,
><oh...@ya...
>> <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I think that I've found how to set/change the
>> max_allowed_packet on
>> >> the client (i.e., EJBCA) side, by editing the Mysql URL
>in the
>> >> standalone/configuration/standalone.xml to:
>> >>
>> >>
>> >
>>
><connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>> >>
>> >> However, when I test the OCSP Responder with a serial
>> number (and CA
>> >> cert) from the CA with the large (>71MB) CRL file, I am
>still
>> >> getting "unknown" and EJBCA OCSP Responder is still NOT
>> FINDING the
>> >> cert serial number.
>> >>
>> >> NOTE, again, that I can test with certs (and CA cert)
>that
>> have
>> >> smaller CRL files, and those tests work.
>> >>
>> >> So it is only when testing with the large CRL file that
>is
>> failing
>> >> with the OCSP Responder.
>> >>
>> >> This seems like a potential BUG for EJBCA OCSP
>Responder?
>> >>
>> >> Please advise.
>> >>
>> >> Thanks,
>> >> Jim
>> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> BTW, just to check, I have actually gone through the
>> process of
>> >> getting the crlData from the MySQL using mysql client (I
>> had to add
>> >> the --max_allowed-packet=100M to the mysql command) and
>I
>> was able
>> >> to actually get the PEM from the Mysql database that
>way.
>> >>
>> >> So, I KNOW that the entire CRL is in the Mysql database,
>> and yet
>> >> when I run an "openssl ocsp" test against the EJBCA OCSP
>> Responder,
>> >> it says it cannot find the revocation.
>> >>
>> >> OCSP Request Data:
>> >> Version: 1 (0x0)
>> >> Requestor List:
>> >> Certificate ID:
>> >> Hash Algorithm: sha1
>> >> Issuer Name Hash:
>37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> >> Issuer Key Hash:
>3929A80864E28F83AA816147119133F85A37A6E4
>> >> Serial Number: 2FFFF4
>> >> Request Extensions:
>> >> OCSP Nonce:
>> >> 04103054A60A55E389B239A55228033D8F62
>> >> Response verify OK
>> >> 0x2FFFF4: unknown
>> >> This Update: Jun 29 19:31:02 2019 GMT
>> >>
>> >>
>> >> In the server log:
>> >>
>> >> 2019-06-29 15:31:02,150 DEBUG
>> >>
>>
>[org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> >> (default task-1) Unable to find revocation information
>for
>> >> certificate with serial '2ffff4' from issuer
>'CN=XXX....'
>> >>
>> >>
>> >> Honestly, I am no 100% sure that the problem is that
>> >> max_allowe_packet is too small on the mysql client side,
>> but that is
>> >> my best guess.
>> >>
>> >> What I AM sure is that the CRL is in mysql, but when I
>do
>> a test
>> >> with OCSP Responder, it cannot find the cert serial
>number
>> in the CRL.
>> >>
>> >> Jim
>> >>
>> >>
>> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi Tomas,
>> >>
>> >> I already had the max_allowed_packet configured on the
>Mysql
>> >> server-side (in /etc/my.cnf).
>> >>
>> >> So as I said, when testing with mysql (client) I had to
>> include
>> >> "--max_allowed_packet=100M" on the mysql (client)
>command
>> line, e.g.:
>> >>
>> >> mysql -u root -p --max_allowed_packet=100M
>> >>
>> >> If I did NOT include the "--max_allowed_packet=100M" on
>> that command
>> >> line, then when I did a "select" on the field base64Crl,
>I
>> would get
>> >> an error.
>> >>
>> >> I suspect that EJBCA is acting as a Mysql client, so, in
>> order to
>> >> work with such large CRLs, there needs to be some way to
>> set the
>> >> equivalent of the "--max_allowed_packet=100M" on the
>EJBCA
>> side.
>> >>
>> >> How can I do that?
>> >>
>> >> If that can't be done, then I think that when EJBCA
>tries to
>> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
>> same error.
>> >>
>> >> Thanks,
>> >> Jim
>> >>
>> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas
>Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >>
>> >> You configure max allowed packet size in you MySQL
>> configuration on the
>> >> MySQL server.
>> >>
>> >> /Tomas
>> >>
>> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> >> > I am not 100% sure, but I think I know what might be
>> causing this
>> >> problem.
>> >> >
>> >> > I did a test, where I just tried to log into mysql and
>> do the
>> >> following
>> >> > select:
>> >> >
>> >> > select base64Crl INTO OUTFILE
>> '/var/lib/mysql-files/xx.out' from
>> >> CRLData
>> >> > where cRLNumber=3167;
>> >> >
>> >> > and I got an error saying that the result was more
>> >> than max_allowed_packet.
>> >> >
>> >> > So then I did the same thing, but for the mysql
>command
>> I used:
>> >> >
>> >> > mysql -u root -p --max_allowed_packet=100M
>> >> >
>> >> > and then when I did that same select, I was able to
>> output the entire
>> >> > CRL as a PEM.
>> >> >
>> >> > So, I am theorizing that that is what is happening
>with
>> EJBCA, i.e.,
>> >> > when I run the "openssl ocsp" command, EJBCA is
>querying the
>> > base64CRL
>> >> > but the result is too large and so that is causing the
>> query to fail.
>> >> >
>> >> > So the question now is: How can I cause EJBCA OCSP
>> Responder to run
>> >> > with a larger max_allowed_packet so that the query
>> succeeds (and then
>> >> > hopefully EJBCA OCSP Responder will be able to check
>the
>> revocation
>> >> > properly)?
>> >> >
>> >> > Does anyone know how that can be done?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
>> > <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>
>> >> <mailto:oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
>> >> >
>> >> >
>> >> > Hi,
>> >> >
>> >> > As discussed in an earlier thread, it looks like I was
>> able to
>> >> import a
>> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
>> Responder), so now,
>> >> I am
>> >> > doing testing, using "openssl ocsp" to send requests
>> into the
>> >> EJBCA OCSP
>> >> > Responder:
>> >> >
>> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> >> -req_text -serial
>> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >> >
>> >> > But it looks like no matter what serial number I use,
>> whether the
>> >> serial
>> >> > number is in the CRL or not, returns a response like:
>> >> >
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash: 37...00
>> >> > Issuer Key Hash: 39...E4
>> >> > Serial Number: 2FFFF9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 041078B1311984A110B5E19C8DC4895F485B
>> >> > Response verify OK
>> >> > 0x2FFFF9: unknown
>> >> > This Update: Jun 28 18:56:20 2019 GMT
>> >> >
>> >> > i.e., it looks like the EJBCA OCSP Responder is not
>able
>> to actually
>> >> > find any revoked certs in the CRL?
>> >> >
>> >> >
>> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is
>functioning
>> >> properly, in
>> >> > general, because I can do the same test against a
>> smaller CRL in the
>> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
>> get an actual
>> >> > "Revoked" response:
>> >> >
>> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
>> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
>> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
>> openssl ocsp
>> >> -issuer simpleca.crt
>> >> > -VAfile simplecabinding2.crt -serial
>> 0x008486394c03e1f5d9 -req_text
>> >> > -host 192.168.0.28:80
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash:
>> 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> >> > Issuer Key Hash:
>> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> >> > Serial Number: 8486394C03E1F5D9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> >> > Response verify OK
>> >> > 0x008486394c03e1f5d9: revoked
>> >> > This Update: Jun 28 19:12:48 2019 GMT
>> >> > Reason: unspecified
>> >> > Revocation Time: May 26 12:30:44 2019 GMT
>> >> >
>> >> > So it seems like the EJBCA OCSP Responder is either
>not
>> > processing the
>> >> > large CRL properly, or maybe the import didn't import
>> the large
>> >> CRL into
>> >> > the OCSP Responder correctly.
>> >> >
>> >> > Can anyone tell me how do I diagnose this problem?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Ejbca-develop mailing list
>> >> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> >
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> >
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >>
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>_______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
|
|
From: <oh...@ya...> - 2019-07-17 14:05:29
|
Hi,
This is an update on the import run I started yesterday evening:
I am still monitoring the import run that I started yesterday. Right now it has processed ~141157 entries, and it has been running about 16 hours, so the processing rate right now is about 141157/16 or ~8822 entries/hours vs. ~17836 entries/hour when the import run first started, so the import processing is slowing down :(!
Jim
On Wednesday, July 17, 2019, 1:52:53 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi Tomas,
I'm sorry that I missed the message below, but I am not clear what you are asking me to check/monitor? Do you mean like running something like "top" command?
Thanks,Jim
On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
To verify that the changes actually happen...
Do you monitor CPU usage to see what processes use CPU and memory?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
The import has been running for about an hour now and has imported 17836 entries.
Per what I had said previously, this CRL file has 1002255entries, that means it would take about 56+ hours to import this CRL.
So, this import processing is still way too slow.
I don't know what is involved, or even how much it would improve things, but I think that the next thing to try is to eliminated both the check for the serial number in the DB AND the dummy cert creation.
Jim
On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-17 13:53:09
|
Hi Tomas,
I'm sorry that I missed the message below, but I am not clear what you are asking me to check/monitor? Do you mean like running something like "top" command?
Thanks,Jim
On Tuesday, July 16, 2019, 7:43:45 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
To verify that the changes actually happen...
Do you monitor CPU usage to see what processes use CPU and memory?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
The import has been running for about an hour now and has imported 17836 entries.
Per what I had said previously, this CRL file has 1002255entries, that means it would take about 56+ hours to import this CRL.
So, this import processing is still way too slow.
I don't know what is involved, or even how much it would improve things, but I think that the next thing to try is to eliminated both the check for the serial number in the DB AND the dummy cert creation.
Jim
On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-16 19:43:51
|
To verify that the changes actually happen...
Do you monitor CPU usage to see what processes use CPU and memory?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
> Hi,
>The import has been running for about an hour now and has imported
>17836 entries.
>Per what I had said previously, this CRL file has 1002255entries, that
>means it would take about 56+ hours to import this CRL.
>So, this import processing is still way too slow.
>I don't know what is involved, or even how much it would improve
>things, but I think that the next thing to try is to eliminated both
>the check for the serial number in the DB AND the dummy cert creation.
>Jim
>
>On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
>
> Hi,
>Ok, I rebuilt the JAR using your Java class, and it is running now, and
>creating rows in the CertificateData, but it still SEEMS kind of slow.
>
>
>I am running now and will try to get a measurement of revoked
>entries/time and will post back.
>Jim
>
>
>
>On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya...
><oh...@ya...> wrote:
>
> Hi,
>So I just tried what I said (emptying the "if") and you are right, it
>doesn't create the rows in the CeritificateData table :(...
>Ok, I will try your version...
>Jim
>
>On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
>
>Hi,
>
>I don't think so, no quite actually. In order to use the API to
>populate
>the database you need the certificate object it seems. Something like
>the attached file should work, it still creates a dummy certificate,
>but
>uses a static hard coded key, so no key generation.
>
>The file assumes you are on 6.15.x.
>
>Regards,
>Tomas
>
>On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
>> For testing (only...), I am thinking that if I modify this If clause
>in
>> CaImportCrlCommand.java:
>>
>> if (adaptive)
>> {
>> Date time = new Date();
>> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>>
>> SubjectPublicKeyInfo pkinfo =
>> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
>> X500Name dnName = new X500Name("CN=Dummy Missing in
>> Imported CRL, serialNumber=" + serialHex);
>> Date notAfter = new Date(time.getTime() +
>315360000000L);
>> X509v3CertificateBuilder certbuilder = new
>> X509v3CertificateBuilder(X500Name.getInstance(cacert
>> .getSubjectX500Principal().getEncoded()), serialNr,
>> time, notAfter, dnName, pkinfo);
>> ContentSigner signer = new BufferingContentSigner(new
>>
>JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
>> .getPrivate()), 20480);
>> X509CertificateHolder certHolder =
>certbuilder.build(signer);
>> X509Certificate certificate =
>>
>(X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
>> X509Certificate.class);
>>
>> String fingerprint =
>> CertTools.getFingerprintAsString(certificate);
>>
>> EndEntityInformation missingUserEndEntityInformation =
>>
>((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
>> missing_user_name);
>> if (missingUserEndEntityInformation == null)
>> {
>> log.debug("Loading/updating user " +
>missing_user_name);
>>
>> EndEntityInformation userdataNew = new
>> EndEntityInformation(missing_user_name,
>> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null,
>10,
>> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0,
>null);
>>
>> userdataNew.setPassword("foo123");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
>> userdataNew, false);
>>
>> log.info("User '" + missing_user_name + "' has been
>> added.");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
>> missing_user_name, 50);
>>
>> log.info("User '" + missing_user_name + "' has been
>> updated.");
>> }
>>
>>
>((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>>
>> EJBTools.wrap(certificate), missing_user_name,
>> fingerprint, 20, 1, 1, 0, null, new Date()
>>
>> .getTime());
>> log.info("Dummy certificate '" + serialHex + "' has
>been
>> stored.");
>> }
>>
>> to (for example):
>>
>> if (adaptive)
>> {
>>
>> log.info("*** SKIPPING *** Dummy certificate '" +
>> serialHex + "' has been stored.");
>> }
>>
>>
>> Would THAT work??
>>
>> Thanks,
>> Jim
>>
>>
>>
>>
>> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> Hi Tomas (or anyone else),
>>
>> Sorry - I was typing my last msg on phone, and it wasn't as detailed
>or
>> clear as I wanted.
>>
>> Basically what I was saying is I was thinking that, as a quick sanity
>> test, if there was one or two places where the code in
>> the CaImportCRLCommand.java was maybe making a method call to create
>the
>> dummy cert and put it into the database, then maybe I could decompile
>> and modify the CaImportCRLCommand.java, and comment out those method
>> calls, then, re-do my large CRL import test and time it?
>>
>> If so, can you let me know specifically where those calls are, and if
>I
>> just comment them out, would that work?
>>
>> Ironically, that (not doing the dummy cert creation and writing) was
>> what I *thought* the "LENIENT" option was doing).
>>
>> Thanks,
>> Jim
>>
>> P. S. I have time today to try this if I figure out what needs to be
>done.
>>
>>
>>
>> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>>
>> Yes, we are definitely interested/willing to test patches. I
>actually
>> was wondering if it would be a quick test to mod the code somewhere
>to
>> avoid a call that creates the dummy cert, to get an idea if that
>would
>> reduce the import time enough??
>>
>>
>> Sent from Yahoo Mail on Android
>>
><https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>>
>> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
>> <to...@pr...> wrote:
>> Most large users, some very very large, typically uses on line
>> updates directly from the CA. Many CAs today don't accept only
>> importing CRLs, as they need to keep track of issued certificates
>as
>> well not only revoked (can't use "non existing is good").
>> Having said thst, it's interesting to be able to import large
>CRLs,
>> so improving it is of interest.
>>
>> Importing CRLs would actually not have to generate a dummy cert at
>> all, since a certificate object is not needed on the responder,
>> which should make it very fast.
>>
>> If you are willing to test patches, we can work with you on
>improving.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>> Hi,
>>
>> I am curious whether you have other users that import large
>CRLs
>> like this?
>>
>> Are we the only ones who would encounter these really long
>> import times, because the other users who might have large
>CRLs
>> use the EJBCA CA also, so they don't run into having the dummy
>> certs being created for every entry in the CRL?
>>
>> Jim
>>
>> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
>> <to...@pr...> wrote:
>>
>>
>>
>> I see...
>>
>> I can see that a new key pair is being generated for each
>dummy
>> cert.
>>
>> In CaImportCRLCommand.java:
>>
>> final KeyPair key_pair = KeyTools.genKeys("2048",
>> AlgorithmConstants.KEYALGORITHM_RSA);
>>
>> Chaning that to 1024 or 512 would be an easy improvement. A
>bit more
>> complex, but even larger speedup would be to use a "hard
>coded"
>> dummy
>> key, i.e. same key for all dummy certs.
>>
>> You can experiment by changing 2048 to 512 (not secure but it
>> doesn't
>> have to be for this dummy cert).
>>
>> Regards,
>> Tomas
>>
>> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
>> > FYI, the is running, but it is REALLY slow. The CRL file
>has
>> 1002255
>> > entries in it, and the import is only up to 8899 now, after
>> about 40
>> > minutes... which would take about 75 hours!!
>> >
>> > In our production systems, we are required to update some
>CRLs
>> every
>> > hour, so that would be a problem.
>> >
>> > What can I do to speed that up??
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > AHHH... Ok, it is running now, and from checking the
>> CertificateData in
>> > the Mysql database, I can see that it is now populating the
>> > CertificateData!!
>> >
>> > Thanks!
>> >
>> > Jim
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > I wasn't sure if we should use ADAPTIVE, since we are not
>> using the CA part.
>> >
>> >
>> > I will try with "-o ADAPTIVE" now.
>> >
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
>> > <to...@pr... <mailto:to...@pr...>> wrote:
>> >
>> >
>> >
>> > It should be ADAPTIVE.
>> >
>> > The documentation says:
>> > bin/ejbca.sh ca importcrl --help
>> > ---
>> > ADAPTIVE means that missing certficates will be replaced by
>dummy
>> > certificates to cater for proper CRLs for missing
>certificates.
>> > ---
>> >
>> > Regards,
>> > Tomas
>> >
>> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> >> Hi,
>> >>
>> >> I've been looking at the code in CaImportCRLCommand, and I
>am
>> wondering
>> >> if, when the command is run in LENIENT mode and it doesn't
>> find the cert
>> >> in the database (and outputs the "missing in database"
>error
>> message),
>> >> what is supposed to happen?
>> >>
>> >> Mind you, the only error messages I am seeing are the
>"missing in
>> >> database" error messages. I don't see any of the other
>error
>> messages
>> >> that are in/from CaImportCRLCommand.java.
>> >>
>> >> I am wondering if, when the command is run with "-o
>LENIENT",
>> and when
>> >> it outputs the "missing in database" error message, then
>the
>> ejbca.sh is
>> >> basically not doing anything (and not storing anything into
>the
>> >> CertificateData table)?
>> >>
>> >> From the lack of any other error messages (vs. what I see
>in the
>> >> CaImportCRLCommand.java code), that is what it looks like
>is
>> happening.
>> >>
>> >> So given I am using EJBCA for OCSP responder only, is
>running the
>> >> ejbca.sh in the "-o LENIENT" mode correct?
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I've been doing a bunch of testing with trying to import
>the
>> large CRL,
>> >> and wanted to summarize:
>> >>
>> >> - The ejbca.sh runs fine - it completes without outputting
>an
>> errors,
>> >> except for outputting a bunch of the "cert xxxx missing in
>> database"
>> >> messages.
>> >>
>> >> - When the import completes, and I check the Mysql
>database,
>> the crlData
>> >> (the full PEM of the CRL) is in the database, but there is
>no
>> rows in
>> >> the CertificateData table for that CA at all.
>> >>
>> >> - I even figured out how to get the ejbca.sh to log using
>> >> log4j.properties, and it output a bunch of logging, mainly
>> the missing
>> >> in database errors, but other than that I didn't see any
>> obvious errors.
>> >>
>> >>
>> >> So, bottom line is that even though the ejbca.sh seems to
>run
>> without
>> >> errors (other than the missing in database errors), the
>import is
>> >> causing the CRL PEM to be stored in the database, but the
>> >> CertificateData table is not being populated at all for
>that CA.
>> >>
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> Ok, I found Tomas' original message... it was in my Spam
>> folder....
>> >>
>> >> I also tried re-running the "./ejbca.sh ca importcrl"
>command
>> a bunch of
>> >> ways. I think it is running without outputting any errors,
>> but still,
>> >> no rows are appearing with that issuerDN at all.
>> >>
>> >>
>> >>
>> >>
>> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> OCSP responses does not have anything to do with the CRL
>> itself. Once
>> >> the CRL has been imported each certificate status is in its
>> own record
>> >> in the CertificateData table. This is where status is read
>> when reponses
>> >> are created.
>> >> If there are issues, it is in that case related to the
>import
>> of the CRL.
>> >>
>> >> Verify that certificate records are created in
>> CertificateData when
>> >> importing the CRL.
>> >> Also, a debug log when making OCSP request will tell exact
>> details of
>> >> what the responder finds in the database or not.
>> >>
>> >> Regards,
>> >> Tomas
>> >>
>> >>
>> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via
>Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >> [Re-sending, as this (handling of large CRLs) is still a
>> problem and
>> >> is still not working.]
>> >>
>> >>
>> >>
>> >> On Monday, July 1, 2019, 12:47:23 PM UTC,
><oh...@ya...
>> <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I think that I've found how to set/change the
>> max_allowed_packet on
>> >> the client (i.e., EJBCA) side, by editing the Mysql URL
>in the
>> >> standalone/configuration/standalone.xml to:
>> >>
>> >>
>> >
>>
><connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>> >>
>> >> However, when I test the OCSP Responder with a serial
>> number (and CA
>> >> cert) from the CA with the large (>71MB) CRL file, I am
>still
>> >> getting "unknown" and EJBCA OCSP Responder is still NOT
>> FINDING the
>> >> cert serial number.
>> >>
>> >> NOTE, again, that I can test with certs (and CA cert)
>that
>> have
>> >> smaller CRL files, and those tests work.
>> >>
>> >> So it is only when testing with the large CRL file that
>is
>> failing
>> >> with the OCSP Responder.
>> >>
>> >> This seems like a potential BUG for EJBCA OCSP
>Responder?
>> >>
>> >> Please advise.
>> >>
>> >> Thanks,
>> >> Jim
>> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> BTW, just to check, I have actually gone through the
>> process of
>> >> getting the crlData from the MySQL using mysql client (I
>> had to add
>> >> the --max_allowed-packet=100M to the mysql command) and
>I
>> was able
>> >> to actually get the PEM from the Mysql database that
>way.
>> >>
>> >> So, I KNOW that the entire CRL is in the Mysql database,
>> and yet
>> >> when I run an "openssl ocsp" test against the EJBCA OCSP
>> Responder,
>> >> it says it cannot find the revocation.
>> >>
>> >> OCSP Request Data:
>> >> Version: 1 (0x0)
>> >> Requestor List:
>> >> Certificate ID:
>> >> Hash Algorithm: sha1
>> >> Issuer Name Hash:
>37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> >> Issuer Key Hash:
>3929A80864E28F83AA816147119133F85A37A6E4
>> >> Serial Number: 2FFFF4
>> >> Request Extensions:
>> >> OCSP Nonce:
>> >> 04103054A60A55E389B239A55228033D8F62
>> >> Response verify OK
>> >> 0x2FFFF4: unknown
>> >> This Update: Jun 29 19:31:02 2019 GMT
>> >>
>> >>
>> >> In the server log:
>> >>
>> >> 2019-06-29 15:31:02,150 DEBUG
>> >>
>>
>[org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> >> (default task-1) Unable to find revocation information
>for
>> >> certificate with serial '2ffff4' from issuer
>'CN=XXX....'
>> >>
>> >>
>> >> Honestly, I am no 100% sure that the problem is that
>> >> max_allowe_packet is too small on the mysql client side,
>> but that is
>> >> my best guess.
>> >>
>> >> What I AM sure is that the CRL is in mysql, but when I
>do
>> a test
>> >> with OCSP Responder, it cannot find the cert serial
>number
>> in the CRL.
>> >>
>> >> Jim
>> >>
>> >>
>> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi Tomas,
>> >>
>> >> I already had the max_allowed_packet configured on the
>Mysql
>> >> server-side (in /etc/my.cnf).
>> >>
>> >> So as I said, when testing with mysql (client) I had to
>> include
>> >> "--max_allowed_packet=100M" on the mysql (client)
>command
>> line, e.g.:
>> >>
>> >> mysql -u root -p --max_allowed_packet=100M
>> >>
>> >> If I did NOT include the "--max_allowed_packet=100M" on
>> that command
>> >> line, then when I did a "select" on the field base64Crl,
>I
>> would get
>> >> an error.
>> >>
>> >> I suspect that EJBCA is acting as a Mysql client, so, in
>> order to
>> >> work with such large CRLs, there needs to be some way to
>> set the
>> >> equivalent of the "--max_allowed_packet=100M" on the
>EJBCA
>> side.
>> >>
>> >> How can I do that?
>> >>
>> >> If that can't be done, then I think that when EJBCA
>tries to
>> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
>> same error.
>> >>
>> >> Thanks,
>> >> Jim
>> >>
>> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas
>Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >>
>> >> You configure max allowed packet size in you MySQL
>> configuration on the
>> >> MySQL server.
>> >>
>> >> /Tomas
>> >>
>> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> >> > I am not 100% sure, but I think I know what might be
>> causing this
>> >> problem.
>> >> >
>> >> > I did a test, where I just tried to log into mysql and
>> do the
>> >> following
>> >> > select:
>> >> >
>> >> > select base64Crl INTO OUTFILE
>> '/var/lib/mysql-files/xx.out' from
>> >> CRLData
>> >> > where cRLNumber=3167;
>> >> >
>> >> > and I got an error saying that the result was more
>> >> than max_allowed_packet.
>> >> >
>> >> > So then I did the same thing, but for the mysql
>command
>> I used:
>> >> >
>> >> > mysql -u root -p --max_allowed_packet=100M
>> >> >
>> >> > and then when I did that same select, I was able to
>> output the entire
>> >> > CRL as a PEM.
>> >> >
>> >> > So, I am theorizing that that is what is happening
>with
>> EJBCA, i.e.,
>> >> > when I run the "openssl ocsp" command, EJBCA is
>querying the
>> > base64CRL
>> >> > but the result is too large and so that is causing the
>> query to fail.
>> >> >
>> >> > So the question now is: How can I cause EJBCA OCSP
>> Responder to run
>> >> > with a larger max_allowed_packet so that the query
>> succeeds (and then
>> >> > hopefully EJBCA OCSP Responder will be able to check
>the
>> revocation
>> >> > properly)?
>> >> >
>> >> > Does anyone know how that can be done?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
>> > <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>
>> >> <mailto:oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
>> >> >
>> >> >
>> >> > Hi,
>> >> >
>> >> > As discussed in an earlier thread, it looks like I was
>> able to
>> >> import a
>> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
>> Responder), so now,
>> >> I am
>> >> > doing testing, using "openssl ocsp" to send requests
>> into the
>> >> EJBCA OCSP
>> >> > Responder:
>> >> >
>> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> >> -req_text -serial
>> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >> >
>> >> > But it looks like no matter what serial number I use,
>> whether the
>> >> serial
>> >> > number is in the CRL or not, returns a response like:
>> >> >
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash: 37...00
>> >> > Issuer Key Hash: 39...E4
>> >> > Serial Number: 2FFFF9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 041078B1311984A110B5E19C8DC4895F485B
>> >> > Response verify OK
>> >> > 0x2FFFF9: unknown
>> >> > This Update: Jun 28 18:56:20 2019 GMT
>> >> >
>> >> > i.e., it looks like the EJBCA OCSP Responder is not
>able
>> to actually
>> >> > find any revoked certs in the CRL?
>> >> >
>> >> >
>> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is
>functioning
>> >> properly, in
>> >> > general, because I can do the same test against a
>> smaller CRL in the
>> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
>> get an actual
>> >> > "Revoked" response:
>> >> >
>> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
>> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
>> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
>> openssl ocsp
>> >> -issuer simpleca.crt
>> >> > -VAfile simplecabinding2.crt -serial
>> 0x008486394c03e1f5d9 -req_text
>> >> > -host 192.168.0.28:80
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash:
>> 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> >> > Issuer Key Hash:
>> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> >> > Serial Number: 8486394C03E1F5D9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> >> > Response verify OK
>> >> > 0x008486394c03e1f5d9: revoked
>> >> > This Update: Jun 28 19:12:48 2019 GMT
>> >> > Reason: unspecified
>> >> > Revocation Time: May 26 12:30:44 2019 GMT
>> >> >
>> >> > So it seems like the EJBCA OCSP Responder is either
>not
>> > processing the
>> >> > large CRL properly, or maybe the import didn't import
>> the large
>> >> CRL into
>> >> > the OCSP Responder correctly.
>> >> >
>> >> > Can anyone tell me how do I diagnose this problem?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Ejbca-develop mailing list
>> >> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> >
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> >
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >>
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>_______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
|
From: Tomas G. <to...@pr...> - 2019-07-16 19:42:59
|
Is there a possibility that you can offer this CRL for download by us?
On July 16, 2019 8:52:37 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
> Hi,
>The import has been running for about an hour now and has imported
>17836 entries.
>Per what I had said previously, this CRL file has 1002255entries, that
>means it would take about 56+ hours to import this CRL.
>So, this import processing is still way too slow.
>I don't know what is involved, or even how much it would improve
>things, but I think that the next thing to try is to eliminated both
>the check for the serial number in the DB AND the dummy cert creation.
>Jim
>
>On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop
><ejb...@li...> wrote:
>
> Hi,
>Ok, I rebuilt the JAR using your Java class, and it is running now, and
>creating rows in the CertificateData, but it still SEEMS kind of slow.
>
>
>I am running now and will try to get a measurement of revoked
>entries/time and will post back.
>Jim
>
>
>
>On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya...
><oh...@ya...> wrote:
>
> Hi,
>So I just tried what I said (emptying the "if") and you are right, it
>doesn't create the rows in the CeritificateData table :(...
>Ok, I will try your version...
>Jim
>
>On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
>
>Hi,
>
>I don't think so, no quite actually. In order to use the API to
>populate
>the database you need the certificate object it seems. Something like
>the attached file should work, it still creates a dummy certificate,
>but
>uses a static hard coded key, so no key generation.
>
>The file assumes you are on 6.15.x.
>
>Regards,
>Tomas
>
>On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
>> For testing (only...), I am thinking that if I modify this If clause
>in
>> CaImportCrlCommand.java:
>>
>> if (adaptive)
>> {
>> Date time = new Date();
>> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>>
>> SubjectPublicKeyInfo pkinfo =
>> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
>> X500Name dnName = new X500Name("CN=Dummy Missing in
>> Imported CRL, serialNumber=" + serialHex);
>> Date notAfter = new Date(time.getTime() +
>315360000000L);
>> X509v3CertificateBuilder certbuilder = new
>> X509v3CertificateBuilder(X500Name.getInstance(cacert
>> .getSubjectX500Principal().getEncoded()), serialNr,
>> time, notAfter, dnName, pkinfo);
>> ContentSigner signer = new BufferingContentSigner(new
>>
>JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
>> .getPrivate()), 20480);
>> X509CertificateHolder certHolder =
>certbuilder.build(signer);
>> X509Certificate certificate =
>>
>(X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
>> X509Certificate.class);
>>
>> String fingerprint =
>> CertTools.getFingerprintAsString(certificate);
>>
>> EndEntityInformation missingUserEndEntityInformation =
>>
>((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
>> missing_user_name);
>> if (missingUserEndEntityInformation == null)
>> {
>> log.debug("Loading/updating user " +
>missing_user_name);
>>
>> EndEntityInformation userdataNew = new
>> EndEntityInformation(missing_user_name,
>> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null,
>10,
>> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0,
>null);
>>
>> userdataNew.setPassword("foo123");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
>> userdataNew, false);
>>
>> log.info("User '" + missing_user_name + "' has been
>> added.");
>>
>>
>((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
>> missing_user_name, 50);
>>
>> log.info("User '" + missing_user_name + "' has been
>> updated.");
>> }
>>
>>
>((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>>
>> EJBTools.wrap(certificate), missing_user_name,
>> fingerprint, 20, 1, 1, 0, null, new Date()
>>
>> .getTime());
>> log.info("Dummy certificate '" + serialHex + "' has
>been
>> stored.");
>> }
>>
>> to (for example):
>>
>> if (adaptive)
>> {
>>
>> log.info("*** SKIPPING *** Dummy certificate '" +
>> serialHex + "' has been stored.");
>> }
>>
>>
>> Would THAT work??
>>
>> Thanks,
>> Jim
>>
>>
>>
>>
>> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> Hi Tomas (or anyone else),
>>
>> Sorry - I was typing my last msg on phone, and it wasn't as detailed
>or
>> clear as I wanted.
>>
>> Basically what I was saying is I was thinking that, as a quick sanity
>> test, if there was one or two places where the code in
>> the CaImportCRLCommand.java was maybe making a method call to create
>the
>> dummy cert and put it into the database, then maybe I could decompile
>> and modify the CaImportCRLCommand.java, and comment out those method
>> calls, then, re-do my large CRL import test and time it?
>>
>> If so, can you let me know specifically where those calls are, and if
>I
>> just comment them out, would that work?
>>
>> Ironically, that (not doing the dummy cert creation and writing) was
>> what I *thought* the "LENIENT" option was doing).
>>
>> Thanks,
>> Jim
>>
>> P. S. I have time today to try this if I figure out what needs to be
>done.
>>
>>
>>
>> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>>
>> Yes, we are definitely interested/willing to test patches. I
>actually
>> was wondering if it would be a quick test to mod the code somewhere
>to
>> avoid a call that creates the dummy cert, to get an idea if that
>would
>> reduce the import time enough??
>>
>>
>> Sent from Yahoo Mail on Android
>>
><https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>>
>> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
>> <to...@pr...> wrote:
>> Most large users, some very very large, typically uses on line
>> updates directly from the CA. Many CAs today don't accept only
>> importing CRLs, as they need to keep track of issued certificates
>as
>> well not only revoked (can't use "non existing is good").
>> Having said thst, it's interesting to be able to import large
>CRLs,
>> so improving it is of interest.
>>
>> Importing CRLs would actually not have to generate a dummy cert at
>> all, since a certificate object is not needed on the responder,
>> which should make it very fast.
>>
>> If you are willing to test patches, we can work with you on
>improving.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...> wrote:
>>
>> Hi,
>>
>> I am curious whether you have other users that import large
>CRLs
>> like this?
>>
>> Are we the only ones who would encounter these really long
>> import times, because the other users who might have large
>CRLs
>> use the EJBCA CA also, so they don't run into having the dummy
>> certs being created for every entry in the CRL?
>>
>> Jim
>>
>> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
>> <to...@pr...> wrote:
>>
>>
>>
>> I see...
>>
>> I can see that a new key pair is being generated for each
>dummy
>> cert.
>>
>> In CaImportCRLCommand.java:
>>
>> final KeyPair key_pair = KeyTools.genKeys("2048",
>> AlgorithmConstants.KEYALGORITHM_RSA);
>>
>> Chaning that to 1024 or 512 would be an easy improvement. A
>bit more
>> complex, but even larger speedup would be to use a "hard
>coded"
>> dummy
>> key, i.e. same key for all dummy certs.
>>
>> You can experiment by changing 2048 to 512 (not secure but it
>> doesn't
>> have to be for this dummy cert).
>>
>> Regards,
>> Tomas
>>
>> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
>> > FYI, the is running, but it is REALLY slow. The CRL file
>has
>> 1002255
>> > entries in it, and the import is only up to 8899 now, after
>> about 40
>> > minutes... which would take about 75 hours!!
>> >
>> > In our production systems, we are required to update some
>CRLs
>> every
>> > hour, so that would be a problem.
>> >
>> > What can I do to speed that up??
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > AHHH... Ok, it is running now, and from checking the
>> CertificateData in
>> > the Mysql database, I can see that it is now populating the
>> > CertificateData!!
>> >
>> > Thanks!
>> >
>> > Jim
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
>> Ejbca-develop
>> > <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>> >
>> >
>> > I wasn't sure if we should use ADAPTIVE, since we are not
>> using the CA part.
>> >
>> >
>> > I will try with "-o ADAPTIVE" now.
>> >
>> >
>> >
>> >
>> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
>> > <to...@pr... <mailto:to...@pr...>> wrote:
>> >
>> >
>> >
>> > It should be ADAPTIVE.
>> >
>> > The documentation says:
>> > bin/ejbca.sh ca importcrl --help
>> > ---
>> > ADAPTIVE means that missing certficates will be replaced by
>dummy
>> > certificates to cater for proper CRLs for missing
>certificates.
>> > ---
>> >
>> > Regards,
>> > Tomas
>> >
>> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> >> Hi,
>> >>
>> >> I've been looking at the code in CaImportCRLCommand, and I
>am
>> wondering
>> >> if, when the command is run in LENIENT mode and it doesn't
>> find the cert
>> >> in the database (and outputs the "missing in database"
>error
>> message),
>> >> what is supposed to happen?
>> >>
>> >> Mind you, the only error messages I am seeing are the
>"missing in
>> >> database" error messages. I don't see any of the other
>error
>> messages
>> >> that are in/from CaImportCRLCommand.java.
>> >>
>> >> I am wondering if, when the command is run with "-o
>LENIENT",
>> and when
>> >> it outputs the "missing in database" error message, then
>the
>> ejbca.sh is
>> >> basically not doing anything (and not storing anything into
>the
>> >> CertificateData table)?
>> >>
>> >> From the lack of any other error messages (vs. what I see
>in the
>> >> CaImportCRLCommand.java code), that is what it looks like
>is
>> happening.
>> >>
>> >> So given I am using EJBCA for OCSP responder only, is
>running the
>> >> ejbca.sh in the "-o LENIENT" mode correct?
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I've been doing a bunch of testing with trying to import
>the
>> large CRL,
>> >> and wanted to summarize:
>> >>
>> >> - The ejbca.sh runs fine - it completes without outputting
>an
>> errors,
>> >> except for outputting a bunch of the "cert xxxx missing in
>> database"
>> >> messages.
>> >>
>> >> - When the import completes, and I check the Mysql
>database,
>> the crlData
>> >> (the full PEM of the CRL) is in the database, but there is
>no
>> rows in
>> >> the CertificateData table for that CA at all.
>> >>
>> >> - I even figured out how to get the ejbca.sh to log using
>> >> log4j.properties, and it output a bunch of logging, mainly
>> the missing
>> >> in database errors, but other than that I didn't see any
>> obvious errors.
>> >>
>> >>
>> >> So, bottom line is that even though the ejbca.sh seems to
>run
>> without
>> >> errors (other than the missing in database errors), the
>import is
>> >> causing the CRL PEM to be stored in the database, but the
>> >> CertificateData table is not being populated at all for
>that CA.
>> >>
>> >>
>> >> Jim
>> >>
>> >>
>> >>
>> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
>> Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> Ok, I found Tomas' original message... it was in my Spam
>> folder....
>> >>
>> >> I also tried re-running the "./ejbca.sh ca importcrl"
>command
>> a bunch of
>> >> ways. I think it is running without outputting any errors,
>> but still,
>> >> no rows are appearing with that issuerDN at all.
>> >>
>> >>
>> >>
>> >>
>> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> OCSP responses does not have anything to do with the CRL
>> itself. Once
>> >> the CRL has been imported each certificate status is in its
>> own record
>> >> in the CertificateData table. This is where status is read
>> when reponses
>> >> are created.
>> >> If there are issues, it is in that case related to the
>import
>> of the CRL.
>> >>
>> >> Verify that certificate records are created in
>> CertificateData when
>> >> importing the CRL.
>> >> Also, a debug log when making OCSP request will tell exact
>> details of
>> >> what the responder finds in the database or not.
>> >>
>> >> Regards,
>> >> Tomas
>> >>
>> >>
>> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via
>Ejbca-develop
>> >> <ejb...@li...
>> <mailto:ejb...@li...>
>> > <mailto:ejb...@li...
>> <mailto:ejb...@li...>>> wrote:
>> >>
>> >> [Re-sending, as this (handling of large CRLs) is still a
>> problem and
>> >> is still not working.]
>> >>
>> >>
>> >>
>> >> On Monday, July 1, 2019, 12:47:23 PM UTC,
><oh...@ya...
>> <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I think that I've found how to set/change the
>> max_allowed_packet on
>> >> the client (i.e., EJBCA) side, by editing the Mysql URL
>in the
>> >> standalone/configuration/standalone.xml to:
>> >>
>> >>
>> >
>>
><connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>> >>
>> >> However, when I test the OCSP Responder with a serial
>> number (and CA
>> >> cert) from the CA with the large (>71MB) CRL file, I am
>still
>> >> getting "unknown" and EJBCA OCSP Responder is still NOT
>> FINDING the
>> >> cert serial number.
>> >>
>> >> NOTE, again, that I can test with certs (and CA cert)
>that
>> have
>> >> smaller CRL files, and those tests work.
>> >>
>> >> So it is only when testing with the large CRL file that
>is
>> failing
>> >> with the OCSP Responder.
>> >>
>> >> This seems like a potential BUG for EJBCA OCSP
>Responder?
>> >>
>> >> Please advise.
>> >>
>> >> Thanks,
>> >> Jim
>> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi,
>> >>
>> >> BTW, just to check, I have actually gone through the
>> process of
>> >> getting the crlData from the MySQL using mysql client (I
>> had to add
>> >> the --max_allowed-packet=100M to the mysql command) and
>I
>> was able
>> >> to actually get the PEM from the Mysql database that
>way.
>> >>
>> >> So, I KNOW that the entire CRL is in the Mysql database,
>> and yet
>> >> when I run an "openssl ocsp" test against the EJBCA OCSP
>> Responder,
>> >> it says it cannot find the revocation.
>> >>
>> >> OCSP Request Data:
>> >> Version: 1 (0x0)
>> >> Requestor List:
>> >> Certificate ID:
>> >> Hash Algorithm: sha1
>> >> Issuer Name Hash:
>37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> >> Issuer Key Hash:
>3929A80864E28F83AA816147119133F85A37A6E4
>> >> Serial Number: 2FFFF4
>> >> Request Extensions:
>> >> OCSP Nonce:
>> >> 04103054A60A55E389B239A55228033D8F62
>> >> Response verify OK
>> >> 0x2FFFF4: unknown
>> >> This Update: Jun 29 19:31:02 2019 GMT
>> >>
>> >>
>> >> In the server log:
>> >>
>> >> 2019-06-29 15:31:02,150 DEBUG
>> >>
>>
>[org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> >> (default task-1) Unable to find revocation information
>for
>> >> certificate with serial '2ffff4' from issuer
>'CN=XXX....'
>> >>
>> >>
>> >> Honestly, I am no 100% sure that the problem is that
>> >> max_allowe_packet is too small on the mysql client side,
>> but that is
>> >> my best guess.
>> >>
>> >> What I AM sure is that the CRL is in mysql, but when I
>do
>> a test
>> >> with OCSP Responder, it cannot find the cert serial
>number
>> in the CRL.
>> >>
>> >> Jim
>> >>
>> >>
>> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
>> <oh...@ya... <mailto:oh...@ya...>
>> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >>
>> >>
>> >> Hi Tomas,
>> >>
>> >> I already had the max_allowed_packet configured on the
>Mysql
>> >> server-side (in /etc/my.cnf).
>> >>
>> >> So as I said, when testing with mysql (client) I had to
>> include
>> >> "--max_allowed_packet=100M" on the mysql (client)
>command
>> line, e.g.:
>> >>
>> >> mysql -u root -p --max_allowed_packet=100M
>> >>
>> >> If I did NOT include the "--max_allowed_packet=100M" on
>> that command
>> >> line, then when I did a "select" on the field base64Crl,
>I
>> would get
>> >> an error.
>> >>
>> >> I suspect that EJBCA is acting as a Mysql client, so, in
>> order to
>> >> work with such large CRLs, there needs to be some way to
>> set the
>> >> equivalent of the "--max_allowed_packet=100M" on the
>EJBCA
>> side.
>> >>
>> >> How can I do that?
>> >>
>> >> If that can't be done, then I think that when EJBCA
>tries to
>> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
>> same error.
>> >>
>> >> Thanks,
>> >> Jim
>> >>
>> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas
>Gustavsson
>> >> <to...@pr... <mailto:to...@pr...>
>> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
>> >>
>> >>
>> >>
>> >> You configure max allowed packet size in you MySQL
>> configuration on the
>> >> MySQL server.
>> >>
>> >> /Tomas
>> >>
>> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> >> > I am not 100% sure, but I think I know what might be
>> causing this
>> >> problem.
>> >> >
>> >> > I did a test, where I just tried to log into mysql and
>> do the
>> >> following
>> >> > select:
>> >> >
>> >> > select base64Crl INTO OUTFILE
>> '/var/lib/mysql-files/xx.out' from
>> >> CRLData
>> >> > where cRLNumber=3167;
>> >> >
>> >> > and I got an error saying that the result was more
>> >> than max_allowed_packet.
>> >> >
>> >> > So then I did the same thing, but for the mysql
>command
>> I used:
>> >> >
>> >> > mysql -u root -p --max_allowed_packet=100M
>> >> >
>> >> > and then when I did that same select, I was able to
>> output the entire
>> >> > CRL as a PEM.
>> >> >
>> >> > So, I am theorizing that that is what is happening
>with
>> EJBCA, i.e.,
>> >> > when I run the "openssl ocsp" command, EJBCA is
>querying the
>> > base64CRL
>> >> > but the result is too large and so that is causing the
>> query to fail.
>> >> >
>> >> > So the question now is: How can I cause EJBCA OCSP
>> Responder to run
>> >> > with a larger max_allowed_packet so that the query
>> succeeds (and then
>> >> > hopefully EJBCA OCSP Responder will be able to check
>the
>> revocation
>> >> > properly)?
>> >> >
>> >> > Does anyone know how that can be done?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
>> > <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>
>> >> <mailto:oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
>> >> >
>> >> >
>> >> > Hi,
>> >> >
>> >> > As discussed in an earlier thread, it looks like I was
>> able to
>> >> import a
>> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
>> Responder), so now,
>> >> I am
>> >> > doing testing, using "openssl ocsp" to send requests
>> into the
>> >> EJBCA OCSP
>> >> > Responder:
>> >> >
>> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> >> -req_text -serial
>> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >> >
>> >> > But it looks like no matter what serial number I use,
>> whether the
>> >> serial
>> >> > number is in the CRL or not, returns a response like:
>> >> >
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash: 37...00
>> >> > Issuer Key Hash: 39...E4
>> >> > Serial Number: 2FFFF9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 041078B1311984A110B5E19C8DC4895F485B
>> >> > Response verify OK
>> >> > 0x2FFFF9: unknown
>> >> > This Update: Jun 28 18:56:20 2019 GMT
>> >> >
>> >> > i.e., it looks like the EJBCA OCSP Responder is not
>able
>> to actually
>> >> > find any revoked certs in the CRL?
>> >> >
>> >> >
>> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is
>functioning
>> >> properly, in
>> >> > general, because I can do the same test against a
>> smaller CRL in the
>> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
>> get an actual
>> >> > "Revoked" response:
>> >> >
>> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
>> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
>> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
>> openssl ocsp
>> >> -issuer simpleca.crt
>> >> > -VAfile simplecabinding2.crt -serial
>> 0x008486394c03e1f5d9 -req_text
>> >> > -host 192.168.0.28:80
>> >> > OCSP Request Data:
>> >> > Version: 1 (0x0)
>> >> > Requestor List:
>> >> > Certificate ID:
>> >> > Hash Algorithm: sha1
>> >> > Issuer Name Hash:
>> 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> >> > Issuer Key Hash:
>> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> >> > Serial Number: 8486394C03E1F5D9
>> >> > Request Extensions:
>> >> > OCSP Nonce:
>> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> >> > Response verify OK
>> >> > 0x008486394c03e1f5d9: revoked
>> >> > This Update: Jun 28 19:12:48 2019 GMT
>> >> > Reason: unspecified
>> >> > Revocation Time: May 26 12:30:44 2019 GMT
>> >> >
>> >> > So it seems like the EJBCA OCSP Responder is either
>not
>> > processing the
>> >> > large CRL properly, or maybe the import didn't import
>> the large
>> >> CRL into
>> >> > the OCSP Responder correctly.
>> >> >
>> >> > Can anyone tell me how do I diagnose this problem?
>> >> >
>> >> > Thanks,
>> >> > Jim
>> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Ejbca-develop mailing list
>> >> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> >
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> >
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >>
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>>
>> >
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >>
>> >> _______________________________________________
>> >> Ejbca-develop mailing list
>> >> Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>_______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
|
From: <oh...@ya...> - 2019-07-16 18:52:51
|
Hi,
The import has been running for about an hour now and has imported 17836 entries.
Per what I had said previously, this CRL file has 1002255entries, that means it would take about 56+ hours to import this CRL.
So, this import processing is still way too slow.
I don't know what is involved, or even how much it would improve things, but I think that the next thing to try is to eliminated both the check for the serial number in the DB AND the dummy cert creation.
Jim
On Tuesday, July 16, 2019, 5:54:54 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-16 17:54:44
|
Hi,
Ok, I rebuilt the JAR using your Java class, and it is running now, and creating rows in the CertificateData, but it still SEEMS kind of slow.
I am running now and will try to get a measurement of revoked entries/time and will post back.
Jim
On Tuesday, July 16, 2019, 5:25:32 PM UTC, oh...@ya... <oh...@ya...> wrote:
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-16 17:25:52
|
Hi,
So I just tried what I said (emptying the "if") and you are right, it doesn't create the rows in the CeritificateData table :(...
Ok, I will try your version...
Jim
On Tuesday, July 16, 2019, 3:57:31 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-16 15:57:03
|
Hi,
I don't think so, no quite actually. In order to use the API to populate
the database you need the certificate object it seems. Something like
the attached file should work, it still creates a dummy certificate, but
uses a static hard coded key, so no key generation.
The file assumes you are on 6.15.x.
Regards,
Tomas
On 2019-07-16 15:02, ohaya--- via Ejbca-develop wrote:
> For testing (only...), I am thinking that if I modify this If clause in
> CaImportCrlCommand.java:
>
> if (adaptive)
> {
> Date time = new Date();
> KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
>
> SubjectPublicKeyInfo pkinfo =
> SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
> X500Name dnName = new X500Name("CN=Dummy Missing in
> Imported CRL, serialNumber=" + serialHex);
> Date notAfter = new Date(time.getTime() + 315360000000L);
> X509v3CertificateBuilder certbuilder = new
> X509v3CertificateBuilder(X500Name.getInstance(cacert
> .getSubjectX500Principal().getEncoded()), serialNr,
> time, notAfter, dnName, pkinfo);
> ContentSigner signer = new BufferingContentSigner(new
> JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
> .getPrivate()), 20480);
> X509CertificateHolder certHolder = certbuilder.build(signer);
> X509Certificate certificate =
> (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(),
> X509Certificate.class);
>
> String fingerprint =
> CertTools.getFingerprintAsString(certificate);
>
> EndEntityInformation missingUserEndEntityInformation =
> ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(),
> missing_user_name);
> if (missingUserEndEntityInformation == null)
> {
> log.debug("Loading/updating user " + missing_user_name);
>
> EndEntityInformation userdataNew = new
> EndEntityInformation(missing_user_name,
> CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10,
> new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
>
> userdataNew.setPassword("foo123");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(),
> userdataNew, false);
>
> log.info("User '" + missing_user_name + "' has been
> added.");
>
> ((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(),
> missing_user_name, 50);
>
> log.info("User '" + missing_user_name + "' has been
> updated.");
> }
>
> ((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
>
> EJBTools.wrap(certificate), missing_user_name,
> fingerprint, 20, 1, 1, 0, null, new Date()
>
> .getTime());
> log.info("Dummy certificate '" + serialHex + "' has been
> stored.");
> }
>
> to (for example):
>
> if (adaptive)
> {
>
> log.info("*** SKIPPING *** Dummy certificate '" +
> serialHex + "' has been stored.");
> }
>
>
> Would THAT work??
>
> Thanks,
> Jim
>
>
>
>
> On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi Tomas (or anyone else),
>
> Sorry - I was typing my last msg on phone, and it wasn't as detailed or
> clear as I wanted.
>
> Basically what I was saying is I was thinking that, as a quick sanity
> test, if there was one or two places where the code in
> the CaImportCRLCommand.java was maybe making a method call to create the
> dummy cert and put it into the database, then maybe I could decompile
> and modify the CaImportCRLCommand.java, and comment out those method
> calls, then, re-do my large CRL import test and time it?
>
> If so, can you let me know specifically where those calls are, and if I
> just comment them out, would that work?
>
> Ironically, that (not doing the dummy cert creation and writing) was
> what I *thought* the "LENIENT" option was doing).
>
> Thanks,
> Jim
>
> P. S. I have time today to try this if I figure out what needs to be done.
>
>
>
> On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
>
> Yes, we are definitely interested/willing to test patches. I actually
> was wondering if it would be a quick test to mod the code somewhere to
> avoid a call that creates the dummy cert, to get an idea if that would
> reduce the import time enough??
>
>
> Sent from Yahoo Mail on Android
> <https://go.onelink.me/107872968?pid=InProduct&c=Global_Internal_YGrowth_AndroidEmailSig__AndroidUsers&af_wl=ym&af_sub1=Internal&af_sub2=Global_YGrowth&af_sub3=EmailSignature>
>
> On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson
> <to...@pr...> wrote:
> Most large users, some very very large, typically uses on line
> updates directly from the CA. Many CAs today don't accept only
> importing CRLs, as they need to keep track of issued certificates as
> well not only revoked (can't use "non existing is good").
> Having said thst, it's interesting to be able to import large CRLs,
> so improving it is of interest.
>
> Importing CRLs would actually not have to generate a dummy cert at
> all, since a certificate object is not needed on the responder,
> which should make it very fast.
>
> If you are willing to test patches, we can work with you on improving.
>
> Regards,
> Tomas
>
>
> On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> Hi,
>
> I am curious whether you have other users that import large CRLs
> like this?
>
> Are we the only ones who would encounter these really long
> import times, because the other users who might have large CRLs
> use the EJBCA CA also, so they don't run into having the dummy
> certs being created for every entry in the CRL?
>
> Jim
>
> On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I see...
>
> I can see that a new key pair is being generated for each dummy
> cert.
>
> In CaImportCRLCommand.java:
>
> final KeyPair key_pair = KeyTools.genKeys("2048",
> AlgorithmConstants.KEYALGORITHM_RSA);
>
> Chaning that to 1024 or 512 would be an easy improvement. A bit more
> complex, but even larger speedup would be to use a "hard coded"
> dummy
> key, i.e. same key for all dummy certs.
>
> You can experiment by changing 2048 to 512 (not secure but it
> doesn't
> have to be for this dummy cert).
>
> Regards,
> Tomas
>
> On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> > FYI, the is running, but it is REALLY slow. The CRL file has
> 1002255
> > entries in it, and the import is only up to 8899 now, after
> about 40
> > minutes... which would take about 75 hours!!
> >
> > In our production systems, we are required to update some CRLs
> every
> > hour, so that would be a problem.
> >
> > What can I do to speed that up??
> >
> > Thanks,
> > Jim
> >
> >
> > On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > AHHH... Ok, it is running now, and from checking the
> CertificateData in
> > the Mysql database, I can see that it is now populating the
> > CertificateData!!
> >
> > Thanks!
> >
> > Jim
> >
> >
> >
> > On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
> Ejbca-develop
> > <ejb...@li...
> <mailto:ejb...@li...>> wrote:
> >
> >
> > I wasn't sure if we should use ADAPTIVE, since we are not
> using the CA part.
> >
> >
> > I will try with "-o ADAPTIVE" now.
> >
> >
> >
> >
> > On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> > <to...@pr... <mailto:to...@pr...>> wrote:
> >
> >
> >
> > It should be ADAPTIVE.
> >
> > The documentation says:
> > bin/ejbca.sh ca importcrl --help
> > ---
> > ADAPTIVE means that missing certficates will be replaced by dummy
> > certificates to cater for proper CRLs for missing certificates.
> > ---
> >
> > Regards,
> > Tomas
> >
> > On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> >> Hi,
> >>
> >> I've been looking at the code in CaImportCRLCommand, and I am
> wondering
> >> if, when the command is run in LENIENT mode and it doesn't
> find the cert
> >> in the database (and outputs the "missing in database" error
> message),
> >> what is supposed to happen?
> >>
> >> Mind you, the only error messages I am seeing are the "missing in
> >> database" error messages. I don't see any of the other error
> messages
> >> that are in/from CaImportCRLCommand.java.
> >>
> >> I am wondering if, when the command is run with "-o LENIENT",
> and when
> >> it outputs the "missing in database" error message, then the
> ejbca.sh is
> >> basically not doing anything (and not storing anything into the
> >> CertificateData table)?
> >>
> >> From the lack of any other error messages (vs. what I see in the
> >> CaImportCRLCommand.java code), that is what it looks like is
> happening.
> >>
> >> So given I am using EJBCA for OCSP responder only, is running the
> >> ejbca.sh in the "-o LENIENT" mode correct?
> >>
> >> Jim
> >>
> >>
> >>
> >> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I've been doing a bunch of testing with trying to import the
> large CRL,
> >> and wanted to summarize:
> >>
> >> - The ejbca.sh runs fine - it completes without outputting an
> errors,
> >> except for outputting a bunch of the "cert xxxx missing in
> database"
> >> messages.
> >>
> >> - When the import completes, and I check the Mysql database,
> the crlData
> >> (the full PEM of the CRL) is in the database, but there is no
> rows in
> >> the CertificateData table for that CA at all.
> >>
> >> - I even figured out how to get the ejbca.sh to log using
> >> log4j.properties, and it output a bunch of logging, mainly
> the missing
> >> in database errors, but other than that I didn't see any
> obvious errors.
> >>
> >>
> >> So, bottom line is that even though the ejbca.sh seems to run
> without
> >> errors (other than the missing in database errors), the import is
> >> causing the CRL PEM to be stored in the database, but the
> >> CertificateData table is not being populated at all for that CA.
> >>
> >>
> >> Jim
> >>
> >>
> >>
> >> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
> Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> Ok, I found Tomas' original message... it was in my Spam
> folder....
> >>
> >> I also tried re-running the "./ejbca.sh ca importcrl" command
> a bunch of
> >> ways. I think it is running without outputting any errors,
> but still,
> >> no rows are appearing with that issuerDN at all.
> >>
> >>
> >>
> >>
> >> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> OCSP responses does not have anything to do with the CRL
> itself. Once
> >> the CRL has been imported each certificate status is in its
> own record
> >> in the CertificateData table. This is where status is read
> when reponses
> >> are created.
> >> If there are issues, it is in that case related to the import
> of the CRL.
> >>
> >> Verify that certificate records are created in
> CertificateData when
> >> importing the CRL.
> >> Also, a debug log when making OCSP request will tell exact
> details of
> >> what the responder finds in the database or not.
> >>
> >> Regards,
> >> Tomas
> >>
> >>
> >> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> >> <ejb...@li...
> <mailto:ejb...@li...>
> > <mailto:ejb...@li...
> <mailto:ejb...@li...>>> wrote:
> >>
> >> [Re-sending, as this (handling of large CRLs) is still a
> problem and
> >> is still not working.]
> >>
> >>
> >>
> >> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> I think that I've found how to set/change the
> max_allowed_packet on
> >> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> >> standalone/configuration/standalone.xml to:
> >>
> >>
> >
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
> >>
> >> However, when I test the OCSP Responder with a serial
> number (and CA
> >> cert) from the CA with the large (>71MB) CRL file, I am still
> >> getting "unknown" and EJBCA OCSP Responder is still NOT
> FINDING the
> >> cert serial number.
> >>
> >> NOTE, again, that I can test with certs (and CA cert) that
> have
> >> smaller CRL files, and those tests work.
> >>
> >> So it is only when testing with the large CRL file that is
> failing
> >> with the OCSP Responder.
> >>
> >> This seems like a potential BUG for EJBCA OCSP Responder?
> >>
> >> Please advise.
> >>
> >> Thanks,
> >> Jim
> >> On Saturday, June 29, 2019, 3:36:18 PM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi,
> >>
> >> BTW, just to check, I have actually gone through the
> process of
> >> getting the crlData from the MySQL using mysql client (I
> had to add
> >> the --max_allowed-packet=100M to the mysql command) and I
> was able
> >> to actually get the PEM from the Mysql database that way.
> >>
> >> So, I KNOW that the entire CRL is in the Mysql database,
> and yet
> >> when I run an "openssl ocsp" test against the EJBCA OCSP
> Responder,
> >> it says it cannot find the revocation.
> >>
> >> OCSP Request Data:
> >> Version: 1 (0x0)
> >> Requestor List:
> >> Certificate ID:
> >> Hash Algorithm: sha1
> >> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> >> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> >> Serial Number: 2FFFF4
> >> Request Extensions:
> >> OCSP Nonce:
> >> 04103054A60A55E389B239A55228033D8F62
> >> Response verify OK
> >> 0x2FFFF4: unknown
> >> This Update: Jun 29 19:31:02 2019 GMT
> >>
> >>
> >> In the server log:
> >>
> >> 2019-06-29 15:31:02,150 DEBUG
> >>
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> >> (default task-1) Unable to find revocation information for
> >> certificate with serial '2ffff4' from issuer 'CN=XXX....'
> >>
> >>
> >> Honestly, I am no 100% sure that the problem is that
> >> max_allowe_packet is too small on the mysql client side,
> but that is
> >> my best guess.
> >>
> >> What I AM sure is that the CRL is in mysql, but when I do
> a test
> >> with OCSP Responder, it cannot find the cert serial number
> in the CRL.
> >>
> >> Jim
> >>
> >>
> >> On Saturday, June 29, 2019, 4:52:20 AM EDT,
> <oh...@ya... <mailto:oh...@ya...>
> > <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
> >>
> >>
> >> Hi Tomas,
> >>
> >> I already had the max_allowed_packet configured on the Mysql
> >> server-side (in /etc/my.cnf).
> >>
> >> So as I said, when testing with mysql (client) I had to
> include
> >> "--max_allowed_packet=100M" on the mysql (client) command
> line, e.g.:
> >>
> >> mysql -u root -p --max_allowed_packet=100M
> >>
> >> If I did NOT include the "--max_allowed_packet=100M" on
> that command
> >> line, then when I did a "select" on the field base64Crl, I
> would get
> >> an error.
> >>
> >> I suspect that EJBCA is acting as a Mysql client, so, in
> order to
> >> work with such large CRLs, there needs to be some way to
> set the
> >> equivalent of the "--max_allowed_packet=100M" on the EJBCA
> side.
> >>
> >> How can I do that?
> >>
> >> If that can't be done, then I think that when EJBCA tries to
> >> retrieve the CRL from the Mysql, it (EJBCA) will see the
> same error.
> >>
> >> Thanks,
> >> Jim
> >>
> >> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> >> <to...@pr... <mailto:to...@pr...>
> <mailto:to...@pr... <mailto:to...@pr...>>> wrote:
> >>
> >>
> >>
> >> You configure max allowed packet size in you MySQL
> configuration on the
> >> MySQL server.
> >>
> >> /Tomas
> >>
> >> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> >> > I am not 100% sure, but I think I know what might be
> causing this
> >> problem.
> >> >
> >> > I did a test, where I just tried to log into mysql and
> do the
> >> following
> >> > select:
> >> >
> >> > select base64Crl INTO OUTFILE
> '/var/lib/mysql-files/xx.out' from
> >> CRLData
> >> > where cRLNumber=3167;
> >> >
> >> > and I got an error saying that the result was more
> >> than max_allowed_packet.
> >> >
> >> > So then I did the same thing, but for the mysql command
> I used:
> >> >
> >> > mysql -u root -p --max_allowed_packet=100M
> >> >
> >> > and then when I did that same select, I was able to
> output the entire
> >> > CRL as a PEM.
> >> >
> >> > So, I am theorizing that that is what is happening with
> EJBCA, i.e.,
> >> > when I run the "openssl ocsp" command, EJBCA is querying the
> > base64CRL
> >> > but the result is too large and so that is causing the
> query to fail.
> >> >
> >> > So the question now is: How can I cause EJBCA OCSP
> Responder to run
> >> > with a larger max_allowed_packet so that the query
> succeeds (and then
> >> > hopefully EJBCA OCSP Responder will be able to check the
> revocation
> >> > properly)?
> >> >
> >> > Does anyone know how that can be done?
> >> >
> >> > Thanks,
> >> > Jim
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> > <oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>
> >> <mailto:oh...@ya... <mailto:oh...@ya...>
> <mailto:oh...@ya... <mailto:oh...@ya...>>>> wrote:
> >> >
> >> >
> >> > Hi,
> >> >
> >> > As discussed in an earlier thread, it looks like I was
> able to
> >> import a
> >> > large (> 71MB) CRL file into EJBCA (as an OCSP
> Responder), so now,
> >> I am
> >> > doing testing, using "openssl ocsp" to send requests
> into the
> >> EJBCA OCSP
> >> > Responder:
> >> >
> >> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> >> -req_text -serial
> >> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >> >
> >> > But it looks like no matter what serial number I use,
> whether the
> >> serial
> >> > number is in the CRL or not, returns a response like:
> >> >
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash: 37...00
> >> > Issuer Key Hash: 39...E4
> >> > Serial Number: 2FFFF9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 041078B1311984A110B5E19C8DC4895F485B
> >> > Response verify OK
> >> > 0x2FFFF9: unknown
> >> > This Update: Jun 28 18:56:20 2019 GMT
> >> >
> >> > i.e., it looks like the EJBCA OCSP Responder is not able
> to actually
> >> > find any revoked certs in the CRL?
> >> >
> >> >
> >> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> >> properly, in
> >> > general, because I can do the same test against a
> smaller CRL in the
> >> > same EJBCA OCSP Responder, for CA simpleca, and I can
> get an actual
> >> > "Revoked" response:
> >> >
> >> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> <mailto:root@ejbca <mailto:root@ejbca>
> > <mailto:root@ejbca <mailto:root@ejbca>>> testocspresponder]#
> openssl ocsp
> >> -issuer simpleca.crt
> >> > -VAfile simplecabinding2.crt -serial
> 0x008486394c03e1f5d9 -req_text
> >> > -host 192.168.0.28:80
> >> > OCSP Request Data:
> >> > Version: 1 (0x0)
> >> > Requestor List:
> >> > Certificate ID:
> >> > Hash Algorithm: sha1
> >> > Issuer Name Hash:
> 0C16107310427EA4ADB3C6436915CE44A15FFE55
> >> > Issuer Key Hash:
> E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> >> > Serial Number: 8486394C03E1F5D9
> >> > Request Extensions:
> >> > OCSP Nonce:
> >> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> >> > Response verify OK
> >> > 0x008486394c03e1f5d9: revoked
> >> > This Update: Jun 28 19:12:48 2019 GMT
> >> > Reason: unspecified
> >> > Revocation Time: May 26 12:30:44 2019 GMT
> >> >
> >> > So it seems like the EJBCA OCSP Responder is either not
> > processing the
> >> > large CRL properly, or maybe the import didn't import
> the large
> >> CRL into
> >> > the OCSP Responder correctly.
> >> >
> >> > Can anyone tell me how do I diagnose this problem?
> >> >
> >> > Thanks,
> >> > Jim
> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > Ejbca-develop mailing list
> >> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> >
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>>
> >
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >>
> >> _______________________________________________
> >> Ejbca-develop mailing list
> >> Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> >> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
|
From: <oh...@ya...> - 2019-07-16 13:02:43
|
For testing (only...), I am thinking that if I modify this If clause in CaImportCrlCommand.java:
if (adaptive)
{
Date time = new Date();
KeyPair key_pair = KeyTools.genKeys("2048", "RSA");
SubjectPublicKeyInfo pkinfo = SubjectPublicKeyInfo.getInstance(key_pair.getPublic().getEncoded());
X500Name dnName = new X500Name("CN=Dummy Missing in Imported CRL, serialNumber=" + serialHex);
Date notAfter = new Date(time.getTime() + 315360000000L);
X509v3CertificateBuilder certbuilder = new X509v3CertificateBuilder(X500Name.getInstance(cacert
.getSubjectX500Principal().getEncoded()), serialNr, time, notAfter, dnName, pkinfo);
ContentSigner signer = new BufferingContentSigner(new JcaContentSignerBuilder("SHA1withRSA").setProvider("BC").build(key_pair
.getPrivate()), 20480);
X509CertificateHolder certHolder = certbuilder.build(signer);
X509Certificate certificate = (X509Certificate)CertTools.getCertfromByteArray(certHolder.getEncoded(), X509Certificate.class);
String fingerprint = CertTools.getFingerprintAsString(certificate);
EndEntityInformation missingUserEndEntityInformation = ((EndEntityAccessSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityAccessSessionRemote.class)).findUser(getAuthenticationToken(), missing_user_name);
if (missingUserEndEntityInformation == null)
{
log.debug("Loading/updating user " + missing_user_name);
EndEntityInformation userdataNew = new EndEntityInformation(missing_user_name, CertTools.getSubjectDN(certificate), cainfo.getCAId(), null, null, 10, new EndEntityType(EndEntityTypes.ENDUSER), 1, 1, null, null, 1, 0, null);
userdataNew.setPassword("foo123");
((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).addUser(getAuthenticationToken(), userdataNew, false);
log.info("User '" + missing_user_name + "' has been added.");
((EndEntityManagementSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(EndEntityManagementSessionRemote.class)).setUserStatus(getAuthenticationToken(), missing_user_name, 50);
log.info("User '" + missing_user_name + "' has been updated.");
}
((CertificateStoreSessionRemote)EjbRemoteHelper.INSTANCE.getRemoteSession(CertificateStoreSessionRemote.class)).storeCertificateRemote(getAuthenticationToken(),
EJBTools.wrap(certificate), missing_user_name, fingerprint, 20, 1, 1, 0, null, new Date()
.getTime());
log.info("Dummy certificate '" + serialHex + "' has been stored.");
}
to (for example):
if (adaptive)
{
log.info("*** SKIPPING *** Dummy certificate '" + serialHex + "' has been stored.");
}
Would THAT work??
Thanks,Jim
On Tuesday, July 16, 2019, 10:59:31 AM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi Tomas (or anyone else),
Sorry - I was typing my last msg on phone, and it wasn't as detailed or clear as I wanted.
Basically what I was saying is I was thinking that, as a quick sanity test, if there was one or two places where the code in the CaImportCRLCommand.java was maybe making a method call to create the dummy cert and put it into the database, then maybe I could decompile and modify the CaImportCRLCommand.java, and comment out those method calls, then, re-do my large CRL import test and time it?
If so, can you let me know specifically where those calls are, and if I just comment them out, would that work?
Ironically, that (not doing the dummy cert creation and writing) was what I *thought* the "LENIENT" option was doing).
Thanks,Jim
P. S. I have time today to try this if I figure out what needs to be done.
On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Yes, we are definitely interested/willing to test patches. I actually was wondering if it would be a quick test to mod the code somewhere to avoid a call that creates the dummy cert, to get an idea if that would reduce the import time enough??
Sent from Yahoo Mail on Android
On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson<to...@pr...> wrote: Most large users, some very very large, typically uses on line updates directly from the CA. Many CAs today don't accept only importing CRLs, as they need to keep track of issued certificates as well not only revoked (can't use "non existing is good").
Having said thst, it's interesting to be able to import large CRLs, so improving it is of interest.
Importing CRLs would actually not have to generate a dummy cert at all, since a certificate object is not needed on the responder, which should make it very fast.
If you are willing to test patches, we can work with you on improving.
Regards,
Tomas
On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
I am curious whether you have other users that import large CRLs like this?
Are we the only ones who would encounter these really long import times, because the other users who might have large CRLs use the EJBCA CA also, so they don't run into having the dummy certs being created for every entry in the CRL?
Jim
On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
I see...
I can see that a new key pair is being generated for each dummy cert.
In CaImportCRLCommand.java:
final KeyPair key_pair = KeyTools.genKeys("2048",
AlgorithmConstants.KEYALGORITHM_RSA);
Chaning that to 1024 or 512 would be an easy improvement. A bit more
complex, but even larger speedup would be to use a "hard coded" dummy
key, i.e. same key for all dummy certs.
You can experiment by changing 2048 to 512 (not secure but it doesn't
have to be for this dummy cert).
Regards,
Tomas
On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
> entries in it, and the import is only up to 8899 now, after about 40
> minutes... which would take about 75 hours!!
>
> In our production systems, we are required to update some CRLs every
> hour, so that would be a problem.
>
> What can I do to speed that up??
>
> Thanks,
> Jim
>
>
> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> AHHH... Ok, it is running now, and from checking the CertificateData in
> the Mysql database, I can see that it is now populating the
> CertificateData!!
>
> Thanks!
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
>
>
> I will try with "-o ADAPTIVE" now.
>
>
>
>
> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> It should be ADAPTIVE.
>
> The documentation says:
> bin/ejbca.sh ca importcrl --help
> ---
> ADAPTIVE means that missing certficates will be replaced by dummy
> certificates to cater for proper CRLs for missing certificates.
> ---
>
> Regards,
> Tomas
>
> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> Hi,
>>
>> I've been looking at the code in CaImportCRLCommand, and I am wondering
>> if, when the command is run in LENIENT mode and it doesn't find the cert
>> in the database (and outputs the "missing in database" error message),
>> what is supposed to happen?
>>
>> Mind you, the only error messages I am seeing are the "missing in
>> database" error messages. I don't see any of the other error messages
>> that are in/from CaImportCRLCommand.java.
>>
>> I am wondering if, when the command is run with "-o LENIENT", and when
>> it outputs the "missing in database" error message, then the ejbca.sh is
>> basically not doing anything (and not storing anything into the
>> CertificateData table)?
>>
>> From the lack of any other error messages (vs. what I see in the
>> CaImportCRLCommand.java code), that is what it looks like is happening.
>>
>> So given I am using EJBCA for OCSP responder only, is running the
>> ejbca.sh in the "-o LENIENT" mode correct?
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> I've been doing a bunch of testing with trying to import the large CRL,
>> and wanted to summarize:
>>
>> - The ejbca.sh runs fine - it completes without outputting an errors,
>> except for outputting a bunch of the "cert xxxx missing in database"
>> messages.
>>
>> - When the import completes, and I check the Mysql database, the crlData
>> (the full PEM of the CRL) is in the database, but there is no rows in
>> the CertificateData table for that CA at all.
>>
>> - I even figured out how to get the ejbca.sh to log using
>> log4j.properties, and it output a bunch of logging, mainly the missing
>> in database errors, but other than that I didn't see any obvious errors.
>>
>>
>> So, bottom line is that even though the ejbca.sh seems to run without
>> errors (other than the missing in database errors), the import is
>> causing the CRL PEM to be stored in the database, but the
>> CertificateData table is not being populated at all for that CA.
>>
>>
>> Jim
>>
>>
>>
>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> Ok, I found Tomas' original message... it was in my Spam folder....
>>
>> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
>> ways. I think it is running without outputting any errors, but still,
>> no rows are appearing with that issuerDN at all.
>>
>>
>>
>>
>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>> Hi,
>>
>> OCSP responses does not have anything to do with the CRL itself. Once
>> the CRL has been imported each certificate status is in its own record
>> in the CertificateData table. This is where status is read when reponses
>> are created.
>> If there are issues, it is in that case related to the import of the CRL.
>>
>> Verify that certificate records are created in CertificateData when
>> importing the CRL.
>> Also, a debug log when making OCSP request will tell exact details of
>> what the responder finds in the database or not.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>> [Re-sending, as this (handling of large CRLs) is still a problem and
>> is still not working.]
>>
>>
>>
>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> I think that I've found how to set/change the max_allowed_packet on
>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>> standalone/configuration/standalone.xml to:
>>
>>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>
>> However, when I test the OCSP Responder with a serial number (and CA
>> cert) from the CA with the large (>71MB) CRL file, I am still
>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
>> cert serial number.
>>
>> NOTE, again, that I can test with certs (and CA cert) that have
>> smaller CRL files, and those tests work.
>>
>> So it is only when testing with the large CRL file that is failing
>> with the OCSP Responder.
>>
>> This seems like a potential BUG for EJBCA OCSP Responder?
>>
>> Please advise.
>>
>> Thanks,
>> Jim
>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> BTW, just to check, I have actually gone through the process of
>> getting the crlData from the MySQL using mysql client (I had to add
>> the --max_allowed-packet=100M to the mysql command) and I was able
>> to actually get the PEM from the Mysql database that way.
>>
>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
>> it says it cannot find the revocation.
>>
>> OCSP Request Data:
>> Version: 1 (0x0)
>> Requestor List:
>> Certificate ID:
>> Hash Algorithm: sha1
>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>> Serial Number: 2FFFF4
>> Request Extensions:
>> OCSP Nonce:
>> 04103054A60A55E389B239A55228033D8F62
>> Response verify OK
>> 0x2FFFF4: unknown
>> This Update: Jun 29 19:31:02 2019 GMT
>>
>>
>> In the server log:
>>
>> 2019-06-29 15:31:02,150 DEBUG
>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> (default task-1) Unable to find revocation information for
>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>
>>
>> Honestly, I am no 100% sure that the problem is that
>> max_allowe_packet is too small on the mysql client side, but that is
>> my best guess.
>>
>> What I AM sure is that the CRL is in mysql, but when I do a test
>> with OCSP Responder, it cannot find the cert serial number in the CRL.
>>
>> Jim
>>
>>
>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi Tomas,
>>
>> I already had the max_allowed_packet configured on the Mysql
>> server-side (in /etc/my.cnf).
>>
>> So as I said, when testing with mysql (client) I had to include
>> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>>
>> mysql -u root -p --max_allowed_packet=100M
>>
>> If I did NOT include the "--max_allowed_packet=100M" on that command
>> line, then when I did a "select" on the field base64Crl, I would get
>> an error.
>>
>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>> work with such large CRLs, there needs to be some way to set the
>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>
>> How can I do that?
>>
>> If that can't be done, then I think that when EJBCA tries to
>> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>>
>> Thanks,
>> Jim
>>
>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>>
>> You configure max allowed packet size in you MySQL configuration on the
>> MySQL server.
>>
>> /Tomas
>>
>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> > I am not 100% sure, but I think I know what might be causing this
>> problem.
>> >
>> > I did a test, where I just tried to log into mysql and do the
>> following
>> > select:
>> >
>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
>> CRLData
>> > where cRLNumber=3167;
>> >
>> > and I got an error saying that the result was more
>> than max_allowed_packet.
>> >
>> > So then I did the same thing, but for the mysql command I used:
>> >
>> > mysql -u root -p --max_allowed_packet=100M
>> >
>> > and then when I did that same select, I was able to output the entire
>> > CRL as a PEM.
>> >
>> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
>> > when I run the "openssl ocsp" command, EJBCA is querying the
> base64CRL
>> > but the result is too large and so that is causing the query to fail.
>> >
>> > So the question now is: How can I cause EJBCA OCSP Responder to run
>> > with a larger max_allowed_packet so that the query succeeds (and then
>> > hopefully EJBCA OCSP Responder will be able to check the revocation
>> > properly)?
>> >
>> > Does anyone know how that can be done?
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >
>> >
>> > Hi,
>> >
>> > As discussed in an earlier thread, it looks like I was able to
>> import a
>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
>> I am
>> > doing testing, using "openssl ocsp" to send requests into the
>> EJBCA OCSP
>> > Responder:
>> >
>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> -req_text -serial
>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >
>> > But it looks like no matter what serial number I use, whether the
>> serial
>> > number is in the CRL or not, returns a response like:
>> >
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 37...00
>> > Issuer Key Hash: 39...E4
>> > Serial Number: 2FFFF9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 041078B1311984A110B5E19C8DC4895F485B
>> > Response verify OK
>> > 0x2FFFF9: unknown
>> > This Update: Jun 28 18:56:20 2019 GMT
>> >
>> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
>> > find any revoked certs in the CRL?
>> >
>> >
>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>> properly, in
>> > general, because I can do the same test against a smaller CRL in the
>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
>> > "Revoked" response:
>> >
>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>> -issuer simpleca.crt
>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
>> > -host 192.168.0.28:80
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> > Serial Number: 8486394C03E1F5D9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> > Response verify OK
>> > 0x008486394c03e1f5d9: revoked
>> > This Update: Jun 28 19:12:48 2019 GMT
>> > Reason: unspecified
>> > Revocation Time: May 26 12:30:44 2019 GMT
>> >
>> > So it seems like the EJBCA OCSP Responder is either not
> processing the
>> > large CRL properly, or maybe the import didn't import the large
>> CRL into
>> > the OCSP Responder correctly.
>> >
>> > Can anyone tell me how do I diagnose this problem?
>> >
>> > Thanks,
>> > Jim
>>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-16 10:59:16
|
Hi Tomas (or anyone else),
Sorry - I was typing my last msg on phone, and it wasn't as detailed or clear as I wanted.
Basically what I was saying is I was thinking that, as a quick sanity test, if there was one or two places where the code in the CaImportCRLCommand.java was maybe making a method call to create the dummy cert and put it into the database, then maybe I could decompile and modify the CaImportCRLCommand.java, and comment out those method calls, then, re-do my large CRL import test and time it?
If so, can you let me know specifically where those calls are, and if I just comment them out, would that work?
Ironically, that (not doing the dummy cert creation and writing) was what I *thought* the "LENIENT" option was doing).
Thanks,Jim
P. S. I have time today to try this if I figure out what needs to be done.
On Monday, July 15, 2019, 6:24:53 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Yes, we are definitely interested/willing to test patches. I actually was wondering if it would be a quick test to mod the code somewhere to avoid a call that creates the dummy cert, to get an idea if that would reduce the import time enough??
Sent from Yahoo Mail on Android
On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson<to...@pr...> wrote: Most large users, some very very large, typically uses on line updates directly from the CA. Many CAs today don't accept only importing CRLs, as they need to keep track of issued certificates as well not only revoked (can't use "non existing is good").
Having said thst, it's interesting to be able to import large CRLs, so improving it is of interest.
Importing CRLs would actually not have to generate a dummy cert at all, since a certificate object is not needed on the responder, which should make it very fast.
If you are willing to test patches, we can work with you on improving.
Regards,
Tomas
On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
I am curious whether you have other users that import large CRLs like this?
Are we the only ones who would encounter these really long import times, because the other users who might have large CRLs use the EJBCA CA also, so they don't run into having the dummy certs being created for every entry in the CRL?
Jim
On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
I see...
I can see that a new key pair is being generated for each dummy cert.
In CaImportCRLCommand.java:
final KeyPair key_pair = KeyTools.genKeys("2048",
AlgorithmConstants.KEYALGORITHM_RSA);
Chaning that to 1024 or 512 would be an easy improvement. A bit more
complex, but even larger speedup would be to use a "hard coded" dummy
key, i.e. same key for all dummy certs.
You can experiment by changing 2048 to 512 (not secure but it doesn't
have to be for this dummy cert).
Regards,
Tomas
On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
> entries in it, and the import is only up to 8899 now, after about 40
> minutes... which would take about 75 hours!!
>
> In our production systems, we are required to update some CRLs every
> hour, so that would be a problem.
>
> What can I do to speed that up??
>
> Thanks,
> Jim
>
>
> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> AHHH... Ok, it is running now, and from checking the CertificateData in
> the Mysql database, I can see that it is now populating the
> CertificateData!!
>
> Thanks!
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
>
>
> I will try with "-o ADAPTIVE" now.
>
>
>
>
> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> It should be ADAPTIVE.
>
> The documentation says:
> bin/ejbca.sh ca importcrl --help
> ---
> ADAPTIVE means that missing certficates will be replaced by dummy
> certificates to cater for proper CRLs for missing certificates.
> ---
>
> Regards,
> Tomas
>
> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> Hi,
>>
>> I've been looking at the code in CaImportCRLCommand, and I am wondering
>> if, when the command is run in LENIENT mode and it doesn't find the cert
>> in the database (and outputs the "missing in database" error message),
>> what is supposed to happen?
>>
>> Mind you, the only error messages I am seeing are the "missing in
>> database" error messages. I don't see any of the other error messages
>> that are in/from CaImportCRLCommand.java.
>>
>> I am wondering if, when the command is run with "-o LENIENT", and when
>> it outputs the "missing in database" error message, then the ejbca.sh is
>> basically not doing anything (and not storing anything into the
>> CertificateData table)?
>>
>> From the lack of any other error messages (vs. what I see in the
>> CaImportCRLCommand.java code), that is what it looks like is happening.
>>
>> So given I am using EJBCA for OCSP responder only, is running the
>> ejbca.sh in the "-o LENIENT" mode correct?
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> I've been doing a bunch of testing with trying to import the large CRL,
>> and wanted to summarize:
>>
>> - The ejbca.sh runs fine - it completes without outputting an errors,
>> except for outputting a bunch of the "cert xxxx missing in database"
>> messages.
>>
>> - When the import completes, and I check the Mysql database, the crlData
>> (the full PEM of the CRL) is in the database, but there is no rows in
>> the CertificateData table for that CA at all.
>>
>> - I even figured out how to get the ejbca.sh to log using
>> log4j.properties, and it output a bunch of logging, mainly the missing
>> in database errors, but other than that I didn't see any obvious errors.
>>
>>
>> So, bottom line is that even though the ejbca.sh seems to run without
>> errors (other than the missing in database errors), the import is
>> causing the CRL PEM to be stored in the database, but the
>> CertificateData table is not being populated at all for that CA.
>>
>>
>> Jim
>>
>>
>>
>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> Ok, I found Tomas' original message... it was in my Spam folder....
>>
>> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
>> ways. I think it is running without outputting any errors, but still,
>> no rows are appearing with that issuerDN at all.
>>
>>
>>
>>
>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>> Hi,
>>
>> OCSP responses does not have anything to do with the CRL itself. Once
>> the CRL has been imported each certificate status is in its own record
>> in the CertificateData table. This is where status is read when reponses
>> are created.
>> If there are issues, it is in that case related to the import of the CRL.
>>
>> Verify that certificate records are created in CertificateData when
>> importing the CRL.
>> Also, a debug log when making OCSP request will tell exact details of
>> what the responder finds in the database or not.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>> [Re-sending, as this (handling of large CRLs) is still a problem and
>> is still not working.]
>>
>>
>>
>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> I think that I've found how to set/change the max_allowed_packet on
>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>> standalone/configuration/standalone.xml to:
>>
>>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>
>> However, when I test the OCSP Responder with a serial number (and CA
>> cert) from the CA with the large (>71MB) CRL file, I am still
>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
>> cert serial number.
>>
>> NOTE, again, that I can test with certs (and CA cert) that have
>> smaller CRL files, and those tests work.
>>
>> So it is only when testing with the large CRL file that is failing
>> with the OCSP Responder.
>>
>> This seems like a potential BUG for EJBCA OCSP Responder?
>>
>> Please advise.
>>
>> Thanks,
>> Jim
>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> BTW, just to check, I have actually gone through the process of
>> getting the crlData from the MySQL using mysql client (I had to add
>> the --max_allowed-packet=100M to the mysql command) and I was able
>> to actually get the PEM from the Mysql database that way.
>>
>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
>> it says it cannot find the revocation.
>>
>> OCSP Request Data:
>> Version: 1 (0x0)
>> Requestor List:
>> Certificate ID:
>> Hash Algorithm: sha1
>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>> Serial Number: 2FFFF4
>> Request Extensions:
>> OCSP Nonce:
>> 04103054A60A55E389B239A55228033D8F62
>> Response verify OK
>> 0x2FFFF4: unknown
>> This Update: Jun 29 19:31:02 2019 GMT
>>
>>
>> In the server log:
>>
>> 2019-06-29 15:31:02,150 DEBUG
>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> (default task-1) Unable to find revocation information for
>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>
>>
>> Honestly, I am no 100% sure that the problem is that
>> max_allowe_packet is too small on the mysql client side, but that is
>> my best guess.
>>
>> What I AM sure is that the CRL is in mysql, but when I do a test
>> with OCSP Responder, it cannot find the cert serial number in the CRL.
>>
>> Jim
>>
>>
>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi Tomas,
>>
>> I already had the max_allowed_packet configured on the Mysql
>> server-side (in /etc/my.cnf).
>>
>> So as I said, when testing with mysql (client) I had to include
>> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>>
>> mysql -u root -p --max_allowed_packet=100M
>>
>> If I did NOT include the "--max_allowed_packet=100M" on that command
>> line, then when I did a "select" on the field base64Crl, I would get
>> an error.
>>
>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>> work with such large CRLs, there needs to be some way to set the
>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>
>> How can I do that?
>>
>> If that can't be done, then I think that when EJBCA tries to
>> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>>
>> Thanks,
>> Jim
>>
>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>>
>> You configure max allowed packet size in you MySQL configuration on the
>> MySQL server.
>>
>> /Tomas
>>
>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> > I am not 100% sure, but I think I know what might be causing this
>> problem.
>> >
>> > I did a test, where I just tried to log into mysql and do the
>> following
>> > select:
>> >
>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
>> CRLData
>> > where cRLNumber=3167;
>> >
>> > and I got an error saying that the result was more
>> than max_allowed_packet.
>> >
>> > So then I did the same thing, but for the mysql command I used:
>> >
>> > mysql -u root -p --max_allowed_packet=100M
>> >
>> > and then when I did that same select, I was able to output the entire
>> > CRL as a PEM.
>> >
>> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
>> > when I run the "openssl ocsp" command, EJBCA is querying the
> base64CRL
>> > but the result is too large and so that is causing the query to fail.
>> >
>> > So the question now is: How can I cause EJBCA OCSP Responder to run
>> > with a larger max_allowed_packet so that the query succeeds (and then
>> > hopefully EJBCA OCSP Responder will be able to check the revocation
>> > properly)?
>> >
>> > Does anyone know how that can be done?
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >
>> >
>> > Hi,
>> >
>> > As discussed in an earlier thread, it looks like I was able to
>> import a
>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
>> I am
>> > doing testing, using "openssl ocsp" to send requests into the
>> EJBCA OCSP
>> > Responder:
>> >
>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> -req_text -serial
>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >
>> > But it looks like no matter what serial number I use, whether the
>> serial
>> > number is in the CRL or not, returns a response like:
>> >
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 37...00
>> > Issuer Key Hash: 39...E4
>> > Serial Number: 2FFFF9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 041078B1311984A110B5E19C8DC4895F485B
>> > Response verify OK
>> > 0x2FFFF9: unknown
>> > This Update: Jun 28 18:56:20 2019 GMT
>> >
>> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
>> > find any revoked certs in the CRL?
>> >
>> >
>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>> properly, in
>> > general, because I can do the same test against a smaller CRL in the
>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
>> > "Revoked" response:
>> >
>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>> -issuer simpleca.crt
>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
>> > -host 192.168.0.28:80
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> > Serial Number: 8486394C03E1F5D9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> > Response verify OK
>> > 0x008486394c03e1f5d9: revoked
>> > This Update: Jun 28 19:12:48 2019 GMT
>> > Reason: unspecified
>> > Revocation Time: May 26 12:30:44 2019 GMT
>> >
>> > So it seems like the EJBCA OCSP Responder is either not
> processing the
>> > large CRL properly, or maybe the import didn't import the large
>> CRL into
>> > the OCSP Responder correctly.
>> >
>> > Can anyone tell me how do I diagnose this problem?
>> >
>> > Thanks,
>> > Jim
>>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-15 18:24:25
|
Yes, we are definitely interested/willing to test patches. I actually was wondering if it would be a quick test to mod the code somewhere to avoid a call that creates the dummy cert, to get an idea if that would reduce the import time enough??
Sent from Yahoo Mail on Android
On Mon, Jul 15, 2019 at 12:10 PM, Tomas Gustavsson<to...@pr...> wrote: Most large users, some very very large, typically uses on line updates directly from the CA. Many CAs today don't accept only importing CRLs, as they need to keep track of issued certificates as well not only revoked (can't use "non existing is good").
Having said thst, it's interesting to be able to import large CRLs, so improving it is of interest.
Importing CRLs would actually not have to generate a dummy cert at all, since a certificate object is not needed on the responder, which should make it very fast.
If you are willing to test patches, we can work with you on improving.
Regards,
Tomas
On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
I am curious whether you have other users that import large CRLs like this?
Are we the only ones who would encounter these really long import times, because the other users who might have large CRLs use the EJBCA CA also, so they don't run into having the dummy certs being created for every entry in the CRL?
Jim
On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
I see...
I can see that a new key pair is being generated for each dummy cert.
In CaImportCRLCommand.java:
final KeyPair key_pair = KeyTools.genKeys("2048",
AlgorithmConstants.KEYALGORITHM_RSA);
Chaning that to 1024 or 512 would be an easy improvement. A bit more
complex, but even larger speedup would be to use a "hard coded" dummy
key, i.e. same key for all dummy certs.
You can experiment by changing 2048 to 512 (not secure but it doesn't
have to be for this dummy cert).
Regards,
Tomas
On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
> entries in it, and the import is only up to 8899 now, after about 40
> minutes... which would take about 75 hours!!
>
> In our production systems, we are required to update some CRLs every
> hour, so that would be a problem.
>
> What can I do to speed that up??
>
> Thanks,
> Jim
>
>
> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> AHHH... Ok, it is running now, and from checking the CertificateData in
> the Mysql database, I can see that it is now populating the
> CertificateData!!
>
> Thanks!
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
>
>
> I will try with "-o ADAPTIVE" now.
>
>
>
>
> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> It should be ADAPTIVE.
>
> The documentation says:
> bin/ejbca.sh ca importcrl --help
> ---
> ADAPTIVE means that missing certficates will be replaced by dummy
> certificates to cater for proper CRLs for missing certificates.
> ---
>
> Regards,
> Tomas
>
> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> Hi,
>>
>> I've been looking at the code in CaImportCRLCommand, and I am wondering
>> if, when the command is run in LENIENT mode and it doesn't find the cert
>> in the database (and outputs the "missing in database" error message),
>> what is supposed to happen?
>>
>> Mind you, the only error messages I am seeing are the "missing in
>> database" error messages. I don't see any of the other error messages
>> that are in/from CaImportCRLCommand.java.
>>
>> I am wondering if, when the command is run with "-o LENIENT", and when
>> it outputs the "missing in database" error message, then the ejbca.sh is
>> basically not doing anything (and not storing anything into the
>> CertificateData table)?
>>
>> From the lack of any other error messages (vs. what I see in the
>> CaImportCRLCommand.java code), that is what it looks like is happening.
>>
>> So given I am using EJBCA for OCSP responder only, is running the
>> ejbca.sh in the "-o LENIENT" mode correct?
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> I've been doing a bunch of testing with trying to import the large CRL,
>> and wanted to summarize:
>>
>> - The ejbca.sh runs fine - it completes without outputting an errors,
>> except for outputting a bunch of the "cert xxxx missing in database"
>> messages.
>>
>> - When the import completes, and I check the Mysql database, the crlData
>> (the full PEM of the CRL) is in the database, but there is no rows in
>> the CertificateData table for that CA at all.
>>
>> - I even figured out how to get the ejbca.sh to log using
>> log4j.properties, and it output a bunch of logging, mainly the missing
>> in database errors, but other than that I didn't see any obvious errors.
>>
>>
>> So, bottom line is that even though the ejbca.sh seems to run without
>> errors (other than the missing in database errors), the import is
>> causing the CRL PEM to be stored in the database, but the
>> CertificateData table is not being populated at all for that CA.
>>
>>
>> Jim
>>
>>
>>
>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> Ok, I found Tomas' original message... it was in my Spam folder....
>>
>> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
>> ways. I think it is running without outputting any errors, but still,
>> no rows are appearing with that issuerDN at all.
>>
>>
>>
>>
>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>> Hi,
>>
>> OCSP responses does not have anything to do with the CRL itself. Once
>> the CRL has been imported each certificate status is in its own record
>> in the CertificateData table. This is where status is read when reponses
>> are created.
>> If there are issues, it is in that case related to the import of the CRL.
>>
>> Verify that certificate records are created in CertificateData when
>> importing the CRL.
>> Also, a debug log when making OCSP request will tell exact details of
>> what the responder finds in the database or not.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>> [Re-sending, as this (handling of large CRLs) is still a problem and
>> is still not working.]
>>
>>
>>
>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> I think that I've found how to set/change the max_allowed_packet on
>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>> standalone/configuration/standalone.xml to:
>>
>>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>
>> However, when I test the OCSP Responder with a serial number (and CA
>> cert) from the CA with the large (>71MB) CRL file, I am still
>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
>> cert serial number.
>>
>> NOTE, again, that I can test with certs (and CA cert) that have
>> smaller CRL files, and those tests work.
>>
>> So it is only when testing with the large CRL file that is failing
>> with the OCSP Responder.
>>
>> This seems like a potential BUG for EJBCA OCSP Responder?
>>
>> Please advise.
>>
>> Thanks,
>> Jim
>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> BTW, just to check, I have actually gone through the process of
>> getting the crlData from the MySQL using mysql client (I had to add
>> the --max_allowed-packet=100M to the mysql command) and I was able
>> to actually get the PEM from the Mysql database that way.
>>
>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
>> it says it cannot find the revocation.
>>
>> OCSP Request Data:
>> Version: 1 (0x0)
>> Requestor List:
>> Certificate ID:
>> Hash Algorithm: sha1
>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>> Serial Number: 2FFFF4
>> Request Extensions:
>> OCSP Nonce:
>> 04103054A60A55E389B239A55228033D8F62
>> Response verify OK
>> 0x2FFFF4: unknown
>> This Update: Jun 29 19:31:02 2019 GMT
>>
>>
>> In the server log:
>>
>> 2019-06-29 15:31:02,150 DEBUG
>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> (default task-1) Unable to find revocation information for
>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>
>>
>> Honestly, I am no 100% sure that the problem is that
>> max_allowe_packet is too small on the mysql client side, but that is
>> my best guess.
>>
>> What I AM sure is that the CRL is in mysql, but when I do a test
>> with OCSP Responder, it cannot find the cert serial number in the CRL.
>>
>> Jim
>>
>>
>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi Tomas,
>>
>> I already had the max_allowed_packet configured on the Mysql
>> server-side (in /etc/my.cnf).
>>
>> So as I said, when testing with mysql (client) I had to include
>> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>>
>> mysql -u root -p --max_allowed_packet=100M
>>
>> If I did NOT include the "--max_allowed_packet=100M" on that command
>> line, then when I did a "select" on the field base64Crl, I would get
>> an error.
>>
>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>> work with such large CRLs, there needs to be some way to set the
>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>
>> How can I do that?
>>
>> If that can't be done, then I think that when EJBCA tries to
>> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>>
>> Thanks,
>> Jim
>>
>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>>
>> You configure max allowed packet size in you MySQL configuration on the
>> MySQL server.
>>
>> /Tomas
>>
>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> > I am not 100% sure, but I think I know what might be causing this
>> problem.
>> >
>> > I did a test, where I just tried to log into mysql and do the
>> following
>> > select:
>> >
>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
>> CRLData
>> > where cRLNumber=3167;
>> >
>> > and I got an error saying that the result was more
>> than max_allowed_packet.
>> >
>> > So then I did the same thing, but for the mysql command I used:
>> >
>> > mysql -u root -p --max_allowed_packet=100M
>> >
>> > and then when I did that same select, I was able to output the entire
>> > CRL as a PEM.
>> >
>> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
>> > when I run the "openssl ocsp" command, EJBCA is querying the
> base64CRL
>> > but the result is too large and so that is causing the query to fail.
>> >
>> > So the question now is: How can I cause EJBCA OCSP Responder to run
>> > with a larger max_allowed_packet so that the query succeeds (and then
>> > hopefully EJBCA OCSP Responder will be able to check the revocation
>> > properly)?
>> >
>> > Does anyone know how that can be done?
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >
>> >
>> > Hi,
>> >
>> > As discussed in an earlier thread, it looks like I was able to
>> import a
>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
>> I am
>> > doing testing, using "openssl ocsp" to send requests into the
>> EJBCA OCSP
>> > Responder:
>> >
>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> -req_text -serial
>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >
>> > But it looks like no matter what serial number I use, whether the
>> serial
>> > number is in the CRL or not, returns a response like:
>> >
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 37...00
>> > Issuer Key Hash: 39...E4
>> > Serial Number: 2FFFF9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 041078B1311984A110B5E19C8DC4895F485B
>> > Response verify OK
>> > 0x2FFFF9: unknown
>> > This Update: Jun 28 18:56:20 2019 GMT
>> >
>> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
>> > find any revoked certs in the CRL?
>> >
>> >
>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>> properly, in
>> > general, because I can do the same test against a smaller CRL in the
>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
>> > "Revoked" response:
>> >
>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>> -issuer simpleca.crt
>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
>> > -host 192.168.0.28:80
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> > Serial Number: 8486394C03E1F5D9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> > Response verify OK
>> > 0x008486394c03e1f5d9: revoked
>> > This Update: Jun 28 19:12:48 2019 GMT
>> > Reason: unspecified
>> > Revocation Time: May 26 12:30:44 2019 GMT
>> >
>> > So it seems like the EJBCA OCSP Responder is either not
> processing the
>> > large CRL properly, or maybe the import didn't import the large
>> CRL into
>> > the OCSP Responder correctly.
>> >
>> > Can anyone tell me how do I diagnose this problem?
>> >
>> > Thanks,
>> > Jim
>>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-15 16:10:59
|
Most large users, some very very large, typically uses on line updates directly from the CA. Many CAs today don't accept only importing CRLs, as they need to keep track of issued certificates as well not only revoked (can't use "non existing is good").
Having said thst, it's interesting to be able to import large CRLs, so improving it is of interest.
Importing CRLs would actually not have to generate a dummy cert at all, since a certificate object is not needed on the responder, which should make it very fast.
If you are willing to test patches, we can work with you on improving.
Regards,
Tomas
On July 15, 2019 5:26:08 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
> Hi,
>I am curious whether you have other users that import large CRLs like
>this?
>
>Are we the only ones who would encounter these really long import
>times, because the other users who might have large CRLs use the EJBCA
>CA also, so they don't run into having the dummy certs being created
>for every entry in the CRL?
>Jim
>
>On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson
><to...@pr...> wrote:
>
>
>I see...
>
>I can see that a new key pair is being generated for each dummy cert.
>
>In CaImportCRLCommand.java:
>
>final KeyPair key_pair = KeyTools.genKeys("2048",
>AlgorithmConstants.KEYALGORITHM_RSA);
>
>Chaning that to 1024 or 512 would be an easy improvement. A bit more
>complex, but even larger speedup would be to use a "hard coded" dummy
>key, i.e. same key for all dummy certs.
>
>You can experiment by changing 2048 to 512 (not secure but it doesn't
>have to be for this dummy cert).
>
>Regards,
>Tomas
>
>On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
>> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
>> entries in it, and the import is only up to 8899 now, after about 40
>> minutes... which would take about 75 hours!!
>>
>> In our production systems, we are required to update some CRLs every
>> hour, so that would be a problem.
>>
>> What can I do to speed that up??
>>
>> Thanks,
>> Jim
>>
>>
>> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> AHHH... Ok, it is running now, and from checking the CertificateData
>in
>> the Mysql database, I can see that it is now populating the
>> CertificateData!!
>>
>> Thanks!
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via
>Ejbca-develop
>> <ejb...@li...> wrote:
>>
>>
>> I wasn't sure if we should use ADAPTIVE, since we are not using the
>CA part.
>>
>>
>> I will try with "-o ADAPTIVE" now.
>>
>>
>>
>>
>> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
>> <to...@pr...> wrote:
>>
>>
>>
>> It should be ADAPTIVE.
>>
>> The documentation says:
>> bin/ejbca.sh ca importcrl --help
>> ---
>> ADAPTIVE means that missing certficates will be replaced by dummy
>> certificates to cater for proper CRLs for missing certificates.
>> ---
>>
>> Regards,
>> Tomas
>>
>> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>>> Hi,
>>>
>>> I've been looking at the code in CaImportCRLCommand, and I am
>wondering
>>> if, when the command is run in LENIENT mode and it doesn't find the
>cert
>>> in the database (and outputs the "missing in database" error
>message),
>>> what is supposed to happen?
>>>
>>> Mind you, the only error messages I am seeing are the "missing in
>>> database" error messages. I don't see any of the other error
>messages
>>> that are in/from CaImportCRLCommand.java.
>>>
>>> I am wondering if, when the command is run with "-o LENIENT", and
>when
>>> it outputs the "missing in database" error message, then the
>ejbca.sh is
>>> basically not doing anything (and not storing anything into the
>>> CertificateData table)?
>>>
>>> From the lack of any other error messages (vs. what I see in the
>>> CaImportCRLCommand.java code), that is what it looks like is
>happening.
>>>
>>> So given I am using EJBCA for OCSP responder only, is running the
>>> ejbca.sh in the "-o LENIENT" mode correct?
>>>
>>> Jim
>>>
>>>
>>>
>>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via
>Ejbca-develop
>>> <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I've been doing a bunch of testing with trying to import the large
>CRL,
>>> and wanted to summarize:
>>>
>>> - The ejbca.sh runs fine - it completes without outputting an
>errors,
>>> except for outputting a bunch of the "cert xxxx missing in database"
>>> messages.
>>>
>>> - When the import completes, and I check the Mysql database, the
>crlData
>>> (the full PEM of the CRL) is in the database, but there is no rows
>in
>>> the CertificateData table for that CA at all.
>>>
>>> - I even figured out how to get the ejbca.sh to log using
>>> log4j.properties, and it output a bunch of logging, mainly the
>missing
>>> in database errors, but other than that I didn't see any obvious
>errors.
>>>
>>>
>>> So, bottom line is that even though the ejbca.sh seems to run
>without
>>> errors (other than the missing in database errors), the import is
>>> causing the CRL PEM to be stored in the database, but the
>>> CertificateData table is not being populated at all for that CA.
>>>
>>>
>>> Jim
>>>
>>>
>>>
>>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via
>Ejbca-develop
>>> <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> Ok, I found Tomas' original message... it was in my Spam folder....
>>>
>>> I also tried re-running the "./ejbca.sh ca importcrl" command a
>bunch of
>>> ways. I think it is running without outputting any errors, but
>still,
>>> no rows are appearing with that issuerDN at all.
>>>
>>>
>>>
>>>
>>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>>> <to...@pr... <mailto:to...@pr...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> OCSP responses does not have anything to do with the CRL itself.
>Once
>>> the CRL has been imported each certificate status is in its own
>record
>>> in the CertificateData table. This is where status is read when
>reponses
>>> are created.
>>> If there are issues, it is in that case related to the import of the
>CRL.
>>>
>>> Verify that certificate records are created in CertificateData when
>>> importing the CRL.
>>> Also, a debug log when making OCSP request will tell exact details
>of
>>> what the responder finds in the database or not.
>>>
>>> Regards,
>>> Tomas
>>>
>>>
>>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>>> <ejb...@li...
>> <mailto:ejb...@li...>> wrote:
>>>
>>> [Re-sending, as this (handling of large CRLs) is still a problem
>and
>>> is still not working.]
>>>
>>>
>>>
>>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
>> <mailto:oh...@ya...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> I think that I've found how to set/change the max_allowed_packet
>on
>>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>>> standalone/configuration/standalone.xml to:
>>>
>>>
>>
><connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>>
>>> However, when I test the OCSP Responder with a serial number (and
>CA
>>> cert) from the CA with the large (>71MB) CRL file, I am still
>>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING
>the
>>> cert serial number.
>>>
>>> NOTE, again, that I can test with certs (and CA cert) that have
>>> smaller CRL files, and those tests work.
>>>
>>> So it is only when testing with the large CRL file that is
>failing
>>> with the OCSP Responder.
>>>
>>> This seems like a potential BUG for EJBCA OCSP Responder?
>>>
>>> Please advise.
>>>
>>> Thanks,
>>> Jim
>>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
>> <mailto:oh...@ya...>> wrote:
>>>
>>>
>>> Hi,
>>>
>>> BTW, just to check, I have actually gone through the process of
>>> getting the crlData from the MySQL using mysql client (I had to
>add
>>> the --max_allowed-packet=100M to the mysql command) and I was
>able
>>> to actually get the PEM from the Mysql database that way.
>>>
>>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>>> when I run an "openssl ocsp" test against the EJBCA OCSP
>Responder,
>>> it says it cannot find the revocation.
>>>
>>> OCSP Request Data:
>>> Version: 1 (0x0)
>>> Requestor List:
>>> Certificate ID:
>>> Hash Algorithm: sha1
>>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>>> Serial Number: 2FFFF4
>>> Request Extensions:
>>> OCSP Nonce:
>>> 04103054A60A55E389B239A55228033D8F62
>>> Response verify OK
>>> 0x2FFFF4: unknown
>>> This Update: Jun 29 19:31:02 2019 GMT
>>>
>>>
>>> In the server log:
>>>
>>> 2019-06-29 15:31:02,150 DEBUG
>>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>>> (default task-1) Unable to find revocation information for
>>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>>
>>>
>>> Honestly, I am no 100% sure that the problem is that
>>> max_allowe_packet is too small on the mysql client side, but that
>is
>>> my best guess.
>>>
>>> What I AM sure is that the CRL is in mysql, but when I do a test
>>> with OCSP Responder, it cannot find the cert serial number in the
>CRL.
>>>
>>> Jim
>>>
>>>
>>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
>> <mailto:oh...@ya...>> wrote:
>>>
>>>
>>> Hi Tomas,
>>>
>>> I already had the max_allowed_packet configured on the Mysql
>>> server-side (in /etc/my.cnf).
>>>
>>> So as I said, when testing with mysql (client) I had to include
>>> "--max_allowed_packet=100M" on the mysql (client) command line,
>e.g.:
>>>
>>> mysql -u root -p --max_allowed_packet=100M
>>>
>>> If I did NOT include the "--max_allowed_packet=100M" on that
>command
>>> line, then when I did a "select" on the field base64Crl, I would
>get
>>> an error.
>>>
>>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>>> work with such large CRLs, there needs to be some way to set the
>>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>>
>>> How can I do that?
>>>
>>> If that can't be done, then I think that when EJBCA tries to
>>> retrieve the CRL from the Mysql, it (EJBCA) will see the same
>error.
>>>
>>> Thanks,
>>> Jim
>>>
>>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>>> <to...@pr... <mailto:to...@pr...>> wrote:
>>>
>>>
>>>
>>> You configure max allowed packet size in you MySQL configuration
>on the
>>> MySQL server.
>>>
>>> /Tomas
>>>
>>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>>> > I am not 100% sure, but I think I know what might be causing
>this
>>> problem.
>>> >
>>> > I did a test, where I just tried to log into mysql and do the
>>> following
>>> > select:
>>> >
>>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out'
>from
>>> CRLData
>>> > where cRLNumber=3167;
>>> >
>>> > and I got an error saying that the result was more
>>> than max_allowed_packet.
>>> >
>>> > So then I did the same thing, but for the mysql command I used:
>>> >
>>> > mysql -u root -p --max_allowed_packet=100M
>>> >
>>> > and then when I did that same select, I was able to output the
>entire
>>> > CRL as a PEM.
>>> >
>>> > So, I am theorizing that that is what is happening with EJBCA,
>i.e.,
>>> > when I run the "openssl ocsp" command, EJBCA is querying the
>> base64CRL
>>> > but the result is too large and so that is causing the query to
>fail.
>>> >
>>> > So the question now is: How can I cause EJBCA OCSP Responder
>to run
>>> > with a larger max_allowed_packet so that the query succeeds
>(and then
>>> > hopefully EJBCA OCSP Responder will be able to check the
>revocation
>>> > properly)?
>>> >
>>> > Does anyone know how that can be done?
>>> >
>>> > Thanks,
>>> > Jim
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
>> <oh...@ya... <mailto:oh...@ya...>
>>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>>> >
>>> >
>>> > Hi,
>>> >
>>> > As discussed in an earlier thread, it looks like I was able to
>>> import a
>>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so
>now,
>>> I am
>>> > doing testing, using "openssl ocsp" to send requests into the
>>> EJBCA OCSP
>>> > Responder:
>>> >
>>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>>> -req_text -serial
>>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>>> >
>>> > But it looks like no matter what serial number I use, whether
>the
>>> serial
>>> > number is in the CRL or not, returns a response like:
>>> >
>>> > OCSP Request Data:
>>> > Version: 1 (0x0)
>>> > Requestor List:
>>> > Certificate ID:
>>> > Hash Algorithm: sha1
>>> > Issuer Name Hash: 37...00
>>> > Issuer Key Hash: 39...E4
>>> > Serial Number: 2FFFF9
>>> > Request Extensions:
>>> > OCSP Nonce:
>>> > 041078B1311984A110B5E19C8DC4895F485B
>>> > Response verify OK
>>> > 0x2FFFF9: unknown
>>> > This Update: Jun 28 18:56:20 2019 GMT
>>> >
>>> > i.e., it looks like the EJBCA OCSP Responder is not able to
>actually
>>> > find any revoked certs in the CRL?
>>> >
>>> >
>>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>>> properly, in
>>> > general, because I can do the same test against a smaller CRL
>in the
>>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an
>actual
>>> > "Revoked" response:
>>> >
>>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
>> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>>> -issuer simpleca.crt
>>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9
>-req_text
>>> > -host 192.168.0.28:80
>>> > OCSP Request Data:
>>> > Version: 1 (0x0)
>>> > Requestor List:
>>> > Certificate ID:
>>> > Hash Algorithm: sha1
>>> > Issuer Name Hash:
>0C16107310427EA4ADB3C6436915CE44A15FFE55
>>> > Issuer Key Hash:
>E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>>> > Serial Number: 8486394C03E1F5D9
>>> > Request Extensions:
>>> > OCSP Nonce:
>>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>>> > Response verify OK
>>> > 0x008486394c03e1f5d9: revoked
>>> > This Update: Jun 28 19:12:48 2019 GMT
>>> > Reason: unspecified
>>> > Revocation Time: May 26 12:30:44 2019 GMT
>>> >
>>> > So it seems like the EJBCA OCSP Responder is either not
>> processing the
>>> > large CRL properly, or maybe the import didn't import the large
>>> CRL into
>>> > the OCSP Responder correctly.
>>> >
>>> > Can anyone tell me how do I diagnose this problem?
>>> >
>>> > Thanks,
>>> > Jim
>>>
>>> >
>>> >
>>> > _______________________________________________
>>> > Ejbca-develop mailing list
>>> > Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>> >
>>>
>>>
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>>
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> <mailto:Ejb...@li...
>> <mailto:Ejb...@li...>>
>>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>>
>>>
>>> _______________________________________________
>>> Ejbca-develop mailing list
>>> Ejb...@li...
>> <mailto:Ejb...@li...>
>>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>>
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
>_______________________________________________
>Ejbca-develop mailing list
>Ejb...@li...
>https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
|
From: <oh...@ya...> - 2019-07-15 15:26:24
|
Hi,
I am curious whether you have other users that import large CRLs like this?
Are we the only ones who would encounter these really long import times, because the other users who might have large CRLs use the EJBCA CA also, so they don't run into having the dummy certs being created for every entry in the CRL?
Jim
On Monday, July 15, 2019, 1:42:57 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
I see...
I can see that a new key pair is being generated for each dummy cert.
In CaImportCRLCommand.java:
final KeyPair key_pair = KeyTools.genKeys("2048",
AlgorithmConstants.KEYALGORITHM_RSA);
Chaning that to 1024 or 512 would be an easy improvement. A bit more
complex, but even larger speedup would be to use a "hard coded" dummy
key, i.e. same key for all dummy certs.
You can experiment by changing 2048 to 512 (not secure but it doesn't
have to be for this dummy cert).
Regards,
Tomas
On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
> entries in it, and the import is only up to 8899 now, after about 40
> minutes... which would take about 75 hours!!
>
> In our production systems, we are required to update some CRLs every
> hour, so that would be a problem.
>
> What can I do to speed that up??
>
> Thanks,
> Jim
>
>
> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> AHHH... Ok, it is running now, and from checking the CertificateData in
> the Mysql database, I can see that it is now populating the
> CertificateData!!
>
> Thanks!
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
>
>
> I will try with "-o ADAPTIVE" now.
>
>
>
>
> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> It should be ADAPTIVE.
>
> The documentation says:
> bin/ejbca.sh ca importcrl --help
> ---
> ADAPTIVE means that missing certficates will be replaced by dummy
> certificates to cater for proper CRLs for missing certificates.
> ---
>
> Regards,
> Tomas
>
> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> Hi,
>>
>> I've been looking at the code in CaImportCRLCommand, and I am wondering
>> if, when the command is run in LENIENT mode and it doesn't find the cert
>> in the database (and outputs the "missing in database" error message),
>> what is supposed to happen?
>>
>> Mind you, the only error messages I am seeing are the "missing in
>> database" error messages. I don't see any of the other error messages
>> that are in/from CaImportCRLCommand.java.
>>
>> I am wondering if, when the command is run with "-o LENIENT", and when
>> it outputs the "missing in database" error message, then the ejbca.sh is
>> basically not doing anything (and not storing anything into the
>> CertificateData table)?
>>
>> From the lack of any other error messages (vs. what I see in the
>> CaImportCRLCommand.java code), that is what it looks like is happening.
>>
>> So given I am using EJBCA for OCSP responder only, is running the
>> ejbca.sh in the "-o LENIENT" mode correct?
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> I've been doing a bunch of testing with trying to import the large CRL,
>> and wanted to summarize:
>>
>> - The ejbca.sh runs fine - it completes without outputting an errors,
>> except for outputting a bunch of the "cert xxxx missing in database"
>> messages.
>>
>> - When the import completes, and I check the Mysql database, the crlData
>> (the full PEM of the CRL) is in the database, but there is no rows in
>> the CertificateData table for that CA at all.
>>
>> - I even figured out how to get the ejbca.sh to log using
>> log4j.properties, and it output a bunch of logging, mainly the missing
>> in database errors, but other than that I didn't see any obvious errors.
>>
>>
>> So, bottom line is that even though the ejbca.sh seems to run without
>> errors (other than the missing in database errors), the import is
>> causing the CRL PEM to be stored in the database, but the
>> CertificateData table is not being populated at all for that CA.
>>
>>
>> Jim
>>
>>
>>
>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> Ok, I found Tomas' original message... it was in my Spam folder....
>>
>> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
>> ways. I think it is running without outputting any errors, but still,
>> no rows are appearing with that issuerDN at all.
>>
>>
>>
>>
>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>> Hi,
>>
>> OCSP responses does not have anything to do with the CRL itself. Once
>> the CRL has been imported each certificate status is in its own record
>> in the CertificateData table. This is where status is read when reponses
>> are created.
>> If there are issues, it is in that case related to the import of the CRL.
>>
>> Verify that certificate records are created in CertificateData when
>> importing the CRL.
>> Also, a debug log when making OCSP request will tell exact details of
>> what the responder finds in the database or not.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>> [Re-sending, as this (handling of large CRLs) is still a problem and
>> is still not working.]
>>
>>
>>
>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> I think that I've found how to set/change the max_allowed_packet on
>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>> standalone/configuration/standalone.xml to:
>>
>>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>
>> However, when I test the OCSP Responder with a serial number (and CA
>> cert) from the CA with the large (>71MB) CRL file, I am still
>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
>> cert serial number.
>>
>> NOTE, again, that I can test with certs (and CA cert) that have
>> smaller CRL files, and those tests work.
>>
>> So it is only when testing with the large CRL file that is failing
>> with the OCSP Responder.
>>
>> This seems like a potential BUG for EJBCA OCSP Responder?
>>
>> Please advise.
>>
>> Thanks,
>> Jim
>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> BTW, just to check, I have actually gone through the process of
>> getting the crlData from the MySQL using mysql client (I had to add
>> the --max_allowed-packet=100M to the mysql command) and I was able
>> to actually get the PEM from the Mysql database that way.
>>
>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
>> it says it cannot find the revocation.
>>
>> OCSP Request Data:
>> Version: 1 (0x0)
>> Requestor List:
>> Certificate ID:
>> Hash Algorithm: sha1
>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>> Serial Number: 2FFFF4
>> Request Extensions:
>> OCSP Nonce:
>> 04103054A60A55E389B239A55228033D8F62
>> Response verify OK
>> 0x2FFFF4: unknown
>> This Update: Jun 29 19:31:02 2019 GMT
>>
>>
>> In the server log:
>>
>> 2019-06-29 15:31:02,150 DEBUG
>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> (default task-1) Unable to find revocation information for
>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>
>>
>> Honestly, I am no 100% sure that the problem is that
>> max_allowe_packet is too small on the mysql client side, but that is
>> my best guess.
>>
>> What I AM sure is that the CRL is in mysql, but when I do a test
>> with OCSP Responder, it cannot find the cert serial number in the CRL.
>>
>> Jim
>>
>>
>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi Tomas,
>>
>> I already had the max_allowed_packet configured on the Mysql
>> server-side (in /etc/my.cnf).
>>
>> So as I said, when testing with mysql (client) I had to include
>> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>>
>> mysql -u root -p --max_allowed_packet=100M
>>
>> If I did NOT include the "--max_allowed_packet=100M" on that command
>> line, then when I did a "select" on the field base64Crl, I would get
>> an error.
>>
>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>> work with such large CRLs, there needs to be some way to set the
>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>
>> How can I do that?
>>
>> If that can't be done, then I think that when EJBCA tries to
>> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>>
>> Thanks,
>> Jim
>>
>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>>
>> You configure max allowed packet size in you MySQL configuration on the
>> MySQL server.
>>
>> /Tomas
>>
>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> > I am not 100% sure, but I think I know what might be causing this
>> problem.
>> >
>> > I did a test, where I just tried to log into mysql and do the
>> following
>> > select:
>> >
>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
>> CRLData
>> > where cRLNumber=3167;
>> >
>> > and I got an error saying that the result was more
>> than max_allowed_packet.
>> >
>> > So then I did the same thing, but for the mysql command I used:
>> >
>> > mysql -u root -p --max_allowed_packet=100M
>> >
>> > and then when I did that same select, I was able to output the entire
>> > CRL as a PEM.
>> >
>> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
>> > when I run the "openssl ocsp" command, EJBCA is querying the
> base64CRL
>> > but the result is too large and so that is causing the query to fail.
>> >
>> > So the question now is: How can I cause EJBCA OCSP Responder to run
>> > with a larger max_allowed_packet so that the query succeeds (and then
>> > hopefully EJBCA OCSP Responder will be able to check the revocation
>> > properly)?
>> >
>> > Does anyone know how that can be done?
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >
>> >
>> > Hi,
>> >
>> > As discussed in an earlier thread, it looks like I was able to
>> import a
>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
>> I am
>> > doing testing, using "openssl ocsp" to send requests into the
>> EJBCA OCSP
>> > Responder:
>> >
>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> -req_text -serial
>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >
>> > But it looks like no matter what serial number I use, whether the
>> serial
>> > number is in the CRL or not, returns a response like:
>> >
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 37...00
>> > Issuer Key Hash: 39...E4
>> > Serial Number: 2FFFF9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 041078B1311984A110B5E19C8DC4895F485B
>> > Response verify OK
>> > 0x2FFFF9: unknown
>> > This Update: Jun 28 18:56:20 2019 GMT
>> >
>> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
>> > find any revoked certs in the CRL?
>> >
>> >
>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>> properly, in
>> > general, because I can do the same test against a smaller CRL in the
>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
>> > "Revoked" response:
>> >
>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>> -issuer simpleca.crt
>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
>> > -host 192.168.0.28:80
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> > Serial Number: 8486394C03E1F5D9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> > Response verify OK
>> > 0x008486394c03e1f5d9: revoked
>> > This Update: Jun 28 19:12:48 2019 GMT
>> > Reason: unspecified
>> > Revocation Time: May 26 12:30:44 2019 GMT
>> >
>> > So it seems like the EJBCA OCSP Responder is either not
> processing the
>> > large CRL properly, or maybe the import didn't import the large
>> CRL into
>> > the OCSP Responder correctly.
>> >
>> > Can anyone tell me how do I diagnose this problem?
>> >
>> > Thanks,
>> > Jim
>>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-15 13:42:31
|
I see...
I can see that a new key pair is being generated for each dummy cert.
In CaImportCRLCommand.java:
final KeyPair key_pair = KeyTools.genKeys("2048",
AlgorithmConstants.KEYALGORITHM_RSA);
Chaning that to 1024 or 512 would be an easy improvement. A bit more
complex, but even larger speedup would be to use a "hard coded" dummy
key, i.e. same key for all dummy certs.
You can experiment by changing 2048 to 512 (not secure but it doesn't
have to be for this dummy cert).
Regards,
Tomas
On 2019-07-11 14:19, ohaya--- via Ejbca-develop wrote:
> FYI, the is running, but it is REALLY slow. The CRL file has 1002255
> entries in it, and the import is only up to 8899 now, after about 40
> minutes... which would take about 75 hours!!
>
> In our production systems, we are required to update some CRLs every
> hour, so that would be a problem.
>
> What can I do to speed that up??
>
> Thanks,
> Jim
>
>
> On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> AHHH... Ok, it is running now, and from checking the CertificateData in
> the Mysql database, I can see that it is now populating the
> CertificateData!!
>
> Thanks!
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
>
>
> I will try with "-o ADAPTIVE" now.
>
>
>
>
> On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> It should be ADAPTIVE.
>
> The documentation says:
> bin/ejbca.sh ca importcrl --help
> ---
> ADAPTIVE means that missing certficates will be replaced by dummy
> certificates to cater for proper CRLs for missing certificates.
> ---
>
> Regards,
> Tomas
>
> On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
>> Hi,
>>
>> I've been looking at the code in CaImportCRLCommand, and I am wondering
>> if, when the command is run in LENIENT mode and it doesn't find the cert
>> in the database (and outputs the "missing in database" error message),
>> what is supposed to happen?
>>
>> Mind you, the only error messages I am seeing are the "missing in
>> database" error messages. I don't see any of the other error messages
>> that are in/from CaImportCRLCommand.java.
>>
>> I am wondering if, when the command is run with "-o LENIENT", and when
>> it outputs the "missing in database" error message, then the ejbca.sh is
>> basically not doing anything (and not storing anything into the
>> CertificateData table)?
>>
>> From the lack of any other error messages (vs. what I see in the
>> CaImportCRLCommand.java code), that is what it looks like is happening.
>>
>> So given I am using EJBCA for OCSP responder only, is running the
>> ejbca.sh in the "-o LENIENT" mode correct?
>>
>> Jim
>>
>>
>>
>> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> I've been doing a bunch of testing with trying to import the large CRL,
>> and wanted to summarize:
>>
>> - The ejbca.sh runs fine - it completes without outputting an errors,
>> except for outputting a bunch of the "cert xxxx missing in database"
>> messages.
>>
>> - When the import completes, and I check the Mysql database, the crlData
>> (the full PEM of the CRL) is in the database, but there is no rows in
>> the CertificateData table for that CA at all.
>>
>> - I even figured out how to get the ejbca.sh to log using
>> log4j.properties, and it output a bunch of logging, mainly the missing
>> in database errors, but other than that I didn't see any obvious errors.
>>
>>
>> So, bottom line is that even though the ejbca.sh seems to run without
>> errors (other than the missing in database errors), the import is
>> causing the CRL PEM to be stored in the database, but the
>> CertificateData table is not being populated at all for that CA.
>>
>>
>> Jim
>>
>>
>>
>> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>>
>> Hi,
>>
>> Ok, I found Tomas' original message... it was in my Spam folder....
>>
>> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
>> ways. I think it is running without outputting any errors, but still,
>> no rows are appearing with that issuerDN at all.
>>
>>
>>
>>
>> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>> Hi,
>>
>> OCSP responses does not have anything to do with the CRL itself. Once
>> the CRL has been imported each certificate status is in its own record
>> in the CertificateData table. This is where status is read when reponses
>> are created.
>> If there are issues, it is in that case related to the import of the CRL.
>>
>> Verify that certificate records are created in CertificateData when
>> importing the CRL.
>> Also, a debug log when making OCSP request will tell exact details of
>> what the responder finds in the database or not.
>>
>> Regards,
>> Tomas
>>
>>
>> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>> [Re-sending, as this (handling of large CRLs) is still a problem and
>> is still not working.]
>>
>>
>>
>> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> I think that I've found how to set/change the max_allowed_packet on
>> the client (i.e., EJBCA) side, by editing the Mysql URL in the
>> standalone/configuration/standalone.xml to:
>>
>>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>>
>> However, when I test the OCSP Responder with a serial number (and CA
>> cert) from the CA with the large (>71MB) CRL file, I am still
>> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
>> cert serial number.
>>
>> NOTE, again, that I can test with certs (and CA cert) that have
>> smaller CRL files, and those tests work.
>>
>> So it is only when testing with the large CRL file that is failing
>> with the OCSP Responder.
>>
>> This seems like a potential BUG for EJBCA OCSP Responder?
>>
>> Please advise.
>>
>> Thanks,
>> Jim
>> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi,
>>
>> BTW, just to check, I have actually gone through the process of
>> getting the crlData from the MySQL using mysql client (I had to add
>> the --max_allowed-packet=100M to the mysql command) and I was able
>> to actually get the PEM from the Mysql database that way.
>>
>> So, I KNOW that the entire CRL is in the Mysql database, and yet
>> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
>> it says it cannot find the revocation.
>>
>> OCSP Request Data:
>> Version: 1 (0x0)
>> Requestor List:
>> Certificate ID:
>> Hash Algorithm: sha1
>> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
>> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
>> Serial Number: 2FFFF4
>> Request Extensions:
>> OCSP Nonce:
>> 04103054A60A55E389B239A55228033D8F62
>> Response verify OK
>> 0x2FFFF4: unknown
>> This Update: Jun 29 19:31:02 2019 GMT
>>
>>
>> In the server log:
>>
>> 2019-06-29 15:31:02,150 DEBUG
>> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
>> (default task-1) Unable to find revocation information for
>> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>>
>>
>> Honestly, I am no 100% sure that the problem is that
>> max_allowe_packet is too small on the mysql client side, but that is
>> my best guess.
>>
>> What I AM sure is that the CRL is in mysql, but when I do a test
>> with OCSP Responder, it cannot find the cert serial number in the CRL.
>>
>> Jim
>>
>>
>> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...
> <mailto:oh...@ya...>> wrote:
>>
>>
>> Hi Tomas,
>>
>> I already had the max_allowed_packet configured on the Mysql
>> server-side (in /etc/my.cnf).
>>
>> So as I said, when testing with mysql (client) I had to include
>> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>>
>> mysql -u root -p --max_allowed_packet=100M
>>
>> If I did NOT include the "--max_allowed_packet=100M" on that command
>> line, then when I did a "select" on the field base64Crl, I would get
>> an error.
>>
>> I suspect that EJBCA is acting as a Mysql client, so, in order to
>> work with such large CRLs, there needs to be some way to set the
>> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>>
>> How can I do that?
>>
>> If that can't be done, then I think that when EJBCA tries to
>> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>>
>> Thanks,
>> Jim
>>
>> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>>
>> You configure max allowed packet size in you MySQL configuration on the
>> MySQL server.
>>
>> /Tomas
>>
>> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
>> > I am not 100% sure, but I think I know what might be causing this
>> problem.
>> >
>> > I did a test, where I just tried to log into mysql and do the
>> following
>> > select:
>> >
>> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
>> CRLData
>> > where cRLNumber=3167;
>> >
>> > and I got an error saying that the result was more
>> than max_allowed_packet.
>> >
>> > So then I did the same thing, but for the mysql command I used:
>> >
>> > mysql -u root -p --max_allowed_packet=100M
>> >
>> > and then when I did that same select, I was able to output the entire
>> > CRL as a PEM.
>> >
>> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
>> > when I run the "openssl ocsp" command, EJBCA is querying the
> base64CRL
>> > but the result is too large and so that is causing the query to fail.
>> >
>> > So the question now is: How can I cause EJBCA OCSP Responder to run
>> > with a larger max_allowed_packet so that the query succeeds (and then
>> > hopefully EJBCA OCSP Responder will be able to check the revocation
>> > properly)?
>> >
>> > Does anyone know how that can be done?
>> >
>> > Thanks,
>> > Jim
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya
> <oh...@ya... <mailto:oh...@ya...>
>> <mailto:oh...@ya... <mailto:oh...@ya...>>> wrote:
>> >
>> >
>> > Hi,
>> >
>> > As discussed in an earlier thread, it looks like I was able to
>> import a
>> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
>> I am
>> > doing testing, using "openssl ocsp" to send requests into the
>> EJBCA OCSP
>> > Responder:
>> >
>> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
>> -req_text -serial
>> > 0x<CERT_SERIAL> -host <HOST:PORT>
>> >
>> > But it looks like no matter what serial number I use, whether the
>> serial
>> > number is in the CRL or not, returns a response like:
>> >
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 37...00
>> > Issuer Key Hash: 39...E4
>> > Serial Number: 2FFFF9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 041078B1311984A110B5E19C8DC4895F485B
>> > Response verify OK
>> > 0x2FFFF9: unknown
>> > This Update: Jun 28 18:56:20 2019 GMT
>> >
>> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
>> > find any revoked certs in the CRL?
>> >
>> >
>> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
>> properly, in
>> > general, because I can do the same test against a smaller CRL in the
>> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
>> > "Revoked" response:
>> >
>> > [root@ejbca <mailto:root@ejbca> <mailto:root@ejbca
> <mailto:root@ejbca>> testocspresponder]# openssl ocsp
>> -issuer simpleca.crt
>> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
>> > -host 192.168.0.28:80
>> > OCSP Request Data:
>> > Version: 1 (0x0)
>> > Requestor List:
>> > Certificate ID:
>> > Hash Algorithm: sha1
>> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
>> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
>> > Serial Number: 8486394C03E1F5D9
>> > Request Extensions:
>> > OCSP Nonce:
>> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
>> > Response verify OK
>> > 0x008486394c03e1f5d9: revoked
>> > This Update: Jun 28 19:12:48 2019 GMT
>> > Reason: unspecified
>> > Revocation Time: May 26 12:30:44 2019 GMT
>> >
>> > So it seems like the EJBCA OCSP Responder is either not
> processing the
>> > large CRL properly, or maybe the import didn't import the large
>> CRL into
>> > the OCSP Responder correctly.
>> >
>> > Can anyone tell me how do I diagnose this problem?
>> >
>> > Thanks,
>> > Jim
>>
>> >
>> >
>> > _______________________________________________
>> > Ejbca-develop mailing list
>> > Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> >
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> <mailto:Ejb...@li...
> <mailto:Ejb...@li...>>
>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>>
>> _______________________________________________
>> Ejbca-develop mailing list
>> Ejb...@li...
> <mailto:Ejb...@li...>
>> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>>
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
|
|
From: <oh...@ya...> - 2019-07-11 12:19:25
|
FYI, the is running, but it is REALLY slow. The CRL file has 1002255 entries in it, and the import is only up to 8899 now, after about 40 minutes... which would take about 75 hours!!
In our production systems, we are required to update some CRLs every hour, so that would be a problem.
What can I do to speed that up??
Thanks,Jim
On Thursday, July 11, 2019, 11:09:33 AM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
AHHH... Ok, it is running now, and from checking the CertificateData in the Mysql database, I can see that it is now populating the CertificateData!!
Thanks!
Jim
On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
I will try with "-o ADAPTIVE" now.
On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson <to...@pr...> wrote:
It should be ADAPTIVE.
The documentation says:
bin/ejbca.sh ca importcrl --help
---
ADAPTIVE means that missing certficates will be replaced by dummy
certificates to cater for proper CRLs for missing certificates.
---
Regards,
Tomas
On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> Hi,
>
> I've been looking at the code in CaImportCRLCommand, and I am wondering
> if, when the command is run in LENIENT mode and it doesn't find the cert
> in the database (and outputs the "missing in database" error message),
> what is supposed to happen?
>
> Mind you, the only error messages I am seeing are the "missing in
> database" error messages. I don't see any of the other error messages
> that are in/from CaImportCRLCommand.java.
>
> I am wondering if, when the command is run with "-o LENIENT", and when
> it outputs the "missing in database" error message, then the ejbca.sh is
> basically not doing anything (and not storing anything into the
> CertificateData table)?
>
> From the lack of any other error messages (vs. what I see in the
> CaImportCRLCommand.java code), that is what it looks like is happening.
>
> So given I am using EJBCA for OCSP responder only, is running the
> ejbca.sh in the "-o LENIENT" mode correct?
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> I've been doing a bunch of testing with trying to import the large CRL,
> and wanted to summarize:
>
> - The ejbca.sh runs fine - it completes without outputting an errors,
> except for outputting a bunch of the "cert xxxx missing in database"
> messages.
>
> - When the import completes, and I check the Mysql database, the crlData
> (the full PEM of the CRL) is in the database, but there is no rows in
> the CertificateData table for that CA at all.
>
> - I even figured out how to get the ejbca.sh to log using
> log4j.properties, and it output a bunch of logging, mainly the missing
> in database errors, but other than that I didn't see any obvious errors.
>
>
> So, bottom line is that even though the ejbca.sh seems to run without
> errors (other than the missing in database errors), the import is
> causing the CRL PEM to be stored in the database, but the
> CertificateData table is not being populated at all for that CA.
>
>
> Jim
>
>
>
> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> Ok, I found Tomas' original message... it was in my Spam folder....
>
> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
> ways. I think it is running without outputting any errors, but still,
> no rows are appearing with that issuerDN at all.
>
>
>
>
> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
> Hi,
>
> OCSP responses does not have anything to do with the CRL itself. Once
> the CRL has been imported each certificate status is in its own record
> in the CertificateData table. This is where status is read when reponses
> are created.
> If there are issues, it is in that case related to the import of the CRL.
>
> Verify that certificate records are created in CertificateData when
> importing the CRL.
> Also, a debug log when making OCSP request will tell exact details of
> what the responder finds in the database or not.
>
> Regards,
> Tomas
>
>
> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> [Re-sending, as this (handling of large CRLs) is still a problem and
> is still not working.]
>
>
>
> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote:
>
>
> Hi,
>
> I think that I've found how to set/change the max_allowed_packet on
> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> standalone/configuration/standalone.xml to:
>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>
> However, when I test the OCSP Responder with a serial number (and CA
> cert) from the CA with the large (>71MB) CRL file, I am still
> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
> cert serial number.
>
> NOTE, again, that I can test with certs (and CA cert) that have
> smaller CRL files, and those tests work.
>
> So it is only when testing with the large CRL file that is failing
> with the OCSP Responder.
>
> This seems like a potential BUG for EJBCA OCSP Responder?
>
> Please advise.
>
> Thanks,
> Jim
> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote:
>
>
> Hi,
>
> BTW, just to check, I have actually gone through the process of
> getting the crlData from the MySQL using mysql client (I had to add
> the --max_allowed-packet=100M to the mysql command) and I was able
> to actually get the PEM from the Mysql database that way.
>
> So, I KNOW that the entire CRL is in the Mysql database, and yet
> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
> it says it cannot find the revocation.
>
> OCSP Request Data:
> Version: 1 (0x0)
> Requestor List:
> Certificate ID:
> Hash Algorithm: sha1
> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> Serial Number: 2FFFF4
> Request Extensions:
> OCSP Nonce:
> 04103054A60A55E389B239A55228033D8F62
> Response verify OK
> 0x2FFFF4: unknown
> This Update: Jun 29 19:31:02 2019 GMT
>
>
> In the server log:
>
> 2019-06-29 15:31:02,150 DEBUG
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> (default task-1) Unable to find revocation information for
> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>
>
> Honestly, I am no 100% sure that the problem is that
> max_allowe_packet is too small on the mysql client side, but that is
> my best guess.
>
> What I AM sure is that the CRL is in mysql, but when I do a test
> with OCSP Responder, it cannot find the cert serial number in the CRL.
>
> Jim
>
>
> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote:
>
>
> Hi Tomas,
>
> I already had the max_allowed_packet configured on the Mysql
> server-side (in /etc/my.cnf).
>
> So as I said, when testing with mysql (client) I had to include
> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>
> mysql -u root -p --max_allowed_packet=100M
>
> If I did NOT include the "--max_allowed_packet=100M" on that command
> line, then when I did a "select" on the field base64Crl, I would get
> an error.
>
> I suspect that EJBCA is acting as a Mysql client, so, in order to
> work with such large CRLs, there needs to be some way to set the
> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>
> How can I do that?
>
> If that can't be done, then I think that when EJBCA tries to
> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>
> Thanks,
> Jim
>
> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> You configure max allowed packet size in you MySQL configuration on the
> MySQL server.
>
> /Tomas
>
> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> > I am not 100% sure, but I think I know what might be causing this
> problem.
> >
> > I did a test, where I just tried to log into mysql and do the
> following
> > select:
> >
> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
> CRLData
> > where cRLNumber=3167;
> >
> > and I got an error saying that the result was more
> than max_allowed_packet.
> >
> > So then I did the same thing, but for the mysql command I used:
> >
> > mysql -u root -p --max_allowed_packet=100M
> >
> > and then when I did that same select, I was able to output the entire
> > CRL as a PEM.
> >
> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
> > when I run the "openssl ocsp" command, EJBCA is querying the base64CRL
> > but the result is too large and so that is causing the query to fail.
> >
> > So the question now is: How can I cause EJBCA OCSP Responder to run
> > with a larger max_allowed_packet so that the query succeeds (and then
> > hopefully EJBCA OCSP Responder will be able to check the revocation
> > properly)?
> >
> > Does anyone know how that can be done?
> >
> > Thanks,
> > Jim
> >
> >
> >
> >
> >
> >
> >
> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya...
> <mailto:oh...@ya...>> wrote:
> >
> >
> > Hi,
> >
> > As discussed in an earlier thread, it looks like I was able to
> import a
> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
> I am
> > doing testing, using "openssl ocsp" to send requests into the
> EJBCA OCSP
> > Responder:
> >
> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> -req_text -serial
> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >
> > But it looks like no matter what serial number I use, whether the
> serial
> > number is in the CRL or not, returns a response like:
> >
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 37...00
> > Issuer Key Hash: 39...E4
> > Serial Number: 2FFFF9
> > Request Extensions:
> > OCSP Nonce:
> > 041078B1311984A110B5E19C8DC4895F485B
> > Response verify OK
> > 0x2FFFF9: unknown
> > This Update: Jun 28 18:56:20 2019 GMT
> >
> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
> > find any revoked certs in the CRL?
> >
> >
> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> properly, in
> > general, because I can do the same test against a smaller CRL in the
> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
> > "Revoked" response:
> >
> > [root@ejbca <mailto:root@ejbca> testocspresponder]# openssl ocsp
> -issuer simpleca.crt
> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
> > -host 192.168.0.28:80
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> > Serial Number: 8486394C03E1F5D9
> > Request Extensions:
> > OCSP Nonce:
> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> > Response verify OK
> > 0x008486394c03e1f5d9: revoked
> > This Update: Jun 28 19:12:48 2019 GMT
> > Reason: unspecified
> > Revocation Time: May 26 12:30:44 2019 GMT
> >
> > So it seems like the EJBCA OCSP Responder is either not processing the
> > large CRL properly, or maybe the import didn't import the large
> CRL into
> > the OCSP Responder correctly.
> >
> > Can anyone tell me how do I diagnose this problem?
> >
> > Thanks,
> > Jim
>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-11 11:09:23
|
AHHH... Ok, it is running now, and from checking the CertificateData in the Mysql database, I can see that it is now populating the CertificateData!!
Thanks!
Jim
On Thursday, July 11, 2019, 11:07:22 AM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
I will try with "-o ADAPTIVE" now.
On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson <to...@pr...> wrote:
It should be ADAPTIVE.
The documentation says:
bin/ejbca.sh ca importcrl --help
---
ADAPTIVE means that missing certficates will be replaced by dummy
certificates to cater for proper CRLs for missing certificates.
---
Regards,
Tomas
On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> Hi,
>
> I've been looking at the code in CaImportCRLCommand, and I am wondering
> if, when the command is run in LENIENT mode and it doesn't find the cert
> in the database (and outputs the "missing in database" error message),
> what is supposed to happen?
>
> Mind you, the only error messages I am seeing are the "missing in
> database" error messages. I don't see any of the other error messages
> that are in/from CaImportCRLCommand.java.
>
> I am wondering if, when the command is run with "-o LENIENT", and when
> it outputs the "missing in database" error message, then the ejbca.sh is
> basically not doing anything (and not storing anything into the
> CertificateData table)?
>
> From the lack of any other error messages (vs. what I see in the
> CaImportCRLCommand.java code), that is what it looks like is happening.
>
> So given I am using EJBCA for OCSP responder only, is running the
> ejbca.sh in the "-o LENIENT" mode correct?
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> I've been doing a bunch of testing with trying to import the large CRL,
> and wanted to summarize:
>
> - The ejbca.sh runs fine - it completes without outputting an errors,
> except for outputting a bunch of the "cert xxxx missing in database"
> messages.
>
> - When the import completes, and I check the Mysql database, the crlData
> (the full PEM of the CRL) is in the database, but there is no rows in
> the CertificateData table for that CA at all.
>
> - I even figured out how to get the ejbca.sh to log using
> log4j.properties, and it output a bunch of logging, mainly the missing
> in database errors, but other than that I didn't see any obvious errors.
>
>
> So, bottom line is that even though the ejbca.sh seems to run without
> errors (other than the missing in database errors), the import is
> causing the CRL PEM to be stored in the database, but the
> CertificateData table is not being populated at all for that CA.
>
>
> Jim
>
>
>
> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> Ok, I found Tomas' original message... it was in my Spam folder....
>
> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
> ways. I think it is running without outputting any errors, but still,
> no rows are appearing with that issuerDN at all.
>
>
>
>
> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
> Hi,
>
> OCSP responses does not have anything to do with the CRL itself. Once
> the CRL has been imported each certificate status is in its own record
> in the CertificateData table. This is where status is read when reponses
> are created.
> If there are issues, it is in that case related to the import of the CRL.
>
> Verify that certificate records are created in CertificateData when
> importing the CRL.
> Also, a debug log when making OCSP request will tell exact details of
> what the responder finds in the database or not.
>
> Regards,
> Tomas
>
>
> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> [Re-sending, as this (handling of large CRLs) is still a problem and
> is still not working.]
>
>
>
> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote:
>
>
> Hi,
>
> I think that I've found how to set/change the max_allowed_packet on
> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> standalone/configuration/standalone.xml to:
>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>
> However, when I test the OCSP Responder with a serial number (and CA
> cert) from the CA with the large (>71MB) CRL file, I am still
> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
> cert serial number.
>
> NOTE, again, that I can test with certs (and CA cert) that have
> smaller CRL files, and those tests work.
>
> So it is only when testing with the large CRL file that is failing
> with the OCSP Responder.
>
> This seems like a potential BUG for EJBCA OCSP Responder?
>
> Please advise.
>
> Thanks,
> Jim
> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote:
>
>
> Hi,
>
> BTW, just to check, I have actually gone through the process of
> getting the crlData from the MySQL using mysql client (I had to add
> the --max_allowed-packet=100M to the mysql command) and I was able
> to actually get the PEM from the Mysql database that way.
>
> So, I KNOW that the entire CRL is in the Mysql database, and yet
> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
> it says it cannot find the revocation.
>
> OCSP Request Data:
> Version: 1 (0x0)
> Requestor List:
> Certificate ID:
> Hash Algorithm: sha1
> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> Serial Number: 2FFFF4
> Request Extensions:
> OCSP Nonce:
> 04103054A60A55E389B239A55228033D8F62
> Response verify OK
> 0x2FFFF4: unknown
> This Update: Jun 29 19:31:02 2019 GMT
>
>
> In the server log:
>
> 2019-06-29 15:31:02,150 DEBUG
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> (default task-1) Unable to find revocation information for
> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>
>
> Honestly, I am no 100% sure that the problem is that
> max_allowe_packet is too small on the mysql client side, but that is
> my best guess.
>
> What I AM sure is that the CRL is in mysql, but when I do a test
> with OCSP Responder, it cannot find the cert serial number in the CRL.
>
> Jim
>
>
> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote:
>
>
> Hi Tomas,
>
> I already had the max_allowed_packet configured on the Mysql
> server-side (in /etc/my.cnf).
>
> So as I said, when testing with mysql (client) I had to include
> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>
> mysql -u root -p --max_allowed_packet=100M
>
> If I did NOT include the "--max_allowed_packet=100M" on that command
> line, then when I did a "select" on the field base64Crl, I would get
> an error.
>
> I suspect that EJBCA is acting as a Mysql client, so, in order to
> work with such large CRLs, there needs to be some way to set the
> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>
> How can I do that?
>
> If that can't be done, then I think that when EJBCA tries to
> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>
> Thanks,
> Jim
>
> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> You configure max allowed packet size in you MySQL configuration on the
> MySQL server.
>
> /Tomas
>
> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> > I am not 100% sure, but I think I know what might be causing this
> problem.
> >
> > I did a test, where I just tried to log into mysql and do the
> following
> > select:
> >
> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
> CRLData
> > where cRLNumber=3167;
> >
> > and I got an error saying that the result was more
> than max_allowed_packet.
> >
> > So then I did the same thing, but for the mysql command I used:
> >
> > mysql -u root -p --max_allowed_packet=100M
> >
> > and then when I did that same select, I was able to output the entire
> > CRL as a PEM.
> >
> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
> > when I run the "openssl ocsp" command, EJBCA is querying the base64CRL
> > but the result is too large and so that is causing the query to fail.
> >
> > So the question now is: How can I cause EJBCA OCSP Responder to run
> > with a larger max_allowed_packet so that the query succeeds (and then
> > hopefully EJBCA OCSP Responder will be able to check the revocation
> > properly)?
> >
> > Does anyone know how that can be done?
> >
> > Thanks,
> > Jim
> >
> >
> >
> >
> >
> >
> >
> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya...
> <mailto:oh...@ya...>> wrote:
> >
> >
> > Hi,
> >
> > As discussed in an earlier thread, it looks like I was able to
> import a
> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
> I am
> > doing testing, using "openssl ocsp" to send requests into the
> EJBCA OCSP
> > Responder:
> >
> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> -req_text -serial
> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >
> > But it looks like no matter what serial number I use, whether the
> serial
> > number is in the CRL or not, returns a response like:
> >
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 37...00
> > Issuer Key Hash: 39...E4
> > Serial Number: 2FFFF9
> > Request Extensions:
> > OCSP Nonce:
> > 041078B1311984A110B5E19C8DC4895F485B
> > Response verify OK
> > 0x2FFFF9: unknown
> > This Update: Jun 28 18:56:20 2019 GMT
> >
> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
> > find any revoked certs in the CRL?
> >
> >
> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> properly, in
> > general, because I can do the same test against a smaller CRL in the
> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
> > "Revoked" response:
> >
> > [root@ejbca <mailto:root@ejbca> testocspresponder]# openssl ocsp
> -issuer simpleca.crt
> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
> > -host 192.168.0.28:80
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> > Serial Number: 8486394C03E1F5D9
> > Request Extensions:
> > OCSP Nonce:
> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> > Response verify OK
> > 0x008486394c03e1f5d9: revoked
> > This Update: Jun 28 19:12:48 2019 GMT
> > Reason: unspecified
> > Revocation Time: May 26 12:30:44 2019 GMT
> >
> > So it seems like the EJBCA OCSP Responder is either not processing the
> > large CRL properly, or maybe the import didn't import the large
> CRL into
> > the OCSP Responder correctly.
> >
> > Can anyone tell me how do I diagnose this problem?
> >
> > Thanks,
> > Jim
>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-11 11:06:58
|
I wasn't sure if we should use ADAPTIVE, since we are not using the CA part.
I will try with "-o ADAPTIVE" now.
On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson <to...@pr...> wrote:
It should be ADAPTIVE.
The documentation says:
bin/ejbca.sh ca importcrl --help
---
ADAPTIVE means that missing certficates will be replaced by dummy
certificates to cater for proper CRLs for missing certificates.
---
Regards,
Tomas
On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> Hi,
>
> I've been looking at the code in CaImportCRLCommand, and I am wondering
> if, when the command is run in LENIENT mode and it doesn't find the cert
> in the database (and outputs the "missing in database" error message),
> what is supposed to happen?
>
> Mind you, the only error messages I am seeing are the "missing in
> database" error messages. I don't see any of the other error messages
> that are in/from CaImportCRLCommand.java.
>
> I am wondering if, when the command is run with "-o LENIENT", and when
> it outputs the "missing in database" error message, then the ejbca.sh is
> basically not doing anything (and not storing anything into the
> CertificateData table)?
>
> From the lack of any other error messages (vs. what I see in the
> CaImportCRLCommand.java code), that is what it looks like is happening.
>
> So given I am using EJBCA for OCSP responder only, is running the
> ejbca.sh in the "-o LENIENT" mode correct?
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> I've been doing a bunch of testing with trying to import the large CRL,
> and wanted to summarize:
>
> - The ejbca.sh runs fine - it completes without outputting an errors,
> except for outputting a bunch of the "cert xxxx missing in database"
> messages.
>
> - When the import completes, and I check the Mysql database, the crlData
> (the full PEM of the CRL) is in the database, but there is no rows in
> the CertificateData table for that CA at all.
>
> - I even figured out how to get the ejbca.sh to log using
> log4j.properties, and it output a bunch of logging, mainly the missing
> in database errors, but other than that I didn't see any obvious errors.
>
>
> So, bottom line is that even though the ejbca.sh seems to run without
> errors (other than the missing in database errors), the import is
> causing the CRL PEM to be stored in the database, but the
> CertificateData table is not being populated at all for that CA.
>
>
> Jim
>
>
>
> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> Ok, I found Tomas' original message... it was in my Spam folder....
>
> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
> ways. I think it is running without outputting any errors, but still,
> no rows are appearing with that issuerDN at all.
>
>
>
>
> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
> Hi,
>
> OCSP responses does not have anything to do with the CRL itself. Once
> the CRL has been imported each certificate status is in its own record
> in the CertificateData table. This is where status is read when reponses
> are created.
> If there are issues, it is in that case related to the import of the CRL.
>
> Verify that certificate records are created in CertificateData when
> importing the CRL.
> Also, a debug log when making OCSP request will tell exact details of
> what the responder finds in the database or not.
>
> Regards,
> Tomas
>
>
> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> [Re-sending, as this (handling of large CRLs) is still a problem and
> is still not working.]
>
>
>
> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote:
>
>
> Hi,
>
> I think that I've found how to set/change the max_allowed_packet on
> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> standalone/configuration/standalone.xml to:
>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>
> However, when I test the OCSP Responder with a serial number (and CA
> cert) from the CA with the large (>71MB) CRL file, I am still
> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
> cert serial number.
>
> NOTE, again, that I can test with certs (and CA cert) that have
> smaller CRL files, and those tests work.
>
> So it is only when testing with the large CRL file that is failing
> with the OCSP Responder.
>
> This seems like a potential BUG for EJBCA OCSP Responder?
>
> Please advise.
>
> Thanks,
> Jim
> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote:
>
>
> Hi,
>
> BTW, just to check, I have actually gone through the process of
> getting the crlData from the MySQL using mysql client (I had to add
> the --max_allowed-packet=100M to the mysql command) and I was able
> to actually get the PEM from the Mysql database that way.
>
> So, I KNOW that the entire CRL is in the Mysql database, and yet
> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
> it says it cannot find the revocation.
>
> OCSP Request Data:
> Version: 1 (0x0)
> Requestor List:
> Certificate ID:
> Hash Algorithm: sha1
> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> Serial Number: 2FFFF4
> Request Extensions:
> OCSP Nonce:
> 04103054A60A55E389B239A55228033D8F62
> Response verify OK
> 0x2FFFF4: unknown
> This Update: Jun 29 19:31:02 2019 GMT
>
>
> In the server log:
>
> 2019-06-29 15:31:02,150 DEBUG
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> (default task-1) Unable to find revocation information for
> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>
>
> Honestly, I am no 100% sure that the problem is that
> max_allowe_packet is too small on the mysql client side, but that is
> my best guess.
>
> What I AM sure is that the CRL is in mysql, but when I do a test
> with OCSP Responder, it cannot find the cert serial number in the CRL.
>
> Jim
>
>
> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote:
>
>
> Hi Tomas,
>
> I already had the max_allowed_packet configured on the Mysql
> server-side (in /etc/my.cnf).
>
> So as I said, when testing with mysql (client) I had to include
> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>
> mysql -u root -p --max_allowed_packet=100M
>
> If I did NOT include the "--max_allowed_packet=100M" on that command
> line, then when I did a "select" on the field base64Crl, I would get
> an error.
>
> I suspect that EJBCA is acting as a Mysql client, so, in order to
> work with such large CRLs, there needs to be some way to set the
> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>
> How can I do that?
>
> If that can't be done, then I think that when EJBCA tries to
> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>
> Thanks,
> Jim
>
> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> You configure max allowed packet size in you MySQL configuration on the
> MySQL server.
>
> /Tomas
>
> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> > I am not 100% sure, but I think I know what might be causing this
> problem.
> >
> > I did a test, where I just tried to log into mysql and do the
> following
> > select:
> >
> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
> CRLData
> > where cRLNumber=3167;
> >
> > and I got an error saying that the result was more
> than max_allowed_packet.
> >
> > So then I did the same thing, but for the mysql command I used:
> >
> > mysql -u root -p --max_allowed_packet=100M
> >
> > and then when I did that same select, I was able to output the entire
> > CRL as a PEM.
> >
> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
> > when I run the "openssl ocsp" command, EJBCA is querying the base64CRL
> > but the result is too large and so that is causing the query to fail.
> >
> > So the question now is: How can I cause EJBCA OCSP Responder to run
> > with a larger max_allowed_packet so that the query succeeds (and then
> > hopefully EJBCA OCSP Responder will be able to check the revocation
> > properly)?
> >
> > Does anyone know how that can be done?
> >
> > Thanks,
> > Jim
> >
> >
> >
> >
> >
> >
> >
> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya...
> <mailto:oh...@ya...>> wrote:
> >
> >
> > Hi,
> >
> > As discussed in an earlier thread, it looks like I was able to
> import a
> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
> I am
> > doing testing, using "openssl ocsp" to send requests into the
> EJBCA OCSP
> > Responder:
> >
> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> -req_text -serial
> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >
> > But it looks like no matter what serial number I use, whether the
> serial
> > number is in the CRL or not, returns a response like:
> >
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 37...00
> > Issuer Key Hash: 39...E4
> > Serial Number: 2FFFF9
> > Request Extensions:
> > OCSP Nonce:
> > 041078B1311984A110B5E19C8DC4895F485B
> > Response verify OK
> > 0x2FFFF9: unknown
> > This Update: Jun 28 18:56:20 2019 GMT
> >
> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
> > find any revoked certs in the CRL?
> >
> >
> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> properly, in
> > general, because I can do the same test against a smaller CRL in the
> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
> > "Revoked" response:
> >
> > [root@ejbca <mailto:root@ejbca> testocspresponder]# openssl ocsp
> -issuer simpleca.crt
> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
> > -host 192.168.0.28:80
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> > Serial Number: 8486394C03E1F5D9
> > Request Extensions:
> > OCSP Nonce:
> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> > Response verify OK
> > 0x008486394c03e1f5d9: revoked
> > This Update: Jun 28 19:12:48 2019 GMT
> > Reason: unspecified
> > Revocation Time: May 26 12:30:44 2019 GMT
> >
> > So it seems like the EJBCA OCSP Responder is either not processing the
> > large CRL properly, or maybe the import didn't import the large
> CRL into
> > the OCSP Responder correctly.
> >
> > Can anyone tell me how do I diagnose this problem?
> >
> > Thanks,
> > Jim
>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: <oh...@ya...> - 2019-07-11 11:05:59
|
Hi,
I wasn't sure if I should use ADAPTIVE, since we are only using EJBCA for the OCSP Responder, I wasn't sure if we wanted to have the dummy certs added.
On Thursday, July 11, 2019, 7:42:14 AM UTC, Tomas Gustavsson <to...@pr...> wrote:
It should be ADAPTIVE.
The documentation says:
bin/ejbca.sh ca importcrl --help
---
ADAPTIVE means that missing certficates will be replaced by dummy
certificates to cater for proper CRLs for missing certificates.
---
Regards,
Tomas
On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote:
> Hi,
>
> I've been looking at the code in CaImportCRLCommand, and I am wondering
> if, when the command is run in LENIENT mode and it doesn't find the cert
> in the database (and outputs the "missing in database" error message),
> what is supposed to happen?
>
> Mind you, the only error messages I am seeing are the "missing in
> database" error messages. I don't see any of the other error messages
> that are in/from CaImportCRLCommand.java.
>
> I am wondering if, when the command is run with "-o LENIENT", and when
> it outputs the "missing in database" error message, then the ejbca.sh is
> basically not doing anything (and not storing anything into the
> CertificateData table)?
>
> From the lack of any other error messages (vs. what I see in the
> CaImportCRLCommand.java code), that is what it looks like is happening.
>
> So given I am using EJBCA for OCSP responder only, is running the
> ejbca.sh in the "-o LENIENT" mode correct?
>
> Jim
>
>
>
> On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> I've been doing a bunch of testing with trying to import the large CRL,
> and wanted to summarize:
>
> - The ejbca.sh runs fine - it completes without outputting an errors,
> except for outputting a bunch of the "cert xxxx missing in database"
> messages.
>
> - When the import completes, and I check the Mysql database, the crlData
> (the full PEM of the CRL) is in the database, but there is no rows in
> the CertificateData table for that CA at all.
>
> - I even figured out how to get the ejbca.sh to log using
> log4j.properties, and it output a bunch of logging, mainly the missing
> in database errors, but other than that I didn't see any obvious errors.
>
>
> So, bottom line is that even though the ejbca.sh seems to run without
> errors (other than the missing in database errors), the import is
> causing the CRL PEM to be stored in the database, but the
> CertificateData table is not being populated at all for that CA.
>
>
> Jim
>
>
>
> On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> Ok, I found Tomas' original message... it was in my Spam folder....
>
> I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of
> ways. I think it is running without outputting any errors, but still,
> no rows are appearing with that issuerDN at all.
>
>
>
>
> On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
> Hi,
>
> OCSP responses does not have anything to do with the CRL itself. Once
> the CRL has been imported each certificate status is in its own record
> in the CertificateData table. This is where status is read when reponses
> are created.
> If there are issues, it is in that case related to the import of the CRL.
>
> Verify that certificate records are created in CertificateData when
> importing the CRL.
> Also, a debug log when making OCSP request will tell exact details of
> what the responder finds in the database or not.
>
> Regards,
> Tomas
>
>
> On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
> [Re-sending, as this (handling of large CRLs) is still a problem and
> is still not working.]
>
>
>
> On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote:
>
>
> Hi,
>
> I think that I've found how to set/change the max_allowed_packet on
> the client (i.e., EJBCA) side, by editing the Mysql URL in the
> standalone/configuration/standalone.xml to:
>
> <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
>
> However, when I test the OCSP Responder with a serial number (and CA
> cert) from the CA with the large (>71MB) CRL file, I am still
> getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the
> cert serial number.
>
> NOTE, again, that I can test with certs (and CA cert) that have
> smaller CRL files, and those tests work.
>
> So it is only when testing with the large CRL file that is failing
> with the OCSP Responder.
>
> This seems like a potential BUG for EJBCA OCSP Responder?
>
> Please advise.
>
> Thanks,
> Jim
> On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote:
>
>
> Hi,
>
> BTW, just to check, I have actually gone through the process of
> getting the crlData from the MySQL using mysql client (I had to add
> the --max_allowed-packet=100M to the mysql command) and I was able
> to actually get the PEM from the Mysql database that way.
>
> So, I KNOW that the entire CRL is in the Mysql database, and yet
> when I run an "openssl ocsp" test against the EJBCA OCSP Responder,
> it says it cannot find the revocation.
>
> OCSP Request Data:
> Version: 1 (0x0)
> Requestor List:
> Certificate ID:
> Hash Algorithm: sha1
> Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
> Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
> Serial Number: 2FFFF4
> Request Extensions:
> OCSP Nonce:
> 04103054A60A55E389B239A55228033D8F62
> Response verify OK
> 0x2FFFF4: unknown
> This Update: Jun 29 19:31:02 2019 GMT
>
>
> In the server log:
>
> 2019-06-29 15:31:02,150 DEBUG
> [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean]
> (default task-1) Unable to find revocation information for
> certificate with serial '2ffff4' from issuer 'CN=XXX....'
>
>
> Honestly, I am no 100% sure that the problem is that
> max_allowe_packet is too small on the mysql client side, but that is
> my best guess.
>
> What I AM sure is that the CRL is in mysql, but when I do a test
> with OCSP Responder, it cannot find the cert serial number in the CRL.
>
> Jim
>
>
> On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote:
>
>
> Hi Tomas,
>
> I already had the max_allowed_packet configured on the Mysql
> server-side (in /etc/my.cnf).
>
> So as I said, when testing with mysql (client) I had to include
> "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
>
> mysql -u root -p --max_allowed_packet=100M
>
> If I did NOT include the "--max_allowed_packet=100M" on that command
> line, then when I did a "select" on the field base64Crl, I would get
> an error.
>
> I suspect that EJBCA is acting as a Mysql client, so, in order to
> work with such large CRLs, there needs to be some way to set the
> equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
>
> How can I do that?
>
> If that can't be done, then I think that when EJBCA tries to
> retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
>
> Thanks,
> Jim
>
> On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> You configure max allowed packet size in you MySQL configuration on the
> MySQL server.
>
> /Tomas
>
> On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> > I am not 100% sure, but I think I know what might be causing this
> problem.
> >
> > I did a test, where I just tried to log into mysql and do the
> following
> > select:
> >
> > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from
> CRLData
> > where cRLNumber=3167;
> >
> > and I got an error saying that the result was more
> than max_allowed_packet.
> >
> > So then I did the same thing, but for the mysql command I used:
> >
> > mysql -u root -p --max_allowed_packet=100M
> >
> > and then when I did that same select, I was able to output the entire
> > CRL as a PEM.
> >
> > So, I am theorizing that that is what is happening with EJBCA, i.e.,
> > when I run the "openssl ocsp" command, EJBCA is querying the base64CRL
> > but the result is too large and so that is causing the query to fail.
> >
> > So the question now is: How can I cause EJBCA OCSP Responder to run
> > with a larger max_allowed_packet so that the query succeeds (and then
> > hopefully EJBCA OCSP Responder will be able to check the revocation
> > properly)?
> >
> > Does anyone know how that can be done?
> >
> > Thanks,
> > Jim
> >
> >
> >
> >
> >
> >
> >
> > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya...
> <mailto:oh...@ya...>> wrote:
> >
> >
> > Hi,
> >
> > As discussed in an earlier thread, it looks like I was able to
> import a
> > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now,
> I am
> > doing testing, using "openssl ocsp" to send requests into the
> EJBCA OCSP
> > Responder:
> >
> > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT>
> -req_text -serial
> > 0x<CERT_SERIAL> -host <HOST:PORT>
> >
> > But it looks like no matter what serial number I use, whether the
> serial
> > number is in the CRL or not, returns a response like:
> >
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 37...00
> > Issuer Key Hash: 39...E4
> > Serial Number: 2FFFF9
> > Request Extensions:
> > OCSP Nonce:
> > 041078B1311984A110B5E19C8DC4895F485B
> > Response verify OK
> > 0x2FFFF9: unknown
> > This Update: Jun 28 18:56:20 2019 GMT
> >
> > i.e., it looks like the EJBCA OCSP Responder is not able to actually
> > find any revoked certs in the CRL?
> >
> >
> > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning
> properly, in
> > general, because I can do the same test against a smaller CRL in the
> > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
> > "Revoked" response:
> >
> > [root@ejbca <mailto:root@ejbca> testocspresponder]# openssl ocsp
> -issuer simpleca.crt
> > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
> > -host 192.168.0.28:80
> > OCSP Request Data:
> > Version: 1 (0x0)
> > Requestor List:
> > Certificate ID:
> > Hash Algorithm: sha1
> > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
> > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> > Serial Number: 8486394C03E1F5D9
> > Request Extensions:
> > OCSP Nonce:
> > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> > Response verify OK
> > 0x008486394c03e1f5d9: revoked
> > This Update: Jun 28 19:12:48 2019 GMT
> > Reason: unspecified
> > Revocation Time: May 26 12:30:44 2019 GMT
> >
> > So it seems like the EJBCA OCSP Responder is either not processing the
> > large CRL properly, or maybe the import didn't import the large
> CRL into
> > the OCSP Responder correctly.
> >
> > Can anyone tell me how do I diagnose this problem?
> >
> > Thanks,
> > Jim
>
> >
> >
> > _______________________________________________
> > Ejbca-develop mailing list
> > Ejb...@li...
> <mailto:Ejb...@li...>
> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> >
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
|
From: Tomas G. <to...@pr...> - 2019-07-11 07:41:46
|
It should be ADAPTIVE. The documentation says: bin/ejbca.sh ca importcrl --help --- ADAPTIVE means that missing certficates will be replaced by dummy certificates to cater for proper CRLs for missing certificates. --- Regards, Tomas On 2019-07-11 08:05, ohaya--- via Ejbca-develop wrote: > Hi, > > I've been looking at the code in CaImportCRLCommand, and I am wondering > if, when the command is run in LENIENT mode and it doesn't find the cert > in the database (and outputs the "missing in database" error message), > what is supposed to happen? > > Mind you, the only error messages I am seeing are the "missing in > database" error messages. I don't see any of the other error messages > that are in/from CaImportCRLCommand.java. > > I am wondering if, when the command is run with "-o LENIENT", and when > it outputs the "missing in database" error message, then the ejbca.sh is > basically not doing anything (and not storing anything into the > CertificateData table)? > > From the lack of any other error messages (vs. what I see in the > CaImportCRLCommand.java code), that is what it looks like is happening. > > So given I am using EJBCA for OCSP responder only, is running the > ejbca.sh in the "-o LENIENT" mode correct? > > Jim > > > > On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop > <ejb...@li...> wrote: > > > Hi, > > I've been doing a bunch of testing with trying to import the large CRL, > and wanted to summarize: > > - The ejbca.sh runs fine - it completes without outputting an errors, > except for outputting a bunch of the "cert xxxx missing in database" > messages. > > - When the import completes, and I check the Mysql database, the crlData > (the full PEM of the CRL) is in the database, but there is no rows in > the CertificateData table for that CA at all. > > - I even figured out how to get the ejbca.sh to log using > log4j.properties, and it output a bunch of logging, mainly the missing > in database errors, but other than that I didn't see any obvious errors. > > > So, bottom line is that even though the ejbca.sh seems to run without > errors (other than the missing in database errors), the import is > causing the CRL PEM to be stored in the database, but the > CertificateData table is not being populated at all for that CA. > > > Jim > > > > On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop > <ejb...@li...> wrote: > > > Hi, > > Ok, I found Tomas' original message... it was in my Spam folder.... > > I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of > ways. I think it is running without outputting any errors, but still, > no rows are appearing with that issuerDN at all. > > > > > On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson > <to...@pr...> wrote: > > > Hi, > > OCSP responses does not have anything to do with the CRL itself. Once > the CRL has been imported each certificate status is in its own record > in the CertificateData table. This is where status is read when reponses > are created. > If there are issues, it is in that case related to the import of the CRL. > > Verify that certificate records are created in CertificateData when > importing the CRL. > Also, a debug log when making OCSP request will tell exact details of > what the responder finds in the database or not. > > Regards, > Tomas > > > On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop > <ejb...@li...> wrote: > > [Re-sending, as this (handling of large CRLs) is still a problem and > is still not working.] > > > > On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote: > > > Hi, > > I think that I've found how to set/change the max_allowed_packet on > the client (i.e., EJBCA) side, by editing the Mysql URL in the > standalone/configuration/standalone.xml to: > > <connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url> > > However, when I test the OCSP Responder with a serial number (and CA > cert) from the CA with the large (>71MB) CRL file, I am still > getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the > cert serial number. > > NOTE, again, that I can test with certs (and CA cert) that have > smaller CRL files, and those tests work. > > So it is only when testing with the large CRL file that is failing > with the OCSP Responder. > > This seems like a potential BUG for EJBCA OCSP Responder? > > Please advise. > > Thanks, > Jim > On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote: > > > Hi, > > BTW, just to check, I have actually gone through the process of > getting the crlData from the MySQL using mysql client (I had to add > the --max_allowed-packet=100M to the mysql command) and I was able > to actually get the PEM from the Mysql database that way. > > So, I KNOW that the entire CRL is in the Mysql database, and yet > when I run an "openssl ocsp" test against the EJBCA OCSP Responder, > it says it cannot find the revocation. > > OCSP Request Data: > Version: 1 (0x0) > Requestor List: > Certificate ID: > Hash Algorithm: sha1 > Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600 > Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4 > Serial Number: 2FFFF4 > Request Extensions: > OCSP Nonce: > 04103054A60A55E389B239A55228033D8F62 > Response verify OK > 0x2FFFF4: unknown > This Update: Jun 29 19:31:02 2019 GMT > > > In the server log: > > 2019-06-29 15:31:02,150 DEBUG > [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean] > (default task-1) Unable to find revocation information for > certificate with serial '2ffff4' from issuer 'CN=XXX....' > > > Honestly, I am no 100% sure that the problem is that > max_allowe_packet is too small on the mysql client side, but that is > my best guess. > > What I AM sure is that the CRL is in mysql, but when I do a test > with OCSP Responder, it cannot find the cert serial number in the CRL. > > Jim > > > On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote: > > > Hi Tomas, > > I already had the max_allowed_packet configured on the Mysql > server-side (in /etc/my.cnf). > > So as I said, when testing with mysql (client) I had to include > "--max_allowed_packet=100M" on the mysql (client) command line, e.g.: > > mysql -u root -p --max_allowed_packet=100M > > If I did NOT include the "--max_allowed_packet=100M" on that command > line, then when I did a "select" on the field base64Crl, I would get > an error. > > I suspect that EJBCA is acting as a Mysql client, so, in order to > work with such large CRLs, there needs to be some way to set the > equivalent of the "--max_allowed_packet=100M" on the EJBCA side. > > How can I do that? > > If that can't be done, then I think that when EJBCA tries to > retrieve the CRL from the Mysql, it (EJBCA) will see the same error. > > Thanks, > Jim > > On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson > <to...@pr...> wrote: > > > > You configure max allowed packet size in you MySQL configuration on the > MySQL server. > > /Tomas > > On 2019-06-29 02:19, o haya via Ejbca-develop wrote: > > I am not 100% sure, but I think I know what might be causing this > problem. > > > > I did a test, where I just tried to log into mysql and do the > following > > select: > > > > select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from > CRLData > > where cRLNumber=3167; > > > > and I got an error saying that the result was more > than max_allowed_packet. > > > > So then I did the same thing, but for the mysql command I used: > > > > mysql -u root -p --max_allowed_packet=100M > > > > and then when I did that same select, I was able to output the entire > > CRL as a PEM. > > > > So, I am theorizing that that is what is happening with EJBCA, i.e., > > when I run the "openssl ocsp" command, EJBCA is querying the base64CRL > > but the result is too large and so that is causing the query to fail. > > > > So the question now is: How can I cause EJBCA OCSP Responder to run > > with a larger max_allowed_packet so that the query succeeds (and then > > hopefully EJBCA OCSP Responder will be able to check the revocation > > properly)? > > > > Does anyone know how that can be done? > > > > Thanks, > > Jim > > > > > > > > > > > > > > > > On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya... > <mailto:oh...@ya...>> wrote: > > > > > > Hi, > > > > As discussed in an earlier thread, it looks like I was able to > import a > > large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now, > I am > > doing testing, using "openssl ocsp" to send requests into the > EJBCA OCSP > > Responder: > > > > openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT> > -req_text -serial > > 0x<CERT_SERIAL> -host <HOST:PORT> > > > > But it looks like no matter what serial number I use, whether the > serial > > number is in the CRL or not, returns a response like: > > > > OCSP Request Data: > > Version: 1 (0x0) > > Requestor List: > > Certificate ID: > > Hash Algorithm: sha1 > > Issuer Name Hash: 37...00 > > Issuer Key Hash: 39...E4 > > Serial Number: 2FFFF9 > > Request Extensions: > > OCSP Nonce: > > 041078B1311984A110B5E19C8DC4895F485B > > Response verify OK > > 0x2FFFF9: unknown > > This Update: Jun 28 18:56:20 2019 GMT > > > > i.e., it looks like the EJBCA OCSP Responder is not able to actually > > find any revoked certs in the CRL? > > > > > > FYI, I *KNOW* that the EJBCA OCSP Responder is functioning > properly, in > > general, because I can do the same test against a smaller CRL in the > > same EJBCA OCSP Responder, for CA simpleca, and I can get an actual > > "Revoked" response: > > > > [root@ejbca <mailto:root@ejbca> testocspresponder]# openssl ocsp > -issuer simpleca.crt > > -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text > > -host 192.168.0.28:80 > > OCSP Request Data: > > Version: 1 (0x0) > > Requestor List: > > Certificate ID: > > Hash Algorithm: sha1 > > Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55 > > Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E > > Serial Number: 8486394C03E1F5D9 > > Request Extensions: > > OCSP Nonce: > > 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F > > Response verify OK > > 0x008486394c03e1f5d9: revoked > > This Update: Jun 28 19:12:48 2019 GMT > > Reason: unspecified > > Revocation Time: May 26 12:30:44 2019 GMT > > > > So it seems like the EJBCA OCSP Responder is either not processing the > > large CRL properly, or maybe the import didn't import the large > CRL into > > the OCSP Responder correctly. > > > > Can anyone tell me how do I diagnose this problem? > > > > Thanks, > > Jim > > > > > > > _______________________________________________ > > Ejbca-develop mailing list > > Ejb...@li... > <mailto:Ejb...@li...> > > https://lists.sourceforge.net/lists/listinfo/ejbca-develop > > > > > _______________________________________________ > Ejbca-develop mailing list > Ejb...@li... > <mailto:Ejb...@li...> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop > > _______________________________________________ > Ejbca-develop mailing list > Ejb...@li... > <mailto:Ejb...@li...> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop > _______________________________________________ > Ejbca-develop mailing list > Ejb...@li... > <mailto:Ejb...@li...> > https://lists.sourceforge.net/lists/listinfo/ejbca-develop > > > _______________________________________________ > Ejbca-develop mailing list > Ejb...@li... > https://lists.sourceforge.net/lists/listinfo/ejbca-develop > |
|
From: <oh...@ya...> - 2019-07-11 06:05:44
|
Hi,
I've been looking at the code in CaImportCRLCommand, and I am wondering if, when the command is run in LENIENT mode and it doesn't find the cert in the database (and outputs the "missing in database" error message), what is supposed to happen?
Mind you, the only error messages I am seeing are the "missing in database" error messages. I don't see any of the other error messages that are in/from CaImportCRLCommand.java.
I am wondering if, when the command is run with "-o LENIENT", and when it outputs the "missing in database" error message, then the ejbca.sh is basically not doing anything (and not storing anything into the CertificateData table)?
>From the lack of any other error messages (vs. what I see in the CaImportCRLCommand.java code), that is what it looks like is happening.
So given I am using EJBCA for OCSP responder only, is running the ejbca.sh in the "-o LENIENT" mode correct?
Jim
On Thursday, July 11, 2019, 5:43:37 AM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
I've been doing a bunch of testing with trying to import the large CRL, and wanted to summarize:
- The ejbca.sh runs fine - it completes without outputting an errors, except for outputting a bunch of the "cert xxxx missing in database" messages.
- When the import completes, and I check the Mysql database, the crlData (the full PEM of the CRL) is in the database, but there is no rows in the CertificateData table for that CA at all.
- I even figured out how to get the ejbca.sh to log using log4j.properties, and it output a bunch of logging, mainly the missing in database errors, but other than that I didn't see any obvious errors.
So, bottom line is that even though the ejbca.sh seems to run without errors (other than the missing in database errors), the import is causing the CRL PEM to be stored in the database, but the CertificateData table is not being populated at all for that CA.
Jim
On Wednesday, July 10, 2019, 7:40:16 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
Hi,
Ok, I found Tomas' original message... it was in my Spam folder....
I also tried re-running the "./ejbca.sh ca importcrl" command a bunch of ways. I think it is running without outputting any errors, but still, no rows are appearing with that issuerDN at all.
On Tuesday, July 9, 2019, 10:45:01 AM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
OCSP responses does not have anything to do with the CRL itself. Once the CRL has been imported each certificate status is in its own record in the CertificateData table. This is where status is read when reponses are created.
If there are issues, it is in that case related to the import of the CRL.
Verify that certificate records are created in CertificateData when importing the CRL.
Also, a debug log when making OCSP request will tell exact details of what the responder finds in the database or not.
Regards,
Tomas
On July 8, 2019 4:59:06 PM GMT+02:00, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
[Re-sending, as this (handling of large CRLs) is still a problem and is still not working.]
On Monday, July 1, 2019, 12:47:23 PM UTC, <oh...@ya...> wrote:
Hi,
I think that I've found how to set/change the max_allowed_packet on the client (i.e., EJBCA) side, by editing the Mysql URL in the standalone/configuration/standalone.xml to:
<connection-url>jdbc:mysql://127.0.0.1:3306/ejbca?maxAllowedPacket=104857600</connection-url>
However, when I test the OCSP Responder with a serial number (and CA cert) from the CA with the large (>71MB) CRL file, I am still getting "unknown" and EJBCA OCSP Responder is still NOT FINDING the cert serial number.
NOTE, again, that I can test with certs (and CA cert) that have smaller CRL files, and those tests work.
So it is only when testing with the large CRL file that is failing with the OCSP Responder.
This seems like a potential BUG for EJBCA OCSP Responder?
Please advise.
Thanks,
Jim
On Saturday, June 29, 2019, 3:36:18 PM EDT, <oh...@ya...> wrote:
Hi,
BTW, just to check, I have actually gone through the process of getting the crlData from the MySQL using mysql client (I had to add the --max_allowed-packet=100M to the mysql command) and I was able to actually get the PEM from the Mysql database that way.
So, I KNOW that the entire CRL is in the Mysql database, and yet when I run an "openssl ocsp" test against the EJBCA OCSP Responder, it says it cannot find the revocation.
OCSP Request Data:
Version: 1 (0x0)
Requestor List:
Certificate ID:
Hash Algorithm: sha1
Issuer Name Hash: 37E3972FD58A03EEC31D8DF5F6F03F6AE4F5C600
Issuer Key Hash: 3929A80864E28F83AA816147119133F85A37A6E4
Serial Number: 2FFFF4
Request Extensions:
OCSP Nonce:
04103054A60A55E389B239A55228033D8F62
Response verify OK
0x2FFFF4: unknown
This Update: Jun 29 19:31:02 2019 GMT
In the server log:
2019-06-29 15:31:02,150 DEBUG [org.cesecore.certificates.ocsp.OcspResponseGeneratorSessionBean] (default task-1) Unable to find revocation information for certificate with serial '2ffff4' from issuer 'CN=XXX....'
Honestly, I am no 100% sure that the problem is that max_allowe_packet is too small on the mysql client side, but that is my best guess.
What I AM sure is that the CRL is in mysql, but when I do a test with OCSP Responder, it cannot find the cert serial number in the CRL.
Jim
On Saturday, June 29, 2019, 4:52:20 AM EDT, <oh...@ya...> wrote:
Hi Tomas,
I already had the max_allowed_packet configured on the Mysql server-side (in /etc/my.cnf).
So as I said, when testing with mysql (client) I had to include "--max_allowed_packet=100M" on the mysql (client) command line, e.g.:
mysql -u root -p --max_allowed_packet=100M
If I did NOT include the "--max_allowed_packet=100M" on that command line, then when I did a "select" on the field base64Crl, I would get an error.
I suspect that EJBCA is acting as a Mysql client, so, in order to work with such large CRLs, there needs to be some way to set the equivalent of the "--max_allowed_packet=100M" on the EJBCA side.
How can I do that?
If that can't be done, then I think that when EJBCA tries to retrieve the CRL from the Mysql, it (EJBCA) will see the same error.
Thanks,
Jim
On Saturday, June 29, 2019, 3:25:47 AM EDT, Tomas Gustavsson <to...@pr...> wrote:
You configure max allowed packet size in you MySQL configuration on the
MySQL server.
/Tomas
On 2019-06-29 02:19, o haya via Ejbca-develop wrote:
> I am not 100% sure, but I think I know what might be causing this problem.
>
> I did a test, where I just tried to log into mysql and do the following
> select:
>
> select base64Crl INTO OUTFILE '/var/lib/mysql-files/xx.out' from CRLData
> where cRLNumber=3167;
>
> and I got an error saying that the result was more than max_allowed_packet.
>
> So then I did the same thing, but for the mysql command I used:
>
> mysql -u root -p --max_allowed_packet=100M
>
> and then when I did that same select, I was able to output the entire
> CRL as a PEM.
>
> So, I am theorizing that that is what is happening with EJBCA, i.e.,
> when I run the "openssl ocsp" command, EJBCA is querying the base64CRL
> but the result is too large and so that is causing the query to fail.
>
> So the question now is: How can I cause EJBCA OCSP Responder to run
> with a larger max_allowed_packet so that the query succeeds (and then
> hopefully EJBCA OCSP Responder will be able to check the revocation
> properly)?
>
> Does anyone know how that can be done?
>
> Thanks,
> Jim
>
>
>
>
>
>
>
> On Friday, June 28, 2019, 07:16:39 PM UTC, o haya <oh...@ya...> wrote:
>
>
> Hi,
>
> As discussed in an earlier thread, it looks like I was able to import a
> large (> 71MB) CRL file into EJBCA (as an OCSP Responder), so now, I am
> doing testing, using "openssl ocsp" to send requests into the EJBCA OCSP
> Responder:
>
> openssl ocsp -issuer <CA_CERT> -VAfile <SIGNING_CERT> -req_text -serial
> 0x<CERT_SERIAL> -host <HOST:PORT>
>
> But it looks like no matter what serial number I use, whether the serial
> number is in the CRL or not, returns a response like:
>
> OCSP Request Data:
> Version: 1 (0x0)
> Requestor List:
> Certificate ID:
> Hash Algorithm: sha1
> Issuer Name Hash: 37...00
> Issuer Key Hash: 39...E4
> Serial Number: 2FFFF9
> Request Extensions:
> OCSP Nonce:
> 041078B1311984A110B5E19C8DC4895F485B
> Response verify OK
> 0x2FFFF9: unknown
> This Update: Jun 28 18:56:20 2019 GMT
>
> i.e., it looks like the EJBCA OCSP Responder is not able to actually
> find any revoked certs in the CRL?
>
>
> FYI, I *KNOW* that the EJBCA OCSP Responder is functioning properly, in
> general, because I can do the same test against a smaller CRL in the
> same EJBCA OCSP Responder, for CA simpleca, and I can get an actual
> "Revoked" response:
>
> [root@ejbca testocspresponder]# openssl ocsp -issuer simpleca.crt
> -VAfile simplecabinding2.crt -serial 0x008486394c03e1f5d9 -req_text
> -host 192.168.0.28:80
> OCSP Request Data:
> Version: 1 (0x0)
> Requestor List:
> Certificate ID:
> Hash Algorithm: sha1
> Issuer Name Hash: 0C16107310427EA4ADB3C6436915CE44A15FFE55
> Issuer Key Hash: E2533BF85F8C7CA60A411BF5458B2DC3B5232B6E
> Serial Number: 8486394C03E1F5D9
> Request Extensions:
> OCSP Nonce:
> 04109A09CC1C56D26AA73CB4C3FB1EC7CF1F
> Response verify OK
> 0x008486394c03e1f5d9: revoked
> This Update: Jun 28 19:12:48 2019 GMT
> Reason: unspecified
> Revocation Time: May 26 12:30:44 2019 GMT
>
> So it seems like the EJBCA OCSP Responder is either not processing the
> large CRL properly, or maybe the import didn't import the large CRL into
> the OCSP Responder correctly.
>
> Can anyone tell me how do I diagnose this problem?
>
> Thanks,
> Jim
>
>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
>
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
106 messages has been excluded from this view by a project administrator.
