Menu
▾
▴
Re: [Ejbca-develop] EJBCA OCSP Responder does not appear to
|
From: <oh...@ya...> - 2019-07-20 15:45:28
|
Hi,
FYI, I got the EJBCA installed on a Redhat 7.4 instance on AWS, and I just started testing with that same CRL, and unfortunately, it is not doing very well... actually even slower than on my earlier dev machine under VBox :(!!
The AWS machine is a t2.medium instance with 70GB disk, 4GB RAM, and 2 CPUs. I converted the tables to InnoDB and also ran the create index SQL script.
It looks like the import is only processing about 5000 entries per hour.
Here's the top:
top - 15:43:29 up 17:25, 7 users, load average: 1.70, 1.60, 0.98
Tasks: 127 total, 1 running, 126 sleeping, 0 stopped, 0 zombie
%Cpu(s): 75.4 us, 0.5 sy, 0.0 ni, 23.2 id, 0.5 wa, 0.0 hi, 0.2 si, 0.2 st
KiB Mem : 3880256 total, 103696 free, 2799736 used, 976824 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 788552 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
12115 root 20 0 3529260 1.074g 15368 S 143.0 29.0 18:35.88 java
7326 root 20 0 3290332 1.038g 21880 S 7.7 28.1 4:02.67 java
1384 mysql 20 0 1580240 231652 8260 S 3.0 6.0 0:41.18 mysqld
264 root 20 0 0 0 0 S 0.3 0.0 0:12.29 xfsaild/xvda2
1 root 20 0 125572 3952 2572 S 0.0 0.1 0:03.04 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.03 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
Tomas, you had mentioned that there were some optimizations for MySQL? Could those be the reason that your import is running so much faster than when I run the import?
Thanks,Jim
On Friday, July 19, 2019, 8:18:13 PM UTC, ohaya--- via Ejbca-develop <ejb...@li...> wrote:
I'm trying to stand up a new EJBCA instance on AWS, to try to see if I can replicate the import speed you are seeing, but ran into an error (see the new thread I just posted).
Thanks,Jim
On Friday, July 19, 2019, 6:39:52 PM UTC, Tomas Gustavsson <to...@pr...> wrote:
Hi,
I just finished importing your CRL.
996549 records in approx. 12 hours.
This is on my laptop, starting with 400K records in the database already
(i.e ending with 1408370). There is no slowdown as number of records grow).
It failed in the end storing the actual CRL due to a limitation on
packet size in my MySQL config, but that could be changed easily in
MySQL (MariaDB) config. Actually the CRL itself is not needed in the
database in order to answer OCSP queries, so there should optimally be a
switch to not import the CRL itself, only the revocation records (to
save a bit on database storage).
The import speed can with some rather simple development be made orders
of magnitude faster (you don't want a roundtrip for every record for
example).
Cheers,
Tomas
On 2019-07-19 18:33, oh...@ya... wrote:
> Started testing with the VM on an SSD and now I am getting ~500 - 550
> per minute, or about 30K per hour. That is faster than before, but
> still way slower than the 100K per hour you mentioned.
>
> Jim
>
> On Friday, July 19, 2019, 2:56:11 PM UTC, ohaya--- via Ejbca-develop
> <ejb...@li...> wrote:
>
>
> Hi,
>
> I don't have any SSDs on the machine that I use as my server... only
> 7200RPM hard drives, except for the OS drive, which is SSD. I can try
> to add an SSD (I have lots of SSDs, just not on that machine).
>
> Jim
>
> On Friday, July 19, 2019, 2:16:04 PM UTC, Tomas Gustavsson
> <to...@pr...> wrote:
>
>
>
> I have now imported 600K of the entries form the CRL. Still running at
> 25 entries per second.
>
> The logging does not normally affect performance, logging is a normal
> part. Not running an SSD for the database typically hugely affect DB
> performance. It's possible to tune logging a lot, For example disable
> logging to console and only log to the file (if your console is really
> slow). It's also possible to set the database to not synchronize on disk
> for every write, during database restore and certain operations for example.
>
> I actually have a worse performing log config than you. I have info log
> to the console and DEBUG log to file, so it's logging a lot. I have a
> SSD disk in my laptop, and 4 cores (multithreaded so shows up as 8 in
> top). But it's still a three year old laptop, so not the fastest you can
> get.
>
> Regards,
> Tomas
>
> On 2019-07-19 15:36, oh...@ya... <mailto:oh...@ya...> wrote:
>> Hi,
>>
>> I got the index done and now am running the CRL import, which has been
>> running about 1/2 hour now, but so far, only about 5500 entries
>> imported, so it doesn't look like it hugely faster than before?
>>
>> I was wondering... on the server side, I think outputting messages like
>> the following to stdout for every entry in the CRL:
>>
>> 09:32:37,008 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default
>> task-1) 2019-07-19
>>
> 09:32:37-04:00;ACCESS_CONTROL;SUCCESS;ACCESSCONTROL;CORE;ejbca;;;;resource0=/endentityprofilesrules/1/revoke_end_entity;resource1=/ra_functionality/revoke_end_entity
>> 09:32:37,025 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default
>> task-1) 2019-07-19
>>
> 09:32:37-04:00;ACCESS_CONTROL;SUCCESS;ACCESSCONTROL;CORE;ejbca;;;;resource0=/ca/1164433895
>> 09:32:37,035 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default
>> task-1) 2019-07-19
>>
> 09:32:37-04:00;CERT_REVOKED;SUCCESS;CERTIFICATE;CORE;ejbca;1164433895;8A296;***
>> Missing During CRL Import to: XXXXXXXXXXXCA_41;msg=Revoked certificate
>> for username '*** Missing During CRL Import to: XXXXXXXXXXXCA_41',
>> fp=667d63d9060a424dd67ed19bdcd25917106eba5a, revocationReason=1,
>> subjectDN 'CN=Dummy Missing in Imported CRL,SN=8A296', issuerDN
>> 'CN=XXXXXX', serialNo=8A296.
>> 09:32:37,079 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default
>> task-1) 2019-07-19
>>
> 09:32:37-04:00;ACCESS_CONTROL;SUCCESS;ACCESSCONTROL;CORE;ejbca;;;;resource0=/ca/1164433895
>> 09:32:37,115 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default
>> task-2) 2019-07-19
>>
> 09:32:37-04:00;ACCESS_CONTROL;SUCCESS;ACCESSCONTROL;CORE;ejbca;;;;resource0=/ca/1164433895
>> 09:32:37,133 INFO [org.cesecore.audit.impl.log4j.Log4jDevice] (default
>> task-2) 2019-07-19
>>
> 09:32:37-04:00;CERT_STORED;SUCCESS;CERTIFICATE;CORE;ejbca;1164433895;4B78C;***
>> Missing During CRL Import to: XXXXXXXXXXXCA_41;msg=Certificate stored
>> for username '*** Missing During CRL Import to: XXXXXXXXXXXCA_41',
>> fp=bf50e62fe1bde64cca1a4ffa050d99bc750c68b1, subjectDN 'CN=Dummy Missin
>>
>>
>> Is it possible that that logging is slowing the CRL import?
>>
>> How is your logging setup (log level) compared to what mine is?
>>
>> Other that, it just may be that my system is much slower than your
>> laptop :)??
>>
>> Jim
>>
>>
>>
>>
>> On Friday, July 19, 2019, 8:18:07 AM UTC, Tomas Gustavsson
>> <to...@pr... <mailto:to...@pr...>> wrote:
>>
>>
>> Ah, eap 7.2 corresponds to wildly 14 as base.
>>
>> Yes, run that to convert, then you should be able to create the index.
>>
>> Cheers,
>> Tomas
>>
>>
>> On July 19, 2019 10:07:01 AM GMT+02:00, ohaya--- via Ejbca-develop
>> <ejb...@li...
> <mailto:ejb...@li...>> wrote:
>>
>> Hi Tomas,
>>
>> No, I am not using Wildfly, but rather JBOSS... specifically
>> "jboss-eap-7.2".
>>
>> So I restored my snapshot from just after I had deleted all the rows
>> in CertificateData that were created before.
>>
>> And it looks like all the tables in the "ejbca" database are MyISAM.
>>
>> So I found a one-liner that will convert them all to InnoDB:
>>
>> mysql -u root -p ejbca -e "show table status where Engine='MyISAM';"
>> | awk 'NR>1 {print "ALTER TABLE "$1" ENGINE = InnoDB;"}' | mysql
>> -u root -p ejbca
>>
>> Should I run that and convert all the tables in the "ejbca" to
> InnoDB???
>>
>>
>> And then try to run the create indexes SQL script?
>>
>>
>> Thanks for all your help!
>>
>> Jim
>>
>>
> _______________________________________________
> Ejbca-develop mailing list
> Ejb...@li...
> <mailto:Ejb...@li...>
> https://lists.sourceforge.net/lists/listinfo/ejbca-develop
_______________________________________________
Ejbca-develop mailing list
Ejb...@li...
https://lists.sourceforge.net/lists/listinfo/ejbca-develop
|
