[Ejbca-develop] Around 20K queries to the DB are being produced even if there is only one successful publishing event

[Jaime H. <hab...@gm...>]

Since r32384, around 20k queries to the DB are being produced from
PublisherQueueSessionBean#plainFifoTryAlwaysLimit100EntriesOrderByTimeCreated
if it succeds at least once in publishing qeued data.

Below is a patch where it is worth noting that the 20K limit has been
removed as it doesn't make sense anymore cause the transactions are
currently circumscribed to the chunks of 100 records processed with each
call to PublisherQueueSessionBean#doChunk:


---
modules/ejbca-ejb-interface/src/org/ejbca/core/ejb/ca/publisher/PublisherQueueSessionLocal.java
(revision 32884)
+++
modules/ejbca-ejb-interface/src/org/ejbca/core/ejb/ca/publisher/PublisherQueueSessionLocal.java
(date 1564089225333)
@@ -102,12 +102,10 @@
     /**
      * Intended for use from PublishQueueProcessWorker.
      *
-     * Publishing algorithm that is a plain fifo queue, but limited to
selecting entries to republish at 100 records at a time. It will select
from the database for this particular publisher id, and process
+     * Publishing algorithm that is a plain fifo queue, but limited to
selecting entries to publish at 100 records at a time. It will select from
the database for this particular publisher, and process
      * the record that is returned one by one. The records are ordered by
date, descending so the oldest record is returned first.
      * Publishing is tried every time for every record returned, with no
limit.
      * Repeat this process as long as we actually manage to publish
something this is because when publishing starts to work we want to publish
everything in one go, if possible.
-     * However we don't want to publish more than 20000 certificates each
time, because we want to commit to the database some time as well.
-     * Now, the OCSP publisher uses a non-transactional data source so it
commits every time so...
      */
     PublishingResult
plainFifoTryAlwaysLimit100EntriesOrderByTimeCreated(AuthenticationToken
admin, BasePublisher publisher);

---
modules/ejbca-ejb-interface/src/org/ejbca/core/ejb/ca/publisher/PublishingResult.java
(revision 32884)
+++
modules/ejbca-ejb-interface/src/org/ejbca/core/ejb/ca/publisher/PublishingResult.java
(date 1564088515443)
@@ -18,8 +18,7 @@

 /**
  * Represents a return type for publishing operations. Successes and
failures are stored in sets containing the fingerprints of what's being
stored
- * due to the fact that the publisher will by default retry 20,000 times
on a failed attempt, so the "failures" will be per attempted certificate
- * and not all attempts.
+ * where the "failures" will be per attempted certificate and not all
attempts.
  *
  * @version $Id: PublishingResult.java 32392 2019-05-22 18:45:21Z
mikekushner $
  *
---
modules/ejbca-ejb/src/org/ejbca/core/ejb/ca/publisher/PublisherQueueSessionBean.java
(revision 32884)
+++
modules/ejbca-ejb/src/org/ejbca/core/ejb/ca/publisher/PublisherQueueSessionBean.java
(date 1564093297113)
@@ -278,12 +278,11 @@
         PublishingResult result = new PublishingResult();
         // Repeat this process as long as we actually manage to publish
something
         // this is because when publishing starts to work we want to
publish everything in one go, if possible.
-        // However we don't want to publish more than 20000 certificates
each time, because we want to commit to the database some time as well.
-        int totalcount = 0;
+        PublishingResult chunkResult;
         do {
-            result.append(publisherQueueSession.doChunk(admin, publisher));
-            totalcount += result.getSuccesses();
-        } while ((result.getSuccesses() > 0) && (totalcount < 20000));
+            chunkResult = publisherQueueSession.doChunk(admin, publisher);
+            result.append(chunkResult);
+        } while (chunkResult.getSuccesses() > 0);
         return result;
     }

@@ -296,7 +295,6 @@

     /**
      * @param admin
-     * @param publisherId
      * @param publisher
      *
      * @return how many publishing operations that succeeded and failed */



-- 
Jaime Hablutzel -  +51 994690880