Menu
▾
▴
hyperic-hq-discuss — Hyperic HQ SVN Commits
You can subscribe to this list here.
| 2007 |
Jan
|
Feb
|
Mar
(927) |
Apr
(419) |
May
(352) |
Jun
(431) |
Jul
(463) |
Aug
(345) |
Sep
(304) |
Oct
(596) |
Nov
(466) |
Dec
(414) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2008 |
Jan
(348) |
Feb
(313) |
Mar
(665) |
Apr
(688) |
May
(434) |
Jun
(311) |
Jul
(540) |
Aug
(554) |
Sep
(467) |
Oct
(341) |
Nov
(365) |
Dec
(272) |
| 2009 |
Jan
(386) |
Feb
(293) |
Mar
(279) |
Apr
(239) |
May
(229) |
Jun
(199) |
Jul
(186) |
Aug
(111) |
Sep
(196) |
Oct
(146) |
Nov
(116) |
Dec
(140) |
| 2010 |
Jan
(170) |
Feb
(159) |
Mar
(151) |
Apr
(161) |
May
(90) |
Jun
(56) |
Jul
(28) |
Aug
(22) |
Sep
(5) |
Oct
|
Nov
(23) |
Dec
(12) |
| 2011 |
Jan
(8) |
Feb
(8) |
Mar
(22) |
Apr
(24) |
May
(4) |
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
|
| 2012 |
Jan
(5) |
Feb
(1) |
Mar
|
Apr
(1) |
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2013 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2014 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: <bo...@hy...> - 2009-12-20 09:15:25
|
Author: bob Date: 2009-12-20 01:15:15 -0800 (Sun, 20 Dec 2009) New Revision: 14108 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14108 Modified: trunk/etc/version.properties Log: Release 4.3.0 build #1294 Modified: trunk/etc/version.properties =================================================================== --- trunk/etc/version.properties 2009-12-19 09:17:03 UTC (rev 14107) +++ trunk/etc/version.properties 2009-12-20 09:15:15 UTC (rev 14108) @@ -1,3 +1,3 @@ -#Sat Dec 19 00:26:03 PST 2009 +#Sun Dec 20 00:30:04 PST 2009 version=4.3.0 -build=1293 +build=1294 |
|
From: <bo...@hy...> - 2009-12-19 09:17:17
|
Author: bob Date: 2009-12-19 01:17:03 -0800 (Sat, 19 Dec 2009) New Revision: 14107 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14107 Modified: trunk/etc/version.properties Log: Release 4.3.0 build #1293 Modified: trunk/etc/version.properties =================================================================== --- trunk/etc/version.properties 2009-12-19 01:08:28 UTC (rev 14106) +++ trunk/etc/version.properties 2009-12-19 09:17:03 UTC (rev 14107) @@ -1,3 +1,3 @@ -#Fri Dec 18 00:24:28 PST 2009 +#Sat Dec 19 00:26:03 PST 2009 version=4.3.0 -build=1292 +build=1293 |
|
From: <no...@gi...> - 2009-12-19 01:52:13
|
Branch: refs/heads/evolution Home: http://github.com/hyperic/hqapi Commit: 71160d1e78ee5f0f153b1d1857408e510621379d http://github.com/hyperic/hqapi/commit/71160d1e78ee5f0f153b1d1857408e510621379d Author: Jennifer Hickey <jen...@sp...> Date: 2009-12-18 (Fri, 18 Dec 2009) Changed paths: M build.xml Log Message: ----------- Temporarily disabled SSL_test until SSL is re-implemented on server side Commit: 85cac1dc75b716d828ebab894b8f406867a5b98a http://github.com/hyperic/hqapi/commit/85cac1dc75b716d828ebab894b8f406867a5b98a Author: Jennifer Hickey <jen...@sp...> Date: 2009-12-18 (Fri, 18 Dec 2009) Changed paths: M hqu/hqapi1/app/ResourceController.groovy Log Message: ----------- Fixed bug caused by refactoring of AppdefEntityID Commit: bab68b1f7846c70c66c0def2a6a291c669ed282f http://github.com/hyperic/hqapi/commit/bab68b1f7846c70c66c0def2a6a291c669ed282f Author: Jennifer Hickey <jen...@sp...> Date: 2009-12-18 (Fri, 18 Dec 2009) Changed paths: M src/org/hyperic/hq/hqapi1/tools/Command.java M xsd/HQApi1.wadl Log Message: ----------- Changed default port to 8080 |
|
From: <sc...@hy...> - 2009-12-19 01:08:41
|
Author: scottmf Date: 2009-12-18 17:08:28 -0800 (Fri, 18 Dec 2009) New Revision: 14106 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14106 Modified: trunk/src/org/hyperic/hq/events/server/session/MultiConditionEvaluator.java trunk/unittest/src/org/hyperic/hq/events/server/session/MultiConditionEvaluatorTest.java Log: [HHQ-3613] removing from a collection while it is being iterated throws ConcurrentModificationException. Need to remove from the iterator itself Modified: trunk/src/org/hyperic/hq/events/server/session/MultiConditionEvaluator.java =================================================================== --- trunk/src/org/hyperic/hq/events/server/session/MultiConditionEvaluator.java 2009-12-18 09:14:07 UTC (rev 14105) +++ trunk/src/org/hyperic/hq/events/server/session/MultiConditionEvaluator.java 2009-12-19 01:08:28 UTC (rev 14106) @@ -114,7 +114,7 @@ // than System.currentTimeMillis Map.Entry entry = (Map.Entry) iter.next(); if (isExpired((AbstractEvent) entry.getValue())) { - events.remove(entry.getKey()); + iter.remove(); } } } Modified: trunk/unittest/src/org/hyperic/hq/events/server/session/MultiConditionEvaluatorTest.java =================================================================== --- trunk/unittest/src/org/hyperic/hq/events/server/session/MultiConditionEvaluatorTest.java 2009-12-18 09:14:07 UTC (rev 14105) +++ trunk/unittest/src/org/hyperic/hq/events/server/session/MultiConditionEvaluatorTest.java 2009-12-19 01:08:28 UTC (rev 14106) @@ -30,6 +30,7 @@ public void setUp() throws Exception { super.setUp(); +Thread.sleep(5000); this.executionStrategy = EasyMock.createMock(ExecutionStrategy.class); } @@ -297,13 +298,20 @@ mockEvent2.setTimestamp(System.currentTimeMillis() - (12 * 60 * 1000)); TriggerFiredEvent triggerFired2 = new TriggerFiredEvent(trigger1Id, mockEvent2); + MockEvent mockEvent3 = new MockEvent(4l, 5); + // trigger fired event occurred 12 minutes ago and is already expired + mockEvent2.setTimestamp(System.currentTimeMillis() - (12 * 60 * 1000)); + TriggerFiredEvent triggerFired3 = new TriggerFiredEvent(trigger1Id, mockEvent3); + EasyMock.replay(executionStrategy); evaluator.triggerFired(triggerFired); + // this setTimestamp() is signifying that time has passed since the last + // trigger fired + mockEvent.setTimestamp(System.currentTimeMillis() - (12 * 60 * 1000)); evaluator.triggerFired(triggerFired2); EasyMock.verify(executionStrategy); Map expectedEvents = new LinkedHashMap(); - expectedEvents.put(trigger2Id, triggerFired); assertEquals(expectedEvents, events); } |
|
From: <bo...@hy...> - 2009-12-18 09:14:26
|
Author: bob Date: 2009-12-18 01:14:07 -0800 (Fri, 18 Dec 2009) New Revision: 14105 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14105 Modified: trunk/etc/version.properties Log: Release 4.3.0 build #1292 Modified: trunk/etc/version.properties =================================================================== --- trunk/etc/version.properties 2009-12-18 05:12:55 UTC (rev 14104) +++ trunk/etc/version.properties 2009-12-18 09:14:07 UTC (rev 14105) @@ -1,3 +1,3 @@ -#Thu Dec 17 00:16:38 PST 2009 +#Fri Dec 18 00:24:28 PST 2009 version=4.3.0 -build=1291 +build=1292 |
|
From: <no...@gi...> - 2009-12-18 05:27:08
|
Branch: refs/heads/master Home: http://github.com/hyperic/hqapi Commit: 6175a74adc177c27e40ea9e607f9ac1194d939ba http://github.com/hyperic/hqapi/commit/6175a74adc177c27e40ea9e607f9ac1194d939ba Author: pnguyen <pnguyen@192.168.1.66> Date: 2009-12-17 (Thu, 17 Dec 2009) Changed paths: M src/org/hyperic/hq/hqapi1/test/AlertFireRecovery_test.java Log Message: ----------- [HQ-1903] Uncomment test case that fires a problem alert and then creates and fires a recovery alert |
|
From: <pn...@hy...> - 2009-12-18 05:13:11
|
Author: pnguyen Date: 2009-12-17 21:12:55 -0800 (Thu, 17 Dec 2009) New Revision: 14104 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14104 Modified: trunk/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImpl.java trunk/unittest/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImplTest.java Log: [HQ-1903] When creating a recovery alert definition, it's initial state should be reconstituted from the db (similar to the startup process) so that it is aware that the problem alert definition has fired but has not been fixed. Modified: trunk/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImpl.java =================================================================== --- trunk/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImpl.java 2009-12-17 08:58:26 UTC (rev 14103) +++ trunk/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImpl.java 2009-12-18 05:12:55 UTC (rev 14104) @@ -47,6 +47,9 @@ import org.hyperic.hq.application.ShutdownCallback; import org.hyperic.hq.application.TransactionListener; import org.hyperic.hq.authz.server.session.AuthzSubject; +import org.hyperic.hq.authz.server.session.AuthzSubjectManagerEJBImpl; +import org.hyperic.hq.authz.shared.AuthzConstants; +import org.hyperic.hq.authz.shared.AuthzSubjectManagerLocal; import org.hyperic.hq.bizapp.server.trigger.conditional.ConditionalTriggerInterface; import org.hyperic.hq.common.SystemException; import org.hyperic.hq.events.AlertFiredEvent; @@ -56,9 +59,11 @@ import org.hyperic.hq.events.ext.RegisterableTriggerInterface; import org.hyperic.hq.events.ext.RegisterableTriggerRepository; import org.hyperic.hq.events.ext.RegisteredTriggers; +import org.hyperic.hq.events.server.session.AlertManagerEJBImpl; import org.hyperic.hq.events.server.session.EventLogManagerEJBImpl; import org.hyperic.hq.events.shared.AlertConditionValue; import org.hyperic.hq.events.shared.AlertDefinitionValue; +import org.hyperic.hq.events.shared.AlertManagerLocal; import org.hyperic.hq.events.shared.EventLogManagerLocal; import org.hyperic.hq.events.shared.RegisteredTriggerManagerLocal; import org.hyperic.hq.events.shared.RegisteredTriggerManagerUtil; @@ -102,6 +107,10 @@ private TriggerDAOInterface triggerDAO; + private AlertManagerLocal alertManager; + + private AuthzSubjectManagerLocal authzSubjectManager; + private EventLogManagerLocal eventLogManager; private ZeventEnqueuer zeventEnqueuer; @@ -163,8 +172,11 @@ log.warn("Unable to find AlertDefinition with id: " + alertDefId + ". These alerts will not fire."); return; } + AlertConditionEvaluator alertConditionEvaluator = alertConditionEvaluatorFactory.create(alertDefinition); + initializeAlertConditionEvaluatorState(alertConditionEvaluator); alertConditionEvaluatorRepository.addAlertConditionEvaluator(alertConditionEvaluator); + for (Iterator i = registeredTriggers.iterator(); i.hasNext();) { // Try to register each trigger, if exception, then move on RegisteredTrigger tv = (RegisteredTrigger) i.next(); @@ -179,6 +191,36 @@ } } + private void initializeAlertConditionEvaluatorState(AlertConditionEvaluator ace) { + Serializable initialState = null; + + if (ace instanceof RecoveryConditionEvaluator) { + // HQ-1903: The initial state of a recovery alert definition + // should be reconstituted from the db so that it is aware of + // the current state of the problem alert definition. + RecoveryConditionEvaluator rce = (RecoveryConditionEvaluator) ace; + Integer alertDefId = rce.getRecoveringFromAlertDefinitionId(); + AuthzSubject hqadmin = getAuthzSubjectManager() + .getSubjectById(AuthzConstants.rootSubjectId); + Alert alert = getAlertManager() + .findLastUnfixedByDefinition(hqadmin, alertDefId); + if (alert != null) { + AlertDefinition ad = alert.getAlertDefinition(); + initialState = + new AlertFiredEvent(alert.getId(), + ad.getId(), + new AppdefEntityID(ad.getResource()), + ad.getName(), + alert.getTimestamp(), + null); + } + } + + if (initialState != null) { + ace.initialize(initialState); + } + } + private void initializeTriggers(Collection registeredTriggers) { final boolean debug = log.isDebugEnabled(); StopWatch watch = new StopWatch(); @@ -320,7 +362,7 @@ } } - Map results = getEventLogManagerLocal() + Map results = getEventLogManager() .findLastUnfixedAlertFiredEvents(new ArrayList(alertDefinitionIds)); if (debug) { @@ -736,14 +778,34 @@ void setTriggerDAO(TriggerDAOInterface triggerDAO) { this.triggerDAO = triggerDAO; } + + AlertManagerLocal getAlertManager() { + return (this.alertManager == null) + ? AlertManagerEJBImpl.getOne() + : this.alertManager; + } - EventLogManagerLocal getEventLogManagerLocal() { + void setAlertManager(AlertManagerLocal alertManagerLocal) { + this.alertManager = alertManagerLocal; + } + + AuthzSubjectManagerLocal getAuthzSubjectManager() { + return (this.authzSubjectManager == null) + ? AuthzSubjectManagerEJBImpl.getOne() + : this.authzSubjectManager; + } + + void setAuthzSubjectManager(AuthzSubjectManagerLocal authzSubjectManagerLocal) { + this.authzSubjectManager = authzSubjectManagerLocal; + } + + EventLogManagerLocal getEventLogManager() { return (this.eventLogManager == null) ? EventLogManagerEJBImpl.getOne() : this.eventLogManager; } - void setEventLogManagerLocal(EventLogManagerLocal eventLogManagerLocal) { + void setEventLogManager(EventLogManagerLocal eventLogManagerLocal) { this.eventLogManager = eventLogManagerLocal; } Modified: trunk/unittest/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImplTest.java =================================================================== --- trunk/unittest/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImplTest.java 2009-12-17 08:58:26 UTC (rev 14103) +++ trunk/unittest/src/org/hyperic/hq/events/server/session/RegisteredTriggerManagerEJBImplTest.java 2009-12-18 05:12:55 UTC (rev 14104) @@ -12,10 +12,12 @@ import junit.framework.TestCase; import org.easymock.EasyMock; +import org.hyperic.hq.authz.shared.AuthzSubjectManagerLocal; import org.hyperic.hq.events.InvalidTriggerDataException; import org.hyperic.hq.events.ext.MockTrigger; import org.hyperic.hq.events.ext.RegisterableTriggerInterface; import org.hyperic.hq.events.ext.RegisterableTriggerRepository; +import org.hyperic.hq.events.shared.AlertManagerLocal; import org.hyperic.hq.events.shared.EventLogManagerLocal; import org.hyperic.hq.events.shared.RegisteredTriggerValue; import org.hyperic.hq.zevents.ZeventEnqueuer; @@ -32,6 +34,8 @@ private AlertConditionEvaluatorFactory alertConditionEvaluatorFactory; private AlertConditionEvaluator alertConditionEvaluator; private TriggerDAOInterface triggerDAO; + private AlertManagerLocal alertManagerLocal; + private AuthzSubjectManagerLocal authzSubjectManagerLocal; private EventLogManagerLocal eventLogManagerLocal; private RegisteredTriggerManagerEJBImpl registeredTriggerManager; private RegisterableTriggerRepository registeredTriggerRepository; @@ -53,6 +57,8 @@ this.alertConditionEvaluatorFactory = EasyMock.createMock(AlertConditionEvaluatorFactory.class); this.alertConditionEvaluator = EasyMock.createMock(AlertConditionEvaluator.class); this.triggerDAO = EasyMock.createMock(TriggerDAOInterface.class); + this.alertManagerLocal = EasyMock.createMock(AlertManagerLocal.class); + this.authzSubjectManagerLocal = EasyMock.createMock(AuthzSubjectManagerLocal.class); this.eventLogManagerLocal = EasyMock.createMock(EventLogManagerLocal.class); this.registeredTriggerRepository = EasyMock.createMock(RegisterableTriggerRepository.class); this.zEventEnqueuer = EasyMock.createMock(ZeventEnqueuer.class); @@ -62,7 +68,9 @@ this.registeredTriggerManager = new RegisteredTriggerManagerEJBImpl(); registeredTriggerManager.setAlertConditionEvaluatorFactory(alertConditionEvaluatorFactory); registeredTriggerManager.setTriggerDAO(triggerDAO); - registeredTriggerManager.setEventLogManagerLocal(eventLogManagerLocal); + registeredTriggerManager.setAlertManager(alertManagerLocal); + registeredTriggerManager.setAuthzSubjectManager(authzSubjectManagerLocal); + registeredTriggerManager.setEventLogManager(eventLogManagerLocal); registeredTriggerManager.setZeventEnqueuer(zEventEnqueuer); registeredTriggerManager.setAlertConditionEvaluatorRepository(alertConditionEvaluatorRepository); MockTrigger.initialized = false; |
|
From: <no...@gi...> - 2009-12-17 18:23:46
|
Branch: refs/heads/evolution Home: http://github.com/hyperic/hqapi Commit: aaaa7d5dcfa3b307d8b47f82466f7ae8622142fb http://github.com/hyperic/hqapi/commit/aaaa7d5dcfa3b307d8b47f82466f7ae8622142fb Author: Jennifer Hickey <jen...@sp...> Date: 2009-12-17 (Thu, 17 Dec 2009) Changed paths: M src/org/hyperic/hq/hqapi1/test/HQApiTestBase.java Log Message: ----------- Changed default port to 8080 (may change back later) |
|
From: <bo...@hy...> - 2009-12-17 08:58:38
|
Author: bob Date: 2009-12-17 00:58:26 -0800 (Thu, 17 Dec 2009) New Revision: 14103 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14103 Modified: trunk/etc/version.properties Log: Release 4.3.0 build #1291 Modified: trunk/etc/version.properties =================================================================== --- trunk/etc/version.properties 2009-12-17 01:21:52 UTC (rev 14102) +++ trunk/etc/version.properties 2009-12-17 08:58:26 UTC (rev 14103) @@ -1,3 +1,3 @@ -#Wed Dec 16 00:16:02 PST 2009 +#Thu Dec 17 00:16:38 PST 2009 version=4.3.0 -build=1290 +build=1291 |
|
From: <no...@gi...> - 2009-12-17 04:29:00
|
Branch: refs/heads/evolution Home: http://github.com/hyperic/hqapi Commit: 6a8d5d047485880d3e357668ea70040fa08cc2b1 http://github.com/hyperic/hqapi/commit/6a8d5d047485880d3e357668ea70040fa08cc2b1 Author: Jennifer Hickey <jhickey@calculon.local> Date: 2009-12-16 (Wed, 16 Dec 2009) Changed paths: M build.xml Log Message: ----------- Updated deploy target for hq.war |
Author: trader Date: 2009-12-16 17:21:52 -0800 (Wed, 16 Dec 2009) New Revision: 14102 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14102 Added: branches/HQ_4_1_2_1/unittest/src/org/hyperic/util/file/ branches/HQ_4_1_2_1/unittest/src/org/hyperic/util/file/DiskListTest.java Modified: branches/HQ_4_1_2_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java branches/HQ_4_1_2_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java branches/HQ_4_1_2_1/src/org/hyperic/util/file/DiskList.java Log: HHQ-3514 patch Modified: branches/HQ_4_1_2_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java =================================================================== --- branches/HQ_4_1_2_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java 2009-12-17 01:14:50 UTC (rev 14101) +++ branches/HQ_4_1_2_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java 2009-12-17 01:21:52 UTC (rev 14102) @@ -51,8 +51,8 @@ implements AgentStorageProvider { private static final int RECSIZE = 1024; - private static final long MAXSIZE = 100 * 1024 * 1024; // 100MB - private static final long CHKSIZE = 20 * 1024 * 1024; // 20MB + private static final long MAXSIZE = 50 * 1024 * 1024; // 50MB + private static final long CHKSIZE = 10 * 1024 * 1024; // 10MB private static final int CHKPERC = 50; // Only allow < 50% free private Log log; // da logger Modified: branches/HQ_4_1_2_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java =================================================================== --- branches/HQ_4_1_2_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java 2009-12-17 01:14:50 UTC (rev 14101) +++ branches/HQ_4_1_2_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java 2009-12-17 01:21:52 UTC (rev 14102) @@ -573,6 +573,10 @@ return; } + if (log.isDebugEnabled()) { + log.debug("Woke up, sending batch of metrics."); + } + lastMetricTime = this.sendBatch(); if(lastMetricTime != null){ String backlogNum = ""; Modified: branches/HQ_4_1_2_1/src/org/hyperic/util/file/DiskList.java =================================================================== --- branches/HQ_4_1_2_1/src/org/hyperic/util/file/DiskList.java 2009-12-17 01:14:50 UTC (rev 14101) +++ branches/HQ_4_1_2_1/src/org/hyperic/util/file/DiskList.java 2009-12-17 01:21:52 UTC (rev 14102) @@ -74,13 +74,14 @@ private static final Log log = LogFactory.getLog(DiskList.class.getName()); private String fileName; + private String idxFileName; private RandomAccessFile indexFile; - private RandomAccessFile dataFile; + protected RandomAccessFile dataFile; private int recordSize; // Size of each record private long firstRec; // IDX of first record private long lastRec; // IDX of last record private byte[] padBytes; // Utility array for padding - private SortedSet freeList; // Set(Long) of free rec idxs + protected SortedSet freeList; // Set(Long) of free rec idxs private int modNum; // Modification random number private long checkSize; // Start to check for unused blocks // when the datafile reaches this @@ -119,10 +120,11 @@ int checkPerc, long maxLength) throws IOException { - File idxFileName; + File idxFile; - idxFileName = new File(dataFile + ".idx"); + idxFile = new File(dataFile + ".idx"); this.fileName = dataFile.getName(); + this.idxFileName = idxFile.getName(); this.rand = new Random(); this.dataFile = new RandomAccessFile(dataFile, "rw"); this.recordSize = recordSize; @@ -135,9 +137,9 @@ " to " + maxLength + " bytes"); this.maxLength = maxLength; - this.genFreeList(idxFileName); + this.indexFile = new RandomAccessFile(idxFile, "rw"); + this.genFreeList(idxFile); - this.indexFile = new RandomAccessFile(idxFileName, "rw"); this.closed = false; } @@ -155,16 +157,16 @@ } /** - * Do maintinece on the data and index files. If the datafile size and - * the free block percentange exceed the defined thresholds, the extra + * Do maintenance on the data and index files. If the datafile size and + * the free block percentage exceed the defined thresholds, the extra * free blocks will be removed by truncating the data and index files. * * Since truncation is used, some times it will be possible that even - * though the criteria are met, we won't be albe to delete the free space. + * though the criteria are met, we won't be able to delete the free space. * This is a recoverable situation though, since new blocks will be * inserted at the beginning of the data file. */ - private void doMaintainence() + private void doMaintenence() throws IOException { long lastData = this.dataFile.length()/this.recordSize; @@ -211,7 +213,7 @@ * buffered input stream, which makes our initial startup much * faster, if there is a lot of data sitting in the list. */ - private void genFreeList(File idxFileName) + private void genFreeList(File idxFile) throws IOException { BufferedInputStream bIs; @@ -226,12 +228,12 @@ this.freeList = new TreeSet(); try { - fIs = new FileInputStream(idxFileName); + fIs = new FileInputStream(idxFile); bIs = new BufferedInputStream(fIs); dIs = new DataInputStream(bIs); - for(int idx=0; ; idx++){ + for(long idx=0; ; idx++){ boolean used; long prev, next; @@ -388,12 +390,20 @@ try { this.indexFile.setLength(0); } catch(IOException exc){ + this.log.error("IOException while truncating file " + idxFileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } sExc = exc; } try { this.dataFile.setLength(0); } catch(IOException exc){ + this.log.error("IOException while truncating file " + fileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } if(sExc != null){ sExc = exc; } @@ -469,9 +479,11 @@ this.freeList.add(new Long(recNo)); } - if ((this.dataFile.length() > this.checkSize) && - (this.getDataFileFreePercentage() > this.checkPerc)) { - this.doMaintainence(); + long length = this.dataFile.length(); + long percFree = this.getDataFileFreePercentage(); + if ((length > this.checkSize) && + (percFree > this.checkPerc)) { + this.doMaintenence(); } } @@ -492,13 +504,21 @@ try { this.dataFile.close(); - } catch(IOException exc){ + } catch(IOException exc){ + this.log.error("IOException while closing file " + fileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } sExc = exc; } try { this.indexFile.close(); } catch(IOException exc){ + this.log.error("IOException while closing file " + idxFileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } if(sExc == null){ sExc = exc; } @@ -550,6 +570,10 @@ try { rec = this.diskList.readRecord(this.curIdx); } catch(IOException exc){ + log.error("IOException while reading record"); + if (log.isDebugEnabled()) { + log.debug(exc); + } throw new NoSuchElementException("Error getting next " + "element: " + exc.getMessage()); @@ -577,6 +601,10 @@ try { this.diskList.removeRecord(this.curIdx); } catch(IOException exc){ + log.error("IOException while removing record"); + if (log.isDebugEnabled()) { + log.debug(exc); + } throw new IllegalStateException("Error removing record: " + exc.getMessage()); } @@ -597,7 +625,7 @@ return new DiskListIterator(this, this.firstRec, this.modNum); } } - + public static void main(String[] args) throws Exception { Added: branches/HQ_4_1_2_1/unittest/src/org/hyperic/util/file/DiskListTest.java =================================================================== --- branches/HQ_4_1_2_1/unittest/src/org/hyperic/util/file/DiskListTest.java (rev 0) +++ branches/HQ_4_1_2_1/unittest/src/org/hyperic/util/file/DiskListTest.java 2009-12-17 01:21:52 UTC (rev 14102) @@ -0,0 +1,438 @@ +package org.hyperic.util.file; + +import java.io.File; +import java.io.IOException; +import java.io.RandomAccessFile; +import java.util.Iterator; + +import junit.framework.TestCase; + +public class DiskListTest extends TestCase +{ + private static final int RECSIZE = 1024; + private static final long MAXSIZE = 50 * 1024 * 1024; // 100MB + private static final long CHKSIZE = 10 * 1024 * 1024; // 20MB + private static final int CHKPERC = 50; // Only allow < 50% free + private static final int MAXRECS = 2000; + + public DiskListTest() {} + + public void setUp() { + } + + public void testReadWriteFile() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + Iterator it = holder.list.getListIterator(); + // Check that we can read the proper number of records back + long i = 0; + while (it.hasNext()) { + it.next(); + i++; + } + assertTrue(i == MAXRECS); + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + it = holder.list.getListIterator(); + i = 0; + while (it.hasNext()) { + it.next(); + it.remove(); + + i++; + } + + assertTrue(i == MAXRECS); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithNoInserts() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + } finally { + + holder.dispose(); + + } + } + + public void testFillAndReopen() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nRecs = 0; + while (holder.list.dataFile.length() < MAXSIZE) { + holder.list.addToList(toPut); + nRecs++; + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + Iterator it = holder.list.getListIterator(); + long nIterated = 0; + while (it.hasNext()) { + it.next(); + nIterated++; + } + + assertTrue("Expected " + nRecs + " records, got " + nIterated, + nIterated == nRecs); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithInsertsAndNoDeletes() throws Exception { + + DiskListDataHolder holder = null; + if (holder == null) return; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithInsertsAndDeletes() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + Iterator it = holder.list.getListIterator(); + int nDeleted = 0; + while (it.hasNext()) { + for (int i = 0; i < 5; ++i) { + it.next(); + } + it.remove(); + nDeleted++; + } + + assertTrue(holder.list.freeList.size() == nDeleted); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListAfterTruncation() throws Exception { + + DiskListDataHolder holder = null; + long epsilon = 50; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + while (holder.list.dataFile.length() + epsilon < MAXSIZE) { + holder.list.addToList(toPut); + } + + assertTrue(holder.list.freeList.size() == 0); + + // Now insert until we spill over, expect a log message from this line + while (holder.list.dataFile.length() > MAXSIZE - epsilon) { + holder.list.addToList(toPut); + } + + assertTrue(holder.list.freeList.size() == 0); + + // After truncation, there should be no records, add one back and then delete it + holder.list.addToList(toPut); + Iterator it = holder.list.getListIterator(); + int nIterated = 0; + while (it.hasNext()) { + it.next(); + + // Each remove should create one spot in the free list + it.remove(); + nIterated++; + } + + assertTrue(nIterated == 1); + assertTrue(holder.list.freeList.size() == 1); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListAfterMaintenance() throws Exception { + + DiskListDataHolder holder = null; + long epsilon = 50; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nInserted = 0; + while (holder.list.dataFile.length() + epsilon < MAXSIZE) { + holder.list.addToList(toPut); + nInserted++; + } + + int freeListSize = holder.list.freeList.size(); + assertTrue(freeListSize == 0); + + // Delete every fourth record as we buzz through the list. Because CHKPERC is + // 50%, deleting every fourth should NOT trigger maintenance + int nIterated = 0; + int nDeleted = 0; + int counter = 0; + Iterator it = holder.list.getListIterator(); + while (it.hasNext()) { + it.next(); + + if (++counter == 4) { + // Each remove should create one spot in the free list + it.remove(); + nDeleted++; + counter = 0; + } + + nIterated++; + } + + assertTrue(nIterated == nInserted); + freeListSize = holder.list.freeList.size(); + assertTrue(freeListSize == nDeleted); + + // Now delete the the same amount - 1: still should NOT trigger maintenance + int nToDelete = nDeleted - 20; + int nPreviouslyDeleted = nDeleted; + nDeleted = 0; + it = holder.list.getListIterator(); + while (it.hasNext() && nDeleted < nToDelete) { + it.next(); + it.remove(); + nDeleted++; + } + + assertTrue(nDeleted == nToDelete); + freeListSize = holder.list.freeList.size(); + assertTrue(nDeleted + nPreviouslyDeleted == freeListSize); + + // Now try to trigger maintenance. First: maintenance is only done if the last block is + // free (maintenance truncates off the end only, no internal compacting), so make sure that + // there is stuff to truncate + int offTheEnd = 20; + it = holder.list.getListIterator(); + for (int i = nDeleted + nPreviouslyDeleted; i < nInserted - offTheEnd; ++i) { + it.next(); + } + while (it.hasNext()) { + it.next(); + it.remove(); + } + + // A few more deletes should trigger maintenance. The calculation uses an integer + // percentage rounded from doubles, so exactly (half - 1) deletes may not be enough. + freeListSize = holder.list.freeList.size(); + int freeListPeakSize = freeListSize; + long oldLength = holder.list.dataFile.length(); + int toTriggerMaintenance = ((int) (nInserted / 100)) / 2 + 1; + it = holder.list.getListIterator(); + for (int i = 0; i < toTriggerMaintenance; ++i) { + it.next(); + it.remove(); + freeListPeakSize = Math.max(freeListPeakSize, holder.list.freeList.size()); + } + + freeListSize = holder.list.freeList.size(); + long length = holder.list.dataFile.length(); + assertTrue("Expected free list size < " + freeListPeakSize + ", actual value was " + freeListSize, + freeListSize < freeListPeakSize); + assertTrue("Expected file length < " + oldLength + ", actual value was " + length, + length < oldLength); + + } finally { + + holder.dispose(); + + } + } + + private static class DiskListDataHolder { + DiskList list; + File dataFile; + File indexFile; + + DiskListDataHolder() throws Exception { + File tmpDirFile; + + String tmpDir = System.getProperty("java.io.tmpdir"); + if (tmpDir == null) { + tmpDir = "/tmp"; + } + + tmpDirFile = new File(tmpDir); + + File dataFile = null; + File indexFile = null; + if (tmpDirFile.isDirectory() && tmpDirFile.canWrite()) { + dataFile = new File(tmpDirFile, "datafile"); + indexFile = new File(tmpDirFile, "datafile.idx"); + dataFile.delete(); + indexFile.delete(); + } else { + throw new IllegalStateException("Non-writeable directory!"); + } + + DiskList list = new DiskList(dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + this.list = list; + this.dataFile = dataFile; + this.indexFile = indexFile; + } + + void dispose() throws Exception { + list.close(); + dataFile.delete(); + indexFile.delete(); + } + } +} |
Author: trader Date: 2009-12-16 17:14:50 -0800 (Wed, 16 Dec 2009) New Revision: 14101 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14101 Added: branches/HQ_4_1_4_1/unittest/src/org/hyperic/util/file/ branches/HQ_4_1_4_1/unittest/src/org/hyperic/util/file/DiskListTest.java Modified: branches/HQ_4_1_4_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java branches/HQ_4_1_4_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java branches/HQ_4_1_4_1/src/org/hyperic/util/file/DiskList.java Log: HHQ-3541, 4.1.4 patch Modified: branches/HQ_4_1_4_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java =================================================================== --- branches/HQ_4_1_4_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java 2009-12-17 01:12:32 UTC (rev 14100) +++ branches/HQ_4_1_4_1/src/org/hyperic/hq/agent/server/AgentDListProvider.java 2009-12-17 01:14:50 UTC (rev 14101) @@ -51,8 +51,8 @@ implements AgentStorageProvider { private static final int RECSIZE = 1024; - private static final long MAXSIZE = 100 * 1024 * 1024; // 100MB - private static final long CHKSIZE = 20 * 1024 * 1024; // 20MB + private static final long MAXSIZE = 50 * 1024 * 1024; // 50MB + private static final long CHKSIZE = 10 * 1024 * 1024; // 10MB private static final int CHKPERC = 50; // Only allow < 50% free private Log log; // da logger Modified: branches/HQ_4_1_4_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java =================================================================== --- branches/HQ_4_1_4_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java 2009-12-17 01:12:32 UTC (rev 14100) +++ branches/HQ_4_1_4_1/src/org/hyperic/hq/measurement/agent/server/SenderThread.java 2009-12-17 01:14:50 UTC (rev 14101) @@ -573,6 +573,10 @@ return; } + if (log.isDebugEnabled()) { + log.debug("Woke up, sending batch of metrics."); + } + lastMetricTime = this.sendBatch(); if(lastMetricTime != null){ String backlogNum = ""; Modified: branches/HQ_4_1_4_1/src/org/hyperic/util/file/DiskList.java =================================================================== --- branches/HQ_4_1_4_1/src/org/hyperic/util/file/DiskList.java 2009-12-17 01:12:32 UTC (rev 14100) +++ branches/HQ_4_1_4_1/src/org/hyperic/util/file/DiskList.java 2009-12-17 01:14:50 UTC (rev 14101) @@ -74,13 +74,14 @@ private static final Log log = LogFactory.getLog(DiskList.class.getName()); private String fileName; + private String idxFileName; private RandomAccessFile indexFile; - private RandomAccessFile dataFile; + protected RandomAccessFile dataFile; private int recordSize; // Size of each record private long firstRec; // IDX of first record private long lastRec; // IDX of last record private byte[] padBytes; // Utility array for padding - private SortedSet freeList; // Set(Long) of free rec idxs + protected SortedSet freeList; // Set(Long) of free rec idxs private int modNum; // Modification random number private long checkSize; // Start to check for unused blocks // when the datafile reaches this @@ -119,10 +120,11 @@ int checkPerc, long maxLength) throws IOException { - File idxFileName; + File idxFile; - idxFileName = new File(dataFile + ".idx"); + idxFile = new File(dataFile + ".idx"); this.fileName = dataFile.getName(); + this.idxFileName = idxFile.getName(); this.rand = new Random(); this.dataFile = new RandomAccessFile(dataFile, "rw"); this.recordSize = recordSize; @@ -135,9 +137,9 @@ " to " + maxLength + " bytes"); this.maxLength = maxLength; - this.genFreeList(idxFileName); + this.indexFile = new RandomAccessFile(idxFile, "rw"); + this.genFreeList(idxFile); - this.indexFile = new RandomAccessFile(idxFileName, "rw"); this.closed = false; } @@ -155,16 +157,16 @@ } /** - * Do maintinece on the data and index files. If the datafile size and - * the free block percentange exceed the defined thresholds, the extra + * Do maintenance on the data and index files. If the datafile size and + * the free block percentage exceed the defined thresholds, the extra * free blocks will be removed by truncating the data and index files. * * Since truncation is used, some times it will be possible that even - * though the criteria are met, we won't be albe to delete the free space. + * though the criteria are met, we won't be able to delete the free space. * This is a recoverable situation though, since new blocks will be * inserted at the beginning of the data file. */ - private void doMaintainence() + private void doMaintenence() throws IOException { long lastData = this.dataFile.length()/this.recordSize; @@ -211,7 +213,7 @@ * buffered input stream, which makes our initial startup much * faster, if there is a lot of data sitting in the list. */ - private void genFreeList(File idxFileName) + private void genFreeList(File idxFile) throws IOException { BufferedInputStream bIs; @@ -226,12 +228,12 @@ this.freeList = new TreeSet(); try { - fIs = new FileInputStream(idxFileName); + fIs = new FileInputStream(idxFile); bIs = new BufferedInputStream(fIs); dIs = new DataInputStream(bIs); - for(int idx=0; ; idx++){ + for(long idx=0; ; idx++){ boolean used; long prev, next; @@ -388,12 +390,20 @@ try { this.indexFile.setLength(0); } catch(IOException exc){ + this.log.error("IOException while truncating file " + idxFileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } sExc = exc; } try { this.dataFile.setLength(0); } catch(IOException exc){ + this.log.error("IOException while truncating file " + fileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } if(sExc != null){ sExc = exc; } @@ -469,9 +479,11 @@ this.freeList.add(new Long(recNo)); } - if ((this.dataFile.length() > this.checkSize) && - (this.getDataFileFreePercentage() > this.checkPerc)) { - this.doMaintainence(); + long length = this.dataFile.length(); + long percFree = this.getDataFileFreePercentage(); + if ((length > this.checkSize) && + (percFree > this.checkPerc)) { + this.doMaintenence(); } } @@ -492,13 +504,21 @@ try { this.dataFile.close(); - } catch(IOException exc){ + } catch(IOException exc){ + this.log.error("IOException while closing file " + fileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } sExc = exc; } try { this.indexFile.close(); } catch(IOException exc){ + this.log.error("IOException while closing file " + idxFileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } if(sExc == null){ sExc = exc; } @@ -550,6 +570,10 @@ try { rec = this.diskList.readRecord(this.curIdx); } catch(IOException exc){ + log.error("IOException while reading record"); + if (log.isDebugEnabled()) { + log.debug(exc); + } throw new NoSuchElementException("Error getting next " + "element: " + exc.getMessage()); @@ -577,6 +601,10 @@ try { this.diskList.removeRecord(this.curIdx); } catch(IOException exc){ + log.error("IOException while removing record"); + if (log.isDebugEnabled()) { + log.debug(exc); + } throw new IllegalStateException("Error removing record: " + exc.getMessage()); } @@ -597,7 +625,7 @@ return new DiskListIterator(this, this.firstRec, this.modNum); } } - + public static void main(String[] args) throws Exception { Added: branches/HQ_4_1_4_1/unittest/src/org/hyperic/util/file/DiskListTest.java =================================================================== --- branches/HQ_4_1_4_1/unittest/src/org/hyperic/util/file/DiskListTest.java (rev 0) +++ branches/HQ_4_1_4_1/unittest/src/org/hyperic/util/file/DiskListTest.java 2009-12-17 01:14:50 UTC (rev 14101) @@ -0,0 +1,438 @@ +package org.hyperic.util.file; + +import java.io.File; +import java.io.IOException; +import java.io.RandomAccessFile; +import java.util.Iterator; + +import junit.framework.TestCase; + +public class DiskListTest extends TestCase +{ + private static final int RECSIZE = 1024; + private static final long MAXSIZE = 50 * 1024 * 1024; // 100MB + private static final long CHKSIZE = 10 * 1024 * 1024; // 20MB + private static final int CHKPERC = 50; // Only allow < 50% free + private static final int MAXRECS = 2000; + + public DiskListTest() {} + + public void setUp() { + } + + public void testReadWriteFile() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + Iterator it = holder.list.getListIterator(); + // Check that we can read the proper number of records back + long i = 0; + while (it.hasNext()) { + it.next(); + i++; + } + assertTrue(i == MAXRECS); + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + it = holder.list.getListIterator(); + i = 0; + while (it.hasNext()) { + it.next(); + it.remove(); + + i++; + } + + assertTrue(i == MAXRECS); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithNoInserts() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + } finally { + + holder.dispose(); + + } + } + + public void testFillAndReopen() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nRecs = 0; + while (holder.list.dataFile.length() < MAXSIZE) { + holder.list.addToList(toPut); + nRecs++; + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + Iterator it = holder.list.getListIterator(); + long nIterated = 0; + while (it.hasNext()) { + it.next(); + nIterated++; + } + + assertTrue("Expected " + nRecs + " records, got " + nIterated, + nIterated == nRecs); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithInsertsAndNoDeletes() throws Exception { + + DiskListDataHolder holder = null; + if (holder == null) return; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithInsertsAndDeletes() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + Iterator it = holder.list.getListIterator(); + int nDeleted = 0; + while (it.hasNext()) { + for (int i = 0; i < 5; ++i) { + it.next(); + } + it.remove(); + nDeleted++; + } + + assertTrue(holder.list.freeList.size() == nDeleted); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListAfterTruncation() throws Exception { + + DiskListDataHolder holder = null; + long epsilon = 50; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + while (holder.list.dataFile.length() + epsilon < MAXSIZE) { + holder.list.addToList(toPut); + } + + assertTrue(holder.list.freeList.size() == 0); + + // Now insert until we spill over, expect a log message from this line + while (holder.list.dataFile.length() > MAXSIZE - epsilon) { + holder.list.addToList(toPut); + } + + assertTrue(holder.list.freeList.size() == 0); + + // After truncation, there should be no records, add one back and then delete it + holder.list.addToList(toPut); + Iterator it = holder.list.getListIterator(); + int nIterated = 0; + while (it.hasNext()) { + it.next(); + + // Each remove should create one spot in the free list + it.remove(); + nIterated++; + } + + assertTrue(nIterated == 1); + assertTrue(holder.list.freeList.size() == 1); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListAfterMaintenance() throws Exception { + + DiskListDataHolder holder = null; + long epsilon = 50; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nInserted = 0; + while (holder.list.dataFile.length() + epsilon < MAXSIZE) { + holder.list.addToList(toPut); + nInserted++; + } + + int freeListSize = holder.list.freeList.size(); + assertTrue(freeListSize == 0); + + // Delete every fourth record as we buzz through the list. Because CHKPERC is + // 50%, deleting every fourth should NOT trigger maintenance + int nIterated = 0; + int nDeleted = 0; + int counter = 0; + Iterator it = holder.list.getListIterator(); + while (it.hasNext()) { + it.next(); + + if (++counter == 4) { + // Each remove should create one spot in the free list + it.remove(); + nDeleted++; + counter = 0; + } + + nIterated++; + } + + assertTrue(nIterated == nInserted); + freeListSize = holder.list.freeList.size(); + assertTrue(freeListSize == nDeleted); + + // Now delete the the same amount - 1: still should NOT trigger maintenance + int nToDelete = nDeleted - 20; + int nPreviouslyDeleted = nDeleted; + nDeleted = 0; + it = holder.list.getListIterator(); + while (it.hasNext() && nDeleted < nToDelete) { + it.next(); + it.remove(); + nDeleted++; + } + + assertTrue(nDeleted == nToDelete); + freeListSize = holder.list.freeList.size(); + assertTrue(nDeleted + nPreviouslyDeleted == freeListSize); + + // Now try to trigger maintenance. First: maintenance is only done if the last block is + // free (maintenance truncates off the end only, no internal compacting), so make sure that + // there is stuff to truncate + int offTheEnd = 20; + it = holder.list.getListIterator(); + for (int i = nDeleted + nPreviouslyDeleted; i < nInserted - offTheEnd; ++i) { + it.next(); + } + while (it.hasNext()) { + it.next(); + it.remove(); + } + + // A few more deletes should trigger maintenance. The calculation uses an integer + // percentage rounded from doubles, so exactly (half - 1) deletes may not be enough. + freeListSize = holder.list.freeList.size(); + int freeListPeakSize = freeListSize; + long oldLength = holder.list.dataFile.length(); + int toTriggerMaintenance = ((int) (nInserted / 100)) / 2 + 1; + it = holder.list.getListIterator(); + for (int i = 0; i < toTriggerMaintenance; ++i) { + it.next(); + it.remove(); + freeListPeakSize = Math.max(freeListPeakSize, holder.list.freeList.size()); + } + + freeListSize = holder.list.freeList.size(); + long length = holder.list.dataFile.length(); + assertTrue("Expected free list size < " + freeListPeakSize + ", actual value was " + freeListSize, + freeListSize < freeListPeakSize); + assertTrue("Expected file length < " + oldLength + ", actual value was " + length, + length < oldLength); + + } finally { + + holder.dispose(); + + } + } + + private static class DiskListDataHolder { + DiskList list; + File dataFile; + File indexFile; + + DiskListDataHolder() throws Exception { + File tmpDirFile; + + String tmpDir = System.getProperty("java.io.tmpdir"); + if (tmpDir == null) { + tmpDir = "/tmp"; + } + + tmpDirFile = new File(tmpDir); + + File dataFile = null; + File indexFile = null; + if (tmpDirFile.isDirectory() && tmpDirFile.canWrite()) { + dataFile = new File(tmpDirFile, "datafile"); + indexFile = new File(tmpDirFile, "datafile.idx"); + dataFile.delete(); + indexFile.delete(); + } else { + throw new IllegalStateException("Non-writeable directory!"); + } + + DiskList list = new DiskList(dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + this.list = list; + this.dataFile = dataFile; + this.indexFile = indexFile; + } + + void dispose() throws Exception { + list.close(); + dataFile.delete(); + indexFile.delete(); + } + } +} |
Author: trader Date: 2009-12-16 17:12:32 -0800 (Wed, 16 Dec 2009) New Revision: 14100 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14100 Added: branches/HQ_4_2_0_PATCH/unittest/src/org/hyperic/util/file/ branches/HQ_4_2_0_PATCH/unittest/src/org/hyperic/util/file/DiskListTest.java Modified: branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/agent/server/AgentDListProvider.java branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/measurement/agent/server/SenderThread.java branches/HQ_4_2_0_PATCH/src/org/hyperic/util/file/DiskList.java Log: HHQ-3541, 4.2.0 patch Modified: branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/agent/server/AgentDListProvider.java =================================================================== --- branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/agent/server/AgentDListProvider.java 2009-12-16 23:59:51 UTC (rev 14099) +++ branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/agent/server/AgentDListProvider.java 2009-12-17 01:12:32 UTC (rev 14100) @@ -51,8 +51,8 @@ implements AgentStorageProvider { private static final int RECSIZE = 1024; - private static final long MAXSIZE = 100 * 1024 * 1024; // 100MB - private static final long CHKSIZE = 20 * 1024 * 1024; // 20MB + private static final long MAXSIZE = 50 * 1024 * 1024; // 50MB + private static final long CHKSIZE = 10 * 1024 * 1024; // 10MB private static final int CHKPERC = 50; // Only allow < 50% free private Log log; // da logger Modified: branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/measurement/agent/server/SenderThread.java =================================================================== --- branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/measurement/agent/server/SenderThread.java 2009-12-16 23:59:51 UTC (rev 14099) +++ branches/HQ_4_2_0_PATCH/src/org/hyperic/hq/measurement/agent/server/SenderThread.java 2009-12-17 01:12:32 UTC (rev 14100) @@ -379,9 +379,9 @@ { try { Record r = SenderThread.decodeRecord((String)it.next()); - if (!records.contains(r)) { - records.add(r); - } else { + boolean didNotAlreadyExist = records.add(r); + if (!didNotAlreadyExist) { + // nuke the dup if (debug) log.debug("Dropping duplicate entry for " + r); numUsed--; } @@ -607,6 +607,10 @@ return; } + if (log.isDebugEnabled()) { + log.debug("Woke up, sending batch of metrics."); + } + lastMetricTime = this.sendBatch(); if(lastMetricTime != null){ String backlogNum = ""; Modified: branches/HQ_4_2_0_PATCH/src/org/hyperic/util/file/DiskList.java =================================================================== --- branches/HQ_4_2_0_PATCH/src/org/hyperic/util/file/DiskList.java 2009-12-16 23:59:51 UTC (rev 14099) +++ branches/HQ_4_2_0_PATCH/src/org/hyperic/util/file/DiskList.java 2009-12-17 01:12:32 UTC (rev 14100) @@ -74,13 +74,14 @@ private static final Log log = LogFactory.getLog(DiskList.class.getName()); private String fileName; + private String idxFileName; private RandomAccessFile indexFile; - private RandomAccessFile dataFile; + protected RandomAccessFile dataFile; private int recordSize; // Size of each record private long firstRec; // IDX of first record private long lastRec; // IDX of last record private byte[] padBytes; // Utility array for padding - private SortedSet freeList; // Set(Long) of free rec idxs + protected SortedSet freeList; // Set(Long) of free rec idxs private int modNum; // Modification random number private long checkSize; // Start to check for unused blocks // when the datafile reaches this @@ -119,10 +120,11 @@ int checkPerc, long maxLength) throws IOException { - File idxFileName; + File idxFile; - idxFileName = new File(dataFile + ".idx"); + idxFile = new File(dataFile + ".idx"); this.fileName = dataFile.getName(); + this.idxFileName = idxFile.getName(); this.rand = new Random(); this.dataFile = new RandomAccessFile(dataFile, "rw"); this.recordSize = recordSize; @@ -135,9 +137,9 @@ " to " + maxLength + " bytes"); this.maxLength = maxLength; - this.genFreeList(idxFileName); + this.indexFile = new RandomAccessFile(idxFile, "rw"); + this.genFreeList(idxFile); - this.indexFile = new RandomAccessFile(idxFileName, "rw"); this.closed = false; } @@ -155,16 +157,16 @@ } /** - * Do maintinece on the data and index files. If the datafile size and - * the free block percentange exceed the defined thresholds, the extra + * Do maintenance on the data and index files. If the datafile size and + * the free block percentage exceed the defined thresholds, the extra * free blocks will be removed by truncating the data and index files. * * Since truncation is used, some times it will be possible that even - * though the criteria are met, we won't be albe to delete the free space. + * though the criteria are met, we won't be able to delete the free space. * This is a recoverable situation though, since new blocks will be * inserted at the beginning of the data file. */ - private void doMaintainence() + private void doMaintenence() throws IOException { long lastData = this.dataFile.length()/this.recordSize; @@ -211,7 +213,7 @@ * buffered input stream, which makes our initial startup much * faster, if there is a lot of data sitting in the list. */ - private void genFreeList(File idxFileName) + private void genFreeList(File idxFile) throws IOException { BufferedInputStream bIs; @@ -226,12 +228,12 @@ this.freeList = new TreeSet(); try { - fIs = new FileInputStream(idxFileName); + fIs = new FileInputStream(idxFile); bIs = new BufferedInputStream(fIs); dIs = new DataInputStream(bIs); - for(int idx=0; ; idx++){ + for(long idx=0; ; idx++){ boolean used; long prev, next; @@ -388,12 +390,20 @@ try { this.indexFile.setLength(0); } catch(IOException exc){ + this.log.error("IOException while truncating file " + idxFileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } sExc = exc; } try { this.dataFile.setLength(0); } catch(IOException exc){ + this.log.error("IOException while truncating file " + fileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } if(sExc != null){ sExc = exc; } @@ -469,9 +479,11 @@ this.freeList.add(new Long(recNo)); } - if ((this.dataFile.length() > this.checkSize) && - (this.getDataFileFreePercentage() > this.checkPerc)) { - this.doMaintainence(); + long length = this.dataFile.length(); + long percFree = this.getDataFileFreePercentage(); + if ((length > this.checkSize) && + (percFree > this.checkPerc)) { + this.doMaintenence(); } } @@ -492,13 +504,21 @@ try { this.dataFile.close(); - } catch(IOException exc){ + } catch(IOException exc){ + this.log.error("IOException while closing file " + fileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } sExc = exc; } try { this.indexFile.close(); } catch(IOException exc){ + this.log.error("IOException while closing file " + idxFileName); + if (this.log.isDebugEnabled()) { + this.log.debug(exc); + } if(sExc == null){ sExc = exc; } @@ -550,6 +570,10 @@ try { rec = this.diskList.readRecord(this.curIdx); } catch(IOException exc){ + log.error("IOException while reading record"); + if (log.isDebugEnabled()) { + log.debug(exc); + } throw new NoSuchElementException("Error getting next " + "element: " + exc.getMessage()); @@ -577,6 +601,10 @@ try { this.diskList.removeRecord(this.curIdx); } catch(IOException exc){ + log.error("IOException while removing record"); + if (log.isDebugEnabled()) { + log.debug(exc); + } throw new IllegalStateException("Error removing record: " + exc.getMessage()); } @@ -597,7 +625,7 @@ return new DiskListIterator(this, this.firstRec, this.modNum); } } - + public static void main(String[] args) throws Exception { Added: branches/HQ_4_2_0_PATCH/unittest/src/org/hyperic/util/file/DiskListTest.java =================================================================== --- branches/HQ_4_2_0_PATCH/unittest/src/org/hyperic/util/file/DiskListTest.java (rev 0) +++ branches/HQ_4_2_0_PATCH/unittest/src/org/hyperic/util/file/DiskListTest.java 2009-12-17 01:12:32 UTC (rev 14100) @@ -0,0 +1,438 @@ +package org.hyperic.util.file; + +import java.io.File; +import java.io.IOException; +import java.io.RandomAccessFile; +import java.util.Iterator; + +import junit.framework.TestCase; + +public class DiskListTest extends TestCase +{ + private static final int RECSIZE = 1024; + private static final long MAXSIZE = 50 * 1024 * 1024; // 100MB + private static final long CHKSIZE = 10 * 1024 * 1024; // 20MB + private static final int CHKPERC = 50; // Only allow < 50% free + private static final int MAXRECS = 2000; + + public DiskListTest() {} + + public void setUp() { + } + + public void testReadWriteFile() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + Iterator it = holder.list.getListIterator(); + // Check that we can read the proper number of records back + long i = 0; + while (it.hasNext()) { + it.next(); + i++; + } + assertTrue(i == MAXRECS); + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + it = holder.list.getListIterator(); + i = 0; + while (it.hasNext()) { + it.next(); + it.remove(); + + i++; + } + + assertTrue(i == MAXRECS); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithNoInserts() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + } finally { + + holder.dispose(); + + } + } + + public void testFillAndReopen() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nRecs = 0; + while (holder.list.dataFile.length() < MAXSIZE) { + holder.list.addToList(toPut); + nRecs++; + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + Iterator it = holder.list.getListIterator(); + long nIterated = 0; + while (it.hasNext()) { + it.next(); + nIterated++; + } + + assertTrue("Expected " + nRecs + " records, got " + nIterated, + nIterated == nRecs); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithInsertsAndNoDeletes() throws Exception { + + DiskListDataHolder holder = null; + if (holder == null) return; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListWithInsertsAndDeletes() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + for (long i = 0; i < MAXRECS; ++i) { + String toPut = String.valueOf(i); + holder.list.addToList(toPut); + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + Iterator it = holder.list.getListIterator(); + int nDeleted = 0; + while (it.hasNext()) { + for (int i = 0; i < 5; ++i) { + it.next(); + } + it.remove(); + nDeleted++; + } + + assertTrue(holder.list.freeList.size() == nDeleted); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListAfterTruncation() throws Exception { + + DiskListDataHolder holder = null; + long epsilon = 50; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + while (holder.list.dataFile.length() + epsilon < MAXSIZE) { + holder.list.addToList(toPut); + } + + assertTrue(holder.list.freeList.size() == 0); + + // Now insert until we spill over, expect a log message from this line + while (holder.list.dataFile.length() > MAXSIZE - epsilon) { + holder.list.addToList(toPut); + } + + assertTrue(holder.list.freeList.size() == 0); + + // After truncation, there should be no records, add one back and then delete it + holder.list.addToList(toPut); + Iterator it = holder.list.getListIterator(); + int nIterated = 0; + while (it.hasNext()) { + it.next(); + + // Each remove should create one spot in the free list + it.remove(); + nIterated++; + } + + assertTrue(nIterated == 1); + assertTrue(holder.list.freeList.size() == 1); + + } finally { + + holder.dispose(); + + } + } + + public void testFreeListAfterMaintenance() throws Exception { + + DiskListDataHolder holder = null; + long epsilon = 50; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nInserted = 0; + while (holder.list.dataFile.length() + epsilon < MAXSIZE) { + holder.list.addToList(toPut); + nInserted++; + } + + int freeListSize = holder.list.freeList.size(); + assertTrue(freeListSize == 0); + + // Delete every fourth record as we buzz through the list. Because CHKPERC is + // 50%, deleting every fourth should NOT trigger maintenance + int nIterated = 0; + int nDeleted = 0; + int counter = 0; + Iterator it = holder.list.getListIterator(); + while (it.hasNext()) { + it.next(); + + if (++counter == 4) { + // Each remove should create one spot in the free list + it.remove(); + nDeleted++; + counter = 0; + } + + nIterated++; + } + + assertTrue(nIterated == nInserted); + freeListSize = holder.list.freeList.size(); + assertTrue(freeListSize == nDeleted); + + // Now delete the the same amount - 1: still should NOT trigger maintenance + int nToDelete = nDeleted - 20; + int nPreviouslyDeleted = nDeleted; + nDeleted = 0; + it = holder.list.getListIterator(); + while (it.hasNext() && nDeleted < nToDelete) { + it.next(); + it.remove(); + nDeleted++; + } + + assertTrue(nDeleted == nToDelete); + freeListSize = holder.list.freeList.size(); + assertTrue(nDeleted + nPreviouslyDeleted == freeListSize); + + // Now try to trigger maintenance. First: maintenance is only done if the last block is + // free (maintenance truncates off the end only, no internal compacting), so make sure that + // there is stuff to truncate + int offTheEnd = 20; + it = holder.list.getListIterator(); + for (int i = nDeleted + nPreviouslyDeleted; i < nInserted - offTheEnd; ++i) { + it.next(); + } + while (it.hasNext()) { + it.next(); + it.remove(); + } + + // A few more deletes should trigger maintenance. The calculation uses an integer + // percentage rounded from doubles, so exactly (half - 1) deletes may not be enough. + freeListSize = holder.list.freeList.size(); + int freeListPeakSize = freeListSize; + long oldLength = holder.list.dataFile.length(); + int toTriggerMaintenance = ((int) (nInserted / 100)) / 2 + 1; + it = holder.list.getListIterator(); + for (int i = 0; i < toTriggerMaintenance; ++i) { + it.next(); + it.remove(); + freeListPeakSize = Math.max(freeListPeakSize, holder.list.freeList.size()); + } + + freeListSize = holder.list.freeList.size(); + long length = holder.list.dataFile.length(); + assertTrue("Expected free list size < " + freeListPeakSize + ", actual value was " + freeListSize, + freeListSize < freeListPeakSize); + assertTrue("Expected file length < " + oldLength + ", actual value was " + length, + length < oldLength); + + } finally { + + holder.dispose(); + + } + } + + private static class DiskListDataHolder { + DiskList list; + File dataFile; + File indexFile; + + DiskListDataHolder() throws Exception { + File tmpDirFile; + + String tmpDir = System.getProperty("java.io.tmpdir"); + if (tmpDir == null) { + tmpDir = "/tmp"; + } + + tmpDirFile = new File(tmpDir); + + File dataFile = null; + File indexFile = null; + if (tmpDirFile.isDirectory() && tmpDirFile.canWrite()) { + dataFile = new File(tmpDirFile, "datafile"); + indexFile = new File(tmpDirFile, "datafile.idx"); + dataFile.delete(); + indexFile.delete(); + } else { + throw new IllegalStateException("Non-writeable directory!"); + } + + DiskList list = new DiskList(dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + this.list = list; + this.dataFile = dataFile; + this.indexFile = indexFile; + } + + void dispose() throws Exception { + list.close(); + dataFile.delete(); + indexFile.delete(); + } + } +} |
|
From: <sc...@hy...> - 2009-12-17 00:00:05
|
Author: scottmf Date: 2009-12-16 15:59:51 -0800 (Wed, 16 Dec 2009) New Revision: 14099 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14099 Modified: trunk/unittest/src/org/hyperic/hq/measurement/server/session/AvailabilityManager_testEJBImpl.java Log: [HHQ-3505] added clearing of the session in the setup() method. This is because our in-container tests only use one session for all the tests. Modified: trunk/unittest/src/org/hyperic/hq/measurement/server/session/AvailabilityManager_testEJBImpl.java =================================================================== --- trunk/unittest/src/org/hyperic/hq/measurement/server/session/AvailabilityManager_testEJBImpl.java 2009-12-16 08:53:11 UTC (rev 14098) +++ trunk/unittest/src/org/hyperic/hq/measurement/server/session/AvailabilityManager_testEJBImpl.java 2009-12-16 23:59:51 UTC (rev 14099) @@ -53,7 +53,6 @@ import org.apache.commons.logging.LogFactory; import org.hyperic.dao.DAOFactory; import org.hyperic.hq.common.SystemException; -import org.hyperic.hq.measurement.MeasurementConstants; import org.hyperic.hq.measurement.TimingVoodoo; import org.hyperic.hq.measurement.shared.AvailabilityManagerLocal; import org.hyperic.hq.measurement.shared.AvailabilityManager_testLocal; @@ -149,10 +148,14 @@ server.invoke(objName, "hitWithDate", obj, str); } + private long now() { + return System.currentTimeMillis(); + } + private void testOverlap() throws Exception { AvailabilityManagerLocal avail = AvailabilityManagerEJBImpl.getOne(); List list = new ArrayList(); - long now = System.currentTimeMillis(); + long now = now(); long baseTime = TimingVoodoo.roundDownTime(now, 60000); DataPoint pt = new DataPoint(PLAT_MEAS_ID, new MetricValue(1.0, baseTime)); pt = new DataPoint(PLAT_MEAS_ID, new MetricValue(0.5, baseTime)); @@ -166,7 +169,7 @@ setupAvailabilityTable(); AvailabilityManagerLocal avail = AvailabilityManagerEJBImpl.getOne(); ArrayList list = new ArrayList(); - long now = System.currentTimeMillis(); + long now = now(); long baseTime = TimingVoodoo.roundDownTime(now, 60000); long incrTime = 60000; long tmpTime = baseTime; @@ -190,7 +193,7 @@ private void stressTest1() throws Exception { setupAvailabilityTable(); - long now = System.currentTimeMillis(); + long now = now(); long baseTime = TimingVoodoo.roundDownTime(now, 60000); long incrTime = 60000; DataPoint pt = testCatchup(PLAT_MEAS_ID, baseTime, incrTime); @@ -206,7 +209,7 @@ Measurement meas = mMan.getMeasurement(measId); long interval = meas.getInterval(); - long now = System.currentTimeMillis(); + long now = now(); long baseTime = TimingVoodoo.roundDownTime(now, 60000); DataPoint pt; invokeBackfiller(baseTime); @@ -297,7 +300,7 @@ Measurement meas = mMan.getMeasurement(measId); long interval = meas.getInterval(); - long now = System.currentTimeMillis(); + long now = now(); long baseTime = TimingVoodoo.roundDownTime(now, 600000); DataPoint pt; List list = new ArrayList(); @@ -345,14 +348,15 @@ } private void testAvailabilityForService(Integer measId) throws Exception{ - setupAvailabilityTable(); - setupAvailabilityTable(measId); + setupAvailabilityTable(); + setupAvailabilityTable(measId); AvailabilityManagerLocal avail = AvailabilityManagerEJBImpl.getOne(); + MeasurementManagerLocal mMan = MeasurementManagerEJBImpl.getOne(); Measurement meas = mMan.getMeasurement(measId); long interval = meas.getInterval(); - long now = System.currentTimeMillis(); + long now = now(); long baseTime = TimingVoodoo.roundDownTime(now, 600000); DataPoint pt; List list = new ArrayList(); @@ -410,7 +414,7 @@ private void testNonOneorZeroDupPtInsertAtBegin() throws Exception { setupAvailabilityTable(); long INCRTIME = 60000; - long baseTime = System.currentTimeMillis(); + long baseTime = now(); baseTime = TimingVoodoo.roundDownTime(baseTime, 60000); long tmpTime = baseTime; @@ -436,7 +440,7 @@ private void testPrependWithDupValue() throws Exception { setupAvailabilityTable(); - long baseTime = System.currentTimeMillis(); + long baseTime = now(); baseTime = TimingVoodoo.roundDownTime(baseTime, 60000); long tmpTime = baseTime; @@ -463,7 +467,7 @@ private void testPrepend() throws Exception { setupAvailabilityTable(); - long baseTime = System.currentTimeMillis(); + long baseTime = now(); baseTime = TimingVoodoo.roundDownTime(baseTime, 60000); long tmpTime = baseTime; @@ -493,7 +497,7 @@ setupAvailabilityTable(); int INCRTIME = 240000; - long baseTime = System.currentTimeMillis(); + long baseTime = now(); baseTime = TimingVoodoo.roundDownTime(baseTime, 60000); long tmpTime = baseTime; @@ -580,6 +584,7 @@ AvailabilityCache cache = AvailabilityCache.getInstance(); cache.clear(); AvailabilityDataDAO dao = getAvailabilityDataDAO(); + dao.getSession().clear(); boolean descending = false; long start = 0l; long end = AvailabilityDataRLE.getLastTimestamp(); @@ -598,6 +603,7 @@ AvailabilityCache cache = AvailabilityCache.getInstance(); cache.clear(); AvailabilityDataDAO dao = getAvailabilityDataDAO(); + dao.getSession().clear(); boolean descending = false; long start = 0l; long end = AvailabilityDataRLE.getLastTimestamp(); |
|
From: lirl98 <li...@12...> - 2009-12-16 09:32:45
|
There is a net work driver G: in my computer, It maps to "\\192.168.0.2\C:\user" So, how to get the net work driver actual path "\\192.168.0.2\C:\user" ? I have tried the sigar.getFileSystemList(), but it can't get that info. Anyone know how to use sigar or java to get it? Thanks a lot~ |
|
From: <bo...@hy...> - 2009-12-16 08:53:23
|
Author: bob Date: 2009-12-16 00:53:11 -0800 (Wed, 16 Dec 2009) New Revision: 14098 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14098 Modified: trunk/etc/version.properties Log: Release 4.3.0 build #1290 Modified: trunk/etc/version.properties =================================================================== --- trunk/etc/version.properties 2009-12-15 23:02:40 UTC (rev 14097) +++ trunk/etc/version.properties 2009-12-16 08:53:11 UTC (rev 14098) @@ -1,3 +1,3 @@ -#Tue Dec 15 00:15:43 PST 2009 +#Wed Dec 16 00:16:02 PST 2009 version=4.3.0 -build=1289 +build=1290 |
|
From: <sc...@hy...> - 2009-12-15 23:02:51
|
Author: scottmf Date: 2009-12-15 15:02:40 -0800 (Tue, 15 Dec 2009) New Revision: 14097 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14097 Modified: trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java Log: revert last change. Change ejb back to NotSupported Modified: trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java =================================================================== --- trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java 2009-12-15 22:42:12 UTC (rev 14096) +++ trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java 2009-12-15 23:02:40 UTC (rev 14097) @@ -30,7 +30,7 @@ * type="Stateless" * * @ejb:util generate="physical" - * @ejb:transaction type="Bean" + * @ejb:transaction type="NotSupported" */ public class EscalationManagerTestEJBImpl implements SessionBean { |
|
From: <sc...@hy...> - 2009-12-15 22:42:25
|
Author: scottmf Date: 2009-12-15 14:42:12 -0800 (Tue, 15 Dec 2009) New Revision: 14096 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14096 Modified: trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java Log: change test from NotSupported to Bean managed Modified: trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java =================================================================== --- trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java 2009-12-15 22:39:54 UTC (rev 14095) +++ trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java 2009-12-15 22:42:12 UTC (rev 14096) @@ -30,7 +30,7 @@ * type="Stateless" * * @ejb:util generate="physical" - * @ejb:transaction type="NotSupported" + * @ejb:transaction type="Bean" */ public class EscalationManagerTestEJBImpl implements SessionBean { |
|
From: <sc...@hy...> - 2009-12-15 22:40:06
|
Author: scottmf Date: 2009-12-15 14:39:54 -0800 (Tue, 15 Dec 2009) New Revision: 14095 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14095 Modified: trunk/unittest/data/escalationManagerTests.xml.gz Log: take away HQ_SEQUENCE table for postgres Modified: trunk/unittest/data/escalationManagerTests.xml.gz =================================================================== (Binary files differ) |
|
From: <no...@gi...> - 2009-12-15 21:27:55
|
Branch: refs/heads/master Home: http://github.com/hyperic/hqapi Commit: c59a3b59a5e8d9190cb1e5dcf782081b7cd6e93f http://github.com/hyperic/hqapi/commit/c59a3b59a5e8d9190cb1e5dcf782081b7cd6e93f Author: Ryan Morgan <rm...@hy...> Date: 2009-12-15 (Tue, 15 Dec 2009) Changed paths: M src/org/hyperic/hq/hqapi1/tools/Command.java Log Message: ----------- Clarify path to client.properties must be absolute. |
|
From: <sc...@hy...> - 2009-12-15 20:51:59
|
Author: scottmf Date: 2009-12-15 12:51:47 -0800 (Tue, 15 Dec 2009) New Revision: 14094 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14094 Modified: trunk/plugins/oracle/etc/hq-plugin.xml trunk/plugins/oracle/src/org/hyperic/hq/plugin/oracle/OracleServerDetector.java Log: [HHQ-3577] fixed up installpath and added the ability to use tnsnames in the jdbc url Modified: trunk/plugins/oracle/etc/hq-plugin.xml =================================================================== --- trunk/plugins/oracle/etc/hq-plugin.xml 2009-12-15 20:44:25 UTC (rev 14093) +++ trunk/plugins/oracle/etc/hq-plugin.xml 2009-12-15 20:51:47 UTC (rev 14094) @@ -77,7 +77,7 @@ </script> <classpath> - <include name="pdk/lib/jdbc/oracle12.jar"/> + <include name="pdk/lib/jdbc/ojdbc14.jar"/> </classpath> <!-- we use a dummy jmx object name --> Modified: trunk/plugins/oracle/src/org/hyperic/hq/plugin/oracle/OracleServerDetector.java =================================================================== --- trunk/plugins/oracle/src/org/hyperic/hq/plugin/oracle/OracleServerDetector.java 2009-12-15 20:44:25 UTC (rev 14093) +++ trunk/plugins/oracle/src/org/hyperic/hq/plugin/oracle/OracleServerDetector.java 2009-12-15 20:51:47 UTC (rev 14094) @@ -67,6 +67,8 @@ private static final String PROP_PROC_PTQL = "process.ptql"; + private static final String PROP_TNSNAMES = "tnsnames"; + private static final String ORATAB = "/etc/oratab"; private static final Pattern _serviceNameEx = @@ -190,8 +192,12 @@ bin = new File(oracle, "bin"); } if (bin.getName().equals("bin")) { - path = oracle.getParent(); + path = bin.getParent(); } + // in HHQ-3577, changed the default installpath. Therefore we need to + // make sure that a new oracle inst is not detected, rather it should + // just modify the existing one. + String aiid = oracle.getParent(); // Make sure that oracle exists, and is a normal file if (oracle.exists() && bin.isDirectory()) { @@ -213,6 +219,7 @@ if (found) { ConfigResponse productConfig = new ConfigResponse(); ServerResource server = createServerResource(path); + server.setIdentifier(aiid); // Set custom properties ConfigResponse cprop = new ConfigResponse(); @@ -225,6 +232,11 @@ servers.add(server); } + // HHQ-3577 allow listener names in tnsnames.ora to be used in the url + String tnsDir = getTnsNamesDir(path, "network/admin/tnsnames.ora"); + if (log.isDebugEnabled()) log.debug("using tns dir as " + tnsDir); + System.setProperty("oracle.net.tns_admin", tnsDir); + return servers; } @@ -290,6 +302,13 @@ protected List discoverServices(ConfigResponse config) throws PluginException { + // HHQ-3577 allow listener names in tnsnames.ora to be used in the url + String tnsDir = getTnsNamesDir( + config.getValue(ProductPlugin.PROP_INSTALLPATH), + config.getValue(PROP_TNSNAMES)); + if (log.isDebugEnabled()) log.debug("using tns dir as " + tnsDir); + System.setProperty("oracle.net.tns_admin", tnsDir); + String url = config.getValue(OracleMeasurementPlugin.PROP_URL); if (url == null) { log.warn("No value for config property " + OracleMeasurementPlugin.PROP_URL + @@ -344,7 +363,7 @@ } // Otherwise, dump the error. throw new PluginException("Error querying for Oracle " + - "services: " + e.getMessage()); + "services: " + e.getMessage(), e); } finally { DBUtil.closeJDBCObjects(log, conn, stmt, null); @@ -555,11 +574,24 @@ return rtn; } + private String getTnsNamesDir(String installpath, String tnsnames) { + if (installpath == null || tnsnames == null) { + return ""; + } + String fs = File.separator; + String[] toks = tnsnames.split(fs); + StringBuilder rtn = new StringBuilder(); + for (int i=0; i<toks.length-1; i++) { + rtn.append(toks[i]).append(fs); + } + return installpath + fs + rtn.toString(); + } + private List getTnsServices(ConfigResponse config) { String line; BufferedReader reader = null; - String tnsnames = config.getValue("tnsnames"), + String tnsnames = config.getValue(PROP_TNSNAMES), installpath = config.getValue(ProductPlugin.PROP_INSTALLPATH); List rtn = new ArrayList(); try |
|
From: <sc...@hy...> - 2009-12-15 20:44:39
|
Author: scottmf Date: 2009-12-15 12:44:25 -0800 (Tue, 15 Dec 2009) New Revision: 14093 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14093 Modified: trunk/unittest/data/escalationManagerTests.README trunk/unittest/data/escalationManagerTests.xml.gz trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java Log: added extra table EAM_REGISTERED_TRIGGER, test should not fail with foreign key violation Modified: trunk/unittest/data/escalationManagerTests.README =================================================================== --- trunk/unittest/data/escalationManagerTests.README 2009-12-15 17:44:35 UTC (rev 14092) +++ trunk/unittest/data/escalationManagerTests.README 2009-12-15 20:44:25 UTC (rev 14093) @@ -1,4 +1,4 @@ -bin/exportTable.sh -l jdbc:oracle:thin:@localhost:1522:hqdb -u hqadmin -p hqadmin \ +bin/exportTable.sh -l jdbc:mysql://localhost:3306/hqdb -u hqadmin -p hqadmin \ -t EAM_AGENT,\ EAM_AUDIT,\ EAM_CONFIG_RESPONSE,\ @@ -14,6 +14,7 @@ EAM_PLATFORM,\ EAM_PLATFORM_SERVER_TYPE_MAP,\ EAM_PLATFORM_TYPE,\ +EAM_REGISTERED_TRIGGER,\ EAM_RES_GRP_RES_MAP,\ EAM_RESOURCE,\ EAM_RESOURCE_EDGE,\ @@ -36,4 +37,5 @@ EAM_ALERT_DEFINITION,\ EAM_ALERT,\ EAM_ALERT_CONDITION,\ -EAM_ACTION -d escalationManagerTest +HQ_SEQUENCE,\ +EAM_ACTION -d escalationManagerTests Modified: trunk/unittest/data/escalationManagerTests.xml.gz =================================================================== (Binary files differ) Modified: trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java =================================================================== --- trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java 2009-12-15 17:44:35 UTC (rev 14092) +++ trunk/unittest/src/org/hyperic/hq/escalation/server/session/EscalationManagerTestEJBImpl.java 2009-12-15 20:44:25 UTC (rev 14093) @@ -52,16 +52,16 @@ state.setAcknowledgedBy(null); state.setNextAction(0); state.setNextActionTime(System.currentTimeMillis()+MeasurementConstants.DAY); - int alertId = 10105; - Escalation esc = eMan.findById(new Integer(100)); + int alertId = 10100; + Escalation esc = eMan.findById(new Integer(10100)); state.setEscalation(esc); state.setAlertId(alertId); - state.setAlertDefinitionId(10001); + state.setAlertDefinitionId(10100); state.setAlertTypeEnum(-559038737); getOne().runEscalation(state); aMan.deleteAlerts(new Integer[] {new Integer(alertId)}); Assert.assertNull( - "alert 10105 should not exist", aMan.getAlertById(new Integer(10105))); + "alert " + alertId + " should not exist", aMan.getAlertById(new Integer(alertId))); try { // should exit without any errors eMan.executeState(state.getId()); |
|
From: <tr...@hy...> - 2009-12-15 17:44:47
|
Author: trader Date: 2009-12-15 09:44:35 -0800 (Tue, 15 Dec 2009) New Revision: 14092 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14092 Modified: trunk/unittest/src/org/hyperic/util/file/DiskListTest.java Log: Added one unittest Modified: trunk/unittest/src/org/hyperic/util/file/DiskListTest.java =================================================================== --- trunk/unittest/src/org/hyperic/util/file/DiskListTest.java 2009-12-15 08:52:17 UTC (rev 14091) +++ trunk/unittest/src/org/hyperic/util/file/DiskListTest.java 2009-12-15 17:44:35 UTC (rev 14092) @@ -104,6 +104,56 @@ } } + public void testFillAndReopen() throws Exception { + + DiskListDataHolder holder = null; + + try { + + try { + holder = new DiskListDataHolder(); + } catch (Exception e) { + e.printStackTrace(); + fail(e.toString()); + } + + String toPut = String.valueOf("dummystring"); + + // Insert until we *almost* spill over + long nRecs = 0; + while (holder.list.dataFile.length() < MAXSIZE) { + holder.list.addToList(toPut); + nRecs++; + } + + holder.list.close(); + + // Check that we can read the proper number after close/reopen, and that they can be cleanly deleted + holder.list = new DiskList(holder.dataFile, + RECSIZE, + CHKSIZE, + CHKPERC, + MAXSIZE); + + assertTrue(holder.list.freeList.size() == 0); + + Iterator it = holder.list.getListIterator(); + long nIterated = 0; + while (it.hasNext()) { + it.next(); + nIterated++; + } + + assertTrue("Expected " + nRecs + " records, got " + nIterated, + nIterated == nRecs); + + } finally { + + holder.dispose(); + + } + } + public void testFreeListWithInsertsAndNoDeletes() throws Exception { DiskListDataHolder holder = null; |
|
From: <bo...@hy...> - 2009-12-15 08:52:30
|
Author: bob Date: 2009-12-15 00:52:17 -0800 (Tue, 15 Dec 2009) New Revision: 14091 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14091 Modified: trunk/etc/version.properties Log: Release 4.3.0 build #1289 Modified: trunk/etc/version.properties =================================================================== --- trunk/etc/version.properties 2009-12-14 23:26:56 UTC (rev 14090) +++ trunk/etc/version.properties 2009-12-15 08:52:17 UTC (rev 14091) @@ -1,3 +1,3 @@ -#Mon Dec 14 00:16:16 PST 2009 +#Tue Dec 15 00:15:43 PST 2009 version=4.3.0 -build=1288 +build=1289 |
|
From: <bo...@hy...> - 2009-12-14 23:27:09
|
Author: bob Date: 2009-12-14 15:26:56 -0800 (Mon, 14 Dec 2009) New Revision: 14090 URL: http://svn.hyperic.org/?view=rev&root=Hyperic+HQ&revision=14090 Modified: tags/HQ_4_2_0_GA/etc/version.properties Log: Release 4.2.0 build #1261 Modified: tags/HQ_4_2_0_GA/etc/version.properties =================================================================== --- tags/HQ_4_2_0_GA/etc/version.properties 2009-12-14 21:43:44 UTC (rev 14089) +++ tags/HQ_4_2_0_GA/etc/version.properties 2009-12-14 23:26:56 UTC (rev 14090) @@ -1,3 +1,3 @@ -#Fri Oct 30 12:14:25 PDT 2009 +#Mon Dec 14 14:58:51 PST 2009 version=4.2.0 -build=1260 +build=1261 |
14 messages has been excluded from this view by a project administrator.
