bigdata-commit — Commit traffic.

[Bigdata-commit] SF.net SVN: bigdata:[3799] branches/QUADS_QUERY_BRANCH/src/resources/config /bigdataStandalone.config

[<tho...@us...>]

Revision: 3799
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3799&view=rev
Author:   thompsonbry
Date:     2010-10-14 14:41:34 +0000 (Thu, 14 Oct 2010)

Log Message:
-----------
Reduced the LBS heap allocation since the federation clients by default now report only the essential performance counters for load balancing.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config

Modified: branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config
===================================================================
--- branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-14 12:58:54 UTC (rev 3798)
+++ branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-14 14:41:34 UTC (rev 3799)
@@ -1152,7 +1152,7 @@
      * keeps the performance counter histories in RAM.  While those histories
      * are bounded, it still uses more RAM than it should.
      */
-	"-Xmx2G",
+	"-Xmx1G",
 	/*
 	 * Enable JXM remote management for the data service.
 	 *


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3798] branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/ bigdata/io/writecache/TestWORMWriteCacheService.java

[<mar...@us...>]

Revision: 3798
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3798&view=rev
Author:   martyncutcher
Date:     2010-10-14 12:58:54 +0000 (Thu, 14 Oct 2010)

Log Message:
-----------
reduce test magnitude to avoid GC issues

Modified Paths:
--------------
    branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/io/writecache/TestWORMWriteCacheService.java

Modified: branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/io/writecache/TestWORMWriteCacheService.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/io/writecache/TestWORMWriteCacheService.java	2010-10-14 12:54:15 UTC (rev 3797)
+++ branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/io/writecache/TestWORMWriteCacheService.java	2010-10-14 12:58:54 UTC (rev 3798)
@@ -129,7 +129,7 @@
      * often absorb the data immediately. 100k is reasonable.
      */
 //    static final int nrecs = 10;
-    static final int nrecs = 100000;
+    static final int nrecs = 10000;
 
     /**
      * The #of records to write for an RW store.


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3797] branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/ com/bigdata/rdf/rules/TestCompareFullAndFastClosure.java

[<mar...@us...>]

Revision: 3797
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3797&view=rev
Author:   martyncutcher
Date:     2010-10-14 12:54:15 +0000 (Thu, 14 Oct 2010)

Log Message:
-----------
Resource path must be absolute for usage - start with "/"

Modified Paths:
--------------
    branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestCompareFullAndFastClosure.java

Modified: branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestCompareFullAndFastClosure.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestCompareFullAndFastClosure.java	2010-10-14 12:33:37 UTC (rev 3796)
+++ branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestCompareFullAndFastClosure.java	2010-10-14 12:54:15 UTC (rev 3797)
@@ -90,7 +90,7 @@
         
         // String[] resource = new String[]{"/com/bigdata/rdf/rules/testOwlSameAs.rdf"};
         // String[] resource = new String[]{"/com/bigdata/rdf/rules/testOwlSameAs.rdf"};
-        String[] resource = new String[]{"com/bigdata/rdf/rules/small.rdf"};
+        String[] resource = new String[]{"/com/bigdata/rdf/rules/small.rdf"};
         String[] baseURL = new String[]{""};
         RDFFormat[] format = new RDFFormat[]{RDFFormat.RDFXML};
 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3796] branches/QUADS_QUERY_BRANCH

[<tho...@us...>]

Revision: 3796
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3796&view=rev
Author:   thompsonbry
Date:     2010-10-14 12:33:37 +0000 (Thu, 14 Oct 2010)

Log Message:
-----------
Added IBigdataClient.Options.REPORT_ALL, which controls whether or not only the minimum set of performance counters are reported to the load balancer for its proper functioning (default true) or whether all collected performance counters are reported.  This may be overridden in the configuration file to collect more information on a cluster.

This change was made to offer a short-term option for reducing the memory burden imposed by collecting all performance counters at the load balancer.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/counters/query/QueryUtil.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/LoadBalancerService.java
    branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/counters/query/QueryUtil.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/counters/query/QueryUtil.java	2010-10-14 11:33:31 UTC (rev 3795)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/counters/query/QueryUtil.java	2010-10-14 12:33:37 UTC (rev 3796)
@@ -56,12 +56,20 @@
 import com.bigdata.counters.HistoryInstrument;
 import com.bigdata.counters.ICounter;
 import com.bigdata.counters.ICounterNode;
+import com.bigdata.counters.ICounterSet;
+import com.bigdata.counters.IHostCounters;
+import com.bigdata.counters.IRequiredHostCounters;
 import com.bigdata.counters.PeriodEnum;
 import com.bigdata.counters.History.SampleIterator;
 import com.bigdata.counters.ICounterSet.IInstrumentFactory;
 import com.bigdata.counters.httpd.DummyEventReportingService;
+import com.bigdata.journal.ConcurrencyManager.IConcurrencyManagerCounters;
 import com.bigdata.rawstore.Bytes;
+import com.bigdata.resources.ResourceManager.IResourceManagerCounters;
+import com.bigdata.resources.StoreManager.IStoreManagerCounters;
 import com.bigdata.service.Event;
+import com.bigdata.service.DataService.IDataServiceCounters;
+import com.bigdata.util.concurrent.IQueueCounters.IThreadPoolExecutorTaskCounters;
 
 /**
  * Some static utility methods.
@@ -578,5 +586,79 @@
         return out;
         
     }
-    
+
+	/**
+	 * Return a {@link Pattern} which will match the minimum set of performance
+	 * counters required by the load balancer to perform its function.
+	 */
+	public static Pattern getRequiredPerformanceCountersFilter() {
+
+		return requiredPerformanceCountersFilter;
+
+	}
+
+	private static final String[] requiredPerformanceCounterPaths = new String[] {
+
+			IRequiredHostCounters.Memory_majorFaultsPerSecond,
+
+			IRequiredHostCounters.LogicalDisk_PercentFreeSpace,
+
+			IRequiredHostCounters.CPU_PercentProcessorTime,
+
+			IHostCounters.CPU_PercentIOWait,
+
+			IDataServiceCounters.concurrencyManager + ICounterSet.pathSeparator
+					+ IConcurrencyManagerCounters.writeService
+					+ ICounterSet.pathSeparator
+					+ IThreadPoolExecutorTaskCounters.AverageQueuingTime,
+
+			IDataServiceCounters.resourceManager + ICounterSet.pathSeparator
+					+ IResourceManagerCounters.StoreManager
+					+ ICounterSet.pathSeparator
+					+ IStoreManagerCounters.DataDirBytesAvailable,
+
+			IDataServiceCounters.resourceManager + ICounterSet.pathSeparator
+					+ IResourceManagerCounters.StoreManager
+					+ ICounterSet.pathSeparator
+					+ IStoreManagerCounters.TmpDirBytesAvailable, };
+
+	private static final Pattern requiredPerformanceCountersFilter = QueryUtil
+			.getPattern(
+					Arrays.asList(requiredPerformanceCounterPaths)/* strings */,
+					null/* regex */);
+
+	/**
+	 * Utility may be used to read the required performance counters for the
+	 * load balancer from zero or more files specified on the command line. The
+	 * results are written using the XML interchange format on stdout.
+	 * 
+	 * @param args The file(s).
+	 * 
+	 * @throws IOException
+	 * @throws SAXException
+	 * @throws ParserConfigurationException
+	 */
+	public static void main(String[] args) throws IOException, SAXException,
+			ParserConfigurationException {
+
+		final Pattern filter = getRequiredPerformanceCountersFilter();
+
+		System.err.println("required counter pattern: " + filter);
+
+		final CounterSet counterSet = new CounterSet();
+
+		for (String s : args) {
+
+			final File file = new File(s);
+
+			readCountersFromFile(file, counterSet, filter,
+					new DefaultInstrumentFactory(60/* slots */,
+							PeriodEnum.Minutes, false/* overwrite */));
+
+		}
+
+		System.out.println("counters: " + counterSet.asXML(null/* filter */));
+
+	}
+
 }

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java	2010-10-14 11:33:31 UTC (rev 3795)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java	2010-10-14 12:33:37 UTC (rev 3796)
@@ -32,6 +32,7 @@
 import java.io.IOException;
 import java.io.UnsupportedEncodingException;
 import java.net.URLEncoder;
+import java.util.Arrays;
 import java.util.HashMap;
 import java.util.LinkedList;
 import java.util.Map;
@@ -45,6 +46,7 @@
 import java.util.concurrent.ScheduledFuture;
 import java.util.concurrent.ThreadPoolExecutor;
 import java.util.concurrent.TimeUnit;
+import java.util.regex.Pattern;
 
 import org.apache.log4j.Logger;
 
@@ -57,12 +59,19 @@
 import com.bigdata.counters.CounterSet;
 import com.bigdata.counters.ICounter;
 import com.bigdata.counters.ICounterSet;
+import com.bigdata.counters.IHostCounters;
+import com.bigdata.counters.IRequiredHostCounters;
 import com.bigdata.counters.IServiceCounters;
 import com.bigdata.counters.OneShotInstrument;
+import com.bigdata.counters.query.QueryUtil;
 import com.bigdata.journal.TemporaryStore;
 import com.bigdata.journal.TemporaryStoreFactory;
+import com.bigdata.journal.ConcurrencyManager.IConcurrencyManagerCounters;
 import com.bigdata.rawstore.Bytes;
 import com.bigdata.relation.locator.DefaultResourceLocator;
+import com.bigdata.resources.ResourceManager.IResourceManagerCounters;
+import com.bigdata.resources.StoreManager.IStoreManagerCounters;
+import com.bigdata.service.DataService.IDataServiceCounters;
 import com.bigdata.service.IBigdataClient.Options;
 import com.bigdata.service.ndx.IClientIndex;
 import com.bigdata.service.ndx.ScaleOutIndexCounters;
@@ -72,6 +81,7 @@
 import com.bigdata.util.concurrent.ShutdownHelper;
 import com.bigdata.util.concurrent.TaskCounters;
 import com.bigdata.util.concurrent.ThreadPoolExecutorStatisticsTask;
+import com.bigdata.util.concurrent.IQueueCounters.IThreadPoolExecutorTaskCounters;
 import com.bigdata.util.httpd.AbstractHTTPD;
 
 /**
@@ -1391,26 +1401,39 @@
 
             }
 
-            /*
-             * @todo this is probably worth compressing as there will be a lot
-             * of redundency.
-             * 
-             * @todo allow filter on what gets sent to the load balancer?
-             */
-            final ByteArrayOutputStream baos = new ByteArrayOutputStream(
-                    Bytes.kilobyte32 * 2);
+			/*
+			 * @todo When sending all counters, this is probably worth
+			 * compressing as there will be a lot of redundancy and a lot of
+			 * data.
+			 */
+			final ByteArrayOutputStream baos = new ByteArrayOutputStream(
+					Bytes.kilobyte32 * 2);
 
-            fed.getCounterSet().asXML(baos, "UTF-8", null/* filter */);
+			final Properties p = fed.getClient().getProperties();
 
+			final boolean reportAll = Boolean.valueOf(p.getProperty(
+					Options.REPORT_ALL, Options.DEFAULT_REPORT_ALL));
+
+			fed.getCounterSet().asXML(
+					baos,
+					"UTF-8",
+					reportAll ? null/* filter */: QueryUtil
+							.getRequiredPerformanceCountersFilter());
+
+			if (log.isInfoEnabled())
+				log.info("reportAll=" + reportAll + ", service="
+						+ fed.getServiceName() + ", #bytesReported="
+						+ baos.size());
+
             loadBalancerService.notify(serviceUUID, baos.toByteArray());
 
             if (log.isInfoEnabled())
                 log.info("Notified the load balancer.");
-            
-        }
 
-    }
+		}
 
+	}
+
     /**
      * @todo it may be possible to optimize this for the jini case.
      */

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java	2010-10-14 11:33:31 UTC (rev 3795)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java	2010-10-14 12:33:37 UTC (rev 3796)
@@ -34,6 +34,7 @@
 import com.bigdata.cache.HardReferenceQueue;
 import com.bigdata.counters.AbstractStatisticsCollector;
 import com.bigdata.counters.CounterSet;
+import com.bigdata.counters.query.QueryUtil;
 import com.bigdata.journal.ITx;
 import com.bigdata.relation.accesspath.IAccessPath;
 import com.bigdata.relation.locator.ILocatableResource;
@@ -456,6 +457,19 @@
          * The default {@link #REPORT_DELAY}.
          */
         String DEFAULT_REPORT_DELAY = ""+(60*1000);
+
+		/**
+		 * When <code>true</code>, all collected performance counters are
+		 * reported (default {@value #DEFAULT_REPORT_ALL)}. Otherwise only the
+		 * {@link QueryUtil#getRequiredPerformanceCountersFilter()} will be
+		 * reported. Reporting all performance counters is useful when
+		 * diagnosing the services in a cluster. However, only a small number of
+		 * performance counters are actually necessary for the functioning of
+		 * the {@link ILoadBalancerService}.
+		 */
+		String REPORT_ALL = IBigdataClient.class.getName() + ".reportAll";
+
+		String DEFAULT_REPORT_ALL = "false";
     
         /**
          * Integer option specifies the port on which an httpd service will be

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/LoadBalancerService.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/LoadBalancerService.java	2010-10-14 11:33:31 UTC (rev 3795)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/LoadBalancerService.java	2010-10-14 12:33:37 UTC (rev 3796)
@@ -36,6 +36,7 @@
 import com.bigdata.counters.IRequiredHostCounters;
 import com.bigdata.counters.PeriodEnum;
 import com.bigdata.counters.ICounterSet.IInstrumentFactory;
+import com.bigdata.counters.query.QueryUtil;
 import com.bigdata.journal.BufferMode;
 import com.bigdata.journal.Journal;
 import com.bigdata.journal.ConcurrencyManager.IConcurrencyManagerCounters;
@@ -932,6 +933,8 @@
      *       service and report the largest value for lastModified() on any
      *       counter for that service.
      * 
+     * @see QueryUtil#getRequiredPerformanceCountersFilter()
+     * 
      * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
      * @version $Id$
      */

Modified: branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config
===================================================================
--- branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-14 11:33:31 UTC (rev 3795)
+++ branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-14 12:33:37 UTC (rev 3796)
@@ -370,8 +370,16 @@
 	 * Turn off periodic performance counter reporting to reduce the memory
 	 * demand on the LBS (performance counters are used for moves on a multi-
 	 * machine cluster and provide detailed reporting on the bigdata internals).
+	 * (default 60 seconds).
 	 */
-	new NV(IBigdataClient.Options.REPORT_DELAY,"0"),
+	//new NV(IBigdataClient.Options.REPORT_DELAY,"0"),
+	
+	/*
+	 * When true, report all performance counters.  When false, report just the
+	 * performance counters required by the load balancer for its correct
+	 * functioning (default false).
+	 */
+	//new NV(IBigdataClient.Options.REPORT_ALL,"false"),
 
     };
 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3795] branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/ bigdata/journal/RWStrategy.java

[<mar...@us...>]

Revision: 3795
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3795&view=rev
Author:   martyncutcher
Date:     2010-10-14 11:33:31 +0000 (Thu, 14 Oct 2010)

Log Message:
-----------
Ensure both rootblocks are correctly initialized on a new RWStore

Modified Paths:
--------------
    branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/journal/RWStrategy.java

Modified: branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/journal/RWStrategy.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/journal/RWStrategy.java	2010-10-14 08:54:19 UTC (rev 3794)
+++ branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/journal/RWStrategy.java	2010-10-14 11:33:31 UTC (rev 3795)
@@ -115,6 +115,8 @@
 		m_store = new RWStore(m_fmv, false, quorum); // not read-only for now
 		m_open = true;
 		
+		m_rb = getRootBlock(); // ensure values correct from create/reopen
+		
 		m_rb0 = copyRootBlock(true);
 		m_rb1 = copyRootBlock(false);
 		


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3794] branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/ com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java

[<mar...@us...>]

Revision: 3794
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3794&view=rev
Author:   martyncutcher
Date:     2010-10-14 08:54:19 +0000 (Thu, 14 Oct 2010)

Log Message:
-----------
Fix second reference to unencoded URI

Modified Paths:
--------------
    branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java

Modified: branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java	2010-10-13 21:27:06 UTC (rev 3793)
+++ branches/JOURNAL_HA_BRANCH/bigdata-rdf/src/test/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java	2010-10-14 08:54:19 UTC (rev 3794)
@@ -721,9 +721,29 @@
                     } catch (FileNotFoundException ex) {
 
                         is = getClass().getResourceAsStream(resource);
-                        baseURI = getClass().getResource(resource).toURI()
-                                .toString();
+                        java.net.URL resourceUrl = 
+                            getClass().getResource(resource);
 
+                        //if the resource couldn't be found in the file system
+                        //and couldn't be found by searching from this class'
+                        //package (com.bigdata.rdf.rules) as root, then use
+                        //the class loader to try searching from the root of
+                        //the JAR itself
+                        if (resourceUrl == null) {
+                            is = getClass().getClassLoader().getResourceAsStream(resource);
+                            resourceUrl = 
+                                getClass().getClassLoader().getResource(resource);
+                        }
+
+                        if (resourceUrl == null) {
+                            log.warn("resource not found ["+resource+"]");
+                            throw new Exception("FAILURE: resource not found ["+resource+"]");
+                        }
+
+                        //must encode spaces in URL before new URI
+                        String encodedUrlStr = resourceUrl.toString().replaceAll(" ", "%20");
+                        java.net.URI resourceUri = new java.net.URI(encodedUrlStr);
+                        baseURI = resourceUri.toString();
                     }
 
                     if (is == null) {


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3793] branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/ bigdata/journal

[<sgo...@us...>]

Revision: 3793
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3793&view=rev
Author:   sgossard
Date:     2010-10-13 21:27:06 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Removing weird directories with java class names.  These appear to have been deleted in r3028, then re-added in a later submit as directories.

Removed Paths:
-------------
    branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/journal/ReplicatedStore.java/
    branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/journal/ReplicatedStoreService.java/


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3792] branches/QUADS_QUERY_BRANCH

[<tho...@us...>]

Revision: 3792
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3792&view=rev
Author:   thompsonbry
Date:     2010-10-13 20:44:28 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
1. Added a cache miss handler to AbstractServiceCache#getService(UUID serviceUUID).  This parallels the existing cache miss handling logic for other methods on this class.  This addresses a problem where a test in BigdataFederationSparqlTest would sometimes fail due to a service not yet being in the JiniFederation's local cache.

2. Made _fed an instance field in BigdataFederationSparqlTest.

All tests are now passing for me against two data services.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/lookup/AbstractCachingServiceClient.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java

Modified: branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/lookup/AbstractCachingServiceClient.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/lookup/AbstractCachingServiceClient.java	2010-10-13 19:39:49 UTC (rev 3791)
+++ branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/lookup/AbstractCachingServiceClient.java	2010-10-13 20:44:28 UTC (rev 3792)
@@ -341,28 +341,102 @@
 
     }
 
-    /**
-     * Return the {@link ServiceItem} associated with the {@link UUID}.
-     * 
-     * @param serviceUUID
-     *            The service {@link UUID}.
-     * 
-     * @return The service item iff it is found in the cache and
-     *         <code>null</code> otherwise.
-     */
+	/**
+	 * Return the {@link ServiceItem} associated with the {@link UUID}. -or-
+	 * <code>null</code> if there is no such service in the cache and a remote
+	 * lookup times out.
+	 * 
+	 * @param serviceUUID
+	 *            The service {@link UUID}.
+	 * 
+	 * @return The service item iff it is found in the cache and
+	 *         <code>null</code> otherwise.
+	 */
     final public ServiceItem getServiceItem(final UUID serviceUUID) {
 
         if (serviceUUID == null)
             throw new IllegalArgumentException();
 
-        final ServiceItem serviceItem = serviceCache
-                .getServiceItemByID(JiniUtil.uuid2ServiceID(serviceUUID));
+		final ServiceID serviceId = JiniUtil.uuid2ServiceID(serviceUUID);
 
+		ServiceItem serviceItem = serviceCache
+				.getServiceItemByID(serviceId);
+		
+        if (serviceItem == null) {
+
+            if (log.isInfoEnabled())
+                log.info("Cache miss.");
+
+            serviceItem = handleCacheMiss(serviceId);
+
+            if (serviceItem == null) {
+
+                log.warn("No matching service.");
+
+                return null;
+
+            }
+
+        }
+
         return serviceItem;
 
     }
 
     /**
+     * Handles a cache miss by a remote query on the managed set of service
+     * registrars.
+     * 
+     * @param filter
+     *            The specific filter to be applied.
+     */
+    final private ServiceItem handleCacheMiss(final ServiceID serviceId) {
+
+        ServiceItem item = null;
+
+        try {
+
+			final ServiceTemplate template = new ServiceTemplate(serviceId,
+					null/* serviceTypes */, null/* attrSetTemplates */);
+        	
+            item = serviceDiscoveryManager.lookup(template, filter,
+                    cacheMissTimeout);
+
+        } catch (RemoteException ex) {
+
+            log.error(ex);
+
+            return null;
+
+        } catch (InterruptedException ex) {
+
+            if (log.isInfoEnabled())
+                log.info("Interrupted - no match.");
+
+            return null;
+
+        }
+
+        if (item == null) {
+
+            // Could not discover a matching service.
+
+			log.warn("Could not discover matching service: serviceID="
+					+ serviceId + ", template=" + template + ", filter="
+					+ filter + ", timeout=" + cacheMissTimeout);
+
+            return null;
+
+        }
+
+        if (log.isInfoEnabled())
+            log.info("Found: " + item);
+
+        return item;
+
+    }
+
+    /**
      * Return an array {@link ServiceItem}s for up to <i>maxCount</i>
      * discovered services which satisify the optional <i>filter</i>.
      * 

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 19:39:49 UTC (rev 3791)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 20:44:28 UTC (rev 3792)
@@ -128,15 +128,6 @@
         throws Exception
     {
         super.tearDown () ;
-		/*
-		 * @todo We should destroy the triple store here, but this is causing
-		 * problems with tear down of the query while it is still running. Once
-		 * that issue has been fixed, uncomment both the line to destroy the
-		 * triple store and the line to shutdown the federation (the latter is
-		 * really optional - it should be Ok to leave the federation up across
-		 * the test runs, but then we will never take it down cleanly when the
-		 * test suite is done. Again, that should be Ok.)
-		 */
 		if (_ts != null) {
 			_ts.destroy();
 			_ts = null;
@@ -313,8 +304,11 @@
 	
     private static final Logger _logger = Logger.getLogger ( BigdataFederationSparqlTest.class ) ;
 
-    private static JiniFederation<Object> _fed = null ;
-
+    /*
+     * Instance fields for the current test.
+     */
+    
+    private JiniFederation<Object> _fed = null ;
     private ScaleOutTripleStore _ts = null ;
     private BigdataSail _sail = null;
     


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3791] branches/QUADS_QUERY_BRANCH

[<tho...@us...>]

Revision: 3791
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3791&view=rev
Author:   thompsonbry
Date:     2010-10-13 19:39:49 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
1. Added toString() to DelegateIndexManager to report the DataService
   UUID in log messages.

2. Added putIfAbsent() pattern to FederatedQueryEngine's proxy cache.

3. Added some log messages to FederatedQueryEngine and
   FederatedRunningQuery.  They are at trace and have been commmented
   out now that I have found the source problem.

3. Added some logging and a query optimization to
   Algorithm_NestedLocatorScan.

4. Modified Bundle to pass in the asBound predicate.

5. Bug fix to MapBindingSetsOverShards.  This was the root cause of
   the failure with 2 data services.  The code was using the predicate
   before the binding set had been applied to decide the shard against
   which the next join would occur.  In fact, it needed to use the
   asBound version of the predicate for that purpose.

   Note: While this fixes the problem, we do not yet have a unit test
   which detects the error.

6. Moved the sort of the Bundles out of MapBindingSetsOverShards.
   This can be done by the individual algorithm implementations.

7. Moved rangeCheck() out of AbstractBTreeTupleCursor into BytesUtil
   so it can be reused by MapBindingSetsOverShards (it was used to
   optimize Algorithm_NestedLocatorScan by deciding when a predicate
   lies entirely within a given shard).

8. Restored shutdown of various things to BigdataFederationSparqlTest
   and SPARQLQueryTest.  This issue was a red-herring.  The actual
   problem was in MapBindingSetsOverShards.

9. Moved the RELAY_DELAY option into the properties NV[] (it was being
   set on the command line using -D but the code did not check System.
   getProperties() for a default so that value was being ignored).

With this change set, I am getting 229 out of 230 tests to pass against two data services.  "Basic - List 4" is the one test which is failing on repeat trials.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/DelegateIndexManager.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedQueryEngine.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Bundle.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/MapBindingSetsOverShardsBuffer.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/AbstractBTreeTupleCursor.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/BytesUtil.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties
    branches/QUADS_QUERY_BRANCH/bigdata-perf/lubm/build.properties
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java
    branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/DelegateIndexManager.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/DelegateIndexManager.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/DelegateIndexManager.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -129,4 +129,11 @@
 
     }
 
+    public String toString() {
+
+    	return super.toString() + "{dataServiceUUID="
+				+ dataService.getServiceUUID() + "}";
+    	
+	}
+
 }

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedQueryEngine.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedQueryEngine.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedQueryEngine.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -540,8 +540,14 @@
      */
     protected IQueryPeer getQueryPeer(final UUID serviceUUID) {
 
+		if (serviceUUID == null)
+			throw new IllegalArgumentException();
+
         IQueryPeer proxy = proxyMap.get(serviceUUID);
 
+//		if(log.isTraceEnabled()) log.trace("serviceUUID=" + serviceUUID
+//				+ (proxy != null ? "cached=" + proxy : " not cached."));
+
         if (proxy == null) {
 
             final IDataService dataService = getFederation().getDataService(
@@ -556,8 +562,15 @@
                 throw new RuntimeException(e);
             }
 
-            proxyMap.put(serviceUUID, proxy);
+            IQueryPeer tmp = proxyMap.putIfAbsent(serviceUUID, proxy);
+            
+			if (tmp != null) {
+				proxy = tmp;
+			}
 
+//			if(log.isTraceEnabled()) log.trace("serviceUUID=" + serviceUUID + ", addedToCache="
+//					+ (tmp == null) + ", proxy=" + proxy);
+
         }
 
         return proxy;

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -347,14 +347,21 @@
 			 */
             queryPeer = getQueryEngine();
             
+//            if(log.isTraceEnabled()) log.trace("Target is self: "+serviceUUID);
+            
         } else if (serviceUUID.equals(queryControllerUUID)) {
         
             // The target is the query controller.
             queryPeer = getQueryController();
-            
+
+//            if(log.isTraceEnabled()) log.trace("Target is controller: "+serviceUUID);
+
         } else {
             
             // The target is some data service.
+        	
+//            if(log.isTraceEnabled()) log.trace("Target is peer: "+serviceUUID);
+
             queryPeer = getQueryEngine().getQueryPeer(serviceUUID);
             
         }

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -1,10 +1,12 @@
 package com.bigdata.bop.fed.shards;
 
+import java.util.Arrays;
 import java.util.Iterator;
 
 import org.apache.log4j.Logger;
 
 import com.bigdata.bop.IBindingSet;
+import com.bigdata.btree.BytesUtil;
 import com.bigdata.mdi.IMetadataIndex;
 import com.bigdata.mdi.PartitionLocator;
 import com.bigdata.relation.accesspath.IBuffer;
@@ -31,8 +33,48 @@
 
     public void mapOverShards(final Bundle<F>[] bundles) {
 
-        for (Bundle<F> bundle : bundles) {
+		/*
+		 * Sort the binding sets in the chunk by the fromKey associated with
+		 * each asBound predicate.
+		 */
+		Arrays.sort(bundles);
 
+		// The most recently discovered locator.
+		PartitionLocator current = null;
+		
+//		// The list of binding sets which are bound for the current locator.
+//		List<IBindingSet> list = new LinkedList<IBindingSet>();
+		
+		final Iterator<Bundle<F>> bitr = Arrays.asList(bundles).iterator();
+
+		while(bitr.hasNext()) {
+			
+			final Bundle<F> bundle = bitr.next();
+
+			if (current != null
+					&& BytesUtil.rangeCheck(bundle.fromKey, current
+							.getLeftSeparatorKey(), current
+							.getRightSeparatorKey())
+					&& BytesUtil.rangeCheck(bundle.toKey, current
+							.getLeftSeparatorKey(), current
+							.getRightSeparatorKey())) {
+
+				/*
+				 * Optimization when the bundle fits inside of the last index
+				 * partition scanned (this optimization is only possible when
+				 * the asBound predicate will be mapped onto a single index
+				 * partition, but this is a very common case since we try to
+				 * choose selective indices for access paths).
+				 */
+				
+                final IBuffer<IBindingSet[]> sink = op.getBuffer(current);
+
+                sink.add(new IBindingSet[] { bundle.bindingSet });
+
+                continue;
+                
+			}
+			
             /*
              * Locator scan for the index partitions for that predicate as
              * bound.
@@ -42,11 +84,12 @@
 
             while (itr.hasNext()) {
 
-                final PartitionLocator locator = itr.next();
+                final PartitionLocator locator = current = itr.next();
 
                 if (log.isTraceEnabled())
-					log.trace("adding bindingSet to buffer" + ": partitionId="
-							+ locator.getPartitionId() + "dataService="
+					log.trace("adding bindingSet to buffer" + ": asBound="
+							+ bundle.asBound + ", partitionId="
+							+ locator.getPartitionId() + ", dataService="
 							+ locator.getDataServiceUUID() + ", bindingSet="
 							+ bundle.bindingSet);
 
@@ -57,6 +100,7 @@
             }
 
         }
+        
     }
 
 }

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Bundle.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Bundle.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Bundle.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -30,12 +30,12 @@
     /** The toKey generated from that asBound predicate. */
     final byte[] toKey;
 
-    public Bundle(final IKeyBuilder keyBuilder, final IPredicate<F> pred,
+    public Bundle(final IKeyBuilder keyBuilder, final IPredicate<F> asBound,
             final IKeyOrder<F> keyOrder, final IBindingSet bindingSet) {
 
         this.bindingSet = bindingSet;
 
-        this.asBound = pred.asBound(bindingSet);
+        this.asBound = asBound;
 
         this.keyOrder = keyOrder;
         

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/MapBindingSetsOverShardsBuffer.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/MapBindingSetsOverShardsBuffer.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/MapBindingSetsOverShardsBuffer.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -292,27 +292,37 @@
         @SuppressWarnings("unchecked")
         final Bundle<F>[] bundles = new Bundle[chunk.length];
 
-        /*
-         * Create the asBound version of the predicate and the associated
-         * fromKey for each bindingSet in the chunk.
-         */
-        for (int i = 0; i < chunk.length; i++) {
+		/*
+		 * Create the asBound version of the predicate and the associated
+		 * fromKey for each bindingSet in the chunk.
+		 */
+		for (int i = 0; i < chunk.length; i++) {
 
-            final IKeyOrder<F> keyOrder = relation.getKeyOrder(pred);
+			// an intermediate solution.
+			final IBindingSet bindingSet = chunk[i];
 
-            final IKeyBuilder keyBuilder = relation.getIndex(keyOrder)
-                    .getIndexMetadata().getKeyBuilder();
-            
-            bundles[i] = new Bundle<F>(keyBuilder, pred, keyOrder, chunk[i]);
-            
-        }
+			// the asBound version of the predicate.
+			final IPredicate<F> asBound = pred.asBound(bindingSet);
 
-        /*
-         * Sort the binding sets in the chunk by the fromKey associated with
-         * each asBound predicate.
-         */
-        Arrays.sort(bundles);
+			// the index which will be used for that asBound predicate.
+			final IKeyOrder<F> keyOrder = relation.getKeyOrder(asBound);
 
+			// the key builder associated with that index.
+			final IKeyBuilder keyBuilder = relation.getIndex(keyOrder)
+					.getIndexMetadata().getKeyBuilder();
+
+			// save the bundle for processing.
+			bundles[i] = new Bundle<F>(keyBuilder, asBound, keyOrder,
+					bindingSet);
+
+		}
+
+//        /*
+//         * Sort the binding sets in the chunk by the fromKey associated with
+//         * each asBound predicate. [Sort is moved into the implementation.]
+//         */
+//        Arrays.sort(bundles);
+
         /*
          * Map the bundles over the shards.
          */

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/AbstractBTreeTupleCursor.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/AbstractBTreeTupleCursor.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/AbstractBTreeTupleCursor.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -254,50 +254,52 @@
      */
     final protected boolean rangeCheck(final byte[] key) {
 
-        if (fromKey == null && toKey == null) {
-
-            // no range constraint.
-            return true;
-            
-        }
-
-        if (fromKey != null) {
-
-            if (BytesUtil.compareBytes(key, fromKey) < 0) {
-
-                if (DEBUG) {
-
-                    log.debug("key=" + BytesUtil.toString(key) + " LT fromKey"
-                            + BytesUtil.toString(fromKey));
-
-                }
-                
-                // key is LT then the optional inclusive lower bound.
-                return false;
-
-            }
-
-        }
-
-        if (toKey != null) {
-
-            if (BytesUtil.compareBytes(key, toKey) >= 0) {
-
-                if (DEBUG) {
-
-                    log.debug("key=" + BytesUtil.toString(key) + " GTE toKey"
-                            + BytesUtil.toString(toKey));
-
-                }
-
-                // key is GTE the optional exclusive upper bound
-                return false;
-
-            }
-
-        }
-
-        return true;
+    	return BytesUtil.rangeCheck(key, fromKey, toKey);
+    	
+//        if (fromKey == null && toKey == null) {
+//
+//            // no range constraint.
+//            return true;
+//            
+//        }
+//
+//        if (fromKey != null) {
+//
+//            if (BytesUtil.compareBytes(key, fromKey) < 0) {
+//
+//                if (DEBUG) {
+//
+//                    log.debug("key=" + BytesUtil.toString(key) + " LT fromKey"
+//                            + BytesUtil.toString(fromKey));
+//
+//                }
+//                
+//                // key is LT then the optional inclusive lower bound.
+//                return false;
+//
+//            }
+//
+//        }
+//
+//        if (toKey != null) {
+//
+//            if (BytesUtil.compareBytes(key, toKey) >= 0) {
+//
+//                if (DEBUG) {
+//
+//                    log.debug("key=" + BytesUtil.toString(key) + " GTE toKey"
+//                            + BytesUtil.toString(toKey));
+//
+//                }
+//
+//                // key is GTE the optional exclusive upper bound
+//                return false;
+//
+//            }
+//
+//        }
+//
+//        return true;
         
     }
 

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/BytesUtil.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/BytesUtil.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/btree/BytesUtil.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -740,6 +740,72 @@
         
     }
     
+	/**
+	 * Return <code>true</code> if the <i>key</i> lies inside of the optional
+	 * half-open range constraint.
+	 * 
+	 * @param key
+	 *            The key.
+	 * @param fromKey
+	 *            The inclusive lower bound -or- <code>null</code> if there is
+	 *            no lower bound.
+	 * @param toKey
+	 *            The exclusive upper bound -or- <code>null</code> if there is
+	 *            no upper bound.
+	 * 
+	 * @return <code>true</code> unless the <i>key</i> is LT [fromKey] or GTE
+	 *         [toKey].
+	 */
+	final static public boolean rangeCheck(final byte[] key,
+			final byte[] fromKey, final byte[] toKey) {
+
+        if (fromKey == null && toKey == null) {
+
+            // no range constraint.
+            return true;
+            
+        }
+
+        if (fromKey != null) {
+
+            if (BytesUtil.compareBytes(key, fromKey) < 0) {
+
+                if (log.isDebugEnabled()) {
+
+                    log.debug("key=" + BytesUtil.toString(key) + " LT fromKey"
+                            + BytesUtil.toString(fromKey));
+
+                }
+                
+                // key is LT then the optional inclusive lower bound.
+                return false;
+
+            }
+
+        }
+
+        if (toKey != null) {
+
+            if (BytesUtil.compareBytes(key, toKey) >= 0) {
+
+                if (log.isDebugEnabled()) {
+
+                    log.debug("key=" + BytesUtil.toString(key) + " GTE toKey"
+                            + BytesUtil.toString(toKey));
+
+                }
+
+                // key is GTE the optional exclusive upper bound
+                return false;
+
+            }
+
+        }
+
+        return true;
+        
+    }
+
     /**
      * This method forces the load of the JNI library and tries to execute the
      * JNI methods.

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties	2010-10-13 19:39:49 UTC (rev 3791)
@@ -89,7 +89,7 @@
 
 #log4j.logger.com.bigdata.service=ALL
 
-log4j.logger.com.bigdata.util.concurrent.Haltable=ALL
+#log4j.logger.com.bigdata.util.concurrent.Haltable=ALL
 
 #log4j.logger.com.bigdata.bop=ALL
 #log4j.logger.com.bigdata.bop.join.PipelineJoin=ALL
@@ -97,7 +97,7 @@
 #log4j.logger.com.bigdata.bop.engine=ALL
 #log4j.logger.com.bigdata.bop.engine.QueryEngine=ALL
 #log4j.logger.com.bigdata.bop.engine.RunningQuery=ALL
-log4j.logger.com.bigdata.bop.engine.RunState=INFO
+#log4j.logger.com.bigdata.bop.engine.RunState=INFO
 #log4j.logger.com.bigdata.bop.engine.RunningQuery$ChunkTask=ALL
 #log4j.logger.com.bigdata.bop.fed.FederatedQueryEngine=ALL
 #log4j.logger.com.bigdata.bop.fed.FederatedRunningQuery=ALL

Modified: branches/QUADS_QUERY_BRANCH/bigdata-perf/lubm/build.properties
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-perf/lubm/build.properties	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata-perf/lubm/build.properties	2010-10-13 19:39:49 UTC (rev 3791)
@@ -48,9 +48,9 @@
 # Laptop benchmark data directory.
 #lubm.baseDir=d:/bigdata-perf-analysis/lubm/U${lubm.univ}
 # Server benchmark directory.
-#lubm.baseDir=/nas/data/lubm/U${lubm.univ}
+lubm.baseDir=/nas/data/lubm/U${lubm.univ}
 # Windows Server 2008 benchmark data directory.
-lubm.baseDir=c:/usr/local/data/lubm/lubm_${lubm.univ}
+#lubm.baseDir=c:/usr/local/data/lubm/lubm_${lubm.univ}
 
 ## Where to put the XML results files.
 #bsbm.resultsDir=${bsbm.baseDir}/..
@@ -80,11 +80,11 @@
 # The name of the file used for the journal.
 #lubm.journalFile=${lubm.baseDir}/bigdata-lubm.${journalMode}.jnl
 # Note: This is on the large volume.
-#lubm.journalFile=/data/lubm/U${lubm.univ}/bigdata-lubm.${journalMode}.jnl
+lubm.journalFile=/data/lubm/U${lubm.univ}/bigdata-lubm.${journalMode}.jnl
 # SSD.
 #lubm.journalFile=e:/data/lubm/U${lubm.univ}/bigdata-lubm.${journalMode}.jnl
 # SAS
-lubm.journalFile=f:/data/lubm/U${lubm.univ}/bigdata-lubm.${journalMode}.jnl
+#lubm.journalFile=f:/data/lubm/U${lubm.univ}/bigdata-lubm.${journalMode}.jnl
 
 # The database to test.
 lubm.configFile=${lubm.dir}/src/resources/config/config.kb.sparql

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -138,13 +138,13 @@
 		 * test suite is done. Again, that should be Ok.)
 		 */
 		if (_ts != null) {
-//			_ts.destroy();
+			_ts.destroy();
 			_ts = null;
 		}
-//		if (_fed != null) {
-//			_fed.shutdownNow();
-//			_fed = null;
-//		}
+		if (_fed != null) {
+			_fed.shutdownNow();
+			_fed = null;
+		}
 	}
 
     @Override protected Repository newRepository ()
@@ -188,7 +188,7 @@
         return _fed ;
     }
 
-    private String getConfiguration ()
+    static private String getConfiguration ()
         throws Exception
     {
         String c = System.getProperty ( CONFIG_PROPERTY ) ;
@@ -318,4 +318,62 @@
     private ScaleOutTripleStore _ts = null ;
     private BigdataSail _sail = null;
     
+//	/**
+//	 * Dumps the locators for an index of a relation.
+//	 * 
+//	 * @param fed
+//	 * @param namespace
+//	 *            The relation namespace.
+//	 * @param timestamp
+//	 *            The timestamp of the view.
+//	 * @param keyOrder
+//	 *            The index.
+//	 */
+//	private static void dumpMDI(AbstractScaleOutFederation<?> fed,
+//			final String namespace, final long timestamp,
+//			final IKeyOrder<?> keyOrder) {
+//
+//		final String name = namespace + "." + keyOrder.getIndexName();
+//
+//		final Iterator<PartitionLocator> itr = fed
+//				.locatorScan(name, timestamp, new byte[] {}/* fromKey */,
+//						null/* toKey */, false/* reverseScan */);
+//
+//		System.err.println("name=" + name + " @ "
+//				+ TimestampUtility.toString(timestamp));
+//		
+//		while (itr.hasNext()) {
+//		
+//			System.err.println(itr.next());
+//			
+//		}
+//
+//	}
+//	
+//	public static void main(String[] args) throws ConfigurationException,
+//			Exception {
+//
+//		String namespace = "SPARQLTest_14e097f5-eba9-4ff8-a250-a5f9f19b9c66.spo";
+//		
+//		long timestamp=1287001362979L;
+//		
+//		IKeyOrder<?> keyOrder = SPOKeyOrder.SPOC;
+//		
+//		JiniClient<Object> jc = new JiniClient<Object>(
+//				new String[] { getConfiguration() });
+//		
+//		try {
+//		
+//			final JiniFederation<?> fed = jc.connect();
+//			
+//			dumpMDI(fed, namespace, timestamp, keyOrder);
+//			
+//		} finally {
+//
+//			jc.disconnect(true/* immediate */);
+//			
+//		}
+//
+//	}
+
 }

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java	2010-10-13 19:39:49 UTC (rev 3791)
@@ -8,6 +8,7 @@
 import info.aduna.io.IOUtil;
 import info.aduna.iteration.Iterations;
 import info.aduna.text.StringUtil;
+
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.InputStreamReader;
@@ -17,8 +18,10 @@
 import java.util.LinkedHashSet;
 import java.util.List;
 import java.util.Set;
+
 import junit.framework.TestCase;
 import junit.framework.TestSuite;
+
 import org.openrdf.model.Resource;
 import org.openrdf.model.Statement;
 import org.openrdf.model.URI;
@@ -61,8 +64,8 @@
 import org.openrdf.sail.memory.MemoryStore;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+
 import com.bigdata.rdf.sail.BigdataSailQuery;
-import com.bigdata.rdf.sail.BigdataSailRepository;
 
 public abstract class SPARQLQueryTest extends TestCase {
 
@@ -295,7 +298,7 @@
 			}
 
             RepositoryConnection con = ((DatasetRepository)dataRep).getDelegate().getConnection();
-            System.err.println(con.getClass());
+//            System.err.println(con.getClass());
             try {
                 String queryString = readQueryString();
                 Query query = con.prepareQuery(QueryLanguage.SPARQL, queryString, queryFileURL);

Modified: branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config
===================================================================
--- branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-13 19:12:52 UTC (rev 3790)
+++ branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-13 19:39:49 UTC (rev 3791)
@@ -303,8 +303,6 @@
 	"-Djava.security.policy="+bigdata.policy,
 	"-Djava.util.logging.config.file="+bigdata.logging,
 	"-Dcom.bigdata.counters.linux.sysstat.path=@SYSSTAT_HOME@",
-	// Turn off periodic performance counter reporting.
-	"-Dcom.bigdata.service.IBigdataClient.reportDelay=0",
 	//bigdata.profilerAgent,
 	};
 
@@ -368,6 +366,13 @@
      */
     new NV(IBigdataClient.Options.CLIENT_MAX_STALE_LOCATOR_RETRIES,"1000"),
 
+	/* 
+	 * Turn off periodic performance counter reporting to reduce the memory
+	 * demand on the LBS (performance counters are used for moves on a multi-
+	 * machine cluster and provide detailed reporting on the bigdata internals).
+	 */
+	new NV(IBigdataClient.Options.REPORT_DELAY,"0"),
+
     };
 
 }


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3790] branches/maven_scaleout/bigdata-integ/src/test/ java/com/bigdata/jini/start/TestServiceConfigurationMonitoring.java

[<ble...@us...>]

Revision: 3790
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3790&view=rev
Author:   blevine218
Date:     2010-10-13 19:12:52 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Increase the delay (sleep) between the ZNode being created and the check that the new child node has appeared under the service config node

Modified Paths:
--------------
    branches/maven_scaleout/bigdata-integ/src/test/java/com/bigdata/jini/start/TestServiceConfigurationMonitoring.java

Modified: branches/maven_scaleout/bigdata-integ/src/test/java/com/bigdata/jini/start/TestServiceConfigurationMonitoring.java
===================================================================
--- branches/maven_scaleout/bigdata-integ/src/test/java/com/bigdata/jini/start/TestServiceConfigurationMonitoring.java	2010-10-13 16:56:24 UTC (rev 3789)
+++ branches/maven_scaleout/bigdata-integ/src/test/java/com/bigdata/jini/start/TestServiceConfigurationMonitoring.java	2010-10-13 19:12:52 UTC (rev 3790)
@@ -119,7 +119,7 @@
         // zpath for the service configuration znode.
         final String serviceConfigurationZPath = zconfig + BigdataZooDefs.ZSLASH + TransactionServer.class.getName();
         
-        // create monitor task that will compete for locks and start procsses.
+        // create monitor task that will compete for locks and start processes.
         MonitorCreatePhysicalServiceLocksTask task1 = new MonitorCreatePhysicalServiceLocksTask(fed, listener);
 
         final Future f1 = fed.getExecutorService().submit(task1);
@@ -160,7 +160,7 @@
          */
         
         // pause a moment.
-        Thread.sleep(1000/*ms*/);
+        Thread.sleep(5000/*ms*/);
 
         System.out.println("logicalServices: " + zookeeper.getChildren(serviceConfigurationZPath, false));
         


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3789] branches/JOURNAL_HA_BRANCH/bigdata/src

[<mar...@us...>]

Revision: 3789
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3789&view=rev
Author:   martyncutcher
Date:     2010-10-13 16:56:24 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Ensure DirectBuffer for BufferedWrites is released

Modified Paths:
--------------
    branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/io/writecache/BufferedWrite.java
    branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/rwstore/RWStore.java
    branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/rwstore/TestRWJournal.java

Modified: branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/io/writecache/BufferedWrite.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/io/writecache/BufferedWrite.java	2010-10-13 15:24:57 UTC (rev 3788)
+++ branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/io/writecache/BufferedWrite.java	2010-10-13 16:56:24 UTC (rev 3789)
@@ -55,6 +55,10 @@
 		m_data = DirectBufferPool.INSTANCE.acquire();
 	}
 	
+	public void release() throws InterruptedException {
+		DirectBufferPool.INSTANCE.release(m_data);
+	}
+	
 	public int write(final long offset, final ByteBuffer data, final IReopenChannel<FileChannel> opener) throws IOException {
 		int nwrites = 0;
 		

Modified: branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/rwstore/RWStore.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/rwstore/RWStore.java	2010-10-13 15:24:57 UTC (rev 3788)
+++ branches/JOURNAL_HA_BRANCH/bigdata/src/java/com/bigdata/rwstore/RWStore.java	2010-10-13 16:56:24 UTC (rev 3789)
@@ -491,7 +491,7 @@
 				m_minFixedAlloc = m_allocSizes[0]*64;
 
 				commitChanges();
-
+				
 			} else {
 				
 				initfromRootBlock();
@@ -509,6 +509,10 @@
 
 	public void close() {
 		try {
+			if (m_bufferedWrite != null) {
+				m_bufferedWrite.release();
+				m_bufferedWrite = null;
+			}
 			m_writeCache.close();
 			m_raf.close();
 		} catch (IOException e) {
@@ -581,7 +585,8 @@
 	 * @throws IOException
 	 */
 	private void initfromRootBlock() throws IOException {
-		m_rb = m_fmv.getRootBlock();
+		// m_rb = m_fmv.getRootBlock();
+		assert(m_rb != null);
 
 		m_commitCounter = m_rb.getCommitCounter();
 

Modified: branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/rwstore/TestRWJournal.java
===================================================================
--- branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/rwstore/TestRWJournal.java	2010-10-13 15:24:57 UTC (rev 3788)
+++ branches/JOURNAL_HA_BRANCH/bigdata/src/test/com/bigdata/rwstore/TestRWJournal.java	2010-10-13 16:56:24 UTC (rev 3789)
@@ -960,57 +960,60 @@
          */
         public void test_allocationContexts() throws IOException {
             Journal store = (Journal) getStore();
+            try {
+            	RWStrategy bs = (RWStrategy) store.getBufferStrategy();
 
-            RWStrategy bs = (RWStrategy) store.getBufferStrategy();
+				RWStore rw = bs.getRWStore();
 
-            RWStore rw = bs.getRWStore();
-            
-            // JournalShadow shadow = new JournalShadow(store);
+				// JournalShadow shadow = new JournalShadow(store);
 
-            // Create a couple of contexts
-            IAllocationContext allocContext1 = new DummyAllocationContext();
-            IAllocationContext allocContext2 = new DummyAllocationContext();
-            
-            int sze = 650;
-            byte[] buf = new byte[sze+4]; // extra for checksum
-            r.nextBytes(buf);
-            
-            long addr1a = bs.write(ByteBuffer.wrap(buf), allocContext1);
-            long addr1b = bs.write(ByteBuffer.wrap(buf), allocContext1);
-            rw.detachContext(allocContext1);
-            
-            
-            long addr2a = bs.write(ByteBuffer.wrap(buf), allocContext2);
-            long addr2b = bs.write(ByteBuffer.wrap(buf), allocContext2);
-            rw.detachContext(allocContext2);           
- 
-            // Re-establish context
-            long addr1c = bs.write(ByteBuffer.wrap(buf), allocContext1);
-            
-            // By detaching contexts we end up using the same allocator
-            assertTrue("allocator re-use", bs.getPhysicalAddress(addr1c) > bs.getPhysicalAddress(addr2b));
-            
-            // Now, prior to commit, try deleting an uncommitted allocation
-            bs.delete(addr1c, allocContext1);
-            // and re-allocating it from the same context
-            long addr1d = bs.write(ByteBuffer.wrap(buf), allocContext1);
+				// Create a couple of contexts
+				IAllocationContext allocContext1 = new DummyAllocationContext();
+				IAllocationContext allocContext2 = new DummyAllocationContext();
 
-            assertTrue("re-allocation", addr1c==addr1d);
+				int sze = 650;
+				byte[] buf = new byte[sze + 4]; // extra for checksum
+				r.nextBytes(buf);
 
-            rw.detachContext(allocContext1);
-            
-            // Now commit
-            store.commit();
-            
-            // now try deleting and re-allocating again, but in a global context
-            bs.delete(addr1d); // this should call deferFree
-            long addr1e = bs.write(ByteBuffer.wrap(buf), allocContext1);
-            
-            assertTrue("deferred-delete", addr1e != addr1d);
-            
-            // Now commit
-            store.commit();
+				long addr1a = bs.write(ByteBuffer.wrap(buf), allocContext1);
+				long addr1b = bs.write(ByteBuffer.wrap(buf), allocContext1);
+				rw.detachContext(allocContext1);
 
+				long addr2a = bs.write(ByteBuffer.wrap(buf), allocContext2);
+				long addr2b = bs.write(ByteBuffer.wrap(buf), allocContext2);
+				rw.detachContext(allocContext2);
+
+				// Re-establish context
+				long addr1c = bs.write(ByteBuffer.wrap(buf), allocContext1);
+
+				// By detaching contexts we end up using the same allocator
+				assertTrue("allocator re-use", bs.getPhysicalAddress(addr1c) > bs.getPhysicalAddress(addr2b));
+
+				// Now, prior to commit, try deleting an uncommitted allocation
+				bs.delete(addr1c, allocContext1);
+				// and re-allocating it from the same context
+				long addr1d = bs.write(ByteBuffer.wrap(buf), allocContext1);
+
+				assertTrue("re-allocation", addr1c == addr1d);
+
+				rw.detachContext(allocContext1);
+
+				// Now commit
+				store.commit();
+
+				// now try deleting and re-allocating again, but in a global
+				// context
+				bs.delete(addr1d); // this should call deferFree
+				long addr1e = bs.write(ByteBuffer.wrap(buf), allocContext1);
+
+				assertTrue("deferred-delete", addr1e != addr1d);
+
+				// Now commit
+				store.commit();
+			} finally {
+            	store.destroy();
+            }
+
         }
         
        public void test_stressAlloc() {


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3788] branches/bbb_cleanup/

[<res...@us...>]

Revision: 3788
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3788&view=rev
Author:   resendes
Date:     2010-10-13 15:24:57 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Deleting temporary branch

Removed Paths:
-------------
    branches/bbb_cleanup/


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3787] branches/maven_scaleout

[<res...@us...>]

Revision: 3787
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3787&view=rev
Author:   resendes
Date:     2010-10-13 15:08:24 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Merge from bbb_cleanup branch
- added com.bigdata.util.config fixes
- added com.bigdata.util.config unit tests

Modified Paths:
--------------
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigDeployUtil.java
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigurationUtil.java
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/Log4jLoggingHandler.java
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/LogUtil.java
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/NicUtil.java

Added Paths:
-----------
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/ConfigDeployUtilTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/ConfigDeployUtilTest2.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/ConfigurationUtilTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/Log4jLoggingHandlerTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/LogUtilTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/NicUtilTest.java

Removed Paths:
-------------
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/ConfigDeployUtilTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/ConfigDeployUtilTest2.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/ConfigurationUtilTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/Log4jLoggingHandlerTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/LogUtilTest.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/util/config/NicUtilTest.java

Property Changed:
----------------
    branches/maven_scaleout/
    branches/maven_scaleout/bigdata-core/bigdata-perf/
    branches/maven_scaleout/bigdata-core/bigdata-perf/lubm/src/resources/
    branches/maven_scaleout/bigdata-core/dsi-utils/LEGAL/
    branches/maven_scaleout/bigdata-core/dsi-utils/lib/
    branches/maven_scaleout/bigdata-core/dsi-utils/src/
    branches/maven_scaleout/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom/
    branches/maven_scaleout/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom/
    branches/maven_scaleout/bigdata-core/osgi/
    branches/maven_scaleout/bigdata-core/src/main/deploy/bin/
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/boot/
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/logging/
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties
    branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties
    branches/maven_scaleout/bigdata-core/src/main/java/
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/attr/
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/disco/
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/rdf/sail/bench/
    branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/rdf/util/
    branches/maven_scaleout/bigdata-core/src/samples-sail/com/bigdata/samples/fastload.properties
    branches/maven_scaleout/bigdata-core/src/test/deploy/testing/conf/bigdataStandaloneTesting.config
    branches/maven_scaleout/bigdata-core/src/test/deploy/testing/data/com/bigdata/jini/start/config/testfed.config
    branches/maven_scaleout/bigdata-core/src/test/deploy/testing/data/com/bigdata/jini/start/testfed.config
    branches/maven_scaleout/bigdata-core/src/test/deploy/testing/data/com/bigdata/service/jini/master/TestMappedRDFDataLoadMaster.config
    branches/maven_scaleout/bigdata-core/src/test/java/


Property changes on: branches/maven_scaleout
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH:2004-2045
/branches/DEV_BRANCH_27_OCT_2009:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH:2633-3304
/branches/bugfix-btm:2594-3237
/branches/dev-btm:2574-3440,3443,3463,3469-3470
/branches/fko:3150-3194
/branches/maven_scaleout:3379-3438
/trunk:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH:2004-2045
/branches/DEV_BRANCH_27_OCT_2009:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH:2633-3304
/branches/bbb_cleanup:3709-3781
/branches/bugfix-btm:2594-3237
/branches/dev-btm:2574-3440,3443,3463,3469-3470
/branches/fko:3150-3194
/branches/maven_scaleout:3379-3438
/trunk:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/bigdata-perf
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/bigdata-perf:3588-3690
/trunk/bigdata-perf:3379-3541
   + /branches/bbb_cleanup/bigdata-core/bigdata-perf:3588-3690,3709-3781
/trunk/bigdata-perf:3379-3541


Property changes on: branches/maven_scaleout/bigdata-core/bigdata-perf/lubm/src/resources
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources:3588-3690
/branches/dev-btm/bigdata-perf/lubm/src/resources:2574-3440
/trunk/bigdata-perf/lubm/src/resources:3379-3541
   + /branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources:3588-3690,3709-3781
/branches/dev-btm/bigdata-perf/lubm/src/resources:2574-3440
/trunk/bigdata-perf/lubm/src/resources:3379-3541


Property changes on: branches/maven_scaleout/bigdata-core/dsi-utils/LEGAL
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/dsi-utils/LEGAL:3588-3690
/trunk/dsi-utils/LEGAL:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/dsi-utils/LEGAL:3588-3690,3709-3781
/trunk/dsi-utils/LEGAL:3379-3430,3499


Property changes on: branches/maven_scaleout/bigdata-core/dsi-utils/lib
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/dsi-utils/lib:3588-3690
/trunk/dsi-utils/lib:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/dsi-utils/lib:3588-3690,3709-3781
/trunk/dsi-utils/lib:3379-3430,3499


Property changes on: branches/maven_scaleout/bigdata-core/dsi-utils/src
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/dsi-utils/src:3588-3690
/trunk/dsi-utils/src:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/dsi-utils/src:3588-3690,3709-3781
/trunk/dsi-utils/src:3379-3430,3499


Property changes on: branches/maven_scaleout/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3588-3690
/trunk/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3588-3690,3709-3781
/trunk/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499


Property changes on: branches/maven_scaleout/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3588-3690
/trunk/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3588-3690,3709-3781
/trunk/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499


Property changes on: branches/maven_scaleout/bigdata-core/osgi
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/osgi:3588-3690
/trunk/osgi:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/osgi:3588-3690,3709-3781
/trunk/osgi:3379-3430,3499


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/bin
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/bin:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/bin:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/bin:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/bin:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/bin:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/bin:2574-3440,3443,3463,3469-3470
/branches/dev-btm/src/resources/bin:3463
/branches/fko/bigdata-core/src/main/deploy/bin:3150-3194
/trunk/bigdata-core/src/main/deploy/bin:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/bin:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/bin:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/bin:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/bin:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/bin:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/bin:2574-3440,3443,3463,3469-3470
/branches/dev-btm/src/resources/bin:3463
/branches/fko/bigdata-core/src/main/deploy/bin:3150-3194
/trunk/bigdata-core/src/main/deploy/bin:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini:3588-3690
/trunk/src/main/deploy/var/config/jini:3499
/trunk/src/resources/config:3516-3528
   + /branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini:3588-3690,3709-3781
/trunk/src/main/deploy/var/config/jini:3499
/trunk/src/resources/config:3516-3528


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2574-3440,3443,3463,3469-3470
/branches/dev-btm/src/resources/config/bigdataCluster.config:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
/trunk/src/resources/config/bigdataCluster.config:3516-3528
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:2574-3440,3443,3463,3469-3470
/branches/dev-btm/src/resources/config/bigdataCluster.config:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/jini/bigdataCluster.config:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
/trunk/src/resources/config/bigdataCluster.config:3516-3528


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/boot
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/boot:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/boot:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/boot/config:3469-3470
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/boot:3150-3194
/branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3438
/trunk/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/boot:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/boot:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/boot/config:3469-3470
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/boot:3150-3194
/branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3438
/trunk/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/metadata/config/shardlocator.config:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/metadata/config/shardlocator.config:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/jini/shardlocator.config:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/logging
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/logging:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/logging:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/logging:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/logging:2594-3237
/branches/dev-btm/bigdata/src/resources/logging:3463
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/logging:2574-3440,3443,3463,3469-3470
/branches/fko/bigdata-core/src/main/deploy/var/config/logging:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/logging:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/logging:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/logging:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/logging:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/logging:2594-3237
/branches/dev-btm/bigdata/src/resources/logging:3463
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/logging:2574-3440,3443,3463,3469-3470
/branches/fko/bigdata-core/src/main/deploy/var/config/logging:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/logging:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2594-3237
/branches/dev-btm/bigdata/src/resources/logging/shardlocator-logging.properties:3463
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/metadata/config/logging.properties:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2594-3237
/branches/dev-btm/bigdata/src/resources/logging/shardlocator-logging.properties:3463
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/metadata/config/logging.properties:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/logging/shardlocator-logging.properties:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2594-3237
/branches/dev-btm/bigdata/src/resources/logging/transaction-logging.properties:3463
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/transaction/config/logging.properties:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2594-3237
/branches/dev-btm/bigdata/src/resources/logging/transaction-logging.properties:3463
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/transaction/config/logging.properties:3463
/branches/fko/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:3150-3194
/trunk/bigdata-core/src/main/deploy/var/config/logging/transaction-logging.properties:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/java
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/java:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/java:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/java:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/java:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/java:2594-3237
/branches/dev-btm/bigdata/src/java:3463
/branches/dev-btm/bigdata-core/src/main/java:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java:3463,3469-3470
/branches/dev-btm/bigdata-rdf/src/java:3463
/branches/dev-btm/bigdata-sails/src/java:3463
/branches/fko/bigdata-core/src/main/java:3150-3194
/trunk/bigdata/src/java:3507
/trunk/bigdata-core/src/main/java:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
/trunk/bigdata-rdf/src/java:3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/java:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/java:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/java:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/java:3588-3690,3709-3781
/branches/bugfix-btm/bigdata-core/src/main/java:2594-3237
/branches/dev-btm/bigdata/src/java:3463
/branches/dev-btm/bigdata-core/src/main/java:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java:3463,3469-3470
/branches/dev-btm/bigdata-rdf/src/java:3463
/branches/dev-btm/bigdata-sails/src/java:3463
/branches/fko/bigdata-core/src/main/java:3150-3194
/trunk/bigdata/src/java:3507
/trunk/bigdata-core/src/main/java:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
/trunk/bigdata-rdf/src/java:3542


Property changes on: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/attr
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/attr:3588-3690
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/attr:3463,3469-3470
/trunk/bigdata-jini/src/java/com/bigdata/attr:3379-3430
   + /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/attr:3588-3690,3709-3781
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/attr:3463,3469-3470
/trunk/bigdata-jini/src/java/com/bigdata/attr:3379-3430


Property changes on: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/disco
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/disco:3588-3690
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/disco:3463,3469-3470
/trunk/bigdata-jini/src/java/com/bigdata/disco:3379-3430
   + /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/disco:3588-3690,3709-3781
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/disco:3463,3469-3470
/trunk/bigdata-jini/src/java/com/bigdata/disco:3379-3430


Property changes on: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/rdf/sail/bench
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/sail/bench:3588-3690
/branches/dev-btm/bigdata-sails/src/java/com/bigdata/rdf/sail/bench:3463
/trunk/bigdata-sails/src/java/com/bigdata/rdf/sail/bench:3379-3430
   + /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/sail/bench:3588-3690,3709-3781
/branches/dev-btm/bigdata-sails/src/java/com/bigdata/rdf/sail/bench:3463
/trunk/bigdata-sails/src/java/com/bigdata/rdf/sail/bench:3379-3430


Property changes on: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/rdf/util
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/util:3588-3690
/branches/dev-btm/bigdata-rdf/src/java/com/bigdata/rdf/util:3463
/trunk/bigdata-rdf/src/java/com/bigdata/rdf/util:3379-3430,3542
   + /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/util:3588-3690,3709-3781
/branches/dev-btm/bigdata-rdf/src/java/com/bigdata/rdf/util:3463
/trunk/bigdata-rdf/src/java/com/bigdata/rdf/util:3379-3430,3542

Modified: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigDeployUtil.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigDeployUtil.java	2010-10-13 14:18:44 UTC (rev 3786)
+++ branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigDeployUtil.java	2010-10-13 15:08:24 UTC (rev 3787)
@@ -34,6 +34,8 @@
 import java.io.FileInputStream;
 import java.io.IOException;
 import java.net.SocketException;
+import java.text.NumberFormat;
+import java.text.ParseException;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Properties;
@@ -46,6 +48,12 @@
  * invoked from within a Jini configuration.
  */
 public class ConfigDeployUtil {
+	
+	private ConfigDeployUtil() {
+		//prevent instantiation
+		throw new AssertionError
+            ("ConfigDeployUtil cannot be instantiated");
+	}
 
     private static Properties deploymentProps = null;
 
@@ -57,8 +65,26 @@
     private static final String TYPE = ".type";
     
     private static final String FALLBACK_FEDNAME_FORMAT = "bigdata.test.group-%s";
-    private static final String TEMPLATE_TOKEN_PATTERN = "@.*@";
+    
+    //use current locale
+    private static final NumberFormat numberFormat = NumberFormat.getInstance();
 
+    /**
+     * Gets the <code>String</code> value associated with the given parameter 
+     * in the configured properties file. The value, if any, returned will be:
+     * 1) the explicitly defined value for the property, or
+     * 2) the default value for that property, if a default exists. 
+     * If the provided parameter has an associated set of valid entries in the
+     * properties files, then the value obtained, above, will be validated 
+     * against those entries. If either the value (explicit or default) can't be
+     * found or fails the validation check,
+     * a <code>ConfigurationException</code> will be thrown.
+     * @param parameter The property name to lookup
+     * @return the <code>String</code> value associated the requested parameter,
+     *     if available
+     * @throws ConfigurationException if the parameter value 
+     *     (explicit or default) was not defined.
+     */
     public static String getString(String parameter) 
                              throws ConfigurationException
     {
@@ -66,7 +92,23 @@
         validateString(parameter, value);
         return value;
     }
-
+    
+    /**
+     * Gets the <code>String[]</code> value associated with the given parameter 
+     * in the configured properties file. The value, if any, returned will be:
+     * 1) the explicitly defined value for the property, or
+     * 2) the default value for that property, if a default exists. 
+     * If the provided parameter has an associated set of valid entries in the
+     * properties files, then the component values obtained, above, will be validated 
+     * against those entries. If either the value (explicit or default) can't be
+     * found or fails the validation check,
+     * a <code>ConfigurationException</code> will be thrown.
+     * @param parameter The property name to lookup
+     * @return the <code>String[]</code> value associated the requested parameter,
+     *     if available
+     * @throws ConfigurationException if the parameter value 
+     *     (explicit or default) was not defined.
+     */    
     public static String[] getStringArray(String parameter)
                                throws ConfigurationException
     {
@@ -75,12 +117,46 @@
         return value;
     }
 
+    /**
+     * Gets the <code>int</code> value associated with the given parameter 
+     * in the configured properties file. The value, if any, returned will be:
+     * 1) the explicitly defined value for the property, or
+     * 2) the default value for that property, if a default exists. 
+     * If the provided parameter has an associated set of valid entries in the
+     * properties files, then the value obtained, above, will be validated 
+     * against those entries. If either the value (explicit or default) can't be
+     * found or fails the validation check,
+     * a <code>ConfigurationException</code> will be thrown.
+     * @param parameter The property name to lookup
+     * @return the <code>int</code> value associated the requested parameter,
+     *     if available
+     * @throws ConfigurationException if the parameter value 
+     *     (explicit or default) was not defined.
+     */    
+
     public static int getInt(String parameter) throws ConfigurationException {
         int value;
         value = validateInt(parameter, get(parameter));
         return value;
     }
 
+    /**
+     * Gets the <code>long</code> value associated with the given parameter 
+     * in the configured properties file. The value, if any, returned will be:
+     * 1) the explicitly defined value for the property, or
+     * 2) the default value for that property, if a default exists. 
+     * If the provided parameter has an associated set of valid entries in the
+     * properties files, then the value obtained, above, will be validated 
+     * against those entries. If either the value (explicit or default) can't be
+     * found or fails the validation check,
+     * a <code>ConfigurationException</code> will be thrown.
+     * @param parameter The property name to lookup
+     * @return the <code>long</code> value associated the requested parameter,
+     *     if available
+     * @throws ConfigurationException if the parameter value 
+     *     (explicit or default) was not defined.
+     */    
+
     public static long getLong(String parameter) throws ConfigurationException
     {
         long value;
@@ -88,60 +164,97 @@
         return value;
     }
 
-    public static boolean getBoolean(String parameter) 
+    /**
+     * Gets the <code>boolean</code> value associated with the given parameter 
+     * in the configured properties file. The value, if any, returned will be:
+     * 1) the explicitly defined value for the property, or
+     * 2) the default value for that property, if a default exists. 
+     * If the provided parameter has an associated set of valid entries in the
+     * properties files, then the value obtained, above, will be validated 
+     * against those entries. If either the value (explicit or default) can't be
+     * found or fails the validation check,
+     * a <code>ConfigurationException</code> will be thrown.
+     * @param parameter The property name to lookup
+     * @return the <code>boolean</code> value associated the requested parameter,
+     *     if available
+     * @throws ConfigurationException if the parameter value 
+     *     (explicit or default) was not defined.
+     */    
+    public static boolean getBoolean(String parameter) throws ConfigurationException
+    {
+        boolean value;
+        value = validateBoolean(parameter, get(parameter));
+        return value;
+    }
+
+    private static boolean validateBoolean(String parameter, String value) 
                               throws ConfigurationException {
         boolean boolValue = false;
-        String value;
-        try {
-            value = get(parameter);
-        } catch (Exception ex) {
-            throw new ConfigurationException("parameter value ["+parameter+"] "
-                                             +"neither 'true' nor 'false'");
-        }
-        if( value != null && value.equalsIgnoreCase("true")
-           || value.equalsIgnoreCase("false") )
+
+        if( value != null && (value.equalsIgnoreCase("true")
+           || value.equalsIgnoreCase("false")) )
         {
             boolValue = Boolean.parseBoolean(value);
         } else {
             throw new ConfigurationException("parameter value ["+parameter+"] "
-                                             +"neither 'true' nor 'false'");
+                                             +"is neither 'true' nor 'false'");
         }
         return boolValue;
     }
 
+    /**
+     * Gets the description value associated with the given parameter 
+     * in the configured properties file. The method returns 
+     * <code>null</code> if the property is not found.
+     * @param parameter The property name to lookup
+     * @return the <code>String</code> value associated the requested parameter,
+     *     if available. Otherwise return <code>null</code>
+     * @throws ConfigurationException if there was a problem accessing
+     *     the parameter value.
+     */    
+
     public static String getDescription(String parameter)
                              throws ConfigurationException
     {
         String value;
-        if(deploymentProps == null) {
-            deploymentProps = new Properties();
-            loadDeployProps(deploymentProps);
-        }
-        value = deploymentProps.getProperty(parameter + DESCRIPTION);
+        value = getDeploymentProperties().getProperty(parameter + DESCRIPTION);
         return value;
     }
 
+    /**
+     * Gets the type value associated with the given parameter 
+     * in the configured properties file. The method returns 
+     * <code>null</code> if the property is not found.
+     * @param parameter The property name to lookup
+     * @return the <code>String</code> value of the type 
+     * associated the requested parameter,
+     *     if available. Otherwise return <code>null</code>
+     * @throws ConfigurationException if there was a problem accessing
+     *     the parameter value.
+     */    
+
     public static String getType(String parameter)
                              throws ConfigurationException
     {
         String value;
-        if (deploymentProps == null) {
-            deploymentProps = new Properties();
-            loadDeployProps(deploymentProps);
-        }
-        value = deploymentProps.getProperty(parameter + TYPE);
+        value = getDeploymentProperties().getProperty(parameter + TYPE);
         return value;
     }
 
+    /**
+     * Gets the default value associated with the given parameter 
+     * in the configured properties file. 
+     * @param parameter The property name to lookup
+     * @return the <code>String</code> value of the type 
+     * associated the requested parameter,
+     *     if available. Otherwise return <code>null</code>
+     * @throws ConfigurationException if no default value was specified
+     */        
     public static String getDefault(String parameter) 
                              throws ConfigurationException
     {
         String value;
-        if (deploymentProps == null) {
-            deploymentProps = new Properties();
-            loadDeployProps(deploymentProps);
-        }
-        value = deploymentProps.getProperty(parameter + DEFAULT);
+        value = getDeploymentProperties().getProperty(parameter + DEFAULT);
         if (value == null) {
             throw new ConfigurationException
                           ("deployment parameter not found ["+parameter+"]");
@@ -150,7 +263,7 @@
     }
 
     /**
-     * Returns a <code>String</code> array whose elments represent the
+     * Returns a <code>String</code> array whose elements represent the
      * lookup service groups to discover. If the system property named
      * "federation.name" is set then that value be used; otherwise,
      * the deployment properties files will be consulted.
@@ -164,7 +277,7 @@
     }
     
     /**
-     * Retrieve the federation name (also used as Jini group name) via this pecking pecking order:
+     * Retrieve the federation name (also used as Jini group name) via this pecking order:
      * <ol>
      * <li>From the Java system property: <code>federation.name</code></li>
      * <li>From the deployment properties file. Note that a value from the deployment
@@ -254,11 +367,7 @@
 
     private static String get(String parameter) throws ConfigurationException {
         String value;
-        if (deploymentProps == null) {
-            deploymentProps = new Properties();
-            loadDeployProps(deploymentProps);
-        }
-        value = deploymentProps.getProperty(parameter);
+        value = getDeploymentProperties().getProperty(parameter);
         if (value == null) value = getDefault(parameter);
         return value;
     }
@@ -267,7 +376,7 @@
                             throws ConfigurationException
     {
         String validValuesStr = 
-            (String) deploymentProps.get(parameter + STRINGVALS);
+            (String) getDeploymentProperties().get(parameter + STRINGVALS);
 
         if (validValuesStr != null) {
             String[] validValues = validValuesStr.split(",");
@@ -284,7 +393,7 @@
                                 throws ConfigurationException
     {
         String validValuesStr = 
-            (String)(deploymentProps.get(parameter + STRINGVALS));
+            (String)(getDeploymentProperties().get(parameter + STRINGVALS));
         String[] values = value.split(",");
 
         if (validValuesStr != null) {
@@ -304,48 +413,77 @@
     private static int validateInt(String parameter, String strvalue)
                            throws ConfigurationException
     {
-        String maxString = (String)(deploymentProps.get(parameter + MAX));
-        String minString = (String)(deploymentProps.get(parameter + MIN));
+        String maxString = (String)(getDeploymentProperties().get(parameter + MAX));
+        String minString = (String)(getDeploymentProperties().get(parameter + MIN));
 
         int value = str2int(strvalue);
 
         if (maxString != null) {
-            int max = Integer.parseInt(maxString);
-            if (value > max) {
-                throw new ConfigurationException("parameter ["+parameter+"] "
-                                                 +"exceeds maximum ["+max+"]");
+            try {
+            	int max = numberFormat.parse(maxString).intValue();          
+	            if (value > max) {
+	                throw new ConfigurationException("parameter ["+parameter+"] "
+	                                                 +"exceeds maximum ["+max+"]");
+	            }
+            } catch (ParseException e) {
+            	throw new NumberFormatException(
+            		"Invalid maximum integer for parameter: " + parameter);
+         	
             }
         }
+        
+        if (minString != null) {
+            try {
+            	int min = numberFormat.parse(minString).intValue();          
+	            if (value < min) {
+	                throw new ConfigurationException("parameter ["+parameter+"] "
+	                    + "is less than manimum ["+min+"]");
+	            }
+            } catch (ParseException e) {
+            	throw new NumberFormatException(
+                		"Invalid minimum integer for parameter: " + parameter);         	
+            }
+        }
+        
         return value;
     }
 
     private static long validateLong(String parameter, String strvalue) 
                             throws ConfigurationException
     {
-        String maxString = (String)(deploymentProps.get(parameter + MAX));
-        String minString = (String)(deploymentProps.get(parameter + MIN));
+        String maxString = (String)(getDeploymentProperties().get(parameter + MAX));
+        String minString = (String)(getDeploymentProperties().get(parameter + MIN));
 
         long value = str2long(strvalue);
 
         if (maxString != null) {
-            long max = Long.parseLong(maxString);
-            if (value > max) {
-                throw new ConfigurationException("parameter ["+parameter+"] "
-                                                 +"exceeds maximum ["+max+"]");
+            try {
+            	long max = numberFormat.parse(maxString).longValue();          
+	            if (value > max) {
+	                throw new ConfigurationException("parameter ["+parameter+"] "
+	                                                 +"exceeds maximum ["+max+"]");
+	            }
+            } catch (ParseException e) {
+            	throw new NumberFormatException(
+            		"Invalid maximum long for parameter: " + parameter);
+         	
             }
         }
         if (minString != null) {
-            long min = Long.parseLong(minString);
-            if (value < min) {
-                throw new ConfigurationException("parameter ["+parameter+"] "
-                                                 +"is less than manimum "
-                                                 +"["+min+"]");
-            }
+            try {
+            	long min = numberFormat.parse(minString).longValue();          
+	            if (value < min) {
+	                throw new ConfigurationException("parameter ["+parameter+"] "
+	                    + "is less than manimum ["+min+"]");
+	            }
+            } catch (ParseException e) {
+            	throw new NumberFormatException(
+                		"Invalid minimum long for parameter: " + parameter);         	
+            }            
         }
         return value;
     }
 
-
     private static File getPropertiesPath() {
         File rootDir = new File("/opt/bigdata"); //real installation
         String appHome = System.getProperty("appHome");//pstart
@@ -372,30 +510,23 @@
     }
 
     private static void loadDefaultProps(Properties deployProps) {
-        FileInputStream fis = null;
-        try {
-            File flnm = new File(getPropertiesPath(), "default-deploy.properties");
-            fis = new FileInputStream(flnm);
-            deployProps.load(fis);
-        } catch (Exception ex) {
-            ex.printStackTrace();
-        } finally {
-            if (fis != null) {
-                try {
-                    fis.close();
-                } catch (IOException ioex) { /* swallow */ }
-            }
-        }
+    	loadPropsInternal("default-deploy.properties", deployProps, true);
     }
 
     private static void loadOverrideProps(Properties deployProps) {
+    	loadPropsInternal("deploy.properties", deployProps, false);
+    }
+    
+    private static void loadPropsInternal(String propFileName, 
+    	Properties deployProps,	boolean prtinTrace) 
+    {    
         FileInputStream fis = null;
         try {
-            File flnm = new File(getPropertiesPath(), "default-deploy.properties");
+            File flnm = new File(getPropertiesPath(), propFileName);
             fis = new FileInputStream(flnm);
             deployProps.load(fis);
         } catch (Exception ex) {
-            // using all defaults
+            if (prtinTrace) ex.printStackTrace();
         } finally {
             if (fis != null) {
                 try {
@@ -407,214 +538,51 @@
 
 
     private static int str2int(String argx) {
-        long l;
-
-        if( argx.trim().equals(Integer.MAX_VALUE) ) return Integer.MAX_VALUE;
-        if( argx.trim().equals(Integer.MIN_VALUE) ) return Integer.MIN_VALUE;
-
-        l = str2long(argx);
-        if (l < Integer.MAX_VALUE && l > Integer.MIN_VALUE) {
-            return (int) l;
-        } else {
-            throw new NumberFormatException("Invalid number:"+argx
-                                            +"  --number out of range");
-        }
+    	Number n = null;
+    	try {	
+    		//TODO - truncation can occur -- check for overflow
+    		n = numberFormat.parse(argx);
+    	} catch (ParseException e) {
+    		throw new NumberFormatException("Invalid integer: " + argx);
+    	}
+    	return n.intValue();
     }
 
     private static long str2long(String argx) {
-
-        int minDigitNumBetwnComma = 3;
-
-        String arg = argx.trim();
-        arg = arg.replaceAll("\"", ""); // strip all quotes
-        int sz = arg.length();
-
-        if( arg.equals("Long.MAX_VALUE") ) return Long.MAX_VALUE;
-
-        if( arg.equals("Long.MIN_VALUE") ) return Long.MIN_VALUE;
-
-        int asterPos = -1;
-        String arg1 = null;
-        String arg2 = null;
-        if( (asterPos = arg.indexOf("*")) != -1) {
-            int dotPos = -1;
-            arg1 = arg.substring(0, asterPos).trim();
-            int denom1 = 1;
-            if( (dotPos = arg1.indexOf(".")) != -1) {
-                StringBuffer tmpBuf = new StringBuffer("1");
-                int hitNumber = 0;
-                for (int i = dotPos + 1; i < (arg1.length() - dotPos); i++) {
-                    if( Character.isDigit(arg1.charAt(i)) ) {
-                        tmpBuf.append("0");
-                    } else {
-                        break;
-                    }
-                }
-                denom1 = Integer.valueOf(tmpBuf.toString());
-                arg1 = arg1.substring(0, dotPos) + arg1.substring(dotPos + 1);
-            }
-
-            arg2 = arg.substring(asterPos + 1).trim();
-            int denom2 = 1;
-            if( (dotPos = arg2.indexOf(".")) != -1) {
-                StringBuffer tmpBuf = new StringBuffer("1");
-                for(int i = dotPos + 1; i <= (arg2.length() - dotPos); i++) {
-                    tmpBuf.append("0");
-                }
-
-                denom2 = Integer.valueOf(tmpBuf.toString());
-                arg2 = arg2.substring(0, dotPos) + arg2.substring(dotPos + 1);
-            }
-
-            long numerator = str2long(arg1) * str2long(arg2);
-            long denom = (denom1 * denom2);
-
-            if (numerator % denom != 0) {
-                throw new NumberFormatException(" Bad value passed in:" +
-                                                ((double) (numerator) /
-                                                 denom) +
-                                                ", expecting a long");
-            }
-            return (numerator / denom);
-        }
-
-        char unit = arg.charAt(sz - 1);
-
-        String valScalar = arg.substring(0, (sz - 1)).trim();
-
-        long factor = 0l;
-
-        switch (Character.toUpperCase(unit)) {
-
-            case 'G':
-                factor = 1000000000l;
-                break;
-            case 'M':
-                factor = 1000000l;
-                break;
-            case 'K':
-                factor = 1000l;
-                break;
-            case 'B':
-                char unitPre = arg.charAt(sz - 2);
-                if (Character.isDigit(unitPre)) {
-                    factor = -1l;
-                } else {
-                    factor =
-                        (Character.toUpperCase(unitPre) ==
-                         'G' ? 1000000000l : (Character.toUpperCase(unitPre) ==
-                                          'M' ? 1000000l : (Character.
-                                                            toUpperCase
-                                                            (unitPre) ==
-                                                            'K' ? 1000l :
-                                                            -1l)));
-                    valScalar = arg.substring(0, (sz - 2)).trim();
-                }
-                break;
-
-            default:
-                if (Character.isDigit(unit)) {
-                    factor = 1l;
-                    valScalar = arg;
-                }
-        }
-        if (factor == -1l) {
-            throw new NumberFormatException("Invalid number:" + arg);
-        }
-
-        int comaPos = -1;
-        if( (comaPos = valScalar.indexOf(',')) != -1) {
-            if(valScalar.indexOf('.') != -1) {
-                throw new NumberFormatException("Invalid number:"+arg
-                                                +" both \",\" and decimal "
-                                                +"point are not supported");
-            }
-            if( comaPos != 0 && comaPos != (valScalar.length() - 1) ) {
-                String[]spltByComa = valScalar.split(",");
-                valScalar = "";
-                for (int i = spltByComa.length - 1; i >= 0; i--) {
-                    if(i > 0 && spltByComa[i].length() < minDigitNumBetwnComma)
-                    {
-                        throw new NumberFormatException("Invalid number:"+arg
-                                                        +"  unexpected comma "
-                                                        +"format");
-                    }
-                    valScalar = spltByComa[i] + valScalar;
-                }
-            } else {
-                throw new NumberFormatException("Invalid number:\"" +arg
-                                                +"\" -unexpected comma in "
-                                                +"position: "+comaPos);
-            }
-        }
-
-        int decimalPos = -1;
-        String valMultiplByFactor = null;
-        int numZero = 0;
-        try {
-            if( (decimalPos = valScalar.indexOf('.')) != -1) {
-                if (decimalPos != valScalar.lastIndexOf('.')) {
-                    throw new NumberFormatException("Invalid number:"
-                                                    +valScalar
-                                                    +"  --invalid decimal "
-                                                    +"number, bad value");
-                }
-
-                String facStr = String.valueOf(factor);
-                int numZeroFactor = facStr.length() - 1;
-                int numDigitsAfterDecimal =
-                    valScalar.length() - decimalPos - 1;
-                int countZero = 0;
-                for(int i = valScalar.length() - 1; i > decimalPos; i--) {
-
-                    if (valScalar.charAt(i) != '0')
-                        break;
-                    --numDigitsAfterDecimal;
-                    countZero++;
-                }
-                numZero = numZeroFactor - numDigitsAfterDecimal;
-                if (numZero == numDigitsAfterDecimal) {
-                    numZero = 0;
-                }
-                if(numZero < 0) {
-                    throw new NumberFormatException("Invalid number:"
-                                                    +valScalar
-                                                    +"  --invalid decimal "
-                                                    +"number, numzero="
-                                                    + numZero);
-                }
-
-                if(numZero >= 0) {
-                    StringBuffer tmpStrNum =
-                        new StringBuffer(20).
-                        append(valScalar.substring(0, decimalPos)).
-                        append(valScalar.substring(decimalPos + 1,
-                                                   decimalPos + 1 +
-                                                   numDigitsAfterDecimal));
-                    for(int i=0; i<numZero; i++) {
-                        tmpStrNum.append('0');
-                    }
-                    valMultiplByFactor = tmpStrNum.toString();
-                }
-
-            }
-        } catch(NumberFormatException nfe) {
-            throw new NumberFormatException("Invalid number:" +valScalar
-                                            +"  --invalid decimal number, "
-                                            +"numZero="+numZero);
-        }
-
-        long ret = -1l;
-
-        Long ll = ((decimalPos != -1) ? Long.valueOf(valMultiplByFactor)
-                   : (Long.valueOf(valScalar) * factor));
-        if( (ret = Long.valueOf(ll)) >= Long.MAX_VALUE
-            || ret <= Long.MIN_VALUE)
-        {
-            throw new NumberFormatException("Invalid number:"+arg
-                                            +"  --absolute value of number "
-                                            +"too big");
-        }
-        return ret;
+    	Number n = null;
+    	try {	
+    		//TODO - truncation can occur -- check for overflow
+    		n = numberFormat.parse(argx);
+    	} catch (ParseException e) {
+    		throw new NumberFormatException("Invalid long: " + argx);
+    	}
+    	return n.longValue();
     }
+    
+    /**
+     * Returns reference to <code>deploymenyProps</code> field. If null, then the
+     * field is populated by looking for the default and override properties files 
+     * (defined in <code>loadDefaultProps</code> and <code>loadOverrideProps</code>). 
+     * This method is synchronized in order to ensure that the returned reference is
+     * a singleton instance.
+     * Note: This method should be private, but is needed by the unit tests in order
+     * to access and modify the underlying <code>Properties</code> object.
+     * @return Properties instance containing the configuration properties.
+     */
+    synchronized static Properties getDeploymentProperties() {    
+	    if(deploymentProps == null) {
+	        deploymentProps = new Properties();
+	        loadDeployProps(deploymentProps);
+	    }
+	    return deploymentProps;
+    }
+    
+    /**
+     * Convenience method intended for use by unit tests only.
+     * @param properties Sets <code>Properties</code> object 
+     */
+    synchronized static void setDeploymentProperties(Properties properties) {    
+    	deploymentProps = properties;
+    }
+    
 }

Modified: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigurationUtil.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigurationUtil.java	2010-10-13 14:18:44 UTC (rev 3786)
+++ branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/ConfigurationUtil.java	2010-10-13 15:08:24 UTC (rev 3787)
@@ -28,10 +28,10 @@
 import org.apache.log4j.Level;
 import org.apache.log4j.Logger;
 
-import com.sun.jini.config.ConfigUtil;
 import net.jini.url.httpmd.HttpmdUtil;
 
 import java.io.IOException;
+import java.net.UnknownHostException;
 
 /**
  * Utility class that provides a set of static convenience methods
@@ -43,6 +43,11 @@
  * This class cannot be instantiated.
  */
 public class ConfigurationUtil {
+	
+	private ConfigurationUtil() {
+		//prevent instantiation
+		throw new AssertionError
+            ("ConfigurationUtil cannot be instantiated");	}
 
     private static final Logger logger = 
         LogUtil.getLog4jLogger( ConfigurationUtil.class );
@@ -95,7 +100,7 @@
      * <p>
      * Note that the <code>name</code> parameter is handled specially by
      * this method. The value input to that <code>String</code> parameter
-     * is interpretted in one of two ways by this method:
+     * is interpreted in one of two ways by this method:
      * <p><ul>
      *   <li> The name of the local <i>network interface</i> over which the
      *        class server will communicate with clients requesting the
@@ -109,7 +114,7 @@
      * This method first treats the value of the <code>name</code> parameter
      * as a network interface, attempting to determine that interface's
      * corresponding IP address. Should that attempt fail, then this method
-     * then assumes that the caller intended this parameter to be interpretted
+     * then assumes that the caller intended this parameter to be interpreted
      * as an IP address or host name; in which case, this method uses
      * the value of the parameter as the <i>address/host</i> component of
      * codebase being constructed.
@@ -181,7 +186,7 @@
      *         any value, including <code>null</code>).
      *
      * @throws IllegalArgumentException if the value input for 
-     *         <code>port</code> is negtive.
+     *         <code>port</code> is negative.
      */
     public static String computeCodebase(String  name,
                                          String  jarFile,
@@ -220,6 +225,10 @@
             logger.log(Level.TRACE, name+" - not a valid "
                            +"network interface, assuming host name");
         }
+        if (ipAddr==null) {
+        	throw new UnknownHostException(
+        		"Could not determine IP address for given name:" + name);
+        }
 
         // Construct the codebase, either httpmd or http 
         if(doHttpmd) {

Modified: branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/Log4jLoggingHandler.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/util/config/Log4jLoggingHan...
 
[truncated message content]

[Bigdata-commit] SF.net SVN: bigdata:[3786] branches/QUADS_QUERY_BRANCH/bigdata-sails/src/ test/com/bigdata/rdf/sail

[<tho...@us...>]

Revision: 3786
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3786&view=rev
Author:   thompsonbry
Date:     2010-10-13 14:18:44 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Modified four tests which were relying on cnx.flush() before issuing a high level query. They now use cxn.commit().

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestOptionals.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestPruneBindingSets.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestQuadsAPI.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestUnions.java

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestOptionals.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestOptionals.java	2010-10-13 14:11:24 UTC (rev 3785)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestOptionals.java	2010-10-13 14:18:44 UTC (rev 3786)
@@ -161,7 +161,8 @@
              * statement buffers to the database before executing any operations
              * that go around the sail.
              */
-            cxn.flush();//commit();
+//            cxn.flush();//commit();
+            cxn.commit();
             
 /**/            
             if (log.isInfoEnabled()) {
@@ -221,6 +222,8 @@
             cxn.add(vf.createURI("u:1"), 
                     vf.createURI("http://www.w3.org/1999/02/22-rdf-syntax-ns#type"), 
                     vf.createURI("u:2"));
+
+            cxn.commit();
             
             String query = 
                 "SELECT REDUCED ?subj ?subj_class ?subj_label " +

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestPruneBindingSets.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestPruneBindingSets.java	2010-10-13 14:11:24 UTC (rev 3785)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestPruneBindingSets.java	2010-10-13 14:18:44 UTC (rev 3786)
@@ -108,7 +108,8 @@
              * statement buffers to the database before executing any operations
              * that go around the sail.
              */
-            cxn.flush();//commit();
+//            cxn.flush();//commit();
+            cxn.commit();
             
 /**/            
             if (log.isInfoEnabled()) {

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestQuadsAPI.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestQuadsAPI.java	2010-10-13 14:11:24 UTC (rev 3785)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestQuadsAPI.java	2010-10-13 14:18:44 UTC (rev 3786)
@@ -242,7 +242,8 @@
              * statement buffers to the database before executing any operations
              * that go around the sail.
              */
-            cxn.flush();//commit();
+//            cxn.flush();//commit();
+            cxn.commit();
             
             assertEquals(2, sail.database.getNamedGraphCount());
             

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestUnions.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestUnions.java	2010-10-13 14:11:24 UTC (rev 3785)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/TestUnions.java	2010-10-13 14:18:44 UTC (rev 3786)
@@ -154,7 +154,8 @@
              * statement buffers to the database before executing any operations
              * that go around the sail.
              */
-            cxn.flush();//commit();
+//            cxn.flush();//commit();
+            cxn.commit();
             
 /**/            
             if (log.isInfoEnabled()) {


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3785] branches/QUADS_QUERY_BRANCH/bigdata/src/ resources/logging/log4j.properties

[<tho...@us...>]

Revision: 3785
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3785&view=rev
Author:   thompsonbry
Date:     2010-10-13 14:11:24 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
logging change.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties	2010-10-13 14:10:53 UTC (rev 3784)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/resources/logging/log4j.properties	2010-10-13 14:11:24 UTC (rev 3785)
@@ -174,6 +174,8 @@
 
 #log4j.logger.com.bigdata.rdf.sail.BigdataSail=DEBUG
 log4j.logger.com.bigdata.rdf.sail.Rule2BOpUtility=INFO
+log4j.logger.com.bigdata.rdf.sail.tck.BigdataFederationSparqlTest=INFO
+#log4j.logger.com.bigdata.bop.fed.shards.Algorithm_NestedLocatorScan=ALL
 #log4j.logger.com.bigdata.rdf.sail.TestNamedGraphs=DEBUG
 #log4j.logger.com.bigdata.rdf.sail.QuadsTestCase=DEBUG
 #log4j.logger.com.bigdata.relation.rule.eval.NestedSubqueryWithJoinThreadsTask=DEBUG


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3784] branches/QUADS_QUERY_BRANCH/bigdata-sails/src/ test/com/bigdata/rdf/sail/tck/

[<tho...@us...>]

Revision: 3784
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3784&view=rev
Author:   thompsonbry
Date:     2010-10-13 14:10:53 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
property reconcile

Property Changed:
----------------
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/


Property changes on: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck
___________________________________________________________________
Modified: svn:ignore
   - DoNotDistribute_QueryTest.java
data-2.ttl
TestSparqlStuff.java

   + DoNotDistribute_QueryTest.java
data-2.ttl
TestSparqlStuff.java
DumpMDI.java



This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3783] branches/QUADS_QUERY_BRANCH/bigdata-sails/src/ test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java

[<tho...@us...>]

Revision: 3783
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3783&view=rev
Author:   thompsonbry
Date:     2010-10-13 14:09:49 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Commented out the dump of the database for each test run.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 13:46:30 UTC (rev 3782)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 14:09:49 UTC (rev 3783)
@@ -165,7 +165,7 @@
 		// return dataRep.getConnection();
 		final BigdataSailRepositoryConnection con = new BigdataSailRepositoryConnection(new BigdataSailRepository(
 				_sail), _sail.getReadOnlyConnection());
-		System.err.println(_sail.getDatabase().dumpStore());
+//		System.err.println(_sail.getDatabase().dumpStore());
 		return con;
 	}
 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3782] branches/QUADS_QUERY_BRANCH/bigdata

[<tho...@us...>]

Revision: 3782
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3782&view=rev
Author:   thompsonbry
Date:     2010-10-13 13:46:30 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Fixed test failure on "normalization" by appropriately configuring Unicode collation for that test.

Fixed (or at least worked around) test failures with two data services by not destroying the triple store in tearDown().  tearDown() is somehow being invoked before the query runs to completion which was causing those "no such index" exceptions to be thrown.

Modified the DataServer to use a single proxy for its QueryEngine for its live span.  Added support for the proxy of the query engine to the MetadataServer, through we are not yet using query against the MDS.

Simplified the query invocation in BigdataEvaluationStrategyImpl.  It now uses QueryEngine.eval/1.

Modified the SAIL to use full read-only transactions when the caller requests a read-only connection.  This asserts the appropriate read lock on the transaction server and prevents commit points against which the query is running from being aged out of the RWStore or the federation.  (This was not an issue in the SPARQL queries, but it is one of the things that I explored.)

Javadoc changes in FederatedRunningQuery.

Log statement changed in Algorithm_NestedLocatorScan.

Added some logic to dump out the partition locators in AccessPath if the index could not be found.  This was eventually traced to the test tear down so the logic is commented out in AccessPath.

Added an explicit getQueryConnection() method to SPARQLQueryTest.

Various changes to the test harness to get things running properly.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java
    branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/DataServer.java
    branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/MetadataServer.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataEvaluationStrategyImpl.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSail.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/FederatedRunningQuery.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -46,6 +46,7 @@
 import com.bigdata.bop.engine.IQueryClient;
 import com.bigdata.bop.engine.IQueryPeer;
 import com.bigdata.bop.engine.LocalChunkMessage;
+import com.bigdata.bop.engine.QueryEngine;
 import com.bigdata.bop.engine.RunningQuery;
 import com.bigdata.bop.fed.shards.MapBindingSetsOverShardsBuffer;
 import com.bigdata.io.DirectBufferPool;
@@ -313,19 +314,20 @@
         
     }
 
-    /**
-     * Resolve the proxy for an {@link IQueryPeer}. This is special cased for
-     * both <i>this</i> service (the actual reference is returned) and the query
-     * controller (we use an alternative path to discover the query controller
-     * since it might not be registered against a lookup service if it is not a
-     * data service).
-     * 
-     * @param serviceUUID
-     *            The service identifier for the peer.
-     * 
-     * @return The proxy for the service or <code>null</code> if the service
-     *         could not be discovered.
-     */
+	/**
+	 * Resolve the proxy for an {@link IQueryPeer}. This is special cased for
+	 * both <i>this</i> service (the actual reference is returned) and the query
+	 * controller (we use an alternative path to discover the query controller
+	 * since it might not be registered against a lookup service if it is not a
+	 * data service).
+	 * 
+	 * @param serviceUUID
+	 *            The service identifier for the peer.
+	 * 
+	 * @return The proxy for the service, the actual {@link QueryEngine}
+	 *         reference if the identified service is <i>this</i> service, or
+	 *         <code>null</code> if the service could not be discovered.
+	 */
     protected IQueryPeer getQueryPeer(final UUID serviceUUID) {
 
         if (serviceUUID == null)
@@ -334,9 +336,16 @@
         final IQueryPeer queryPeer;
         
         if(serviceUUID.equals(getQueryEngine().getServiceUUID())) {
-        
-            // Return a hard reference to this query engine (NOT a proxy).
-            return getQueryEngine();
+
+			/*
+			 * Return a hard reference to this query engine (NOT a proxy).
+			 * 
+			 * Note: This is used to avoid RMI when the message will be consumed
+			 * by the service which produced that message. This is a deliberate
+			 * performance optimization which is supported by all of the data
+			 * structures involved.
+			 */
+            queryPeer = getQueryEngine();
             
         } else if (serviceUUID.equals(queryControllerUUID)) {
         

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/fed/shards/Algorithm_NestedLocatorScan.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -45,9 +45,10 @@
                 final PartitionLocator locator = itr.next();
 
                 if (log.isTraceEnabled())
-                    log.trace("adding bindingSet to buffer" + ": partitionId="
-                            + locator.getPartitionId() + ", bindingSet="
-                            + bundle.bindingSet);
+					log.trace("adding bindingSet to buffer" + ": partitionId="
+							+ locator.getPartitionId() + "dataService="
+							+ locator.getDataServiceUUID() + ", bindingSet="
+							+ bundle.bindingSet);
 
                 final IBuffer<IBindingSet[]> sink = op.getBuffer(locator);
 

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -390,6 +390,11 @@
 
             if (ndx == null) {
 
+//            	// For debugging only - comment this out.
+//				dumpMDI((AbstractScaleOutFederation<?>) relation
+//						.getIndexManager(), relation.getNamespace(), timestamp,
+//						keyOrder);
+
                 throw new RuntimeException("No such index: relation="
                         + relation.getNamespace() + ", timestamp=" + timestamp
                         + ", keyOrder=" + keyOrder + ", pred=" + predicate
@@ -1688,4 +1693,33 @@
      */
     private static final DiskCostModel diskCostModel = DiskCostModel.DEFAULT;
 
+//	/**
+//	 * Dumps the locators for an index of a relation.
+//	 * 
+//	 * @param fed
+//	 * @param namespace
+//	 *            The relation namespace.
+//	 * @param timestamp
+//	 *            The timestamp of the view.
+//	 * @param keyOrder
+//	 *            The index.
+//	 */
+//	private static void dumpMDI(AbstractScaleOutFederation<?> fed,
+//			final String namespace, final long timestamp,
+//			final IKeyOrder<?> keyOrder) {
+//
+//		final String name = namespace + "." + keyOrder.getIndexName();
+//
+//		final Iterator<PartitionLocator> itr = fed
+//				.locatorScan(name, timestamp, new byte[] {}/* fromKey */,
+//						null/* toKey */, false/* reverseScan */);
+//
+//		System.err.println("name=" + name + " @ "
+//				+ TimestampUtility.toString(timestamp));
+//		while (itr.hasNext()) {
+//			System.err.println(itr.next());
+//		}
+//
+//	}
+
 }

Modified: branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/DataServer.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/DataServer.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/DataServer.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -372,17 +372,27 @@
 
         }
 
+        /**
+         * Extends the base behavior to return an RMI compatible proxy for the
+         * {@link IQueryEngine}.
+         */
         @Override
         public IQueryPeer getQueryEngine() {
             
-            /*
-             * Note: DGC is not necessary since the DataService has a hard
-             * reference to the QueryEngine.
-             */
-            return getFederation()
-                    .getProxy(super.getQueryEngine(), false/* enableDGC */);
-            
+			synchronized (this) {
+				if (queryPeerProxy == null) {
+					/*
+					 * Note: DGC is not necessary since the DataService has a
+					 * hard reference to the QueryEngine.
+					 */
+					queryPeerProxy = getFederation().getProxy(
+							super.getQueryEngine(), false/* enableDGC */);
+				}
+				return queryPeerProxy;
+			}
+
         }
+        private IQueryPeer queryPeerProxy = null;
         
         /**
          * Extends the base behavior to return a {@link Name} of the service

Modified: branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/MetadataServer.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/MetadataServer.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-jini/src/java/com/bigdata/service/jini/MetadataServer.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -43,9 +43,9 @@
 
 import org.apache.log4j.MDC;
 
+import com.bigdata.bop.engine.IQueryPeer;
 import com.bigdata.btree.proc.IIndexProcedure;
 import com.bigdata.service.IDataService;
-import com.bigdata.service.IMetadataService;
 import com.bigdata.service.MetadataService;
 import com.bigdata.service.DataService.DataServiceFederationDelegate;
 import com.sun.jini.start.LifeCycle;
@@ -302,6 +302,28 @@
 
         /**
          * Extends the base behavior to return an RMI compatible proxy for the
+         * {@link IQueryEngine}.
+         */
+        @Override
+        public IQueryPeer getQueryEngine() {
+            
+			synchronized (this) {
+				if (queryPeerProxy == null) {
+					/*
+					 * Note: DGC is not necessary since the DataService has a
+					 * hard reference to the QueryEngine.
+					 */
+					queryPeerProxy = getFederation().getProxy(
+							super.getQueryEngine(), false/* enableDGC */);
+				}
+				return queryPeerProxy;
+			}
+
+        }
+        private IQueryPeer queryPeerProxy = null;
+        
+        /**
+         * Extends the base behavior to return an RMI compatible proxy for the
          * {@link Future}.
          */
         @Override

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataEvaluationStrategyImpl.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataEvaluationStrategyImpl.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataEvaluationStrategyImpl.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -14,7 +14,6 @@
 import java.util.Map;
 import java.util.Properties;
 import java.util.Set;
-import java.util.UUID;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicInteger;
 
@@ -55,7 +54,6 @@
 import com.bigdata.BigdataStatics;
 import com.bigdata.bop.BOp;
 import com.bigdata.bop.Constant;
-import com.bigdata.bop.HashBindingSet;
 import com.bigdata.bop.IBindingSet;
 import com.bigdata.bop.IConstraint;
 import com.bigdata.bop.IPredicate;
@@ -70,7 +68,6 @@
 import com.bigdata.bop.constraint.NE;
 import com.bigdata.bop.constraint.NEConstant;
 import com.bigdata.bop.constraint.OR;
-import com.bigdata.bop.engine.LocalChunkMessage;
 import com.bigdata.bop.engine.QueryEngine;
 import com.bigdata.bop.engine.RunningQuery;
 import com.bigdata.bop.solutions.ISortOrder;
@@ -103,7 +100,6 @@
 import com.bigdata.relation.accesspath.IAsynchronousIterator;
 import com.bigdata.relation.accesspath.IBuffer;
 import com.bigdata.relation.accesspath.IElementFilter;
-import com.bigdata.relation.accesspath.ThickAsynchronousIterator;
 import com.bigdata.relation.rule.IAccessPathExpander;
 import com.bigdata.relation.rule.IProgram;
 import com.bigdata.relation.rule.IQueryOptions;
@@ -1718,12 +1714,14 @@
 
         }
 
-        final UUID queryId = UUID.randomUUID();
-        final RunningQuery runningQuery = queryEngine.eval(queryId, query,
-                new LocalChunkMessage<IBindingSet>(queryEngine, queryId,
-                        startId, -1/* partitionId */,
-                        newBindingSetIterator(new HashBindingSet())));
-        
+//        final UUID queryId = UUID.randomUUID();
+//        final RunningQuery runningQuery = queryEngine.eval(queryId, query,
+//                new LocalChunkMessage<IBindingSet>(queryEngine, queryId,
+//                        startId, -1/* partitionId */,
+//                        newBindingSetIterator(new HashBindingSet())));
+
+        final RunningQuery runningQuery = queryEngine.eval(query);
+
         final IAsynchronousIterator<IBindingSet[]> it1 = 
             runningQuery.iterator();
         
@@ -1817,21 +1815,21 @@
         
     }
 
-    /**
-     * Return an {@link IAsynchronousIterator} that will read a single,
-     * empty {@link IBindingSet}.
-     * 
-     * @param bindingSet
-     *            the binding set.
-     */
-    protected ThickAsynchronousIterator<IBindingSet[]> newBindingSetIterator(
-            final IBindingSet bindingSet) {
+//    /**
+//     * Return an {@link IAsynchronousIterator} that will read a single,
+//     * empty {@link IBindingSet}.
+//     * 
+//     * @param bindingSet
+//     *            the binding set.
+//     */
+//    protected ThickAsynchronousIterator<IBindingSet[]> newBindingSetIterator(
+//            final IBindingSet bindingSet) {
+//
+//        return new ThickAsynchronousIterator<IBindingSet[]>(
+//                new IBindingSet[][] { new IBindingSet[] { bindingSet } });
+//
+//    }
 
-        return new ThickAsynchronousIterator<IBindingSet[]>(
-                new IBindingSet[][] { new IBindingSet[] { bindingSet } });
-
-    }
-
     @SuppressWarnings("serial")
     private class UnknownOperatorException extends RuntimeException {
         private TupleExpr operator;

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSail.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSail.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/java/com/bigdata/rdf/sail/BigdataSail.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -114,11 +114,11 @@
 
 import com.bigdata.bop.engine.QueryEngine;
 import com.bigdata.bop.fed.QueryEngineFactory;
+import com.bigdata.journal.AbstractJournal;
 import com.bigdata.journal.IIndexManager;
 import com.bigdata.journal.ITransactionService;
 import com.bigdata.journal.ITx;
 import com.bigdata.journal.Journal;
-import com.bigdata.journal.TimestampUtility;
 import com.bigdata.rdf.axioms.NoAxioms;
 import com.bigdata.rdf.inf.TruthMaintenance;
 import com.bigdata.rdf.internal.IV;
@@ -153,6 +153,7 @@
 import com.bigdata.relation.accesspath.IAccessPath;
 import com.bigdata.relation.accesspath.IElementFilter;
 import com.bigdata.relation.rule.IRule;
+import com.bigdata.service.AbstractFederation;
 import com.bigdata.service.IBigdataFederation;
 import com.bigdata.striterator.CloseableIteratorWrapper;
 import com.bigdata.striterator.IChunkedIterator;
@@ -1163,7 +1164,7 @@
     public BigdataSailConnection getReadOnlyConnection() {
         
         final long timestamp = database.getIndexManager().getLastCommitTime();
-        
+
         return getReadOnlyConnection(timestamp);
         
     }
@@ -1178,15 +1179,118 @@
      * 
      * @return The view.
      */
-    public BigdataSailConnection getReadOnlyConnection(long timestamp) {
+    public BigdataSailConnection getReadOnlyConnection(final long timestamp) {
         
-        AbstractTripleStore view = (AbstractTripleStore) database
-            .getIndexManager().getResourceLocator().locate(
-                    database.getNamespace(),
-                    TimestampUtility.asHistoricalRead(timestamp));
+//        AbstractTripleStore view = (AbstractTripleStore) database
+//            .getIndexManager().getResourceLocator().locate(
+//                    database.getNamespace(),
+//                    TimestampUtility.asHistoricalRead(timestamp));
+//
+//        return new BigdataSailConnection(view, null);
 
-        return new BigdataSailConnection(view, null);
+    	try {
+			return _getReadOnlyConnection(timestamp);
+		} catch (IOException e) {
+			throw new RuntimeException(e);
+		}
+    	
+    }
         
+    /**
+     * Return a read-only connection backed by a read-only transaction.  The
+     * transaction will be closed when the connection is closed.
+     * @param timestamp The timestamp.
+     * @return The transaction.
+     * @throws IOException
+     * @see ITransactionService#newTx(long)
+     */
+    private BigdataSailConnection _getReadOnlyConnection(final long timestamp) throws IOException {
+    	
+        final String namespace = database.getNamespace();
+
+        final IIndexManager indexManager = database.getIndexManager();
+
+        final ITransactionService txService = getTxService();
+        
+        return new BigdataSailConnection(null/*lock*/) {
+
+            /**
+             * The transaction id.
+             */
+            private long tx;
+
+            /**
+             * Constructor starts a new transaction.
+             */
+            {
+                newTx();
+            }
+            
+            /**
+             * Obtain a new read-only transaction from the journal's
+             * transaction service, and attach this SAIL connection to the new
+             * view of the database. 
+             */
+            protected void newTx() throws IOException {
+                
+                this.tx = txService.newTx(timestamp);
+
+                final AbstractTripleStore txView = (AbstractTripleStore) indexManager
+                        .getResourceLocator().locate(namespace, tx);
+                
+                attach(txView);
+                
+            }
+
+            /**
+             * NOP
+             */
+            @Override
+            public synchronized void commit() throws SailException {
+
+            	// NOP.
+            	
+            }
+            
+            /**
+             * NOP
+             */
+            @Override
+            public synchronized void rollback() throws SailException {
+
+            	// NOP
+            	
+            }
+            
+            /**
+             * A specialized close that will also abort the current read-only
+             * transaction.
+             */
+            @Override
+            public synchronized void close() throws SailException {
+
+                if (!isOpen()) {
+                    
+                    return;
+                    
+                }
+                
+                super.close();
+                
+                try {
+
+                    txService.abort(tx);
+                
+                } catch(IOException ex) {
+                        
+                    throw new SailException(ex);
+                    
+                }
+                
+            }
+
+        };
+        
     }
     
     /**
@@ -1204,10 +1308,14 @@
         
         final IIndexManager indexManager = database.getIndexManager();
 
-        // @todo no way to get the txService here w/o a cast?
-        final ITransactionService txService = ((Journal) indexManager)
-                .getTransactionManager().getTransactionService();
+        if(indexManager instanceof IBigdataFederation<?>) {
+
+        	throw new UnsupportedOperationException("Read/write transactions are not yet supported in scale-out.");
+        	
+        }
         
+        final ITransactionService txService = getTxService();
+        
         final String namespace = database.getNamespace();
         
         final Lock readLock = lock.readLock();
@@ -1344,13 +1452,37 @@
         
     }
     
+    /**
+     * Return the {@link ITransactionService}.
+     */
+	protected ITransactionService getTxService() {
+
+		final IIndexManager indexManager = database.getIndexManager();
+
+		final ITransactionService txService;
+
+		if (indexManager instanceof AbstractJournal) {
+
+			txService = ((Journal) indexManager).getTransactionManager()
+					.getTransactionService();
+
+		} else {
+
+			txService = ((AbstractFederation<?>) indexManager)
+					.getTransactionService();
+
+		}
+
+		return txService;
+
+	}
+    
     public QueryEngine getQueryEngine() {
         
         return queryEngine;
         
     }
     
-    
     /**
      * Inner class implements the {@link SailConnection}. Some additional
      * functionality is available on this class, including

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -35,12 +35,16 @@
 import org.openrdf.query.parser.sparql.ManifestTest;
 import org.openrdf.query.parser.sparql.SPARQLQueryTest;
 import org.openrdf.repository.Repository;
+import org.openrdf.repository.RepositoryConnection;
 import org.openrdf.repository.dataset.DatasetRepository;
 
+import com.bigdata.btree.keys.CollatorEnum;
 import com.bigdata.btree.keys.KeyBuilder;
+import com.bigdata.btree.keys.StrengthEnum;
 import com.bigdata.journal.ITx;
 import com.bigdata.rdf.sail.BigdataSail;
 import com.bigdata.rdf.sail.BigdataSailRepository;
+import com.bigdata.rdf.sail.BigdataSailRepositoryConnection;
 import com.bigdata.rdf.sail.BigdataSail.Options;
 import com.bigdata.rdf.store.ScaleOutTripleStore;
 import com.bigdata.service.jini.JiniClient;
@@ -52,6 +56,7 @@
  * in its own bigdata namespace.
  * 
  * @author <a href="mailto:dm...@us...">David MacMillan</a>
+ * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
  * @version $Id$
  */
 public class BigdataFederationSparqlTest extends SPARQLQueryTest
@@ -123,14 +128,29 @@
         throws Exception
     {
         super.tearDown () ;
-        _ts.destroy () ;
-        _ts = null ;
-    }
+		/*
+		 * @todo We should destroy the triple store here, but this is causing
+		 * problems with tear down of the query while it is still running. Once
+		 * that issue has been fixed, uncomment both the line to destroy the
+		 * triple store and the line to shutdown the federation (the latter is
+		 * really optional - it should be Ok to leave the federation up across
+		 * the test runs, but then we will never take it down cleanly when the
+		 * test suite is done. Again, that should be Ok.)
+		 */
+		if (_ts != null) {
+//			_ts.destroy();
+			_ts = null;
+		}
+//		if (_fed != null) {
+//			_fed.shutdownNow();
+//			_fed = null;
+//		}
+	}
 
     @Override protected Repository newRepository ()
         throws Exception
     {
-        return new DatasetRepository ( new BigdataSailRepository ( new BigdataSail ( newTripleStore () ) ) ) ;
+        return new DatasetRepository ( new BigdataSailRepository ( _sail = new BigdataSail ( newTripleStore () ) ) ) ;
     }
 
     @Override
@@ -140,6 +160,15 @@
         return repo;
     }
 
+	protected RepositoryConnection getQueryConnection(Repository dataRep)
+			throws Exception {
+		// return dataRep.getConnection();
+		final BigdataSailRepositoryConnection con = new BigdataSailRepositoryConnection(new BigdataSailRepository(
+				_sail), _sail.getReadOnlyConnection());
+		System.err.println(_sail.getDatabase().dumpStore());
+		return con;
+	}
+
     private ScaleOutTripleStore newTripleStore ()
         throws Exception
     {
@@ -176,163 +205,105 @@
 	/**
 	 * Configuration options for the KB instances used to run the SPARQL
 	 * compliance test suite.
+	 * <p>
+	 * Note: These properties can not be cached across tests since they have to
+	 * be slightly different for some of the tests to handle things like tests
+	 * which will fail with inlining enabled ot tests which require Unicode
+	 * collation strength of IDENTICAL.
 	 */
     private Properties getProperties () throws Exception
     {
-//    	Note: This approach does not work because we are using a different namespace for each test.
-//        /*
-//         * Pick up properties configured for the client as defaults.
-//         * 
-//         * You can specify those properties using NV[] for the component.
-//         */
-//        final String component = System.getProperty ( COMPONENT_PROPERTY, DEFAULT_COMPONENT_PROPERTY ) ;
-//        final Properties properties = getFederation().getClient().getProperties(
-//                component);
-//        return properties;
-        if ( null == _properties )
-        {
-        	
-//    	    /* Multiplier for the scatter effect.
-//    	     */
-//    	    final int scatterFactor = 1;
-//    	    final int scatterFactor_term2id = 1;
-//    	    final int dataServiceCount = 2;
-//
-//    	    /* The #of index partitions to allocate on a scatter split.  ZERO
-//    	     * (0) means that 2 index partitions will be allocated per
-//    	     * data service which partiticpates in the scatter split.
-//    	     * Non-zero values directly give the #of index partitions to
-//    	     * create.  
-//    	     */
-//    	    final int scatterSplitIndexPartitionCount = ConfigMath.multiply
-//    		( scatterFactor,
-//    		  dataServiceCount
-//    		  );
-//    	    final int scatterSplitIndexPartitionCount_term2id = ConfigMath.multiply
-//    		( scatterFactor_term2id,
-//    		  dataServiceCount
-//    		  );
-//
-//    	    // Use all discovered data services when scattering an index.
-//    	    final int scatterSplitDataServiceCount = 0;
-//
-//    	    /* Scatter split trigger point.  The scatter split will not be
-//    	     * triggered until the initial index partition has reached
-//    	     * this percentage of a nominal index partition in size.
-//    	     */
-//    	    final double scatterSplitPercentOfSplitThreshold = 0.5;//was .5
-//
-//    		/* 
-//    		 * Multipliers that compensate for the consumer/producer ratio for
-//    		 * the asynchronous index write API.  These are empirical factors
-//    		 * based on observing the ratio (chunkWritingTime/chunkWaitingTime).
-//    		 * Assuming a constant chunk writing time, if the chunk size for each
-//    		 * index is adjusted by its multiplier then this ratio would be 1:1.
-//    		 * In practice, the chunk writing time is not a linear function of
-//    		 * the chunk size, which is one reason why we prefer larger chunks
-//    		 * and why the asynchronous write API is a win.
-//    		 *
-//    		 * Note: These factors were set relative to TERM2ID.  However, when
-//    		 * I reduced the scatterFactor for TERM2ID by 1/2, I doubled its
-//    		 * chunk size to keep up the same throughput so it is now at 2.00
-//    		 * rather than 1.00.
-//    		 */
-//    	    final double chunkSizeFactor_id2term =  1.79;
-//    		final double chunkSizeFactor_term2id =  2.00;
-//    		final double chunkSizeFactor_stmts   =  8.00;
-//    		
-//    		/* The nominal sink chunk size.  For each index, this is adjusted
-//    		 * by the factor specified above.
-//    		 */
-////        		static private sinkChunkSize = 10000;
-//    		final int sinkChunkSize = 1000;
-        		 
-    	    /*
-    	     * Specify / override some triple store properties.
-    	     *
-    	     * Note: You must reference this object in the section for the
-    	     * component which will actually create the KB instance, e.g.,
-    	     * either the RDFDataLoadMaster or the LubmGeneratorMaster.
-    	     */
-            _properties = new Properties ();
-            
-            /*
-             * Setup for quads.
-             */
-            _properties.put ( BigdataSail.Options.QUADS_MODE, "true" );
-            _properties.put ( BigdataSail.Options.TRUTH_MAINTENANCE, "false" );
-            _properties.put ( BigdataSail.Options.QUERY_TIME_EXPANDER, "false" );
-            
-			/*
-			 * The Sesame TCK forces statement level connection auto-commit so
-			 * we set a flag to permit that here. However, auto-commit and this
-			 * flag SHOULD NOT be used outside of the test suite as they provide
-			 * an extreme performance penalty.
-			 */
-            //_properties.put ( BigdataSail.Options.ALLOW_AUTO_COMMIT, "true" ) ;
+		final Properties _properties;
 
-			/*
-			 * Provide Unicode support for keys with locale-based string
-			 * collation. This is more expensive in key creation during loading,
-			 * but allows key comparison and sorting in the specified locale in
-			 * queries.
-			 * 
-			 * @see com.bigdata.btree.keys.CollatorEnum
-			 */
-    	    _properties.put(KeyBuilder.Options.COLLATOR,"ICU");
-    	    _properties.put(KeyBuilder.Options.USER_LANGUAGE,"en");
-    	    _properties.put(KeyBuilder.Options.USER_COUNTRY,"US");
-    	    _properties.put(KeyBuilder.Options.USER_VARIANT,"");
+		/*
+		 * Specify / override some triple store properties.
+		 * 
+		 * Note: You must reference this object in the section for the component
+		 * which will actually create the KB instance, e.g., either the
+		 * RDFDataLoadMaster or the LubmGeneratorMaster.
+		 */
+		_properties = new Properties();
 
-			/*
-			 * Turn off the full text index (search for literals by keyword).
-			 */
-			_properties.put(BigdataSail.Options.TEXT_INDEX, "false");
+		/*
+		 * Setup for quads.
+		 */
+		_properties.put(BigdataSail.Options.QUADS_MODE, "true");
+		_properties.put(BigdataSail.Options.TRUTH_MAINTENANCE, "false");
+		_properties.put(BigdataSail.Options.QUERY_TIME_EXPANDER, "false");
 
-			/*
-			 * Turn on bloom filter for the SPO index (good up to ~2M index
-			 * entries for scale-up -or- for any size index for scale-out). This
-			 * is a big win for some queries on scale-out indices since we can
-			 * avoid touching the disk if the bloom filter reports "false" for a
-			 * key.
-			 */
-			_properties.put(BigdataSail.Options.BLOOM_FILTER, "true");
+		/*
+		 * The Sesame TCK forces statement level connection auto-commit so we
+		 * set a flag to permit that here. However, auto-commit and this flag
+		 * SHOULD NOT be used outside of the test suite as they provide an
+		 * extreme performance penalty.
+		 */
+		// _properties.put ( BigdataSail.Options.ALLOW_AUTO_COMMIT, "true" ) ;
 
-			/*
-			 * The #of low order bits from the TERM2ID index partition local
-			 * counter that will be reversed and written into the high-order
-			 * bits of the term identifier. This has a strong effect on the
-			 * distribution of bulk index read/write operations for the triple
-			 * store. For a given value of N, a bulk write will tend to touch
-			 * 2^N index partitions. Therefore if this is even roughly on the
-			 * order of the number of index partitions, each bulk write will
-			 * tend to be scattered to all index partitions.
-			 * 
-			 * Note: If this value is too large then the writes WITHIN the index
-			 * partitions will become uniformly distributed, which will
-			 * negatively impact index performance.
-			 */
-			_properties.put(BigdataSail.Options.TERMID_BITS_TO_REVERSE,"0"); // was 2.
-        	        
-			/*
-			 * Option may be enabled to store blank nodes such that they are
-			 * stable (they are not stored by default).
-			 */
-			// new NV(BigdataSail.Options.STORE_BLANK_NODES,"true");
+		/*
+		 * Provide Unicode support for keys with locale-based string collation.
+		 * This is more expensive in key creation during loading, but allows key
+		 * comparison and sorting in the specified locale in queries.
+		 * 
+		 * @see com.bigdata.btree.keys.CollatorEnum
+		 */
+		_properties.put(KeyBuilder.Options.COLLATOR, "ICU");
+		_properties.put(KeyBuilder.Options.USER_LANGUAGE, "en");
+		_properties.put(KeyBuilder.Options.USER_COUNTRY, "US");
+		_properties.put(KeyBuilder.Options.USER_VARIANT, "");
 
-        }
+		/*
+		 * Turn off the full text index (search for literals by keyword).
+		 */
+		_properties.put(BigdataSail.Options.TEXT_INDEX, "false");
 
 		/*
-		 * Turn inlining on or off depending on _this_ test. This is outside of
-		 * the if block above because _properties is cached.
+		 * Turn on bloom filter for the SPO index (good up to ~2M index entries
+		 * for scale-up -or- for any size index for scale-out). This is a big
+		 * win for some queries on scale-out indices since we can avoid touching
+		 * the disk if the bloom filter reports "false" for a key.
 		 */
-        if (BigdataSparqlTest.cannotInlineTests.contains(testURI)) {
-            _properties.setProperty(Options.INLINE_LITERALS, "false");
-        } else {
-        	_properties.setProperty(Options.INLINE_LITERALS, "true");
-        }
-        
-        return _properties ;
+		_properties.put(BigdataSail.Options.BLOOM_FILTER, "true");
+
+		/*
+		 * The #of low order bits from the TERM2ID index partition local counter
+		 * that will be reversed and written into the high-order bits of the
+		 * term identifier. This has a strong effect on the distribution of bulk
+		 * index read/write operations for the triple store. For a given value
+		 * of N, a bulk write will tend to touch 2^N index partitions. Therefore
+		 * if this is even roughly on the order of the number of index
+		 * partitions, each bulk write will tend to be scattered to all index
+		 * partitions.
+		 * 
+		 * Note: If this value is too large then the writes WITHIN the index
+		 * partitions will become uniformly distributed, which will negatively
+		 * impact index performance.
+		 */
+		_properties.put(BigdataSail.Options.TERMID_BITS_TO_REVERSE, "0");
+
+		/*
+		 * Option may be enabled to store blank nodes such that they are stable
+		 * (they are not stored by default).
+		 */
+		// new NV(BigdataSail.Options.STORE_BLANK_NODES,"true");
+
+		/*
+		 * Turn inlining on or off depending on _this_ test.
+		 */
+		if (BigdataSparqlTest.cannotInlineTests.contains(testURI)) {
+			_properties.setProperty(Options.INLINE_LITERALS, "false");
+		} else {
+			_properties.setProperty(Options.INLINE_LITERALS, "true");
+		}
+
+		if (BigdataSparqlTest.unicodeStrengthIdentical.contains(testURI)) {
+			// Force identical Unicode comparisons.
+			_properties.setProperty(Options.COLLATOR, CollatorEnum.JDK
+					.toString());
+			_properties.setProperty(Options.STRENGTH, StrengthEnum.Identical
+					.toString());
+		}
+
+		return _properties;
     }
 
 	/**
@@ -340,21 +311,11 @@
 	 */
 	public static final String CONFIG_PROPERTY = "bigdata.configuration";
 	
-//	/**
-//	 * The name of the component in that configuration file whose "properties"
-//	 * field will be used to initialize the KB.
-//	 */
-//    public static final String COMPONENT_PROPERTY = "bigdata.component" ;
-//
-//    /**
-//     * The default value for {@link #COMPONENT_PROPERTY}.
-//     */
-//    public static final String DEFAULT_COMPONENT_PROPERTY = "SparqlQuadsTestSuite";
-    
     private static final Logger _logger = Logger.getLogger ( BigdataFederationSparqlTest.class ) ;
 
     private static JiniFederation<Object> _fed = null ;
-    private static Properties _properties = null ;
 
     private ScaleOutTripleStore _ts = null ;
+    private BigdataSail _sail = null;
+    
 }

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -87,6 +87,15 @@
           "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/distinct/manifest#no-distinct-9",
           "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/distinct/manifest#distinct-9",
     });
+
+	/**
+	 * The following tests require Unicode configuration for identical
+	 * comparisons. This appears to work with {ASCII,IDENTICAL} or
+	 * {JDK,IDENTICAL} but not with {ICU,IDENTICAL} for some reason.
+	 */
+    static final Collection<String> unicodeStrengthIdentical = Arrays.asList(new String[] {
+    		"http://www.w3.org/2001/sw/DataAccess/tests/data-r2/i18n/manifest#normalization-1"
+    });
     
 //    private static String datasetTests = "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/dataset";
 
@@ -176,9 +185,9 @@
         "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/optional-filter/manifest#dawg-optional-filter-004",
 */        
 //      Dataset crap
-        // "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/i18n/manifest#normalization-1"
+//         "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/i18n/manifest#normalization-1",
             
-        // "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/dataset/manifest#dawg-dataset-01"
+//         "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/dataset/manifest#dawg-dataset-01",
             
 //        "http://www.w3.org/2001/sw/DataAccess/tests/data-r2//manifest#",
 //            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/expr-builtin/manifest#dawg-str-1",
@@ -191,6 +200,13 @@
 //            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/expr-equals/manifest#eq-graph-1",
 //            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/expr-equals/manifest#eq-graph-2",
             
+    		/*
+    		 * Tests which fail with 2 data services.
+    		 */
+//    		"http://www.w3.org/2001/sw/DataAccess/tests/data-r2/basic/manifest#bgp-no-match",//Ok
+//    		"http://www.w3.org/2001/sw/DataAccess/tests/data-r2/basic/manifest#prefix-name-1",//OK
+//    		"http://www.w3.org/2001/sw/DataAccess/tests/data-r2/basic/manifest#spoo-1",//BOOM
+    		
     });
 
     /**
@@ -351,10 +367,10 @@
         // exact size only there for TCK
         props.setProperty(Options.EXACT_SIZE, "true");
         
-        props.setProperty(Options.COLLATOR, CollatorEnum.ASCII.toString());
+//        props.setProperty(Options.COLLATOR, CollatorEnum.ASCII.toString());
         
 //      Force identical unicode comparisons (assuming default COLLATOR setting).
-        props.setProperty(Options.STRENGTH, StrengthEnum.Identical.toString());
+//        props.setProperty(Options.STRENGTH, StrengthEnum.Identical.toString());
         
         // disable read/write transactions
         props.setProperty(Options.ISOLATABLE_INDICES, "false");
@@ -372,9 +388,17 @@
         if (true) {
             final Properties props = getProperties();
             
-            if (cannotInlineTests.contains(testURI))
+            if (cannotInlineTests.contains(testURI)){
+            	// The test can not be run using XSD inlining.
                 props.setProperty(Options.INLINE_LITERALS, "false");
+            }
             
+            if(unicodeStrengthIdentical.contains(testURI)) {
+            	// Force identical Unicode comparisons.
+            	props.setProperty(Options.COLLATOR, CollatorEnum.JDK.toString());
+            	props.setProperty(Options.STRENGTH, StrengthEnum.Identical.toString());
+            }
+            
             final BigdataSail sail = new BigdataSail(props);
             return new DatasetRepository(new BigdataSailRepository(sail));
         } else {

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java	2010-10-13 13:27:48 UTC (rev 3781)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java	2010-10-13 13:46:30 UTC (rev 3782)
@@ -62,6 +62,7 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import com.bigdata.rdf.sail.BigdataSailQuery;
+import com.bigdata.rdf.sail.BigdataSailRepository;
 
 public abstract class SPARQLQueryTest extends TestCase {
 
@@ -164,11 +165,23 @@
 		}
 	}
 
+	/**
+	 * Return the connection which will be used to query the repository.
+	 * @param dataRep The repository.
+	 * @return The connection.
+	 * @throws Exception
+	 */
+    protected RepositoryConnection getQueryConnection(Repository dataRep)
+    	throws Exception
+	{
+    	return dataRep.getConnection();
+    }
+	
 	@Override
 	protected void runTest()
 		throws Exception
 	{
-		RepositoryConnection con = dataRep.getConnection();
+		RepositoryConnection con = getQueryConnection(dataRep);
 		try {
 			String queryString = readQueryString();
 			Query query = con.prepareQuery(QueryLanguage.SPARQL, queryString, queryFileURL);


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3781] branches/bbb_cleanup

[<res...@us...>]

Revision: 3781
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3781&view=rev
Author:   resendes
Date:     2010-10-13 13:27:48 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Merge "up" from maven_scaleout branch

Modified Paths:
--------------
    branches/bbb_cleanup/bigdata-core/bigdata-perf/bsbm/src/resources/logging/log4j.properties
    branches/bbb_cleanup/bigdata-core/bigdata-perf/btc/src/resources/logging/log4j.properties
    branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources/logging/log4j.properties
    branches/bbb_cleanup/bigdata-core/bigdata-perf/uniprot/src/resources/logging/log4j.properties
    branches/bbb_cleanup/bigdata-core/pom.xml
    branches/bbb_cleanup/bigdata-core/src/main/deploy/legacy/bigdata-resources/logging/log4j-perf-tests.properties
    branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/log4j.properties
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/bfs/BigdataFileSystem.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTree.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTreeTupleCursor.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/BTree.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/IndexMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/IndexSegment.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/IndexSegmentBuilder.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/IndexSegmentStore.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/KeyAfterPartitionException.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/KeyBeforePartitionException.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/Node.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/ReadOnlyIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/ViewStatistics.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/proc/RangeCountProcedure.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/view/FusedView.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/cache/HardReferenceGlobalLRU.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/cache/HardReferenceGlobalLRURecycler.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/cache/HardReferenceGlobalLRURecyclerExplicitDeleteRequired.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/cache/LRUNexus.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/concurrent/NonBlockingLockManagerWithNewDesign.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/counters/httpd/DummyEventReportingService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/AbstractJournal.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/AbstractTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/CommitRecordIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/ConcurrencyManager.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/DumpJournal.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/Name2Addr.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/RegisterIndexTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/journal/WriteExecutorService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/loadbalancer/EmbeddedLoadBalancer.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/AbstractResourceMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/IPartitionMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/IndexPartitionCause.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/PartitionLocator.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/SegmentMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/metadata/EmbeddedIndexStore.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/metadata/EmbeddedShardLocator.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/rules/BackchainAccessPath.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/relation/accesspath/AbstractAccessPath.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/relation/rule/eval/IJoinNexus.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/AsynchronousOverflowTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/BTreeMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/BuildResult.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/CompactingMergeTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/IncrementalBuildTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/IndexManager.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/IndexSegmentIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/JoinIndexPartitionTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/JournalIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/MoveTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/OverflowManager.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/PurgeResult.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/ResourceEvents.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/ResourceManager.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/ScatterSplitTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/SplitIndexPartitionTask.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/SplitResult.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/SplitUtility.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/StoreManager.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/resources/ViewMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/AbstractScaleOutClient.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/AbstractScaleOutFederation.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/AbstractTransactionService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/CacheOnceMetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/CachingMetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/CommitTimeIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/DistributedTransactionService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/EventReceiver.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/EventResource.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/IBigdataFederation.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/IDataService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/IMetadataService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/IndexCache.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/LoadBalancerService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/MetadataIndexCache.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/MetadataService.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/NoCacheMetadataIndexView.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ShardLocator.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ShardManagement.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/jini/util/DumpFederation.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/AbstractScaleOutClientIndexView.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/AbstractScaleOutClientIndexView2.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/AbstractSplitter.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/ClientIndexView.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/ClientIndexViewRefactor.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/IScaleOutClientIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/ndx/ISplitter.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/transaction/EmbeddedTransactionService.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/boot/BootComponentTest.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/btree/TestIndexPartitionFencePosts.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/btree/TestRemoveAll.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/cache/TestHardReferenceQueue.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/concurrent/AbstractStressTestNonBlockingLockManager.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/concurrent/TestAll.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/counters/TestAll.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/journal/TestCommitRecordIndex.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/rdf/rio/TestLoadAndVerify.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestTruthMaintenance.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/rdf/store/AbstractServerTestCase.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/AbstractResourceManagerTestCase.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/TestBuildTask.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/TestBuildTask2.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/TestMergeTask.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/TestOverflow.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/TestResourceManagerBootstrap.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/resources/TestSegSplitter.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/AbstractEmbeddedFederationTestCase.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/StressTestConcurrent.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestDistributedTransactionServiceRestart.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestEmbeddedClient.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestEventReceiver.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestMetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestMove.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestOverflow.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestScatterSplit.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestSnapshotHelper.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/TestSplitJoin.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/ndx/TestSplitter.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/ndx/pipeline/AbstractKeyRangeMasterTestCase.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/service/ndx/pipeline/TestMasterTaskWithSplits.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/util/TestEntryUtil.java
    branches/bbb_cleanup/bigdata-integ/pom.xml
    branches/bbb_cleanup/bigdata-integ/src/test/java/com/bigdata/service/jini/AbstractServerTestCase.java
    branches/bbb_cleanup/pom.xml

Added Paths:
-----------
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/IMetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/JournalMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/LocalPartitionMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/MetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/MetadataIndexView.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/event/
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/event/IEventParams.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/EventBTree.java
    branches/bbb_cleanup/bigdata-core/src/test/deploy/testing/conf/log4j.properties
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalParameterized.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalRunner.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/FalseCondition.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/IgnoreIf.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIf.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnIgnoredClass.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnNotIgnoredClass.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TrueCondition.java

Removed Paths:
-------------
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/DelegateIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/concurrent/NonBlockingLockManager.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/counters/query/CounterSetBTreeSelector.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/counters/query/CounterSetLoader.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/counters/store/CounterSetBTree.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/event/IEventParams.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/IMetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/JournalMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/LocalPartitionMetadata.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/MetadataIndex.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/mdi/MetadataIndexView.java
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/service/Params.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/concurrent/TestNonBlockingLockManager.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/counters/store/TestAll.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/counters/store/TestCounterSetBTree.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalParameterized.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalRunner.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/FalseCondition.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/IgnoreIf.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIf.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnIgnoredClass.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnNotIgnoredClass.java
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/test/conditional/TrueCondition.java

Property Changed:
----------------
    branches/bbb_cleanup/
    branches/bbb_cleanup/bigdata-core/bigdata-perf/
    branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources/
    branches/bbb_cleanup/bigdata-core/dsi-utils/LEGAL/
    branches/bbb_cleanup/bigdata-core/dsi-utils/lib/
    branches/bbb_cleanup/bigdata-core/dsi-utils/src/
    branches/bbb_cleanup/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom/
    branches/bbb_cleanup/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom/
    branches/bbb_cleanup/bigdata-core/osgi/
    branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/boot/
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/attr/
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/disco/
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/sail/bench/
    branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/rdf/util/
    branches/bbb_cleanup/bigdata-core/src/samples-sail/com/bigdata/samples/fastload.properties


Property changes on: branches/bbb_cleanup
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH:2004-2045
/branches/DEV_BRANCH_27_OCT_2009:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH:2633-3304
/branches/bugfix-btm:2594-3237
/branches/dev-btm:2574-3440,3443,3463,3469-3470
/branches/fko:3150-3194
/branches/maven_scaleout:3379-3438
/trunk:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH:2004-2045
/branches/DEV_BRANCH_27_OCT_2009:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH:2633-3304
/branches/bugfix-btm:2594-3237
/branches/dev-btm:2574-3440,3443,3463,3469-3470
/branches/fko:3150-3194
/branches/maven_scaleout:3379-3438,3709-3780
/trunk:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542


Property changes on: branches/bbb_cleanup/bigdata-core/bigdata-perf
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/bigdata-perf:3588-3690
/trunk/bigdata-perf:3379-3541
   + /branches/bbb_cleanup/bigdata-core/bigdata-perf:3588-3690
/branches/maven_scaleout/bigdata-core/bigdata-perf:3709-3780
/trunk/bigdata-perf:3379-3541

Modified: branches/bbb_cleanup/bigdata-core/bigdata-perf/bsbm/src/resources/logging/log4j.properties
===================================================================
--- branches/bbb_cleanup/bigdata-core/bigdata-perf/bsbm/src/resources/logging/log4j.properties	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/bigdata-perf/bsbm/src/resources/logging/log4j.properties	2010-10-13 13:27:48 UTC (rev 3781)
@@ -68,7 +68,6 @@
 #log4j.logger.com.bigdata.service.AbstractTransactionService=INFO
 #log4j.logger.com.bigdata.journal.AbstractLocalTransactionManager=INFO
 #log4j.logger.com.bigdata.concurrent.TxDag=WARN
-#log4j.logger.com.bigdata.concurrent.NonBlockingLockManager=WARN
 #log4j.logger.com.bigdata.concurrent.TestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.AbstractStressTestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.LockManager=INFO

Modified: branches/bbb_cleanup/bigdata-core/bigdata-perf/btc/src/resources/logging/log4j.properties
===================================================================
--- branches/bbb_cleanup/bigdata-core/bigdata-perf/btc/src/resources/logging/log4j.properties	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/bigdata-perf/btc/src/resources/logging/log4j.properties	2010-10-13 13:27:48 UTC (rev 3781)
@@ -67,7 +67,6 @@
 #log4j.logger.com.bigdata.service.AbstractTransactionService=INFO
 #log4j.logger.com.bigdata.journal.AbstractLocalTransactionManager=INFO
 #log4j.logger.com.bigdata.concurrent.TxDag=WARN
-#log4j.logger.com.bigdata.concurrent.NonBlockingLockManager=WARN
 #log4j.logger.com.bigdata.concurrent.TestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.AbstractStressTestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.LockManager=INFO


Property changes on: branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources:3588-3690
/branches/dev-btm/bigdata-perf/lubm/src/resources:2574-3440
/trunk/bigdata-perf/lubm/src/resources:3379-3541
   + /branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources:3588-3690
/branches/dev-btm/bigdata-perf/lubm/src/resources:2574-3440
/branches/maven_scaleout/bigdata-core/bigdata-perf/lubm/src/resources:3709-3780
/trunk/bigdata-perf/lubm/src/resources:3379-3541

Modified: branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources/logging/log4j.properties
===================================================================
--- branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources/logging/log4j.properties	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/bigdata-perf/lubm/src/resources/logging/log4j.properties	2010-10-13 13:27:48 UTC (rev 3781)
@@ -57,7 +57,6 @@
 #log4j.logger.com.bigdata.service.AbstractTransactionService=INFO
 #log4j.logger.com.bigdata.journal.AbstractLocalTransactionManager=INFO
 log4j.logger.com.bigdata.concurrent.TxDag=WARN
-log4j.logger.com.bigdata.concurrent.NonBlockingLockManager=WARN
 log4j.logger.com.bigdata.concurrent.TestNonBlockingLockManager=INFO
 log4j.logger.com.bigdata.concurrent.AbstractStressTestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.LockManager=INFO

Modified: branches/bbb_cleanup/bigdata-core/bigdata-perf/uniprot/src/resources/logging/log4j.properties
===================================================================
--- branches/bbb_cleanup/bigdata-core/bigdata-perf/uniprot/src/resources/logging/log4j.properties	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/bigdata-perf/uniprot/src/resources/logging/log4j.properties	2010-10-13 13:27:48 UTC (rev 3781)
@@ -67,7 +67,6 @@
 #log4j.logger.com.bigdata.service.AbstractTransactionService=INFO
 #log4j.logger.com.bigdata.journal.AbstractLocalTransactionManager=INFO
 #log4j.logger.com.bigdata.concurrent.TxDag=WARN
-#log4j.logger.com.bigdata.concurrent.NonBlockingLockManager=WARN
 #log4j.logger.com.bigdata.concurrent.TestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.AbstractStressTestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.LockManager=INFO


Property changes on: branches/bbb_cleanup/bigdata-core/dsi-utils/LEGAL
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/dsi-utils/LEGAL:3588-3690
/trunk/dsi-utils/LEGAL:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/dsi-utils/LEGAL:3588-3690
/branches/maven_scaleout/bigdata-core/dsi-utils/LEGAL:3709-3780
/trunk/dsi-utils/LEGAL:3379-3430,3499


Property changes on: branches/bbb_cleanup/bigdata-core/dsi-utils/lib
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/dsi-utils/lib:3588-3690
/trunk/dsi-utils/lib:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/dsi-utils/lib:3588-3690
/branches/maven_scaleout/bigdata-core/dsi-utils/lib:3709-3780
/trunk/dsi-utils/lib:3379-3430,3499


Property changes on: branches/bbb_cleanup/bigdata-core/dsi-utils/src
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/dsi-utils/src:3588-3690
/trunk/dsi-utils/src:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/dsi-utils/src:3588-3690
/branches/maven_scaleout/bigdata-core/dsi-utils/src:3709-3780
/trunk/dsi-utils/src:3379-3430,3499


Property changes on: branches/bbb_cleanup/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3588-3690
/trunk/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3588-3690
/branches/maven_scaleout/bigdata-core/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3709-3780
/trunk/lgpl-utils/src/java/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499


Property changes on: branches/bbb_cleanup/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3588-3690
/trunk/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3588-3690
/branches/maven_scaleout/bigdata-core/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3709-3780
/trunk/lgpl-utils/src/test/it/unimi/dsi/fastutil/bytes/custom:3379-3430,3499


Property changes on: branches/bbb_cleanup/bigdata-core/osgi
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/osgi:3588-3690
/trunk/osgi:3379-3430,3499
   + /branches/bbb_cleanup/bigdata-core/osgi:3588-3690
/branches/maven_scaleout/bigdata-core/osgi:3709-3780
/trunk/osgi:3379-3430,3499

Modified: branches/bbb_cleanup/bigdata-core/pom.xml
===================================================================
--- branches/bbb_cleanup/bigdata-core/pom.xml	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/pom.xml	2010-10-13 13:27:48 UTC (rev 3781)
@@ -14,12 +14,41 @@
 
     <name>Bigdata Core</name>
     <url>http://www.bigdata.com</url>
+    <description>The Bigdata Core</description>
 
     <properties>
-        <thirdParty.groupId>com.bigdata.thirdparty</thirdParty.groupId><!-- group ID for non-public bigdata dependencies. -->
+        <!-- group ID for non-public bigdata dependencies. -->
+        <thirdParty.groupId>com.bigdata.thirdparty</thirdParty.groupId>
+
+        <!--
+            These properties are used to set system properties that are required by the unit tests. See the surefire plugin configuration below. Any of these can be overridden on the command-line by
+            adding -D<property-name>=<propert-value>.
+        -->
+
+        <!--
+            This directory is created when the deployment tarball is created during the 'package' phase by the maven assembly plugin. If the naming convention as defined in
+            src/main/assembly/deploy.xml changes, this property needs to change as well.
+            
+            TODO: May want to see if these can be promoted to the parent POM and shared with the bigdata-integ module
+        -->
+        <deploy.root.dir>${project.build.directory}/${project.artifactId}-${project.version}-deploy</deploy.root.dir>
+
+        <deploy.dir>${deploy.root.dir}/${project.artifactId}-${project.version}</deploy.dir>
+        <app.home>${deploy.dir}</app.home>
+        <test.dir>${deploy.dir}/testing</test.dir>
+        <test.conf.dir>${test.dir}/conf</test.conf.dir>
+        <deploy.lib>${deploy.dir}/lib</deploy.lib>
+        <test.codebase.dir>${deploy.lib.dl}</test.codebase.dir>
+        <test.codebase.port>23333</test.codebase.port>
+        <java.security.policy>${test.conf.dir}/policy.all</java.security.policy>
+        <log4j.configuration>${test.conf.dir}/log4j.properties</log4j.configuration>
+        <java.net.preferIPv4Stack>true</java.net.preferIPv4Stack>
+        <default.nic>eth0</default.nic>
+
+        <!--  Set to empty string to indicate "unset." Application code will set a reasonable default -->
+        <federation.name></federation.name>
     </properties>
 
-
     <build>
         <plugins>
             <plugin>
@@ -31,6 +60,8 @@
                             java files were getting recompiled and put into the bigdata jar. This setting forces javac to only look for source in the current maven source directory.
                         -->
                         <sourcepath>${project.build.sourceDirectory}</sourcepath>
+                        <!-- <Xlint></Xlint> -->
+                        <!-- <Xlint:unchecked></Xlint:unchecked> -->
                     </compilerArguments>
                 </configuration>
             </plugin>
@@ -82,48 +113,71 @@
                 </executions>
             </plugin>
 
+            <!--
+                Configure the surefire plugin to run in the 'integration-test' phase rather than the 'test' phase. This allows us to run the unit tests after packaging has been done, but before the
+                build artifacts are installed during the 'install' phase
+            -->
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-surefire-plugin</artifactId>
                 <configuration>
-                    <!--
-                        TODO: BLECH! All the tests are excluded from the regular unit test phase. TODO: Need to split out the unit tests and the heavier integration tests, plus TODO: get all the unit
-                        tests passing so regressions can properly fail the build.
-                    -->
-                    <testFailureIgnore>true</testFailureIgnore>
-                    <includes />
-                    <excludes>
-                        <exclude>**/*</exclude>
-                    </excludes>
+                    <skip>true</skip>
                 </configuration>
-            </plugin>
-
-            <plugin>
-                <!--
-                    These are where the heavier tests can be run. Right now failsafe looks for tests starting or ending with IT, aka FooIT.java or ITFoo.java, which don't exist yet, so nothing runs.
-                -->
-                <groupId>org.apache.maven.plugins</groupId>
-                <artifactId>maven-failsafe-plugin</artifactId>
                 <executions>
                     <execution>
-                        <id>integration-test</id>
+                        <id>unit-tests</id>
+                        <phase>integration-test</phase>
                         <goals>
-                            <goal>integration-test</goal>
+                            <goal>test</goal>
                         </goals>
+                        <configuration>
+                            <testFailureIgnore>true</testFailureIgnore>
+                            <skip>false</skip>
+                            <!-- Run only the top-level suite which in turn runs the other suites and tests -->
+                            <includes>
+                                <include>com/bigdata/TestAll.java</include>
+                            </includes>
+                            
+                            <forkMode>pertest</forkMode>
+
+                            <!-- These system properties are required by the unit tests. -->
+                            <systemPropertyVariables>
+                                <java.security.policy>${java.security.policy}</java.security.policy>
+                                <java.net.preferIPv4Stack>${java.net.preferIPv4Stack}</java.net.preferIPv4Stack>
+                                <log4j.configuration>${log4j.configuration}</log4j.configuration>
+
+                                <app.home>${app.home}</app.home> <!-- This is the deployment directory, easily accessed by the DataFinder class. -->
+                                <log4j.path>${log4j.configuration}</log4j.path>
+                                <default.nic>${default.nic}</default.nic>
+                                <federation.name>${federation.name}</federation.name>
+
+                                <classserver.jar>${deploy.lib}/classserver.jar</classserver.jar>
+                                <colt.jar>${deploy.lib}/colt.jar</colt.jar>
+                                <ctc_utils.jar>${deploy.lib}/ctc_utils.jar</ctc_utils.jar>
+                                <cweb-commons.jar>${deploy.lib}/cweb-commons.jar</cweb-commons.jar>
+                                <cweb-extser.jar>${deploy.lib}/cweb-extser.jar</cweb-extser.jar>
+                                <highscalelib.jar>${deploy.lib}/highscalelib.jar</highscalelib.jar>
+                                <dsiutils.jar>${deploy.lib}/dsiutils.jar</dsiutils.jar>
+                                <lgplutils.jar>${deploy.lib}/lgplutils.jar</lgplutils.jar>
+                                <fastutil.jar>${deploy.lib}/fastutil.jar</fastutil.jar>
+                                <icu4j.jar>${deploy.lib}/icu4j.jar</icu4j.jar>
+                                <jsk-lib.jar>${deploy.lib}/jsk-lib.jar</jsk-lib.jar>
+                                <jsk-platform.jar>${deploy.lib}/jsk-platform.jar</jsk-platform.jar>
+                                <log4j.jar>${deploy.lib}/log4j.jar</log4j.jar>
+                                <iris.jar>${deploy.lib}/iris.jar</iris.jar>
+                                <jgrapht.jar>${deploy.lib}/jgrapht.jar</jgrapht.jar>
+                                <openrdf-sesame.jar>${deploy.lib}/openrdf-sesame.jar</openrdf-sesame.jar>
+                                <slf4j.jar>${deploy.lib}/slf4j.jar</slf4j.jar>
+                                <nxparser.jar>${deploy.lib}/nxparser.jar</nxparser.jar>
+                                <zookeeper.jar>${deploy.lib}/zookeeper.jar</zookeeper.jar>
+                            </systemPropertyVariables>
+                        </configuration>
                     </execution>
-                    <execution>
-                        <id>verify</id>
-                        <goals>
-                            <goal>verify</goal>
-                        </goals>
-                    </execution>
                 </executions>
             </plugin>
 
             <plugin>
-                <!-- Workaround assembly issue for nicely organized libs in the deployment tarball.  For more info
-                     see src/main/assemblies/deploy.xml
-                -->
+                <!-- Workaround assembly issue for nicely organized libs in the deployment tarball. For more info see src/main/assemblies/deploy.xml -->
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-dependency-plugin</artifactId>
                 <executions>
@@ -189,13 +243,9 @@
                     </execution>
                 </executions>
             </plugin>
-
-
         </plugins>
-
     </build>
 
-
     <dependencies>
 
 
@@ -204,7 +254,7 @@
         <!-- ************************ Start of non-public dependencies ************************ -->
         <!-- ************************ Start of non-public dependencies ************************ -->
 
-        <!-- TODO: look at maven-bundle-plugin from felix to provide osgi support.  bndlib version 0.0.357 in central. -->
+        <!-- TODO: look at maven-bundle-plugin from felix to provide osgi support. bndlib version 0.0.357 in central. -->
 
         <dependency>
             <groupId>${thirdParty.groupId}</groupId>
@@ -259,7 +309,7 @@
             <version>[3.2.1,3.3)</version>  <!-- TODO: This artifact is older, but the new 3.3.1 artifact in central is incompatible -->
         </dependency>
 
-        <!--  Apache River -->
+        <!-- Apache River -->
         <dependency>
             <groupId>org.apache.river</groupId>
             <artifactId>tools</artifactId>
@@ -551,7 +601,13 @@
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-surefire-report-plugin</artifactId>
-                <version>2.5</version>
+                <reportSets>
+                    <reportSet>
+                        <reports>
+                            <report>report-only</report>
+                        </reports>
+                    </reportSet>
+                </reportSets>
             </plugin>
 
             <plugin>
@@ -584,9 +640,7 @@
                         <groupId>com.atlassian.maven.plugins</groupId>
                         <artifactId>maven-clover2-plugin</artifactId>
                         <configuration>
-                            <!--
-                                Details on configuring clover are in the parent pom.
-                            -->
+                            <!-- Details on configuring clover are in the parent pom. -->
                             <excludes><!-- These are stuffed into the lookup starter jar, which uses a stripped classpath without clover. -->
                                 <exclude>**/LookupStarter*.java</exclude>
                                 <exclude>**/LogUtil.java</exclude>

Modified: branches/bbb_cleanup/bigdata-core/src/main/deploy/legacy/bigdata-resources/logging/log4j-perf-tests.properties
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/main/deploy/legacy/bigdata-resources/logging/log4j-perf-tests.properties	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/src/main/deploy/legacy/bigdata-resources/logging/log4j-perf-tests.properties	2010-10-13 13:27:48 UTC (rev 3781)
@@ -52,7 +52,6 @@
 #log4j.logger.com.bigdata.service.AbstractTransactionService=INFO
 #log4j.logger.com.bigdata.journal.AbstractLocalTransactionManager=INFO
 log4j.logger.com.bigdata.concurrent.TxDag=WARN
-log4j.logger.com.bigdata.concurrent.NonBlockingLockManager=WARN
 log4j.logger.com.bigdata.concurrent.TestNonBlockingLockManager=INFO
 log4j.logger.com.bigdata.concurrent.AbstractStressTestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.LockManager=INFO


Property changes on: branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/boot
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/boot:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/boot:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/boot/config:3469-3470
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/boot:3150-3194
/branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3438
/trunk/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542
   + /branches/BTREE_BUFFER_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2004-2045
/branches/DEV_BRANCH_27_OCT_2009/bigdata-core/src/main/deploy/var/config/jini/boot:2270-2546,2548-2782
/branches/LEXICON_REFACTOR_BRANCH/bigdata-core/src/main/deploy/var/config/jini/boot:2633-3304
/branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/jini/boot:3588-3690
/branches/bugfix-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2594-3237
/branches/dev-btm/bigdata-core/src/main/deploy/var/config/jini/boot:2574-3440,3443,3463,3469-3470
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/boot/config:3469-3470
/branches/fko/bigdata-core/src/main/deploy/var/config/jini/boot:3150-3194
/branches/maven_scaleout/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3438,3709-3780
/trunk/bigdata-core/src/main/deploy/var/config/jini/boot:3379-3430,3432-3460,3476-3499,3503,3507,3516-3528,3542

Modified: branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/log4j.properties
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/log4j.properties	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/src/main/deploy/var/config/logging/log4j.properties	2010-10-13 13:27:48 UTC (rev 3781)
@@ -56,7 +56,6 @@
 #log4j.logger.com.bigdata.service.AbstractTransactionService=INFO
 #log4j.logger.com.bigdata.journal.AbstractLocalTransactionManager=INFO
 log4j.logger.com.bigdata.concurrent.TxDag=WARN
-log4j.logger.com.bigdata.concurrent.NonBlockingLockManager=WARN
 log4j.logger.com.bigdata.concurrent.TestNonBlockingLockManager=INFO
 log4j.logger.com.bigdata.concurrent.AbstractStressTestNonBlockingLockManager=INFO
 #log4j.logger.com.bigdata.concurrent.LockManager=INFO


Property changes on: branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/attr
___________________________________________________________________
Modified: svn:mergeinfo
   - /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/attr:3588-3690
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/attr:3463,3469-3470
/trunk/bigdata-jini/src/java/com/bigdata/attr:3379-3430
   + /branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/attr:3588-3690
/branches/dev-btm/bigdata-jini/src/java/com/bigdata/attr:3463,3469-3470
/branches/maven_scaleout/bigdata-core/src/main/java/com/bigdata/attr:3709-3780
/trunk/bigdata-jini/src/java/com/bigdata/attr:3379-3430

Modified: branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/bfs/BigdataFileSystem.java
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/bfs/BigdataFileSystem.java	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/bfs/BigdataFileSystem.java	2010-10-13 13:27:48 UTC (rev 3781)
@@ -34,7 +34,7 @@
 import com.bigdata.journal.IResourceLock;
 import com.bigdata.journal.ITx;
 import com.bigdata.journal.Journal;
-import com.bigdata.mdi.MetadataIndex;
+import com.bigdata.btree.MetadataIndex;
 import com.bigdata.rawstore.Bytes;
 import com.bigdata.rawstore.IBlock;
 import com.bigdata.rawstore.WormAddressManager;

Modified: branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTree.java
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTree.java	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTree.java	2010-10-13 13:27:48 UTC (rev 3781)
@@ -81,7 +81,6 @@
 import com.bigdata.journal.IIndexManager;
 import com.bigdata.journal.Journal;
 import com.bigdata.mdi.IResourceMetadata;
-import com.bigdata.mdi.LocalPartitionMetadata;
 import com.bigdata.rawstore.IRawStore;
 import com.bigdata.resources.IndexManager;
 import com.bigdata.resources.OverflowManager;
@@ -129,7 +128,6 @@
  * </p>
  * 
  * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- * @version $Id$
  * 
  * @see KeyBuilder
  */
@@ -165,7 +163,7 @@
     /**
      * Log for btree opeations.
      */
-    protected static final Logger log = Logger.getLogger(AbstractBTree.class);
+    private static final Logger log = Logger.getLogger(AbstractBTree.class);
 
     /**
      * True iff the {@link #log} level is INFO or less.
@@ -339,7 +337,6 @@
      * 
      * @author <a href="mailto:tho...@us...">Bryan
      *         Thompson</a>
-     * @version $Id$
      */
     static class ChildMemoizer extends
             Memoizer<LoadChildRequest/* request */, AbstractNode<?>/* child */> {
@@ -1363,7 +1360,6 @@
      * Static class since must be {@link Serializable}.
      * 
      * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
-     * @version $Id$
      */
     static final class TransientResourceMetadata implements IResourceMetadata {
 
@@ -2589,6 +2585,7 @@
 
     }
 
+    //fko ===== needs generic type =====
     final public ITupleIterator rangeIterator() {
 
         return rangeIterator(null, null);
@@ -2603,6 +2600,7 @@
      * @param toKey
      * @return
      */
+    //fko ===== needs generic type =====
     final public ITupleIterator rangeIterator(Object fromKey, Object toKey) {
 
         fromKey = fromKey == null ? null : metadata.getTupleSerializer()
@@ -2615,6 +2613,7 @@
         
     }
     
+    //fko ===== needs generic type =====
     final public ITupleIterator rangeIterator(byte[] fromKey, byte[] toKey) {
 
         return rangeIterator(fromKey, toKey, 0/* capacity */,
@@ -2666,6 +2665,7 @@
      * @param toKey
      * @return
      */
+    //fko ===== needs generic type =====
     final public ITupleIterator rangeIterator(Object fromKey, Object toKey,
             final int capacity,//
             final int flags,//
@@ -2706,208 +2706,13 @@
      * @todo add support to the iterator construct for filtering by a tuple
      *       revision timestamp range.
      */
-    public ITupleIterator rangeIterator(//
-            final byte[] fromKey,//
-            final byte[] toKey,//
-            final int capacityIsIgnored,//
-            final int flags,//
-            final IFilterConstructor filter//
-            ) {
+    //fko ===== needs generic type =====
+    abstract public ITupleIterator rangeIterator(final byte[] fromKey,
+                                                 final byte[] toKey,
+                                                 final int capacityIsIgnored,
+                                                 final int flags,
+                                                 final IFilterConstructor filter);
 
-//        btreeCounters.nrangeIterator.incrementAndGet();
-
-        /*
-         * Does the iterator declare that it will not write back on the index?
-         */
-        final boolean readOnly = ((flags & IRangeQuery.READONLY) != 0);
-
-        if (readOnly && ((flags & IRangeQuery.REMOVEALL) != 0)) {
-
-            throw new IllegalArgumentException();
-
-        }
-
-        /*
-         * Note: this does not work out since it is not so easy to determine when
-         * the iterator is a point test as toKey is the exclusive upper bound.
-         */
-//        * Note: this method will automatically apply the optional bloom filter to
-//        * reject range iterator requests that correspond to a point test. However
-//        * this can only be done when the fromKey and toKey are both non-null and
-//        * equals and further when the iterator was not requested with any options
-//        * that would permit concurrent modification of the index.
-//        if (isBloomFilter()
-//                && fromKey != null
-//                && toKey != null
-//                && (readOnly || (((flags & REMOVEALL) == 0) && ((flags & CURSOR) == 0)))
-//                && BytesUtil.bytesEqual(fromKey, toKey)) {
-//
-//            /*
-//             * Do a fast rejection test using the bloom filter.
-//             */
-//            if(!getBloomFilter().contains(fromKey)) {
-//                
-//                /*
-//                 * The key is known to not be in the index so return an empty
-//                 * iterator.
-//                 */
-//                return EmptyTupleIterator.INSTANCE;
-//                
-//            }
-//            
-//            /*
-//             * Since the bloom filter accepts the key we fall through into the
-//             * normal iterator logic. Using this code path is still possible
-//             * that the filter gave us a false positive and that the key is not
-//             * (in fact) in the index. Either way, the logic below will sort
-//             * things out.
-//             */
-//            
-//        }
-        
-        /*
-         * Figure out what base iterator implementation to use.  We will layer
-         * on the optional filter(s) below. 
-         */
-        ITupleIterator src;
-
-        if ((this instanceof BTree) && ((flags & REVERSE) == 0)
-                && ((flags & REMOVEALL) == 0) && ((flags & CURSOR) == 0)) {
-
-            /*
-             * Use the recursion-based striterator since it is faster for a
-             * BTree (but not for an IndexSegment).
-             * 
-             * Note: The recursion-based striterator does not support remove()!
-             * 
-             * @todo we could pass in the Tuple here to make the APIs a bit more
-             * consistent across the recursion-based and the cursor based
-             * iterators.
-             * 
-             * @todo when the capacity is one and REVERSE is specified then we
-             * can optimize this using a reverse traversal striterator - this
-             * will have lower overhead than the cursor for the BTree (but not
-             * for an IndexSegment).
-             */
-
-//            src = fastForwardIterator(fromKey, toKey, capacity, flags);
-
-            src = getRoot().rangeIterator(fromKey, toKey, flags);
-
-        } else {
-
-            final Tuple tuple = new Tuple(this, flags);
-
-            if (this instanceof IndexSegment) {
-                
-                final IndexSegment seg = (IndexSegment) this;
-
-                /*
-                 * @todo we could scan the list of pools and chose the best fit
-                 * pool and then allocate a buffer from that pool. Best fit
-                 * would mean either the byte range fits without "too much" slop
-                 * or the #of reads will have to perform is not too large. We
-                 * might also want to limit the maximum size of the reads.
-                 */
-
-//                final DirectBufferPool pool = DirectBufferPool.INSTANCE_10M;
-                final DirectBufferPool pool = DirectBufferPool.INSTANCE;
-                
-                if (true
-                        && ((flags & REVERSE) == 0)
-                        && ((flags & CURSOR) == 0)
-                        && (seg.getStore().getCheckpoint().maxNodeOrLeafLength <= pool
-                                .getBufferCapacity())
-                        && ((rangeCount(fromKey, toKey) / branchingFactor) > 2)) {
-
-                    src = new IndexSegmentMultiBlockIterator(seg, pool,
-                            fromKey, toKey, flags);
-
-                } else {
-
-                    src = new IndexSegmentTupleCursor(seg, tuple, fromKey,
-                            toKey);
-
-                }
-
-            } else if (this instanceof BTree) {
-
-                if (isReadOnly()) {
-
-                    // Note: this iterator does not allow removal.
-                    src = new ReadOnlyBTreeTupleCursor(((BTree) this), tuple,
-                            fromKey, toKey);
-
-                } else {
-
-                    // Note: this iterator supports traversal with concurrent
-                    // modification.
-                    src = new MutableBTreeTupleCursor(((BTree) this),
-                            new Tuple(this, flags), fromKey, toKey);
-
-                }
-
-            } else {
-
-                throw new UnsupportedOperationException(
-                        "Unknown B+Tree implementation: "
-                                + this.getClass().getName());
-
-            }
-
-            if ((flags & REVERSE) != 0) {
-
-                /*
-                 * Reverse scan iterator.
-                 * 
-                 * Note: The reverse scan MUST be layered directly over the
-                 * ITupleCursor. Most critically, REMOVEALL combined with a
-                 * REVERSE scan needs to process the tuples in reverse index
-                 * order and then delete them as it goes.
-                 */
-
-                src = new Reverserator((ITupleCursor) src);
-
-            }
-
-        }
-
-        if (filter != null) {
-
-            /*
-             * Apply the optional filter.
-             * 
-             * Note: This needs to be after the reverse scan and before
-             * REMOVEALL (those are the assumptions for the flags).
-             */
-            
-            src = filter.newInstance(src);
-            
-        }
-        
-        if ((flags & REMOVEALL) != 0) {
-            
-            assertNotReadOnly();
-            
-            /*
-             * Note: This iterator removes each tuple that it visits from the
-             * source iterator.
-             */
-            
-            src = new TupleRemover() {
-                @Override
-                protected boolean remove(ITuple e) {
-                    // remove all visited tuples.
-                    return true;
-                }
-            }.filter(src);
-
-        }
-        
-        return src;
-
-    }
-
     /**
      * Copy all data, including deleted index entry markers and timestamps iff
      * supported by the source and target. The goal is an exact copy of the data
@@ -3894,7 +3699,6 @@
      * {@link Reference} (a runtime security manager exception will result).
      * 
      * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
-     * @version $Id$
      * 
      * @param <T>
      */

Modified: branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTreeTupleCursor.java
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTreeTupleCursor.java	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/AbstractBTreeTupleCursor.java	2010-10-13 13:27:48 UTC (rev 3781)
@@ -37,7 +37,6 @@
 import com.bigdata.btree.isolation.IsolatedFusedView;
 import com.bigdata.btree.view.FusedView;
 import com.bigdata.io.DataOutputBuffer;
-import com.bigdata.mdi.LocalPartitionMetadata;
 
 /**
  * Class supporting random access to tuples and sequential tuple-based cursor

Modified: branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/BTree.java
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/BTree.java	2010-10-13 12:37:58 UTC (rev 3780)
+++ branches/bbb_cleanup/bigdata-core/src/main/java/com/bigdata/btree/BTree.java	2010-10-13 13:27:48 UTC (rev 3781)
@@ -32,18 +32,21 @@
 
 import com.bigdata.BigdataStatics;
 import com.bigdata.btree.AbstractBTreeTupleCursor.MutableBTreeTupleCursor;
+import com.bigdata.btree.AbstractBTreeTupleCursor.ReadOnlyBTreeTupleCursor;
 import com.bigdata.btree.Leaf.ILeafListener;
 import com.bigdata.btree.data.ILeafData;
 import com.bigdata.btree.data.INodeData;
+import com.bigdata.btree.filter.IFilterConstructor;
+import com.bigdata.btree.filter.Reverserator;
+import com.bigdata.btree.filter.TupleRemover;
 import com.bigdata.journal.AbstractJournal;
 import com.bigdata.journal.ICommitter;
 import com.bigdata.journal.IIndexManager;
 import com.bigdata.journal.Name2Addr;
 import com.bigdata.journal.Name2Addr.Entry;
 import com.bigdata.mdi.IResourceMetadata;
-import com.bigdata.mdi.JournalMetadata;
-import com.bigdata.mdi.LocalPartitionMetadata;
 import com.bigdata.rawstore.IRawStore;
+import org.apache.log4j.Logger;
 
 /**
  * <p>
@@ -153,10 +156,14 @@
  *       several published papers.
  * 
  * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
- * @version $Id$
  */
 public class BTree extends AbstractBTree implements ICommitter, ILocalBTreeView {
     
+    /**
+     * Log for btree operations.
+     */
+    private static final Logger log = Logger.getLogger(BTree.class);
+
     final public int getHeight() {
         
         return height;
@@ -190,6 +197,7 @@
 
     }
 
+    @Override
     public final IResourceMetadata[] getResourceMetadata() {
         //override to make final so sub-classes cannot modify behavior.
         return super.getResourceMetadata();
@@ -206,23 +214,23 @@
      */
     public ICounter getCounter() {
 
-        ICounter counter = new Counter(this);
+        ICounter tmpCounter = new Counter(this);
         
         final LocalPartitionMetadata pmd = metadata.getPartitionMetadata();
 
         if (pmd != null) {
 
-            counter = new PartitionedCounter(pmd.getPartitionId(), counter);
+            tmpCounter = new PartitionedCounter(pmd.getPartitionId(), tmpCounter);
 
         }
 
         if (isReadOnly()) {
 
-            return new ReadOnlyCounter(counter);
+            return new ReadOnlyCounter(tmpCounter);
 
         }
 
-        return counter;
+        return tmpCounter;
 
     }
     
@@ -395,7 +403,7 @@
      * <p>
      * Note: The {@link #getCounter()} is NOT changed by this method.
      */
-    final private void newRootLeaf() {
+    private void newRootLeaf() {
 
         height = 0;
 
@@ -1341,14 +1349,8 @@
             // view of this BTree.
             newResources[1] = new JournalMetadata((AbstractJournal) getStore(),
                     priorCommitTime);
+            System.arraycopy(oldResources, 1, newResources, 2, oldResources.length - 1);
 
-            // any other stores in the view are copied.
-            for (int i = 1; i < oldResources.length; i++) {
-
-                newResources[i + 1] = oldResources[i];
-
-            }
-
             final LocalPartitionMetadata newPmd = new LocalPartitionMetadata(
                     oldPmd.getPartitionId(), // partitionId
                     -1, // sourcePartitionId
@@ -1569,7 +1571,6 @@
 	 * @throws IllegalArgumentException
 	 *             if store is <code>null</code>.
 	 */
-    @SuppressWarnings("unchecked")
     public static BTree load(final IRawStore store, final long addrCheckpoint,
             final boolean readOnly) {
 
@@ -1655,6 +1656,147 @@
     }
     
     /**
+     * Core implementation.
+     * <p>
+     * Note: If {@link IRangeQuery#CURSOR} is specified the returned iterator
+     * supports traversal with concurrent modification by a single-threaded
+     * process (the {@link BTree} is NOT thread-safe for writers). Write are
+     * permitted iff {@link AbstractBTree} allows writes.
+     * <p>
+     * Note: {@link IRangeQuery#REVERSE} is handled here by wrapping the
+     * underlying {@link ITupleCursor}.
+     * <p>
+     * Note: {@link IRangeQuery#REMOVEALL} is handled here by wrapping the
+     * iterator.
+     * <p>
+     * Note:
+     * {@link FusedView#rangeIterator(byte[], byte[], int, int, IFilterConstructor)}
+     * is also responsible for constructing an {@link ITupleIterator} in a
+     * manner similar to this method. If you are updating the logic here, then
+     * check the logic in that method as well!
+     * 
+     * @todo add support to the iterator construct for filtering by a tuple
+     *       revision timestamp range.
+     */
+    public ITupleIterator rangeIterator(//
+            final byte[] fromKey,//
+            final byte[] toKey,//
+            final int capacityIsIgnored,//
+            final int flags,//
+            final IFilterConstructor filter//
+            ) {
+
+        /*
+         * Does the iterator declare that it will not write back on the index?
+         */
+        final boolean ro = ((flags & IRangeQuery.READONLY) != 0);
+
+        if (ro && ((flags & IRangeQuery.REMOVEALL) != 0)) {
+
+            throw new IllegalArgumentException();
+
+        }
+
+        /*
+         * Figure out what base iterator implementation to use.  We will layer
+         * on the optional filter(s) below. 
+         ...
 
[truncated message content]

[Bigdata-commit] SF.net SVN: bigdata:[3780] branches/bbb_cleanup/bigdata-core/src/test/java/ com/bigdata/util/config/NicUtilTest.java

[<res...@us...>]

Revision: 3780
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3780&view=rev
Author:   resendes
Date:     2010-10-13 12:37:58 +0000 (Wed, 13 Oct 2010)

Log Message:
-----------
Added ipv4LoopbackAddress() method to NicUtilTest. Three tests were failing under Ubuntu wrt loopback addresses.
- NicUtil path was returning InetAddress.getLocalHost() (e.g. 127.0.1.1) 
- The test was traversing  NetworkInterface.getNetworkInterfaces() and finding the associated IPv4 addresses (e.g. 127.0.0.1)
The test was imply doing a string compare, which failed because 127.0.0.1 != 127.0.1.1. 

Notes: 
- All 127.* IPv4 addresses are effectively the loopback address
- Ubuntu /etc/hosts (which apparetnly getLocalHost() interrogates) had the following entries:
127.0.0.1	localhost
127.0.1.1	<hostname-of-this-machine>

Modified Paths:
--------------
    branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/util/config/NicUtilTest.java

Modified: branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/util/config/NicUtilTest.java
===================================================================
--- branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/util/config/NicUtilTest.java	2010-10-12 21:13:14 UTC (rev 3779)
+++ branches/bbb_cleanup/bigdata-core/src/test/java/com/bigdata/util/config/NicUtilTest.java	2010-10-13 12:37:58 UTC (rev 3780)
@@ -995,10 +995,15 @@
 			String ip = 
 				NicUtil.getIpAddress(
 					propName, (String)arg[0], ((Boolean)arg[1]).booleanValue());
-			assertTrue(ipv4Exists(ip, true));		
+			assertTrue(ipv4LoopbackAddress(ip));		
 		}
 	}	
 
+	private boolean ipv4LoopbackAddress(String ip) {
+		//The whole 127.* range is considered loopback
+		return ip.startsWith("127.");
+	}
+
 	@Test
 	public void testGetIpAddressStringStringBoolean_unset_prop_valid_default() 
 		throws SocketException, IOException 
@@ -1035,7 +1040,7 @@
 		String propName = UUID.randomUUID().toString();
 		String invalidDefault = propName;
 		String actual = NicUtil.getIpAddress(propName, invalidDefault, true);
-		assertTrue(ipv4Exists(actual, true));
+		assertTrue(ipv4LoopbackAddress(actual));		
 	}		
 	
 	@Test
@@ -1117,7 +1122,7 @@
 		String propName = null;
 		String invalidDefault = null;
 		String actual = NicUtil.getIpAddress(propName, invalidDefault, true);
-		assertTrue(ipv4Exists(actual, true));
+		assertTrue(ipv4LoopbackAddress(actual));		
 	}		
 
 	@Test


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3779] branches/maven_scaleout/bigdata-core/src/test/ java/com/bigdata/rdf

[<ble...@us...>]

Revision: 3779
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3779&view=rev
Author:   blevine218
Date:     2010-10-12 21:13:14 +0000 (Tue, 12 Oct 2010)

Log Message:
-----------
Added @IgnoreIf to certain long-running testcases to conditionally filter them out

Modified Paths:
--------------
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rio/TestLoadAndVerify.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestTruthMaintenance.java

Modified: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rio/TestLoadAndVerify.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rio/TestLoadAndVerify.java	2010-10-12 20:33:16 UTC (rev 3778)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rio/TestLoadAndVerify.java	2010-10-12 21:13:14 UTC (rev 3779)
@@ -39,6 +39,10 @@
 import com.bigdata.rdf.store.AbstractTestCase;
 import com.bigdata.rdf.store.AbstractTripleStore;
 import com.bigdata.rdf.store.ProxyTestCase;
+import com.bigdata.test.conditional.ConditionalParameterized;
+import com.bigdata.test.conditional.IgnoreIf;
+import com.bigdata.test.conditional.LongTestsExcluded;
+
 import java.util.Collection;
 import org.junit.Test;
 import org.junit.runner.RunWith;
@@ -51,7 +55,7 @@
  * 
  * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
  */
-@RunWith(Parameterized.class)
+@RunWith(ConditionalParameterized.class)
 public class TestLoadAndVerify extends AbstractRIOTestCase {
 
     /**
@@ -167,6 +171,7 @@
      * LUBM U(1)
      */
     @Test
+    @IgnoreIf(LongTestsExcluded.class)
     public void test_loadAndVerify_U1() throws Exception {
         
         final String file = DataFinder.bestPath("testing/data/lehigh/U1");

Modified: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java	2010-10-12 20:33:16 UTC (rev 3778)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestDatabaseAtOnceClosure.java	2010-10-12 21:13:14 UTC (rev 3779)
@@ -40,6 +40,10 @@
 import com.bigdata.relation.rule.eval.ActionEnum;
 import com.bigdata.relation.rule.eval.IJoinNexus;
 import com.bigdata.relation.rule.eval.IJoinNexusFactory;
+import com.bigdata.test.conditional.ConditionalParameterized;
+import com.bigdata.test.conditional.IgnoreIf;
+import com.bigdata.test.conditional.LongTestsExcluded;
+
 import java.util.Collection;
 import org.junit.Test;
 import org.junit.runner.RunWith;
@@ -52,7 +56,7 @@
  * 
  * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
  */
-@RunWith(Parameterized.class)
+@RunWith(ConditionalParameterized.class)
 public class TestDatabaseAtOnceClosure extends AbstractRuleTestCase {
 
     /**
@@ -382,6 +386,7 @@
      */
 
     @Test
+    @IgnoreIf(LongTestsExcluded.class)
     public void test_fixedPoint_LUBM_U1_As_Full_NestedSubquery()
             throws Exception {
 
@@ -410,6 +415,7 @@
     }
 
     @Test
+    @IgnoreIf(LongTestsExcluded.class)
     public void test_fixedPoint_LUBM_U1_As_Full_PipelineJoins()
             throws Exception {
 
@@ -438,6 +444,7 @@
     }
 
     @Test
+    @IgnoreIf(LongTestsExcluded.class)
     public void test_fixedPoint_LUBM_U1_As_Fast_NestedSubquery() throws Exception {
 
         final String[] resources = readFiles(new File( DataFinder.bestPath("testing/data/lehigh/U1" ) ),
@@ -464,6 +471,7 @@
     }
 
     @Test
+    @IgnoreIf(LongTestsExcluded.class)
     public void test_fixedPoint_LUBM_U1_As_Fast_PipelineJoins() throws Exception {
 
         final String[] resources = readFiles(new File( DataFinder.bestPath("testing/data/lehigh/U1" ) ),

Modified: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestTruthMaintenance.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestTruthMaintenance.java	2010-10-12 20:33:16 UTC (rev 3778)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/rdf/rules/TestTruthMaintenance.java	2010-10-12 21:13:14 UTC (rev 3779)
@@ -67,6 +67,10 @@
 import com.bigdata.striterator.ChunkedArrayIterator;
 import com.bigdata.striterator.IChunkedIterator;
 import com.bigdata.striterator.IChunkedOrderedIterator;
+import com.bigdata.test.conditional.ConditionalParameterized;
+import com.bigdata.test.conditional.IgnoreIf;
+import com.bigdata.test.conditional.LongTestsExcluded;
+
 import java.util.Collection;
 import org.junit.Test;
 import org.junit.runner.RunWith;
@@ -83,9 +87,10 @@
  * 
  * @author <a href="mailto:tho...@us...">Bryan Thompson</a>
  */
-@RunWith(Parameterized.class)
+@RunWith(ConditionalParameterized.class)
 public class TestTruthMaintenance extends AbstractInferenceEngineTestCase {
 
+
     /**
      * 
      */
@@ -928,6 +933,7 @@
      * @todo use data files that we can bundle with the distribution.
      */
     @Test
+    @IgnoreIf(LongTestsExcluded.class)
     public void test_stress() {
 
 //        fail("enable test");


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3778] branches/QUADS_QUERY_BRANCH/src/resources/config /bigdataStandalone.config

[<tho...@us...>]

Revision: 3778
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3778&view=rev
Author:   thompsonbry
Date:     2010-10-12 20:33:16 +0000 (Tue, 12 Oct 2010)

Log Message:
-----------
File should have been in the last commit.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config

Modified: branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config
===================================================================
--- branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-12 20:33:07 UTC (rev 3777)
+++ branches/QUADS_QUERY_BRANCH/src/resources/config/bigdataStandalone.config	2010-10-12 20:33:16 UTC (rev 3778)
@@ -303,6 +303,8 @@
 	"-Djava.security.policy="+bigdata.policy,
 	"-Djava.util.logging.config.file="+bigdata.logging,
 	"-Dcom.bigdata.counters.linux.sysstat.path=@SYSSTAT_HOME@",
+	// Turn off periodic performance counter reporting.
+	"-Dcom.bigdata.service.IBigdataClient.reportDelay=0",
 	//bigdata.profilerAgent,
 	};
 
@@ -911,14 +913,14 @@
 
 	/* Maximum #of index partition moves per overflow.
 	 */
-	new NV(DataServer.Options.MAXIMUM_MOVES,"1"),
+	new NV(DataServer.Options.MAXIMUM_MOVES,"0"), // was 1
 
 	/* Option controls how many index partitions may be moved onto
 	 * any given target data service in a single overflow cycle
 	 * and may be used to disable index partition moves (for
 	 * debugging purposes).
 	 */
-	new NV(DataServer.Options.MAXIMUM_MOVES_PER_TARGET,"1"),
+	new NV(DataServer.Options.MAXIMUM_MOVES_PER_TARGET,"0"), // was 1
 
 	/* The minimum CPU activity on a host before it will consider moving an
 	* index partition to shed some load.
@@ -1137,7 +1139,7 @@
      * keeps the performance counter histories in RAM.  While those histories
      * are bounded, it still uses more RAM than it should.
      */
-	"-Xmx1G",
+	"-Xmx2G",
 	/*
 	 * Enable JXM remote management for the data service.
 	 *
@@ -1435,7 +1437,7 @@
 	 * index partitions will become uniformly distributed, which
 	 * will negatively impact index performance.
 	 */
-        new NV(BigdataSail.Options.TERMID_BITS_TO_REVERSE,"2"),
+        new NV(BigdataSail.Options.TERMID_BITS_TO_REVERSE,"0"), // was 2.
 
         /*
          * Turn off statement identifiers (support for statements


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3777] branches/QUADS_QUERY_BRANCH/bigdata

[<tho...@us...>]

Revision: 3777
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3777&view=rev
Author:   thompsonbry
Date:     2010-10-12 20:33:07 +0000 (Tue, 12 Oct 2010)

Log Message:
-----------
Fixed problem in BOpBase which was resulting in misreporting of the predicate flags in scale-out.

Moved set of the INLINE property outside of the caching of the properties in BigdataFederationSparqlTest so that its value is now correctly set for each test.

SparqlQueryTest#createRepository() now throws UnsupportedOperationException and must be explicitly overridden.

SparqlQueryTest#upload() no longer turns on autoCommit so we do not have to allow it for the SPARQL test suites.

Modified AccessPath to use ITx.READ_COMMITTED if the timestamp is UNISOLATED and the flags includes READONLY.

Modified AbstractFederation to not report performance counters to the load balancer when the REPORT_DELAY is set to zero.

Changed TERMID_BITS_TO_REVERSE to ZERO (0) in bigdataStandalone.config.  This is much more performant for small data sets.

Turned MOVES off in bigdataStandalone.config.

Modified Paths:
--------------
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/BOpBase.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java
    branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java
    branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/BOpBase.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/BOpBase.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/bop/BOpBase.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -572,7 +572,7 @@
                 if (e.getValue() != null && e.getValue().getClass().isArray()) {
                     sb.append(e.getKey() + "="
                             + Arrays.toString((Object[]) e.getValue()));
-                } else if (e.getKey() == IPredicate.Annotations.FLAGS) {
+                } else if (e.getKey().equals(IPredicate.Annotations.FLAGS)) {
                     sb.append(e.getKey() + "="
                             + Tuple.flagString((Integer) e.getValue()));
                 } else {

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/relation/accesspath/AccessPath.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -64,6 +64,7 @@
 import com.bigdata.btree.view.FusedView;
 import com.bigdata.io.DirectBufferPool;
 import com.bigdata.journal.IIndexManager;
+import com.bigdata.journal.ITx;
 import com.bigdata.journal.Journal;
 import com.bigdata.journal.NoSuchIndexException;
 import com.bigdata.journal.TimestampUtility;
@@ -328,12 +329,35 @@
             this.indexManager = localIndexManager;
         }
         
-        this.timestamp = relation.getTimestamp();
-        
         this.predicate = predicate;
 
         this.keyOrder = keyOrder;
 
+        final int flags = predicate.getProperty(
+                IPredicate.Annotations.FLAGS,
+                IPredicate.Annotations.DEFAULT_FLAGS);
+        
+        this.flags = flags;
+
+		/*
+		 * Choose the timestamp of the view. If the request is for the
+		 * unisolated index but the predicate was flagged as READONLY then
+		 * automatically choose READ_COMMITTED instead.
+		 */
+		{
+
+			long timestamp = relation.getTimestamp();
+
+			timestamp = (timestamp == ITx.UNISOLATED
+					&& (flags & IRangeQuery.READONLY) != 0 ? ITx.READ_COMMITTED
+					: timestamp);
+			
+			this.timestamp = timestamp;
+			
+		}
+        
+        this.historicalRead = TimestampUtility.isReadOnly(timestamp);
+        
         final int partitionId = predicate.getPartitionId();
         
         final IIndex ndx;
@@ -414,10 +438,6 @@
 
         this.ndx = ndx;
 
-        final int flags = predicate.getProperty(
-                IPredicate.Annotations.FLAGS,
-                IPredicate.Annotations.DEFAULT_FLAGS);
-        
         final int chunkOfChunksCapacity = predicate.getProperty(
                 BufferAnnotations.CHUNK_OF_CHUNKS_CAPACITY,
                 BufferAnnotations.DEFAULT_CHUNK_OF_CHUNKS_CAPACITY);
@@ -430,16 +450,12 @@
                 IPredicate.Annotations.FULLY_BUFFERED_READ_THRESHOLD,
                 IPredicate.Annotations.DEFAULT_FULLY_BUFFERED_READ_THRESHOLD);
 
-        this.flags = flags;
-
         this.chunkOfChunksCapacity = chunkOfChunksCapacity;
 
         this.chunkCapacity = chunkCapacity;
 
         this.fullyBufferedReadThreshold = fullyBufferedReadThreshold;
         
-        this.historicalRead = TimestampUtility.isReadOnly(timestamp);
-
         this.isFullyBoundForKey = predicate.isFullyBound(keyOrder);
 
         {

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/AbstractFederation.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -1219,19 +1219,23 @@
             final long delay = Long.parseLong(p.getProperty(
                     Options.REPORT_DELAY, Options.DEFAULT_REPORT_DELAY));
 
-            if (log.isInfoEnabled())
-                log.info(Options.REPORT_DELAY + "=" + delay);
+			if (log.isInfoEnabled())
+				log.info(Options.REPORT_DELAY + "=" + delay);
 
-            final TimeUnit unit = TimeUnit.MILLISECONDS;
+			if (delay > 0L) {
 
-            final long initialDelay = delay;
+				final TimeUnit unit = TimeUnit.MILLISECONDS;
 
-            addScheduledTask(new ReportTask(AbstractFederation.this),
-                    initialDelay, delay, unit);
+				final long initialDelay = delay;
 
-            if (log.isInfoEnabled())
-                log.info("Started ReportTask.");
+				addScheduledTask(new ReportTask(AbstractFederation.this),
+						initialDelay, delay, unit);
 
+				if (log.isInfoEnabled())
+					log.info("Started ReportTask.");
+
+			}
+
         }
 
         /**
@@ -1346,7 +1350,7 @@
                 /*
                  * Report the performance counters to the load balancer.
                  */
-                
+
                 reportPerformanceCounters();
                 
             } catch (Throwable t) {

Modified: branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata/src/java/com/bigdata/service/IBigdataClient.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -442,12 +442,14 @@
 
         String DEFAULT_COLLECT_QUEUE_STATISTICS = "true";
 
-        /**
-         * The delay between reports of performance counters to the
-         * {@link ILoadBalancerService} in milliseconds ({@value #DEFAULT_REPORT_DELAY}).
-         * 
-         * @see #DEFAULT_REPORT_DELAY
-         */
+		/**
+		 * The delay between reports of performance counters to the
+		 * {@link ILoadBalancerService} in milliseconds (
+		 * {@value #DEFAULT_REPORT_DELAY}). When ZERO (0L), performance counter
+		 * reporting will be disabled.
+		 * 
+		 * @see #DEFAULT_REPORT_DELAY
+		 */
         String REPORT_DELAY = IBigdataClient.class.getName() + ".reportDelay";
 
         /**

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataFederationSparqlTest.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -133,6 +133,13 @@
         return new DatasetRepository ( new BigdataSailRepository ( new BigdataSail ( newTripleStore () ) ) ) ;
     }
 
+    @Override
+    protected Repository createRepository() throws Exception {
+        Repository repo = newRepository();
+        repo.initialize();
+        return repo;
+    }
+
     private ScaleOutTripleStore newTripleStore ()
         throws Exception
     {
@@ -256,16 +263,13 @@
             _properties.put ( BigdataSail.Options.TRUTH_MAINTENANCE, "false" );
             _properties.put ( BigdataSail.Options.QUERY_TIME_EXPANDER, "false" );
             
-            if (BigdataSparqlTest.cannotInlineTests.contains(testURI))
-                _properties.setProperty(Options.INLINE_LITERALS, "false");
-
 			/*
 			 * The Sesame TCK forces statement level connection auto-commit so
 			 * we set a flag to permit that here. However, auto-commit and this
 			 * flag SHOULD NOT be used outside of the test suite as they provide
 			 * an extreme performance penalty.
 			 */
-            _properties.put ( BigdataSail.Options.ALLOW_AUTO_COMMIT, "true" ) ;
+            //_properties.put ( BigdataSail.Options.ALLOW_AUTO_COMMIT, "true" ) ;
 
 			/*
 			 * Provide Unicode support for keys with locale-based string
@@ -308,7 +312,7 @@
 			 * partitions will become uniformly distributed, which will
 			 * negatively impact index performance.
 			 */
-			_properties.put(BigdataSail.Options.TERMID_BITS_TO_REVERSE,"2");
+			_properties.put(BigdataSail.Options.TERMID_BITS_TO_REVERSE,"0"); // was 2.
         	        
 			/*
 			 * Option may be enabled to store blank nodes such that they are
@@ -317,6 +321,17 @@
 			// new NV(BigdataSail.Options.STORE_BLANK_NODES,"true");
 
         }
+
+		/*
+		 * Turn inlining on or off depending on _this_ test. This is outside of
+		 * the if block above because _properties is cached.
+		 */
+        if (BigdataSparqlTest.cannotInlineTests.contains(testURI)) {
+            _properties.setProperty(Options.INLINE_LITERALS, "false");
+        } else {
+        	_properties.setProperty(Options.INLINE_LITERALS, "true");
+        }
+        
         return _properties ;
     }
 

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/com/bigdata/rdf/sail/tck/BigdataSparqlTest.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -191,19 +191,6 @@
 //            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/expr-equals/manifest#eq-graph-1",
 //            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/expr-equals/manifest#eq-graph-2",
             
-            /*
-             * busted with scale-out quads query (problem was that the
-             * subqueries did not have a top-level operator which ran on
-             * the query controller).
-             */
-//            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/optional/manifest#dawg-union-001",
-//            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/graph/manifest#dawg-graph-07",
-//            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/graph/manifest#dawg-graph-11",
-//            "http://www.w3.org/2001/sw/DataAccess/tests/data-r2/distinct/manifest#distinct-star-1"
-    		
-    		/*
-    		 * Problems with JiniFederation.
-    		 */
     });
 
     /**
@@ -358,8 +345,8 @@
         // no query time inference
         props.setProperty(Options.QUERY_TIME_EXPANDER, "false");
         
-        // auto-commit only there for TCK
-        props.setProperty(Options.ALLOW_AUTO_COMMIT, "true");
+//        // auto-commit only there for TCK
+//        props.setProperty(Options.ALLOW_AUTO_COMMIT, "true");
         
         // exact size only there for TCK
         props.setProperty(Options.EXACT_SIZE, "true");

Modified: branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java
===================================================================
--- branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java	2010-10-12 19:40:58 UTC (rev 3776)
+++ branches/QUADS_QUERY_BRANCH/bigdata-sails/src/test/org/openrdf/query/parser/sparql/SPARQLQueryTest.java	2010-10-12 20:33:07 UTC (rev 3777)
@@ -133,17 +133,22 @@
 	protected Repository createRepository()
 		throws Exception
 	{
-		Repository repo = newRepository();
-		repo.initialize();
-		RepositoryConnection con = repo.getConnection();
-		try {
-			con.clear();
-			con.clearNamespaces();
-		}
-		finally {
-			con.close();
-		}
-		return repo;
+		/*
+		 * Note: We override this for bigdata and use a new repository instance
+		 * for each test.  See the various subclasses for examples.
+		 */
+		throw new UnsupportedOperationException();
+//		Repository repo = newRepository();
+//		repo.initialize();
+//		RepositoryConnection con = repo.getConnection();
+//		try {
+//			con.clear();
+//			con.clearNamespaces();
+//		}
+//		finally {
+//			con.close();
+//		}
+//		return repo;
 	}
 
 	protected abstract Repository newRepository()
@@ -402,7 +407,12 @@
 				in.close();
 			}
 
-			con.setAutoCommit(true);
+			/*
+			 * Modified Oct 11th 2010 by BBT.  Do not enable auto-commit. Just
+			 * commit the connection.
+			 */
+//			con.setAutoCommit(true);
+			con.commit();
 		}
 		finally {
 			con.close();


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3776] branches/maven_scaleout/bigdata-core/src/test/ java/com/bigdata/test/conditional/LongTestsExcluded.java

[<ble...@us...>]

Revision: 3776
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3776&view=rev
Author:   blevine218
Date:     2010-10-12 19:40:58 +0000 (Tue, 12 Oct 2010)

Log Message:
-----------
reverse sense of test

Modified Paths:
--------------
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java

Modified: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java	2010-10-12 19:31:09 UTC (rev 3775)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java	2010-10-12 19:40:58 UTC (rev 3776)
@@ -32,9 +32,9 @@
  */
 public class LongTestsExcluded
 {
-    private final static String INCLUDE_LONG_TESTS_PROP = "com.bigdata.test.includeLongTests";    
+    private final static String EXCLUDE_LONG_TESTS_PROP = "com.bigdata.test.excludeLongTests";    
     public final static boolean test()
     {
-        return !Boolean.getBoolean(INCLUDE_LONG_TESTS_PROP);
+        return Boolean.getBoolean(EXCLUDE_LONG_TESTS_PROP);
     }
 }


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3775] branches/maven_scaleout/bigdata-core/src/test/ java/com/bigdata/test

[<ble...@us...>]

Revision: 3775
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3775&view=rev
Author:   blevine218
Date:     2010-10-12 19:31:09 +0000 (Tue, 12 Oct 2010)

Log Message:
-----------
Re-whack @IngoreIf to take a class that executes the test.
Move all classes into new com.bigdata.test.conditional sub-package.

Added Paths:
-----------
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalParameterized.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalRunner.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/FalseCondition.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/IgnoreIf.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIf.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnIgnoredClass.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnNotIgnoredClass.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TrueCondition.java

Removed Paths:
-------------
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalParameterized.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalRunner.java
    branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/IgnoreIf.java

Deleted: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalParameterized.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalParameterized.java	2010-10-12 17:53:12 UTC (rev 3774)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalParameterized.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -1,127 +0,0 @@
-package com.bigdata.test;
-
-import java.lang.reflect.Modifier;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.List;
-
-import org.junit.runner.Runner;
-import org.junit.runner.notification.RunNotifier;
-import org.junit.runners.Suite;
-import org.junit.runners.Parameterized.Parameters;
-import org.junit.runners.model.FrameworkMethod;
-import org.junit.runners.model.InitializationError;
-import org.junit.runners.model.Statement;
-import org.junit.runners.model.TestClass;
-
-/**
- * Similar to the Parameterized runner but uses a ConditionalRunner rather than the default
- * runner when generating the parameterized test instances.
- * 
- * @author blevine
- *
- */
-public class ConditionalParameterized extends Suite
-{
-
-    private final ArrayList<Runner> runners = new ArrayList<Runner>();
-    
-    /**
-     * Constructor
-     * 
-     * @param klass
-     * @throws Throwable
-     */
-    public ConditionalParameterized(Class<?> klass) throws Throwable
-    {
-        super(klass, Collections.<Runner> emptyList());
-        List<Object[]> parametersList = getParametersList(getTestClass());
-        
-        for (int i = 0; i < parametersList.size(); i++)
-        {
-            runners.add(new TestClassRunnerForParameters(getTestClass().getJavaClass(), parametersList, i));
-        }
-    }
-
-    @Override
-    protected List<Runner> getChildren()
-    {
-        return runners;
-    }
-
-    @SuppressWarnings("unchecked")
-    private List<Object[]> getParametersList(TestClass klass) throws Throwable
-    {
-        return (List<Object[]>) getParametersMethod(klass).invokeExplosively(null);
-    }
-
-    private FrameworkMethod getParametersMethod(TestClass testClass) throws Exception
-    {
-        List<FrameworkMethod> methods = testClass.getAnnotatedMethods(Parameters.class);
-        for (FrameworkMethod each : methods)
-        {
-            int modifiers = each.getMethod().getModifiers();
-            if (Modifier.isStatic(modifiers) && Modifier.isPublic(modifiers))
-                return each;
-        }
-
-        throw new Exception("No public static parameters method on class " + testClass.getName());
-    }
-    
-
-    private class TestClassRunnerForParameters extends ConditionalRunner
-    {
-        private final int parmSetNumber;
-        private final List<Object[]> parmList;
-
-        TestClassRunnerForParameters(Class<?> type, List<Object[]> parameterList, int i) throws InitializationError
-        {
-            super(type);
-            parmList = parameterList;
-            parmSetNumber = i;
-        }
-
-        @Override
-        public Object createTest() throws Exception
-        {
-            return getTestClass().getOnlyConstructor().newInstance(computeParams());
-        }
-
-        private Object[] computeParams() throws Exception
-        {
-            try
-            {
-                return parmList.get(parmSetNumber);
-            }
-            catch (ClassCastException e)
-            {
-                throw new Exception(String.format("%s.%s() must return a Collection of arrays.", getTestClass().getName(), getParametersMethod(getTestClass()).getName()));
-            }
-        }
-
-        @Override
-        protected String getName()
-        {
-            return String.format("[%s]", parmSetNumber);
-        }
-
-        @Override
-        protected String testName(final FrameworkMethod method)
-        {
-            return String.format("%s[%s]", method.getName(), parmSetNumber);
-        }
-
-        @Override
-        protected void validateConstructor(List<Throwable> errors)
-        {
-            validateOnlyOneConstructor(errors);
-        }
-
-        @Override
-        protected Statement classBlock(RunNotifier notifier)
-        {
-            return childrenInvoker(notifier);
-        }
-    }
-
-}

Deleted: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalRunner.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalRunner.java	2010-10-12 17:53:12 UTC (rev 3774)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalRunner.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -1,100 +0,0 @@
-/**
-
-Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
-
-Contact:
-     SYSTAP, LLC
-     4501 Tower Road
-     Greensboro, NC 27410
-     lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
- */
-package com.bigdata.test;
-
-import org.junit.internal.runners.model.EachTestNotifier;
-import org.junit.runner.Description;
-import org.junit.runner.notification.RunNotifier;
-import org.junit.runners.BlockJUnit4ClassRunner;
-import org.junit.runners.model.FrameworkMethod;
-import org.junit.runners.model.InitializationError;
-
-/**
- * A subclass of the default JUnit runner that checks for and process the @IgnoreIf conditional annotation
- * 
- * @author blevine
- * 
- */
-public class ConditionalRunner extends BlockJUnit4ClassRunner
-{
-
-    /**
-     * Constructor 
-     * 
-     * @param klass
-     * @throws InitializationError
-     */
-    public ConditionalRunner(Class<?> klass) throws InitializationError
-    {
-        super(klass);
-    }
-
-    /* (non-Javadoc)
-     * @see org.junit.runners.BlockJUnit4ClassRunner#runChild(org.junit.runners.model.FrameworkMethod, org.junit.runner.notification.RunNotifier)
-     */
-    @Override
-    protected void runChild(FrameworkMethod method, RunNotifier notifier)
-    {
-        // Is the annotation on the method?
-        IgnoreIf ann = method.getAnnotation(IgnoreIf.class);
-        
-        // Is the annotation on the class?
-        if (ann == null)
-        {
-            ann = method.getMethod().getDeclaringClass().getAnnotation(IgnoreIf.class);
-        }
-        
-        // If we found the annotation on the method or it's declaring class, check the condition
-        // and if true, skip this method (test case).
-        if (ann != null)
-        {
-            boolean condition = ann.value();
-            
-            if (condition)
-            {
-                EachTestNotifier eachNotifier = makeNotifier(method, notifier);
-                eachNotifier.fireTestIgnored();
-                return;
-            }
-        }
-        
-        // If we didn't find the annotation or if the annotation's condition evaluated to false process
-        // the test case normally.
-        
-        super.runChild(method, notifier);
-    }
-
-    /**
-     * Unfortunately declared private in super-class. Cloned from there
-     * 
-     * @param method
-     * @param notifier
-     * @return
-     */
-    private EachTestNotifier makeNotifier(FrameworkMethod method, RunNotifier notifier)
-    {
-        Description description = describeChild(method);
-        return new EachTestNotifier(notifier, description);
-    }
-}

Deleted: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/IgnoreIf.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/IgnoreIf.java	2010-10-12 17:53:12 UTC (rev 3774)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/IgnoreIf.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -1,41 +0,0 @@
-/**
-
-Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
-
-Contact:
-     SYSTAP, LLC
-     4501 Tower Road
-     Greensboro, NC 27410
-     lic...@bi...
-
-This program is free software; you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation; version 2 of the License.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; if not, write to the Free Software
-Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
-*/
-package com.bigdata.test;
-
-import java.lang.annotation.ElementType;
-import java.lang.annotation.Retention;
-import java.lang.annotation.RetentionPolicy;
-import java.lang.annotation.Target;
-
-/**
- * A conditional implementation of Junit's @Ignore annotation
- * @author blevine
- *
- */
-@Retention(RetentionPolicy.RUNTIME)
-@Target({ElementType.METHOD, ElementType.TYPE})
-public @interface IgnoreIf {
-    boolean value() default true;
-}
-

Copied: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalParameterized.java (from rev 3772, branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalParameterized.java)
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalParameterized.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalParameterized.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,150 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+package com.bigdata.test.conditional;
+
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.junit.runner.Runner;
+import org.junit.runner.notification.RunNotifier;
+import org.junit.runners.Suite;
+import org.junit.runners.Parameterized.Parameters;
+import org.junit.runners.model.FrameworkMethod;
+import org.junit.runners.model.InitializationError;
+import org.junit.runners.model.Statement;
+import org.junit.runners.model.TestClass;
+
+/**
+ * Similar to the standard JUnit Parameterized runner but uses a ConditionalRunner rather than the default
+ * runner when generating the parameterized test proxies.
+ * 
+ * @author blevine
+ *
+ */
+public class ConditionalParameterized extends Suite
+{
+
+    private final ArrayList<Runner> runners = new ArrayList<Runner>();
+    
+    /**
+     * Constructor
+     * 
+     * @param klass
+     * @throws Throwable
+     */
+    public ConditionalParameterized(Class<?> klass) throws Throwable
+    {
+        super(klass, Collections.<Runner> emptyList());
+        List<Object[]> parametersList = getParametersList(getTestClass());
+        
+        for (int i = 0; i < parametersList.size(); i++)
+        {
+            runners.add(new TestClassRunnerForParameters(getTestClass().getJavaClass(), parametersList, i));
+        }
+    }
+
+    @Override
+    protected List<Runner> getChildren()
+    {
+        return runners;
+    }
+
+    @SuppressWarnings("unchecked")
+    private List<Object[]> getParametersList(TestClass klass) throws Throwable
+    {
+        return (List<Object[]>) getParametersMethod(klass).invokeExplosively(null);
+    }
+
+    private FrameworkMethod getParametersMethod(TestClass testClass) throws Exception
+    {
+        List<FrameworkMethod> methods = testClass.getAnnotatedMethods(Parameters.class);
+        for (FrameworkMethod each : methods)
+        {
+            int modifiers = each.getMethod().getModifiers();
+            if (Modifier.isStatic(modifiers) && Modifier.isPublic(modifiers))
+                return each;
+        }
+
+        throw new Exception("No public static parameters method on class " + testClass.getName());
+    }
+    
+
+    private class TestClassRunnerForParameters extends ConditionalRunner
+    {
+        private final int parmSetNumber;
+        private final List<Object[]> parmList;
+
+        TestClassRunnerForParameters(Class<?> type, List<Object[]> parameterList, int i) throws InitializationError
+        {
+            super(type);
+            parmList = parameterList;
+            parmSetNumber = i;
+        }
+
+        @Override
+        public Object createTest() throws Exception
+        {
+            return getTestClass().getOnlyConstructor().newInstance(computeParams());
+        }
+
+        private Object[] computeParams() throws Exception
+        {
+            try
+            {
+                return parmList.get(parmSetNumber);
+            }
+            catch (ClassCastException e)
+            {
+                throw new Exception(String.format("%s.%s() must return a Collection of arrays.", getTestClass().getName(), getParametersMethod(getTestClass()).getName()));
+            }
+        }
+
+        @Override
+        protected String getName()
+        {
+            return String.format("[%s]", parmSetNumber);
+        }
+
+        @Override
+        protected String testName(final FrameworkMethod method)
+        {
+            return String.format("%s[%s]", method.getName(), parmSetNumber);
+        }
+
+        @Override
+        protected void validateConstructor(List<Throwable> errors)
+        {
+            validateOnlyOneConstructor(errors);
+        }
+
+        @Override
+        protected Statement classBlock(RunNotifier notifier)
+        {
+            return childrenInvoker(notifier);
+        }
+    }
+
+}

Copied: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalRunner.java (from rev 3772, branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/ConditionalRunner.java)
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalRunner.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/ConditionalRunner.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,132 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+package com.bigdata.test.conditional;
+
+import java.lang.reflect.Method;
+
+import org.junit.internal.runners.model.EachTestNotifier;
+import org.junit.runner.Description;
+import org.junit.runner.notification.RunNotifier;
+import org.junit.runners.BlockJUnit4ClassRunner;
+import org.junit.runners.model.FrameworkMethod;
+import org.junit.runners.model.InitializationError;
+
+/**
+ * A subclass of the default JUnit standard runner that checks for and processes the @IgnoreIf 
+ * conditional annotation
+ * 
+ * @author blevine
+ * 
+ */
+public class ConditionalRunner extends BlockJUnit4ClassRunner
+{
+
+    /**
+     * Constructor 
+     * 
+     * @param klass
+     * @throws InitializationError
+     */
+    public ConditionalRunner(Class<?> klass) throws InitializationError
+    {
+        super(klass);
+    }
+
+    /* (non-Javadoc)
+     * @see org.junit.runners.BlockJUnit4ClassRunner#runChild(org.junit.runners.model.FrameworkMethod, org.junit.runner.notification.RunNotifier)
+     */
+    @Override
+    protected void runChild(FrameworkMethod method, RunNotifier notifier)
+    {
+        // Is the annotation on the method?
+        IgnoreIf ann = method.getAnnotation(IgnoreIf.class);
+        
+        // Is the annotation on the class?
+        if (ann == null)
+        {
+            ann = method.getMethod().getDeclaringClass().getAnnotation(IgnoreIf.class);
+        }
+        
+        // If we found the annotation on the method or it's declaring class, check the condition
+        // and if true, skip this method (test case).
+        if (ann != null)
+        {
+            Class<?> conditionalClass = ann.value();
+            
+            if (conditionalClass != null)
+            {
+                String conditionalClassName = conditionalClass.getName();
+                // Does the conditional class have a public static boolean test() method?  If so, invoke it.
+                // If not, it's an error.
+                try
+                {
+                    Method testMethod = conditionalClass.getMethod("test");
+                    Class<?> returnType = testMethod.getReturnType();
+                    
+                    if ((returnType) == null || (returnType != Boolean.TYPE))
+                    {
+                        throw new IllegalArgumentException("@IgnoreIf conditional class, " + conditionalClassName + 
+                                                           "  must have a static no arg 'test' method that returns boolean");                       
+                    }
+                    
+                    Boolean result = (Boolean)testMethod.invoke(null);
+                    
+                    if (result)
+                    {
+                        EachTestNotifier eachNotifier = makeNotifier(method, notifier);
+                        eachNotifier.fireTestIgnored();
+                        return;
+                    }
+                }
+                catch (Exception e)
+                {
+                    throw new IllegalArgumentException("@IgnoreIf conditional class, " + conditionalClassName + 
+                                                       "  must have a static no arg 'test' method that returns boolean");
+                }
+            }
+            else
+            {
+                throw new IllegalArgumentException("No class specified for @IgnoreIf annotation.");
+            }
+        }
+        
+        // If we didn't find the annotation or if the annotation's condition evaluated to false process
+        // the test case normally.
+        
+        super.runChild(method, notifier);
+    }
+
+    /**
+     * Unfortunately declared private in super-class. Cloned from there
+     * 
+     * @param method
+     * @param notifier
+     * @return
+     */
+    private EachTestNotifier makeNotifier(FrameworkMethod method, RunNotifier notifier)
+    {
+        Description description = describeChild(method);
+        return new EachTestNotifier(notifier, description);
+    }
+}

Added: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/FalseCondition.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/FalseCondition.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/FalseCondition.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,36 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+package com.bigdata.test.conditional;
+
+/**
+ * @author blevine
+ *
+ */
+public class FalseCondition
+{
+    public static boolean test()
+    {
+        return false;
+    }
+}

Copied: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/IgnoreIf.java (from rev 3772, branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/IgnoreIf.java)
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/IgnoreIf.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/IgnoreIf.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,41 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+package com.bigdata.test.conditional;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * A conditional implementation of Junit's @Ignore annotation
+ * @author blevine
+ *
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target({ElementType.METHOD, ElementType.TYPE})
+public @interface IgnoreIf {
+    Class<?> value();
+}
+

Added: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/LongTestsExcluded.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,40 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+package com.bigdata.test.conditional;
+
+/**
+ * Tests whether long-running unit tests should be included in the set of executed tests.
+ * Used in conjunction with the @IgnoreIf annotation.
+ * 
+ * @author blevine
+ *
+ */
+public class LongTestsExcluded
+{
+    private final static String INCLUDE_LONG_TESTS_PROP = "com.bigdata.test.includeLongTests";    
+    public final static boolean test()
+    {
+        return !Boolean.getBoolean(INCLUDE_LONG_TESTS_PROP);
+    }
+}

Added: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIf.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIf.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIf.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,57 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+package com.bigdata.test.conditional;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+
+/**
+ * @author blevine
+ *
+ */
+@RunWith(ConditionalRunner.class)
+public class TestIgnoreIf
+{
+    @Test
+    public void includedTest()
+    {
+        System.out.println("Included test!");
+    }
+    
+    @Test
+    @IgnoreIf(FalseCondition.class)
+    public void includedTest2()
+    {
+        System.out.println("Included test 2!");
+    }
+   
+    @Test
+    @IgnoreIf(TrueCondition.class)
+    public void excludedTest()
+    {
+        Assert.fail("This test should have been excluded!");
+    }
+}

Added: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnIgnoredClass.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnIgnoredClass.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnIgnoredClass.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,58 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+package com.bigdata.test.conditional;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+
+/**
+ * @author blevine
+ *
+ */
+@RunWith(ConditionalRunner.class)
+@IgnoreIf(TrueCondition.class)
+public class TestIgnoreIfOnIgnoredClass
+{
+    @Test
+    public void test1()
+    {
+        Assert.fail("Test should have been ignored");
+    }
+    
+    @Test
+    public void test2()
+    {
+        Assert.fail("Test should have been ignored");;
+    }
+    
+    @Test
+    @IgnoreIf(FalseCondition.class)
+    public void test3()
+    {
+        System.out.println("Test3!!");
+    }
+    
+}

Added: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnNotIgnoredClass.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnNotIgnoredClass.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TestIgnoreIfOnNotIgnoredClass.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,58 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+package com.bigdata.test.conditional;
+
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+
+
+/**
+ * @author blevine
+ *
+ */
+@RunWith(ConditionalRunner.class)
+@IgnoreIf(FalseCondition.class)
+public class TestIgnoreIfOnNotIgnoredClass
+{
+    @Test
+    public void test1()
+    {
+        System.out.println("Test1!");
+    }
+    
+    @Test
+    public void test2()
+    {
+        System.out.println("Test2!");
+    }
+    
+    @Test
+    @IgnoreIf(TrueCondition.class)
+    public void test3()
+    {
+        Assert.fail("This test should have been ignored.");
+    }
+    
+}

Added: branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TrueCondition.java
===================================================================
--- branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TrueCondition.java	                        (rev 0)
+++ branches/maven_scaleout/bigdata-core/src/test/java/com/bigdata/test/conditional/TrueCondition.java	2010-10-12 19:31:09 UTC (rev 3775)
@@ -0,0 +1,36 @@
+/**
+
+Copyright (C) SYSTAP, LLC 2006-2007.  All rights reserved.
+
+Contact:
+     SYSTAP, LLC
+     4501 Tower Road
+     Greensboro, NC 27410
+     lic...@bi...
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+*/
+package com.bigdata.test.conditional;
+
+/**
+ * @author blevine
+ *
+ */
+public class TrueCondition
+{
+    public static boolean test()
+    {
+        return true;
+    }
+}


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.