bigdata-commit

[Bigdata-commit] SF.net SVN: bigdata:[3333] trunk/src/resources/config

[<tho...@us...>]

Revision: 3333
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3333&view=rev
Author:   thompsonbry
Date:     2010-07-28 13:07:06 +0000 (Wed, 28 Jul 2010)

Log Message:
-----------
cleaned up the comments concerning the journalExtent parameter.  the 200MB value is advised.  The 1000MB value has not been tested in quite some time and I have no reason to believe that it would perform better with the changes to the dynamic sharding logic this year.

Modified Paths:
--------------
    trunk/src/resources/config/bigdataCluster.config
    trunk/src/resources/config/bigdataCluster16.config
    trunk/src/resources/config/standalone/bigdataStandalone.config

Modified: trunk/src/resources/config/bigdataCluster.config
===================================================================
--- trunk/src/resources/config/bigdataCluster.config	2010-07-27 21:33:26 UTC (rev 3332)
+++ trunk/src/resources/config/bigdataCluster.config	2010-07-28 13:07:06 UTC (rev 3333)
@@ -184,11 +184,7 @@
 
     // @todo also specify k (replicationCount)
 
-    /* Sets the initial and maximum journal extents.
-     * 
-     * Note: 200M is good for 32bit machines.  Higher end server machines with
-     * more on-disk cache work well with 1000M (1GB).
-     */
+    // Sets the initial and maximum journal extents.
     static private journalExtent = ConfigMath.multiply(200, Bytes.megabyte);
 
     /**

Modified: trunk/src/resources/config/bigdataCluster16.config
===================================================================
--- trunk/src/resources/config/bigdataCluster16.config	2010-07-27 21:33:26 UTC (rev 3332)
+++ trunk/src/resources/config/bigdataCluster16.config	2010-07-28 13:07:06 UTC (rev 3333)
@@ -193,16 +193,8 @@
     static private maxDataServicesPerHost = 2;
     static private dataServiceCount = 22; // was 24; was 20; was 18; was 20;//28;//14;//28
 
-    /* Sets the initial and maximum journal extents.
-     * 
-     * Note: 200M is good for 32bit machines.  Higher end server machines with
-     * more on-disk cache work well with 1000M (1GB).
-     *
-     * Note: I have seen modestly significant IOWait associated with overflow
-     * of a 1G journal at 8B+ triples, so I am reducing the journal size to
-     * compensate.
-     */
-    static private journalExtent = ConfigMath.multiply(200/*was 512*/, Bytes.megabyte);
+    // Sets the initial and maximum journal extents.
+    static private journalExtent = ConfigMath.multiply(200, Bytes.megabyte);
 
     /**
      * zookeeper configuration properties.

Modified: trunk/src/resources/config/standalone/bigdataStandalone.config
===================================================================
--- trunk/src/resources/config/standalone/bigdataStandalone.config	2010-07-27 21:33:26 UTC (rev 3332)
+++ trunk/src/resources/config/standalone/bigdataStandalone.config	2010-07-28 13:07:06 UTC (rev 3333)
@@ -138,8 +138,7 @@
      */
     static private dataServiceCount = 1;
     
-    /* Sets the initial and maximum journal extents.
-     */
+    // Sets the initial and maximum journal extents.
     static private journalExtent = ConfigMath.multiply(200, Bytes.megabyte);
 
     /**


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Bigdata-commit] SF.net SVN: bigdata:[3528] trunk/src/resources/config

[<tho...@us...>]

Revision: 3528
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3528&view=rev
Author:   thompsonbry
Date:     2010-09-10 16:02:26 +0000 (Fri, 10 Sep 2010)

Log Message:
-----------
Modified the configuration files to no longer specify the JVM -Xms argument for the data services per https://sourceforge.net/apps/trac/bigdata/ticket/157.

Modified Paths:
--------------
    trunk/src/resources/config/bigdataCluster.config
    trunk/src/resources/config/bigdataCluster16.config
    trunk/src/resources/config/bigdataStandalone.config

Modified: trunk/src/resources/config/bigdataCluster.config
===================================================================
--- trunk/src/resources/config/bigdataCluster.config	2010-09-10 14:26:34 UTC (rev 3527)
+++ trunk/src/resources/config/bigdataCluster.config	2010-09-10 16:02:26 UTC (rev 3528)
@@ -758,10 +758,11 @@
 	 * have for your applications!
 	 */
 	"-Xmx1600m",// was 800
-	/* Optionally, grab all/most of the max heap at once.  This makes sense for
-	 * DS but is less necessary for other bigdata services.
+	/* Pre-allocation of the DS heap is no longer recommended.
+	 *
+	 * See https://sourceforge.net/apps/trac/bigdata/ticket/157
+	"-Xms800m",
 	 */
-	"-Xms800m", // 1/2 of the max heap is a good value. 
 	/*
 	 * This option will keep the JVM "alive" even when it is memory starved
 	 * but perform of a memory starved JVM is terrible.

Modified: trunk/src/resources/config/bigdataCluster16.config
===================================================================
--- trunk/src/resources/config/bigdataCluster16.config	2010-09-10 14:26:34 UTC (rev 3527)
+++ trunk/src/resources/config/bigdataCluster16.config	2010-09-10 16:02:26 UTC (rev 3528)
@@ -813,12 +813,11 @@
      * http://blogs.msdn.com/ntdebugging/archive/2009/02/06/microsoft-windows-dynamic-cache-service.aspx
 	 */
 	"-Xmx9G", // Note: out of 32 available!
-	/* Optionally, grab all/most of the max heap at once.  This makes sense for
-	 * DS, but is less necessary for other bigdata services.  If the machine is
-	 * dedicated to the DataService then use the maximum heap.  Otherwise 1/2 of
-	 * the maximum heap is a good value.
-	 */
+	/* Pre-allocation of the DS heap is no longer recommended.
+	 *
+	 * See https://sourceforge.net/apps/trac/bigdata/ticket/157
 	"-Xms9G", 
+	 */
 	/*
 	 * FIXME This might not be required, so that should be tested.
 	 * However, you don't want the JVM to just die if it is being

Modified: trunk/src/resources/config/bigdataStandalone.config
===================================================================
--- trunk/src/resources/config/bigdataStandalone.config	2010-09-10 14:26:34 UTC (rev 3527)
+++ trunk/src/resources/config/bigdataStandalone.config	2010-09-10 16:02:26 UTC (rev 3528)
@@ -781,10 +781,11 @@
 	 * have for your applications!
 	 */
 	"-Xmx4g",// was 800
-	/* Optionally, grab all/most of the max heap at once.  This makes sense for
-	 * DS but is less necessary for other bigdata services.
+	/* Pre-allocation of the DS heap is no longer recommended.
+	 *
+	 * See https://sourceforge.net/apps/trac/bigdata/ticket/157
+	"-Xms2G",
 	 */
-	"-Xms2G", // 1/2 of the max heap is a good value.
 	/*
 	 * This option will keep the JVM "alive" even when it is memory starved
 	 * but perform of a memory starved JVM is terrible.


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.