Thread: [Bigdata-commit] SF.net SVN: bigdata:[3257] trunk/build.xml

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Revision: 3257
          http://bigdata.svn.sourceforge.net/bigdata/?rev=3257&view=rev
Author:   thompsonbry
Date:     2010-07-21 12:59:52 +0000 (Wed, 21 Jul 2010)

Log Message:
-----------
Merged updates from BrianM and attempted to reconcile the standalone-XXX targets for benchmarking a federation running on a single machine with the LookupStarter as used by the unit tests.  I am still missing something critical about how to setup the LUS as it fails with exceptions such as below.  

standalone-setup:
standalone-startLookup:
     [java] Jul 21, 2010 8:58:57 AM com.sun.jini.start.HTTPDStatus httpdWarning
     [java] WARNING: Problem accessing desired URL[http://192.168.1.102:23333/reggie-dl.jar]: java.net.ConnectException: Connection refused: connect.
     [java] Jul 21, 2010 8:58:58 AM com.sun.jini.start.HTTPDStatus httpdWarning
     [java] WARNING: Problem accessing desired URL[http://192.168.1.102:23333/jsk-dl.jar]: java.net.ConnectException: Connection refused: connect.
     [java] Jul 21, 2010 8:59:00 AM com.sun.jini.reggie.RegistrarImpl init
     [java] INFO: started Reggie: c8778e6d-8df4-468b-bf41-96167ed1a2d5, [testFed], ConstrainableLookupLocator[[jini://192.168.1.102/], [null]]
     [java] java.lang.ClassNotFoundException: com.sun.jini.reggie.ConstrainableRegistrarProxy (could not determine preferred setting; original codebase: "http://192.168.1.102:23333/reggie-dl.jar http://192.168.1.102:23333/jsk-dl.jar")
     [java] 	at net.jini.loader.pref.PreferredClassProvider.loadClass(PreferredClassProvider.java:580)
     [java] 	at java.rmi.server.RMIClassLoader.loadClass(RMIClassLoader.java:247)
     [java] 	at sun.rmi.server.MarshalInputStream.resolveClass(MarshalInputStream.java:197)
     [java] 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1575)
     [java] 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
     [java] 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
     [java] 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
     [java] 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
     [java] 	at java.rmi.MarshalledObject.get(MarshalledObject.java:142)
     [java] 	at com.sun.jini.start.NonActivatableServiceDescriptor.create(NonActivatableServiceDescriptor.java:644)
     [java] 	at com.bigdata.service.jini.util.LookupStarter.startLookupService(LookupStarter.java:228)
     [java] 	at com.bigdata.service.jini.util.LookupStarter.access$100(LookupStarter.java:58)
     [java] 	at com.bigdata.service.jini.util.LookupStarter$LookupStartThread.run(LookupStarter.java:152)
     [java] Caused by: java.net.ConnectException: Connection refused: connect
     [java] 	at java.net.PlainSocketImpl.socketConnect(Native Method)
     [java] 	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
     [java] 	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
     [java] 	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
     [java] 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
     [java] 	at java.net.Socket.connect(Socket.java:525)
     [java] 	at java.net.Socket.connect(Socket.java:475)
     [java] 	at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
     [java] 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
     [java] 	at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
     [java] 	at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
     [java] 	at sun.net.www.http.HttpClient.New(HttpClient.java:306)
     [java] 	at sun.net.www.http.HttpClient.New(HttpClient.java:323)
     [java] 	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:860)
     [java] 	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:801)
     [java] 	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:726)
     [java] 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1049)
     [java] 	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.getPreferredConnection(PreferredClassLoader.java:656)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.jarExists(PreferredClassLoader.java:584)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.getPreferredInputStream(PreferredClassLoader.java:534)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.initializePreferredResources(PreferredClassLoader.java:482)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.isPreferredResource0(PreferredClassLoader.java:760)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.access$100(PreferredClassLoader.java:226)
     [java] 	at net.jini.loader.pref.PreferredClassLoader$3.run(PreferredClassLoader.java:742)
     [java] 	at java.security.AccessController.doPrivileged(Native Method)
     [java] 	at net.jini.loader.pref.PreferredClassLoader.isPreferredResource(PreferredClassLoader.java:739)
     [java] 	at net.jini.loader.pref.PreferredClassProvider.loadClass(PreferredClassProvider.java:576)
     [java] 	... 12 more
     [java] Java Result: -2

Modified Paths:
--------------
    trunk/build.xml

Modified: trunk/build.xml
===================================================================

--- trunk/build.xml	2010-07-20 22:31:05 UTC (rev 3256)
+++ trunk/build.xml	2010-07-21 12:59:52 UTC (rev 3257)
@@ -849,12 +849,13 @@
 
 	<!--                                                                   -->
 	<!--                    STANDALONE FEDERATION TARGETS                  -->
-	<!--                                                                   -->
+	<!--                         (test/benchamarking)                      -->
 
 <target name="standalone-setup" description="Setup properties used by the standalone federation and LUS start/stop.">
     <property name="app.home" location="${bigdata.dir}" />
     <property name="test.codebase.port" value="23333"/>
-    <property name="test.codebase.dir" location="${bigdata.dir}/bigdata-jini/lib/jini/lib-dl" />
+    <property name="jini.lib" location="${bigdata.dir}/bigdata-jini/lib/jini/lib" />
+    <property name="jini.lib.dl" location="${bigdata.dir}/bigdata-jini/lib/jini/lib-dl" />
     <property name="test.codebase" value="http://${this.hostname}:${test.codebase.port}/jsk-dl.jar"/>
     <property name="java.security.policy" value="${bigdata.dir}/policy.all"/>
 	<property name="log4j.configuration" value="${bigdata.dir}/src/resources/config/standalone/log4j.properties"/>
@@ -864,11 +865,11 @@
 
 <target name="standalone-startLookup" depends="jar,standalone-setup" description="Start the lookup service for the standalone federation.">
   <java classname="com.bigdata.service.jini.util.LookupStarter"
-		fork="true" spawn="true">
+		fork="true" spawn="false">
 		<classpath refid="runtime.classpath" />
     <sysproperty key="app.home" value="${app.home}"/>
-    <sysproperty key="jini.lib" value="${dist.lib}"/>
-    <sysproperty key="jini.lib.dl" value="${dist.lib.dl}"/>
+    <sysproperty key="jini.lib" value="${jini.lib}"/>
+    <sysproperty key="jini.lib.dl" value="${jini.lib.dl}"/>
     <sysproperty key="java.security.policy" value="${java.security.policy}"/>
     <sysproperty key="java.security.debug" value="off"/>
     <sysproperty key="java.protocol.handler.pkgs" value="net.jini.url"/>
@@ -933,25 +934,22 @@
     <antcall target="standalone-stopLookup" />
 </target>
 
-<!-- @todo Add target to stand up a SPARQL endpoint for a KB namespace. Could be
-     Sesame Server or bigdata REST API. -->
-	
-<!-- Note: The source files must be "reasonable" for the bulk loader.  Each file
-     will be fully parsed into memory before writing onto the database.  Very
-     large source files must be split into smaller files.  See the 'split'
-     command under unix, but be aware that it will not generate more than 576
-     output files (24 x 24 = 576) and only 100 if you are using number suffixes.
-     For example, split an NT file into files of 200000 lines each: 
-     
-     mkdir splits; cd splits
-     split -l 200000 ../dataset.nt dataset
-     
-     You will also have to fix up the file extensions.  You can do this using
-     bash as follows:
-     
-     for file in *; do mv $file $file.nt; done
-     
-     -->
+<target name="standalone-start-nano-server" depends="jar" description="Start a small http server fronting for a bigdata database instance.">
+	<java classname="com.bigdata.rdf.sail.bench.NanoSparqlServer" fork="true" failonerror="true">
+		<arg line="${standalone.nanoServerPort} ${standalone.namespace} src/resources/config/standalone/bigdataStandalone.config" />
+		<jvmarg line="-server"/>
+		<jvmarg line="-Xmx200M"/>
+		<classpath refid="runtime.classpath" />
+	</java>
+</target>
+
+<target name="standalone-stop-nano-server" depends="jar" description="Stop the small http server running at the configured port.">
+	<java classname="com.bigdata.rdf.sail.bench.NanoSparqlServer" fork="true" failonerror="true">
+		<arg line="${standalone.nanoServerPort} -stop" />
+		<classpath refid="runtime.classpath" />
+	</java>
+</target>
+
 <target name="standalone-bulk-load" depends="jar" description="Bulk load RDF data into the standalone federation.">
 	<java classname="com.bigdata.rdf.load.MappedRDFDataLoadMaster"
 		failonerror="true" fork="true" logerror="true">
@@ -967,30 +965,14 @@
 		<!--                    -->
 		<!-- Per job parameters -->
 		<!--                    -->
-		<!-- The KB namespace ("kb" is a common default). -->
-		<jvmarg value="-Dbigdata.rdf.namespace=U10"/>
+		<!-- The namespace of the target KB. -->
+		<jvmarg value="-Dbigdata.rdf.namespace=${standalone.namespace}"/>
 		<!-- The job name (same as the KB namespace is a common default). -->
-		<jvmarg value="-Dbigdata.rdf.job.name=bulk-load-kb"/>
-		<!-- The file or directory containing RDF data to be loaded. 
-		
-		     Possible interesting data:
-		
-		     d:/bigdata-perf-analysis/lubm/U10
-		     d:/bigdata-perf-analysis/bsbm/bsbm_2785/dataset.nt
-		-->
-		<jvmarg value="-Dbigdata.rdf.data=d:/bigdata-perf-analysis/lubm/U10"/>
-		<!-- The file or directory containing zero or more RDF ontology files to be loaded.
-		
-			 Note: If you do not want to load any "ontology" data you can point this at an
-			 empty directory.  "ontology" data is loaded once - when a new KB instance is
-			 created.
-
-			 Possible interesting ontologies:
-		
-			bigdata-rdf/src/resources/data/lehigh/univ-bench.owl
-			d:/bigdata-perf-analysis/emptyDir
-		 -->
-		<jvmarg value="-Dbigdata.rdf.ontology=bigdata-rdf/src/resources/data/lehigh/univ-bench.owl"/>
+		<jvmarg value="-Dbigdata.rdf.job.name=bulk-load-kb-${standalone-namespace}"/>
+		<!-- The file or directory containing zero or more files to be loaded first. -->
+		<jvmarg value="-Dbigdata.rdf.ontology=${standalone.bulkLoad.ontology}"/>
+		<!-- The file or directory containing RDF data to be loaded. --> 
+		<jvmarg value="-Dbigdata.rdf.data=${standalone.bulkLoad.data}"/>
 		<!-- The main configuration file. -->
 		<arg value="src/resources/config/standalone/bigdataStandalone.config" />
 	</java>


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.




Thread: [Bigdata-commit] SF.net SVN: bigdata:[3257] trunk/build.xml

Fast, scalable, robust graph database platform

bigdata-commit