SmartFrog

Revision: 7326
          http://smartfrog.svn.sourceforge.net/smartfrog/?rev=7326&view=rev
Author:   steve_l
Date:     2009-01-29 15:56:29 +0000 (Thu, 29 Jan 2009)

Log Message:
-----------
updated the todo list

Modified Paths:
--------------
    trunk/core/components/hadoop/doc/managing_hadoop.ppt

Modified: trunk/core/components/hadoop/doc/managing_hadoop.ppt
===================================================================
(Binary files differ)


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[ http://jira.smartfrog.org/jira/browse/SFOS-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=11494#action_11494 ] 

Steve Loughran commented on SFOS-788:
-------------------------------------

The current design is no good. We should not have a separate "job" and JobSubmission. The JobSubmission should contain all information about the Job, and take a cluster parameter to point to a specific cluster configuration to use/overwrite

> write component to submit jobs to a Hadoop cluster
> --------------------------------------------------
>
>                 Key: SFOS-788
>                 URL: http://jira.smartfrog.org/jira/browse/SFOS-788
>             Project: SmartFrog
>          Issue Type: Sub-task
>          Components: _service_hadoop
>    Affects Versions: 3.17.004
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>             Fix For: 3.17.x
>
>


-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.smartfrog.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.smartfrog.org/jira/browse/SFOS-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved SFOS-1090.
----------------------------------

    Resolution: Fixed

Checked in; docs updated

> Move Hadoop service lifecycle from "terminated" to close; update state diagrams
> -------------------------------------------------------------------------------
>
>                 Key: SFOS-1090
>                 URL: http://jira.smartfrog.org/jira/browse/SFOS-1090
>             Project: SmartFrog
>          Issue Type: Improvement
>          Components: _service_hadoop
>    Affects Versions: 3.17.x
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>             Fix For: 3.17.x
>
>   Original Estimate: 2 hours
>  Remaining Estimate: 2 hours
>
> Make the hadoop state diagrams/tables consistent across all docs, use CLOSED instead of terminated, make the changes in the (uncommitted) code and the SF subclasses

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.smartfrog.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[ http://jira.smartfrog.org/jira/browse/SFOS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=11492#action_11492 ] 

Steve Loughran commented on SFOS-839:
-------------------------------------

Or the test runner should grab any attached exceptions and log them. That keeps the toString() terse but makes the junit logs useful

> TerminationRecord.toString() should include the stack trace of all nested exceptions
> ------------------------------------------------------------------------------------
>
>                 Key: SFOS-839
>                 URL: http://jira.smartfrog.org/jira/browse/SFOS-839
>             Project: SmartFrog
>          Issue Type: Improvement
>          Components: .sfCore
>    Affects Versions: 3.12.040
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>             Fix For: 3.17.x
>
>
> uncommitted enhancements to the testing framework cause early startup failures to get turned into termination records and propagated back to the caller -but now the termination record doesn't print the full stack trace
> TerminationRecord.toString() should do so

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.smartfrog.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Revision: 7325
          http://smartfrog.svn.sourceforge.net/smartfrog/?rev=7325&view=rev
Author:   steve_l
Date:     2009-01-29 15:37:40 +0000 (Thu, 29 Jan 2009)

Log Message:
-----------
SFOS-788 	 write component to submit jobs to a Hadoop cluster

Modified Paths:
--------------
    trunk/core/components/hadoop/test/org/smartfrog/services/hadoop/test/system/local/tracker/components.sf

Modified: trunk/core/components/hadoop/test/org/smartfrog/services/hadoop/test/system/local/tracker/components.sf
===================================================================
--- trunk/core/components/hadoop/test/org/smartfrog/services/hadoop/test/system/local/tracker/components.sf	2009-01-29 15:30:19 UTC (rev 7324)
+++ trunk/core/components/hadoop/test/org/smartfrog/services/hadoop/test/system/local/tracker/components.sf	2009-01-29 15:37:40 UTC (rev 7325)
@@ -44,7 +44,7 @@
  * Test the local node
  */
 testJobSubmission extends ExpectDeploy {
-  description "bring up a Task Tracker";
+  description "Submit a job to a cluster and copy out the result";
 
   action extends LocalClusterCompound {
   


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[ http://jira.smartfrog.org/jira/browse/SFOS-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved SFOS-1076.
----------------------------------

    Fix Version/s: 3.17.006
       Resolution: Fixed

> restlet JARs are in smartfrog lib under SVN
> -------------------------------------------
>
>                 Key: SFOS-1076
>                 URL: http://jira.smartfrog.org/jira/browse/SFOS-1076
>             Project: SmartFrog
>          Issue Type: Bug
>          Components: .sfCore
>    Affects Versions: 3.17.x
>            Reporter: Steve Loughran
>            Assignee: andrew farrell
>             Fix For: 3.17.006
>
>
> the restlet JARs have crept into core/smartfrog/lib. Do we need them? If not, they should be deleted

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.smartfrog.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

Revision: 7324
          http://smartfrog.svn.sourceforge.net/smartfrog/?rev=7324&view=rev
Author:   steve_l
Date:     2009-01-29 15:30:19 +0000 (Thu, 29 Jan 2009)

Log Message:
-----------
SFOS-1083   	 Release SmartFrog 3.16.004; a maintenance release of the 3.12.044 branch; updating notes

Modified Paths:
--------------
    trunk/core/release/doc/creating_release_artifacts.sxw

Modified: trunk/core/release/doc/creating_release_artifacts.sxw
===================================================================
(Binary files differ)


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

Revision: 7323
          http://smartfrog.svn.sourceforge.net/smartfrog/?rev=7323&view=rev
Author:   steve_l
Date:     2009-01-29 15:28:37 +0000 (Thu, 29 Jan 2009)

Log Message:
-----------
SFOS-1090  Move Hadoop service lifecycle from "terminated" to close; update state diagrams
SFOS-788 	 SFOS-780
write component to submit jobs to a Hadoop cluster

Modified Paths:
--------------
    trunk/core/components/libraries.properties

Added Paths:
-----------
    trunk/core/antbuild/repository/org.apache.hadoop/hadoop-core/0.21.0-alpha-3/
    trunk/core/antbuild/repository/org.apache.hadoop/hadoop-core/0.21.0-alpha-3/hadoop-core-0.21.0-alpha-3.jar

Added: trunk/core/antbuild/repository/org.apache.hadoop/hadoop-core/0.21.0-alpha-3/hadoop-core-0.21.0-alpha-3.jar
===================================================================
(Binary files differ)


Property changes on: trunk/core/antbuild/repository/org.apache.hadoop/hadoop-core/0.21.0-alpha-3/hadoop-core-0.21.0-alpha-3.jar
___________________________________________________________________
Added: svn:mime-type
   + application/octet-stream

Modified: trunk/core/components/libraries.properties
===================================================================
--- trunk/core/components/libraries.properties	2009-01-29 13:06:23 UTC (rev 7322)
+++ trunk/core/components/libraries.properties	2009-01-29 15:28:37 UTC (rev 7323)
@@ -97,7 +97,7 @@
 
 #hadoop artifacts
 commons-cli.version=2.0-SNAPSHOT
-hadoop.version=0.21.0-alpha-2
+hadoop.version=0.21.0-alpha-3
 #hadoop.jetty.version=5.1.4
 #hadoop.servlet-api.version=2.4
 #hadoop.tomcat.version=5.5.12


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

To whom it may engage...
        
This is an automated request, but not an unsolicited one. For 
more information please visit http://gump.apache.org/nagged.html, 
and/or contact the folk at general@....

Project smartfrog-components has an issue affecting its community integration.
This issue affects 1 projects,
 and has been outstanding for 29 runs.
The current state of this project is 'Failed', with reason 'Build Failed'.
For reference only, the following projects are affected by this:
    - smartfrog-components :  Smartfrog: Application Deployment from HP Laboratories


Full details are available at:
    http://vmgump.apache.org/gump/public/smartfrog/smartfrog-components/index.html

That said, some information snippets are provided here.

The following annotations (debug/informational/warning/error messages) were provided:
 -INFO- Optional dependency jetty failed with reason missing build outputs
 -INFO- Failed with reason build failed
 -INFO- Project Reports in: /srv/gump/public/workspace/smartfrog/components/junit/build/test/reports
 -WARNING- No directory [/srv/gump/public/workspace/smartfrog/components/junit/build/test/reports]
 -INFO- Failed to extract fallback artifacts from Gump Repository



The following work was performed:
http://vmgump.apache.org/gump/public/smartfrog/smartfrog-components/gump_work/build_smartfrog_smartfrog-components.html
Work Name: build_smartfrog_smartfrog-components (Type: Build)
Work ended in a state of : Failed
Elapsed: 2 secs
Command Line: /usr/lib/jvm/java-6-sun/bin/java -Djava.awt.headless=true -Xbootclasspath/p:/srv/gump/public/workspace/xml-xalan/build/serializer.jar:/srv/gump/public/workspace/xml-xalan/build/xalan-unbundled.jar:/srv/gump/public/workspace/xml-commons/java/external/build/xml-apis.jar:/srv/gump/public/workspace/xml-xerces2/build/xercesImpl.jar org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml -Dbuild.sysclasspath=only -Dsystem.tests=false published 
[Working Directory: /srv/gump/public/workspace/smartfrog]
CLASSPATH: /usr/lib/jvm/java-6-sun/lib/tools.jar:/srv/gump/public/workspace/smartfrog/components/scripting/lib/bsh-1.3.0.jar:/srv/gump/public/workspace/smartfrog/components/anubis/build/classes:/srv/gump/public/workspace/smartfrog/components/anubisdeployer/build/classes:/srv/gump/public/workspace/smartfrog/components/dns/build/classes:/srv/gump/public/workspace/smartfrog/components/emailer/build/classes:/srv/gump/public/workspace/smartfrog/components/loggingservices/build/classes:/srv/gump/public/workspace/smartfrog/components/net/build/classes:/srv/gump/public/workspace/smartfrog/components/scripting/build/classes:/srv/gump/public/workspace/smartfrog/components/sfinstaller/build/classes:/srv/gump/public/workspace/smartfrog/components/slp/build/classes:/srv/gump/public/workspace/smartfrog/components/ssh/build/classes:/srv/gump/public/workspace/smartfrog/components/tomcat/build/classes:/srv/gump/public/workspace/smartfrog/components/www/build/classes:/srv/gump/public/workspace/smartfrog/components/xml/build/classes:/srv/gump/public/workspace/smartfrog/components/xml/build/test/classes:/srv/gump/public/workspace/smartfrog/components/junit/build/classes:/srv/gump/public/workspace/smartfrog/components/junit/build/test/classes:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/ant/dist/lib/ant-trax.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-nodeps.jar:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ivy/build/artifact/jars/ivy-29012009.jar:/srv/gump/public/workspace/smartfrog/smartfrog/dist/lib/smartfrog-29012009.jar:/srv/gump/public/workspace/smartfrog/smartfrog/dist/lib/sfServices-29012009.jar:/srv/gump/public/workspace/smartfrog/smartfrog/dist/lib/sfExamples-29012009.jar:/srv/gump/packages/junit3.8.1/junit.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/apache-commons/logging/target/commons-logging-29012009.jar:/srv/gump/public/workspace/apache-commons/logging/target/commons-logging-api-29012009.jar:/srv/gump/public/workspace/logging-log4j-12/dist/lib/log4j-29012009.jar:/srv/gump/packages/javacc-3.1/bin/lib/javacc.jar:/srv/gump/public/workspace/httpcomponents/oac.hc3x/dist/commons-httpclient.jar:/srv/gump/public/workspace/apache-commons/codec/dist/commons-codec-29012009.jar:/srv/gump/public/workspace/jakarta-oro/jakarta-oro-29012009.jar:/srv/gump/public/workspace/apache-commons/vfs/core/target/commons-vfs-2.0-SNAPSHOT.jar:/srv/gump/public/workspace/apache-commons/vfs/sandbox/target/commons-vfs-sandbox-29012009.jar:/srv/gump/public/workspace/jakarta-slide/webdavclient/dist/lib/jakarta-slide-webdavlib-29012009.jar:/srv/gump/packages/jsch/jsch-0.1.38.jar:/srv/gump/public/workspace/smartfrog/extras/ant/dist/lib/sf-tasks-29012009.jar:/srv/gump/public/workspace/ant/build/lib/ant-testutil.jar:/srv/gump/public/workspace/smartfrog/testharness/dist/lib/sf-testharness-29012009.jar:/srv/gump/packages/javamail-1.4/mail.jar:/srv/gump/packages/javamail-1.4/lib/mailapi.jar:/srv/gump/packages/jaf-1.1ea/activation.jar:/srv/gump/public/workspace/apache-commons/net/dist/commons-net-29012009.jar:/srv/gump/public/workspace/jakarta-bsf/build/lib/bsf.jar:/srv/gump/public/workspace/velocity-engine/bin/velocity-29012009.jar:/srv/gump/public/workspace/james-server/stage/dnsjava/jars/dnsjava-2.0.6.jar:/srv/gump/packages/jaxen-1.1-beta-6/jaxen-1.1-beta-6.jar:/srv/gump/public/workspace/xom/build/xom-29012009.jar:/srv/gump/public/workspace/xml-commons/java/external/build/xml-apis-ext.jar:/srv/gump/packages/cargo/cargo-core-uberjar-0.8.jar:/srv/gump/public/workspace/jetty/modules/extra/jetty-rewrite/target/jetty-rewrite-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/plus/jetty-annotations/target/jetty-annotations-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/server/jetty-util/target/jetty-util-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/plus/jetty-jndi/target/jetty-jndi-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/extra/jetty-servlet/target/jetty-servlet-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/server/jetty-start/target/jetty-start-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/extra/jetty-threadpool/target/jetty-threadpool-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/extra/jetty-ajp/target/jetty-ajp-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/extra/jetty-html/target/jetty-html-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/plus/jetty-plus/target/jetty-plus-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/extra/jetty-client/target/jetty-client-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/plus/jetty-jmx/target/jetty-jmx-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/server/jetty-ssl/target/jetty-ssl-7.0-SNAPSHOT.jar:/srv/gump/public/workspace/jetty/modules/server/jetty/target/jetty-7.0-SNAPSHOT.jar
---------------------------------------------
Buildfile: /srv/gump/public/workspace/smartfrog/build.xml
  [taskdef] Could not load definitions from resource cloverlib.xml. It could not be found.

init:
     [echo]   
     [echo]   
     [echo]  =======================================================
     [echo]  Build info - Project: SmartFrog ALL
     [echo]    (C) Copyright 1998-2009 Hewlett-Packard Development Company, LP
     [echo]       Java: 1.6.0 Sun Microsystems Inc. 
     [echo]             /usr/lib/jvm/java-6-sun-1.6.0.00/jre 
     [echo]       OS: Linux 2.6.20-16-server
     [echo]  =======================================================
     [echo]   

ivy-properties:

ivy-dirs:

ivy-init:
    [mkdir] Created dir: /srv/gump/public/workspace/smartfrog/external
[ivy:configure] :: Ivy 2.1.x-local-20090129032757 - 20090129032757 :: http://ant.apache.org/ivy/ ::
:: loading settings :: file = /srv/gump/public/workspace/smartfrog/antbuild/ivy/ivyconf.xml
DEPRECATED: 'ivyconf' element is deprecated, use 'ivysettings' instead (file:/srv/gump/public/workspace/smartfrog/antbuild/ivy/ivyconf.xml)

BUILD FAILED
/srv/gump/public/workspace/smartfrog/build.xml:497: impossible to configure ivy:settings with given file: /srv/gump/public/workspace/smartfrog/antbuild/ivy/ivyconf.xml : java.text.ParseException: failed to load settings from file:/srv/gump/public/workspace/smartfrog/antbuild/ivy/ivyconf.xml: impossible to add configured child for artifact on class org.apache.ivy.plugins.resolver.FileSystemResolver: artifact pattern must be absolute: ${root.dir}/antbuild/repository/[organisation]/[module]/[revision]/[module]-[revision].[ext]

Total time: 1 second
---------------------------------------------

To subscribe to this information via syndicated feeds:
- RSS: http://vmgump.apache.org/gump/public/smartfrog/smartfrog-components/rss.xml
- Atom: http://vmgump.apache.org/gump/public/smartfrog/smartfrog-components/atom.xml

============================== Gump Tracking Only ===
Produced by Gump version 2.3.
Gump Run 13000029012009, vmgump:vmgump-public:13000029012009
Gump E-mail Identifier (unique within run) #54.

--
Apache Gump
http://gump.apache.org/ [Instance: vmgump]

Revision: 7322
          http://smartfrog.svn.sourceforge.net/smartfrog/?rev=7322&view=rev
Author:   steve_l
Date:     2009-01-29 13:06:23 +0000 (Thu, 29 Jan 2009)

Log Message:
-----------
SFOS-1062    Presentation on Dynamic Hadoop Clusters

Modified Paths:
--------------
    trunk/core/components/hadoop/doc/managing_hadoop.ppt
    trunk/core/components/hadoop/doc/managing_illustrations.vsd

Modified: trunk/core/components/hadoop/doc/managing_hadoop.ppt
===================================================================
(Binary files differ)

Modified: trunk/core/components/hadoop/doc/managing_illustrations.vsd
===================================================================
(Binary files differ)


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

Revision: 7321
          http://smartfrog.svn.sourceforge.net/smartfrog/?rev=7321&view=rev
Author:   steve_l
Date:     2009-01-29 11:24:52 +0000 (Thu, 29 Jan 2009)

Log Message:
-----------
getting artifact number right for hadoop

Modified Paths:
--------------
    trunk/core/components/libraries.properties

Modified: trunk/core/components/libraries.properties
===================================================================
--- trunk/core/components/libraries.properties	2009-01-28 17:42:31 UTC (rev 7320)
+++ trunk/core/components/libraries.properties	2009-01-29 11:24:52 UTC (rev 7321)
@@ -97,7 +97,7 @@
 
 #hadoop artifacts
 commons-cli.version=2.0-SNAPSHOT
-hadoop.version=0.21.0-alpha-6
+hadoop.version=0.21.0-alpha-2
 #hadoop.jetty.version=5.1.4
 #hadoop.servlet-api.version=2.4
 #hadoop.tomcat.version=5.5.12


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.