treebase-guts — Commit and tracker messages

[Treebase-guts] [ treebase-Bugs-3089438 ] Find journal name in Dryad bagit metadata

[SourceForge.net <no...@so...>]

Bugs item #3089438, was opened at 2010-10-17 21:04
Message generated for change (Settings changed) made by hlapp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089438&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
Status: Open
>Priority: 7
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
>Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: Find journal name in Dryad bagit metadata

Initial Comment:
Dryad returns to TreeBASE information about a submission in the bagit format.  This format contains a data/dryadpub.xml record that contains a metadata element with journal information.  The element is <bibo:Journal>  Bill Piel has mentioned that the TreeBASE side is not finding the journal information.  TreeBASE code needs to be updated to look in the data/dryadpub.xml record (after unpacking the bagit format) for this information.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089438&group_id=248804

[Treebase-guts] [ treebase-Bugs-3089436 ] URL that is returned to Dryad should be treebase.org/...

[SourceForge.net <no...@so...>]

Bugs item #3089436, was opened at 2010-10-17 21:00
Message generated for change (Settings changed) made by hlapp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089436&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
Status: Open
>Priority: 7
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
>Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: URL that is returned to Dryad should be treebase.org/...

Initial Comment:
In the submission integration process between Dryad and TreeBASE, Dryad connects to TreeBASE at:

http://treebase.org/treebase-web

When a successful connection is made, TreeBASE returns a URL that is then passed to the Dryad user so s/he can complete the submission process at TreeBASE.  The URL that is being returned is currently:

http://treebase.nescent.org/treebase-web/login.jsp?importKey=KEY_GOES_HERE

We've heard from Bill Piel that this URL should be to treebase.org not treebase.nescent.org

Would you change the TreeBASE code (or server setup?) to return this URL?  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089436&group_id=248804

[Treebase-guts] [ treebase-Bugs-3089438 ] Find journal name in Dryad bagit metadata

[SourceForge.net <no...@so...>]

Bugs item #3089438, was opened at 2010-10-17 18:04
Message generated for change (Tracker Item Submitted) made by ksclarke
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089438&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
Assigned to: Nobody/Anonymous (nobody)
Summary: Find journal name in Dryad bagit metadata

Initial Comment:
Dryad returns to TreeBASE information about a submission in the bagit format.  This format contains a data/dryadpub.xml record that contains a metadata element with journal information.  The element is <bibo:Journal>  Bill Piel has mentioned that the TreeBASE side is not finding the journal information.  TreeBASE code needs to be updated to look in the data/dryadpub.xml record (after unpacking the bagit format) for this information.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089438&group_id=248804

[Treebase-guts] [ treebase-Bugs-3089436 ] URL that is returned to Dryad should be treebase.org/...

[SourceForge.net <no...@so...>]

Bugs item #3089436, was opened at 2010-10-17 18:00
Message generated for change (Tracker Item Submitted) made by ksclarke
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089436&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
Assigned to: Nobody/Anonymous (nobody)
Summary: URL that is returned to Dryad should be treebase.org/...

Initial Comment:
In the submission integration process between Dryad and TreeBASE, Dryad connects to TreeBASE at:

http://treebase.org/treebase-web

When a successful connection is made, TreeBASE returns a URL that is then passed to the Dryad user so s/he can complete the submission process at TreeBASE.  The URL that is being returned is currently:

http://treebase.nescent.org/treebase-web/login.jsp?importKey=KEY_GOES_HERE

We've heard from Bill Piel that this URL should be to treebase.org not treebase.nescent.org

Would you change the TreeBASE code (or server setup?) to return this URL?  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3089436&group_id=248804

[Treebase-guts] SF.net SVN: treebase:[758] trunk/treebase-core/db/cleaning/ 2010-10-06_delete_treeblock/delete_treeblock.pl

[<sfr...@us...>]

Revision: 758
          http://treebase.svn.sourceforge.net/treebase/?rev=758&view=rev
Author:   sfrgpiel
Date:     2010-10-07 17:25:58 +0000 (Thu, 07 Oct 2010)

Log Message:
-----------
Fixed the code to ensure that all nodes get deleted

Modified Paths:
--------------
    trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl

Modified: trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl
===================================================================
--- trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl	2010-10-07 01:00:23 UTC (rev 757)
+++ trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl	2010-10-07 17:25:58 UTC (rev 758)
@@ -56,17 +56,17 @@
 				print "Lets attempt to do this ". int($totRec/$sets) ." times in sets of $sets records\n";
 				
 				$statement = "DELETE FROM phylotreenode WHERE phylotree_id IN (
-				   SELECT phylotree_id FROM phylotree WHERE treeblock_id = ? LIMIT $sets 
+				   SELECT phylotree_id FROM phylotree WHERE treeblock_id = ? LIMIT ? 
 				)";
 				
 				my $delete_phylonodes =  $dbh->prepare("$statement");
 
 				foreach my $cnt (1 .. int($totRec/$sets) ) {
-					$delete_phylonodes->execute( $treeblock_id );
+					$delete_phylonodes->execute( $treeblock_id, $sets );
 					print "Deletion $cnt for batch of $sets phylotreenode records \n";
 				}
 				# one more for good measure
-				$delete_phylonodes->execute( $treeblock_id );
+				$delete_phylonodes->execute( $treeblock_id, $totRec );
 
 				# taxonlabels are referenced by nodes, matrix rows, taxon blocks *and* submissions 
 				# let's remove the connection to submissions. This assumes that there are no matrices 


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[757] trunk/treebase-core/db/cleaning

[<sfr...@us...>]

Revision: 757
          http://treebase.svn.sourceforge.net/treebase/?rev=757&view=rev
Author:   sfrgpiel
Date:     2010-10-07 01:00:23 +0000 (Thu, 07 Oct 2010)

Log Message:
-----------
Perl script for deleting treeblocks that otherwise can't be deleted using the usual user interface

Modified Paths:
--------------
    trunk/treebase-core/db/cleaning/2010-03-03_fixlabels/fixlabels_trans.pl

Added Paths:
-----------
    trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/
    trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/README.txt
    trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl

Modified: trunk/treebase-core/db/cleaning/2010-03-03_fixlabels/fixlabels_trans.pl
===================================================================
--- trunk/treebase-core/db/cleaning/2010-03-03_fixlabels/fixlabels_trans.pl	2010-10-06 17:31:22 UTC (rev 756)
+++ trunk/treebase-core/db/cleaning/2010-03-03_fixlabels/fixlabels_trans.pl	2010-10-07 01:00:23 UTC (rev 757)
@@ -15,19 +15,11 @@
 # constraint that taxonlabel_id in sub_taxonlabel be unique. Instead, we delete any 
 # sub_taxonlabel records that have a taxonlabel_id that is determined to be redundant. 
 
-# For a local database:
-
-my $database = "tb2synops";
-my $username = "piel";
+my $database = "";
+my $username = "treebase_app";
 my $password = "";
 
-# For the remote database, uncomment this (and also uncomment line 142):
 
-# my $database = "treebasedev";
-# my $username = "treebase_app";
-# my $password = "tim5tema";
-
-
 my $dbh = &ConnectToPg($database, $username, $password);
 
 # Let's start by counting the number of redundant taxonlabel_ids:
@@ -155,7 +147,7 @@
 	
 	$cstr = "DBI:Pg:dbname="."$cstr";
 	# uncomment this to run against the server at NESCent:
-	# $cstr .= ";host=treebasedb-dev.nescent.org";
+	$cstr .= ";host=treebasedb-dev.nescent.org";
 
 	
 	my $dbh = DBI->connect($cstr, $user, $pass, {AutoCommit => 0, PrintError => 1, RaiseError => 1});

Added: trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/README.txt
===================================================================
--- trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/README.txt	                        (rev 0)
+++ trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/README.txt	2010-10-07 01:00:23 UTC (rev 757)
@@ -0,0 +1,10 @@
+A perl script from Bill to delete treeblocks and the trees inside them. 
+
+Before running, adjust database name and password in delete_treeblock.pl. 
+
+The parameter to provide is the treeblock_id, e.g.:
+
+perl delete_treeblock.pl 12016
+
+... where 12016 is the treeblock_id. You are then asked to confirm the number of trees in the tree block. 
+

Added: trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl
===================================================================
--- trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl	                        (rev 0)
+++ trunk/treebase-core/db/cleaning/2010-10-06_delete_treeblock/delete_treeblock.pl	2010-10-07 01:00:23 UTC (rev 757)
@@ -0,0 +1,163 @@
+#!/usr/bin/perl
+
+use strict;
+use DBI;
+
+# This script is intended to delete a treeblock, cascading to all dependent tables
+
+my $database = "";
+my $username = "treebase_app";
+my $password = "";
+
+my $treeblock_id = shift;
+
+# Number of nodes to delete at a time
+my $sets = 1000;
+
+# check that the treeblock_id looks like a number:
+if ( $treeblock_id =~ m/^\d+$/ ) {
+
+	my $dbh = &ConnectToPg($database, $username, $password);
+		
+	my $count = "SELECT COUNT(*) FROM treeblock WHERE treeblock_id = $treeblock_id ";
+	my $totRec = $dbh->selectrow_array ($count);
+
+	# check to see that the treeblock in question exists
+	if ($totRec == 1) {
+
+		$count = "SELECT COUNT(*) FROM phylotree WHERE treeblock_id = $treeblock_id ";
+		$totRec = $dbh->selectrow_array ($count);
+
+		my $a;
+		do {
+			print "There are $totRec trees in treeblock $treeblock_id, does that sound right? [yn]\n";
+			$a = <STDIN>;
+			chop $a;
+		} until (($a eq "y") || ($a eq "n"));
+		
+		# offer a chance to cancel
+		if ($a eq "y") {
+
+			# run the whole thing in a single transaction 
+			eval {
+				
+				# phylotreenode and phylotree have mutual FK constraints 
+				# first NULL the phylotree.rootnode_id to remove constraint
+				$dbh->do( "UPDATE phylotree SET rootnode_id = DEFAULT WHERE treeblock_id = ?", undef, $treeblock_id );
+
+				# since we've been getting a time-out when deleting phylonodes, lets delete them in 
+				# batches of $sets
+
+				my $statement = "SELECT COUNT(*) FROM phylotreenode WHERE phylotree_id IN (
+					SELECT phylotree_id FROM phylotree WHERE treeblock_id = $treeblock_id ) ";				
+				$totRec = $dbh->selectrow_array ($statement);
+				
+				print "There are $totRec phylotreenode records that we need to delete.\n";
+				print "Lets attempt to do this ". int($totRec/$sets) ." times in sets of $sets records\n";
+				
+				$statement = "DELETE FROM phylotreenode WHERE phylotree_id IN (
+				   SELECT phylotree_id FROM phylotree WHERE treeblock_id = ? LIMIT $sets 
+				)";
+				
+				my $delete_phylonodes =  $dbh->prepare("$statement");
+
+				foreach my $cnt (1 .. int($totRec/$sets) ) {
+					$delete_phylonodes->execute( $treeblock_id );
+					print "Deletion $cnt for batch of $sets phylotreenode records \n";
+				}
+				# one more for good measure
+				$delete_phylonodes->execute( $treeblock_id );
+
+				# taxonlabels are referenced by nodes, matrix rows, taxon blocks *and* submissions 
+				# let's remove the connection to submissions. This assumes that there are no matrices 
+				# or other treeblocks that will continue to reference these taxonlabels
+				$statement = "DELETE FROM sub_taxonlabel WHERE taxonlabel_id IN (
+				   SELECT DISTINCT taxonlabel_id 
+				   FROM taxonlabelset_taxonlabel JOIN taxonlabelset USING (taxonlabelset_id) 
+				   JOIN treeblock USING (taxonlabelset_id) 
+				   WHERE treeblock_id = ? 
+				)";
+				$dbh->do( $statement, undef, $treeblock_id );
+
+				# remove the many-to-many join between taxonlabelset and taxonlabel
+				$statement = "DELETE FROM taxonlabelset_taxonlabel WHERE taxonlabelset_id IN (
+				   SELECT taxonlabelset_id FROM treeblock WHERE treeblock_id = ? 
+				)";
+				$dbh->do( $statement, undef, $treeblock_id );
+
+				# delete taxonlabel records after having deleted the taxonlabelset_taxonlabel records
+				$statement = "DELETE FROM taxonlabel WHERE taxonlabel_id IN (
+				   SELECT tl.taxonlabel_id 
+				   FROM taxonlabel tl LEFT JOIN taxonlabelset_taxonlabel USING (taxonlabel_id) 
+				   WHERE study_id = (SELECT DISTINCT study_id FROM phylotree WHERE treeblock_id = ?) 
+				   AND taxonlabelset_id IS NULL 
+				   AND NOT EXISTS (SELECT 1 FROM matrixrow mr WHERE mr.taxonlabel_id = tl.taxonlabel_id) 
+				   AND NOT EXISTS (SELECT 1 FROM phylotreenode ptn WHERE ptn.taxonlabel_id = tl.taxonlabel_id) 
+				)";
+				$dbh->do( $statement, undef, $treeblock_id );
+				
+				# Before we delete the taxonlabelset, we need to remove the FK constraint
+				# with treeblock
+				$statement = "UPDATE treeblock SET taxonlabelset_id = DEFAULT WHERE treeblock_id = ?";
+				$dbh->do( $statement, undef, $treeblock_id );
+
+				# Now delete the taxonlabelset 
+				$statement = "DELETE FROM taxonlabelset WHERE taxonlabelset_id IN (
+				   SELECT DISTINCT taxonlabelset_id 
+				   FROM taxonlabelset tls LEFT JOIN taxonlabelset_taxonlabel tltl USING (taxonlabelset_id) 
+				   WHERE study_id = (
+					  SELECT DISTINCT study_id FROM phylotree  
+					  WHERE treeblock_id = ?
+					  ) 
+				   AND tltl.taxonlabel_id IS NULL 
+				   AND NOT EXISTS (SELECT 1 FROM treeblock tb WHERE tb.taxonlabelset_id = tls.taxonlabelset_id AND tb.treeblock_id <> ?) 
+				   AND NOT EXISTS (SELECT 1 FROM matrix mx WHERE mx.taxonlabelset_id = tls.taxonlabelset_id ) 
+				)";
+				$dbh->do( $statement, undef, $treeblock_id, $treeblock_id );
+
+				$dbh->do( "DELETE FROM phylotree WHERE treeblock_id = ?", undef, $treeblock_id );
+				$dbh->do( "DELETE FROM sub_treeblock WHERE treeblock_id = ?", undef, $treeblock_id );
+				$dbh->do( "DELETE FROM treeblock WHERE treeblock_id = ?", undef, $treeblock_id );
+
+				# If no errors so far, let's commit
+				$dbh->commit();
+				print "Congratulations: treeblock_id $treeblock_id has been deleted and the deletes committed\n";
+			};
+	
+			if ($@) {
+			   warn "Failed to delete treeblock_id $treeblock_id: $@\n";
+			   $dbh->rollback();
+			   print "rollback!!\n";
+			}
+			
+		} else {
+			print "Deletion cancelled on user request\n";
+		}
+	} else {
+		print "$totRec record(s) found, there should be only one.\n";
+	}
+	
+	my $rc = $dbh->disconnect;
+} else {
+	print "The treeblock_id ($treeblock_id) does not look like a number.\n";
+}
+
+
+# Connect to Postgres using DBI
+#==============================================================
+sub ConnectToPg {
+
+	my ($cstr, $user, $pass) = @_;
+	
+	$cstr = "DBI:Pg:dbname="."$cstr";
+	# uncomment this to run against the server at NESCent:
+	$cstr .= ";host=treebasedb-dev.nescent.org";
+
+	
+	my $dbh = DBI->connect($cstr, $user, $pass, {AutoCommit => 0, PrintError => 1, RaiseError => 1});
+	$dbh || &error("DBI connect failed : ",$dbh->errstr);
+	 
+	return($dbh);
+}
+
+


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[756] trunk/treebase-web/src/main/webapp/WEB-INF/pages /uploadFile.jsp

[<sfr...@us...>]

Revision: 756
          http://treebase.svn.sourceforge.net/treebase/?rev=756&view=rev
Author:   sfrgpiel
Date:     2010-10-06 17:31:22 +0000 (Wed, 06 Oct 2010)

Log Message:
-----------
Fixing two typos

Modified Paths:
--------------
    trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp

Modified: trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp
===================================================================
--- trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp	2010-10-06 15:33:22 UTC (rev 755)
+++ trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp	2010-10-06 17:31:22 UTC (rev 756)
@@ -26,7 +26,7 @@
 <body id="submissions"/>
 <p>Use the following form to upload your Nexus files for <b>submission ${studyMap['id']} - ${studyMap['name']}</b><br>
 Please limit the number of trees you upload for parsing to  ~30, otherwise  large numbers of trees resulting from the 
-same analysis will overwhelm the user experience in TreeBASE's search interface with what are largley redudant trees. 
+same analysis will overwhelm the user experience in TreeBASE's search interface with what are largely redundant trees. 
 If you have a large number of trees, please either select a few preferred trees and upload those for parsing, or upload 
 a consensus tree. For information on making large numbers of trees available to users, please see the help 
 <a href="#" class="openHelp" onclick="openHelp('uploadFile')"><img class="iconButton" src="<fmt:message key="icons.help"/>" /></a>.</p>	


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[755] trunk/treebase-web/src/main/webapp/WEB-INF/pages /uploadFile.jsp

[<sfr...@us...>]

Revision: 755
          http://treebase.svn.sourceforge.net/treebase/?rev=755&view=rev
Author:   sfrgpiel
Date:     2010-10-06 15:33:22 +0000 (Wed, 06 Oct 2010)

Log Message:
-----------
Moving some excess instructions to the help window.

Modified Paths:
--------------
    trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp

Modified: trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp
===================================================================
--- trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp	2010-10-06 14:23:25 UTC (rev 754)
+++ trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp	2010-10-06 15:33:22 UTC (rev 755)
@@ -28,12 +28,8 @@
 Please limit the number of trees you upload for parsing to  ~30, otherwise  large numbers of trees resulting from the 
 same analysis will overwhelm the user experience in TreeBASE's search interface with what are largley redudant trees. 
 If you have a large number of trees, please either select a few preferred trees and upload those for parsing, or upload 
-a consensus tree. Keep in mind that users will be able to download your original, un-parsed upload file, so if you have 
-a 10,000-tree posterior distribution that you would like to make available, consider creating two tree blocks: one with 
-the majority rule consensus tree, and the other with the 10,000 trees \xD1 but instead of writing 
-"BEGIN TREES;" [...] "END;" put something like "BEGIN EXTRATREES;" [...] "END;". 
-This allows a large batch of trees to be made available to users but they won't actually be parsed into TreeBASE, and 
-thus they won't  overwhelm the user experience.  </p>	
+a consensus tree. For information on making large numbers of trees available to users, please see the help 
+<a href="#" class="openHelp" onclick="openHelp('uploadFile')"><img class="iconButton" src="<fmt:message key="icons.help"/>" /></a>.</p>	
 <!--form method="post" enctype="multipart/form-data" onsubmit="setTimeout('queryStatus()', 200);"-->
 <!-- Line above is needed to show the actal amount of data uploaded using Ajax -->
 <!-- Since we are not ready for it that is why line is commented -->


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[754] trunk/treebase-web/src/main/webapp/WEB-INF/pages /uploadFile.jsp

[<sfr...@us...>]

Revision: 754
          http://treebase.svn.sourceforge.net/treebase/?rev=754&view=rev
Author:   sfrgpiel
Date:     2010-10-06 14:23:25 +0000 (Wed, 06 Oct 2010)

Log Message:
-----------
Adding instructions to prevent people from uploading loads and loads of trees.

Modified Paths:
--------------
    trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp

Modified: trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp
===================================================================
--- trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp	2010-09-24 22:10:26 UTC (rev 753)
+++ trunk/treebase-web/src/main/webapp/WEB-INF/pages/uploadFile.jsp	2010-10-06 14:23:25 UTC (rev 754)
@@ -24,7 +24,16 @@
 
 
 <body id="submissions"/>
-<p>Use the following form to upload your Nexus files for <b>submission ${studyMap['id']} - ${studyMap['name']}</b></p>	
+<p>Use the following form to upload your Nexus files for <b>submission ${studyMap['id']} - ${studyMap['name']}</b><br>
+Please limit the number of trees you upload for parsing to  ~30, otherwise  large numbers of trees resulting from the 
+same analysis will overwhelm the user experience in TreeBASE's search interface with what are largley redudant trees. 
+If you have a large number of trees, please either select a few preferred trees and upload those for parsing, or upload 
+a consensus tree. Keep in mind that users will be able to download your original, un-parsed upload file, so if you have 
+a 10,000-tree posterior distribution that you would like to make available, consider creating two tree blocks: one with 
+the majority rule consensus tree, and the other with the 10,000 trees \xD1 but instead of writing 
+&quot;BEGIN TREES;&quot; [...] &quot;END;&quot; put something like &quot;BEGIN EXTRATREES;&quot; [...] &quot;END;&quot;. 
+This allows a large batch of trees to be made available to users but they won't actually be parsed into TreeBASE, and 
+thus they won't  overwhelm the user experience.  </p>	
 <!--form method="post" enctype="multipart/form-data" onsubmit="setTimeout('queryStatus()', 200);"-->
 <!-- Line above is needed to show the actal amount of data uploaded using Ajax -->
 <!-- Since we are not ready for it that is why line is commented -->


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] [ treebase-Bugs-3079602 ] OAI records contain all subjects in a single field

[SourceForge.net <no...@so...>]

Bugs item #3079602, was opened at 2010-10-01 15:13
Message generated for change (Settings changed) made by hlapp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3079602&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
Status: Open
>Priority: 7
Private: No
Submitted By: Ryan Scherle (rscherle)
Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: OAI records contain all subjects in a single field

Initial Comment:
In the OAI records, each <dc:subject> field contains many keywords, separated by commas, like this:

<dc:subject>
Ascomycota, Pezizomycotina, Dothideomyceta, fungal evolution, lichens, multigene phylogeny, phylogenomics, plant pathogens, saprobes, Tree of Life
</dc:subject>

It is best practice to put each keyword into a separate <dc:subject> field. This allows harvesting systems (like Dryad) to accurately separate the keywords, and not worry about keywords that may contain commas.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3079602&group_id=248804

[Treebase-guts] [ treebase-Bugs-3079602 ] OAI records contain all subjects in a single field

[SourceForge.net <no...@so...>]

Bugs item #3079602, was opened at 2010-10-01 14:13
Message generated for change (Tracker Item Submitted) made by rscherle
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3079602&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Ryan Scherle (rscherle)
Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: OAI records contain all subjects in a single field

Initial Comment:
In the OAI records, each <dc:subject> field contains many keywords, separated by commas, like this:

<dc:subject>
Ascomycota, Pezizomycotina, Dothideomyceta, fungal evolution, lichens, multigene phylogeny, phylogenomics, plant pathogens, saprobes, Tree of Life
</dc:subject>

It is best practice to put each keyword into a separate <dc:subject> field. This allows harvesting systems (like Dryad) to accurately separate the keywords, and not worry about keywords that may contain commas.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3079602&group_id=248804

[Treebase-guts] SF.net SVN: treebase:[753] trunk/treebase-web/src/main/webapp/login.jsp

[<vga...@us...>]

Revision: 753
          http://treebase.svn.sourceforge.net/treebase/?rev=753&view=rev
Author:   vgapeyev
Date:     2010-09-24 22:10:26 +0000 (Fri, 24 Sep 2010)

Log Message:
-----------
Re-enabling logins back

Modified Paths:
--------------
    trunk/treebase-web/src/main/webapp/login.jsp

Modified: trunk/treebase-web/src/main/webapp/login.jsp
===================================================================
--- trunk/treebase-web/src/main/webapp/login.jsp	2010-09-22 21:42:26 UTC (rev 752)
+++ trunk/treebase-web/src/main/webapp/login.jsp	2010-09-24 22:10:26 UTC (rev 753)
@@ -101,10 +101,6 @@
 <body id="info"/>
 
 <!-- loginForm.jsp -->
-<%-- VG: delete this after TB disk move --%>
-<b>Logging into Treebase is currently disabled in order to perform necessary maintenance. Please try again a few hours later.  We expect logins re-enabled by noon 24 September 2010, or earlier.</b> 
-<%-- VG: restore this after TB disk move
 <c:import url="/WEB-INF/pages/loginForm.jsp"/>
---%>
 <!-- password -->
 <p>Forgot your password? Have your <a href="<c:url value="passwordForm.html"/>">password e-mailed to you</a></p>


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[752] trunk/treebase-web/src/main/webapp/login.jsp

[<vga...@us...>]

Revision: 752
          http://treebase.svn.sourceforge.net/treebase/?rev=752&view=rev
Author:   vgapeyev
Date:     2010-09-22 21:42:26 +0000 (Wed, 22 Sep 2010)

Log Message:
-----------
Temporarily disabling TB logins, for maintenance. 

Modified Paths:
--------------
    trunk/treebase-web/src/main/webapp/login.jsp

Modified: trunk/treebase-web/src/main/webapp/login.jsp
===================================================================
--- trunk/treebase-web/src/main/webapp/login.jsp	2010-09-16 15:47:41 UTC (rev 751)
+++ trunk/treebase-web/src/main/webapp/login.jsp	2010-09-22 21:42:26 UTC (rev 752)
@@ -101,6 +101,10 @@
 <body id="info"/>
 
 <!-- loginForm.jsp -->
+<%-- VG: delete this after TB disk move --%>
+<b>Logging into Treebase is currently disabled in order to perform necessary maintenance. Please try again a few hours later.  We expect logins re-enabled by noon 24 September 2010, or earlier.</b> 
+<%-- VG: restore this after TB disk move
 <c:import url="/WEB-INF/pages/loginForm.jsp"/>
+--%>
 <!-- password -->
 <p>Forgot your password? Have your <a href="<c:url value="passwordForm.html"/>">password e-mailed to you</a></p>


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] [ treebase-Bugs-3043116 ] DC Date in OAI records has "extra" text

[SourceForge.net <no...@so...>]

Bugs item #3043116, was opened at 2010-08-11 10:53
Message generated for change (Comment added) made by vgapeyev
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3043116&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
>Status: Closed
Priority: 7
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: DC Date in OAI records has "extra" text

Initial Comment:
Hi, in harvesting TreeBASE, I've noticed the value for the dc:date element in the records has content like: "Published in 1983" instead of a plain date like "1983".  This is valid according to the OAI DC XML Schema, but DC does have a best practice that suggests that the content should just be a date in a standard encoding scheme: "Recommended best practice is to use an encoding scheme, such as the W3CDTF profile of ISO 8601" Cf. http://dublincore.org/documents/dces/

I think putting a date in dc:date that matches a standard encoding scheme would be less likely to cause problems with harvesters that make assumptions about data content based on best practice recommendations (many of the standard OAI tools do this unfortunately).  If you're only using dc:date for publication dates (is it currently used for anything else?) then people who harvest will be able to make that assumption without relying on the "Published in" text.

Thanks for considering this.

----------------------------------------------------------------------

>Comment By: Vladimir Gapeyev (vgapeyev)
Date: 2010-09-16 12:04

Message:
As of SVN#750, the element contains just a decimal for the year, such as
<dc:date>2009</dc:date>. 
This is pulled from CITATION.PUBLISHYEAR field; there is no data in the
record for a more precise date. 
This field is nullable in the DB, so the element is not guaranteed to be
non-empty, although all records in treebase-dev do have a value there. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3043116&group_id=248804

[Treebase-guts] [ treebase-Bugs-3033740 ] Some OAI records missing required datestamp element

[SourceForge.net <no...@so...>]

Bugs item #3033740, was opened at 2010-07-23 15:55
Message generated for change (Comment added) made by vgapeyev
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3033740&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: APIs
Group: None
>Status: Closed
Priority: 7
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: Some OAI records missing required datestamp element

Initial Comment:
Some of the records coming out via OAI are missing a required datestamp element in their header.

For example: http://www.treebase.org/treebase-web/top/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=TB:s1236

These records have an empty <datestamp></datestamp> element, but this isn't acceptable according to the UTCdatetimeType defined in the schema.  The record, as a result, is not valid and may cause OAI ingesters to choke.

Thanks.

----------------------------------------------------------------------

>Comment By: Vladimir Gapeyev (vgapeyev)
Date: 2010-09-16 12:04

Message:
This problem was only in the production instance.  Newer versions, as
deployed on treebase-dev, return non-empty <datestamp>. 
FYI, the <datestamp> value comes from STUDY.LASTMODIFIEDDATE table field.
According to a comment in the source code, it is the date "the author last
changed the status field (i.e. from "in progress" to "ready")".
This field is nullable in the DB, so the element is not guaranteed to be
non-empty, although all records in treebase-dev do have a value there. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3033740&group_id=248804

[Treebase-guts] SF.net SVN: treebase:[751] trunk/treebase-web/src/main/java/org/cipres/ treebase/web/controllers/OAIPMHController.java

[<vga...@us...>]

Revision: 751
          http://treebase.svn.sourceforge.net/treebase/?rev=751&view=rev
Author:   vgapeyev
Date:     2010-09-16 15:47:41 +0000 (Thu, 16 Sep 2010)

Log Message:
-----------
Fixing TB#3043116: removing the extra 'published on' text from the <dc:date> element in OAI. (Previous commit 'fixed' code in a dead source branch) 

Modified Paths:
--------------
    trunk/treebase-web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java

Modified: trunk/treebase-web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java
===================================================================
--- trunk/treebase-web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java	2010-09-16 15:44:51 UTC (rev 750)
+++ trunk/treebase-web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java	2010-09-16 15:47:41 UTC (rev 751)
@@ -242,7 +242,7 @@
 		    else
 		    	map.put("description",IdentifyUtil.escape4XML(study.getId(),study.getNotes()));
 			map.put("publisher", IdentifyUtil.escape4XML(study.getId(),publisher));						
-			map.put("date", "published on "+citation.getPublishYear());
+			map.put("date", citation.getPublishYear());
 			map.put("identifier", "purl.org/phylo/treebase/phylows/study/TB2:S"+study.getId());
 			map.put("datestamp", study.getLastModifiedDate());
 			


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[750] trunk/oai-pmh_data_provider/data_provider_web/ src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java

[<vga...@us...>]

Revision: 750
          http://treebase.svn.sourceforge.net/treebase/?rev=750&view=rev
Author:   vgapeyev
Date:     2010-09-16 15:44:51 +0000 (Thu, 16 Sep 2010)

Log Message:
-----------
Fixing TB#3043116: removing the extra 'published on' text from the <dc:date> element in OAI

Modified Paths:
--------------
    trunk/oai-pmh_data_provider/data_provider_web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java

Modified: trunk/oai-pmh_data_provider/data_provider_web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java
===================================================================
--- trunk/oai-pmh_data_provider/data_provider_web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java	2010-09-15 18:35:46 UTC (rev 749)
+++ trunk/oai-pmh_data_provider/data_provider_web/src/main/java/org/cipres/treebase/web/controllers/OAIPMHController.java	2010-09-16 15:44:51 UTC (rev 750)
@@ -241,7 +241,7 @@
 		    else
 		    	map.put("description",study.getNotes());
 			map.put("publisher", publisher);						
-			map.put("date", "published on "+citation.getPublishYear());
+			map.put("date", citation.getPublishYear());
 			map.put("identifier", "TreeBASE.org/study/TB2:s"+study.getId());
 			map.put("datestamp", study.getReleaseDate());
 			


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] SF.net SVN: treebase:[749] trunk

[<vga...@us...>]

Revision: 749
          http://treebase.svn.sourceforge.net/treebase/?rev=749&view=rev
Author:   vgapeyev
Date:     2010-09-15 18:35:46 +0000 (Wed, 15 Sep 2010)

Log Message:
-----------
Dropped trailing slash on some repository URLs -- seems to reduce verbiage during maven builds

Modified Paths:
--------------
    trunk/oai-pmh_data_provider/data_provider_web/pom.xml
    trunk/treebase-core/pom.xml
    trunk/treebase-web/pom.xml

Modified: trunk/oai-pmh_data_provider/data_provider_web/pom.xml
===================================================================
--- trunk/oai-pmh_data_provider/data_provider_web/pom.xml	2010-07-22 16:04:41 UTC (rev 748)
+++ trunk/oai-pmh_data_provider/data_provider_web/pom.xml	2010-09-15 18:35:46 UTC (rev 749)
@@ -49,7 +49,7 @@
       <plugin>
 	  	<groupId>org.codehaus.mojo</groupId>
 		<artifactId>tomcat-maven-plugin</artifactId>
-	  	<version>1.0-SNAPSHOT</version>
+	  	<version>1.0</version>
       </plugin>
 	  
     </plugins>

Modified: trunk/treebase-core/pom.xml
===================================================================
--- trunk/treebase-core/pom.xml	2010-07-22 16:04:41 UTC (rev 748)
+++ trunk/treebase-core/pom.xml	2010-09-15 18:35:46 UTC (rev 749)
@@ -35,13 +35,13 @@
 		<repository>
 			<id>m2.remote.repos</id>
 			<name>Maven 2 Remote Repository</name>
-			<url>http://treebase.sourceforge.net/maven2/</url>
+			<url>http://treebase.sourceforge.net/maven2</url>
 		</repository>
 
 		<repository>
 			<id>m2.nexml.repos</id>
 			<name>NeXML Remote Repository</name>
-			<url>http://nexml-dev.nescent.org/.m2/repository/</url>
+			<url>http://nexml-dev.nescent.org/.m2/repository</url>
 		</repository>
 
 	</repositories>

Modified: trunk/treebase-web/pom.xml
===================================================================
--- trunk/treebase-web/pom.xml	2010-07-22 16:04:41 UTC (rev 748)
+++ trunk/treebase-web/pom.xml	2010-09-15 18:35:46 UTC (rev 749)
@@ -55,7 +55,7 @@
 			<plugin>
 				<groupId>org.codehaus.mojo</groupId>
 				<artifactId>tomcat-maven-plugin</artifactId>
-				<version>1.0-SNAPSHOT</version>
+				<version>1.0</version>
 			</plugin>
 
 		</plugins>
@@ -72,17 +72,17 @@
 		<repository>
 			<id>m2.remote.repos</id>
 			<name>Maven 2 Remote Repository</name>
-			<url>http://treebase.sourceforge.net/maven2/</url>
+			<url>http://treebase.sourceforge.net/maven2</url>
 		</repository>
 
 		<repository>
 			<id>m2.nexml.repos</id>
 			<name>NeXML Remote Repository</name>
-			<url>http://nexml-dev.nescent.org/.m2/repository/</url>
+			<url>http://nexml-dev.nescent.org/.m2/repository</url>
 		</repository>
 		<repository>
 			<id>org.tuckey.urlrewritefilter</id>
-			<url>http://repository.jboss.org/maven2/org/tuckey/urlrewritefilter/</url>
+			<url>http://repository.jboss.org/maven2/org/tuckey/urlrewritefilter</url>
 		</repository>      
     	<repository>
       		<id>java.net</id>


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.

[Treebase-guts] [ treebase-Bugs-3033740 ] Some OAI records missing required datestamp element

[SourceForge.net <no...@so...>]

Bugs item #3033740, was opened at 2010-07-23 15:55
Message generated for change (Settings changed) made by hlapp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3033740&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: APIs
Group: None
Status: Open
>Priority: 7
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
>Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: Some OAI records missing required datestamp element

Initial Comment:
Some of the records coming out via OAI are missing a required datestamp element in their header.

For example: http://www.treebase.org/treebase-web/top/oai?verb=GetRecord&metadataPrefix=oai_dc&identifier=TB:s1236

These records have an empty <datestamp></datestamp> element, but this isn't acceptable according to the UTCdatetimeType defined in the schema.  The record, as a result, is not valid and may cause OAI ingesters to choke.

Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3033740&group_id=248804

[Treebase-guts] [ treebase-Bugs-3043116 ] DC Date in OAI records has "extra" text

[SourceForge.net <no...@so...>]

Bugs item #3043116, was opened at 2010-08-11 10:53
Message generated for change (Settings changed) made by hlapp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3043116&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: APIs
Group: None
Status: Open
>Priority: 7
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
>Assigned to: Vladimir Gapeyev (vgapeyev)
Summary: DC Date in OAI records has "extra" text

Initial Comment:
Hi, in harvesting TreeBASE, I've noticed the value for the dc:date element in the records has content like: "Published in 1983" instead of a plain date like "1983".  This is valid according to the OAI DC XML Schema, but DC does have a best practice that suggests that the content should just be a date in a standard encoding scheme: "Recommended best practice is to use an encoding scheme, such as the W3CDTF profile of ISO 8601" Cf. http://dublincore.org/documents/dces/

I think putting a date in dc:date that matches a standard encoding scheme would be less likely to cause problems with harvesters that make assumptions about data content based on best practice recommendations (many of the standard OAI tools do this unfortunately).  If you're only using dc:date for publication dates (is it currently used for anything else?) then people who harvest will be able to make that assumption without relying on the "Published in" text.

Thanks for considering this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3043116&group_id=248804

[Treebase-guts] [ treebase-Bugs-3043116 ] DC Date in OAI records has "extra" text

[SourceForge.net <no...@so...>]

Bugs item #3043116, was opened at 2010-08-11 07:53
Message generated for change (Tracker Item Submitted) made by ksclarke
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3043116&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: data
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Kevin S. Clarke (ksclarke)
Assigned to: Mark Dominus (mjdominus)
Summary: DC Date in OAI records has "extra" text

Initial Comment:
Hi, in harvesting TreeBASE, I've noticed the value for the dc:date element in the records has content like: "Published in 1983" instead of a plain date like "1983".  This is valid according to the OAI DC XML Schema, but DC does have a best practice that suggests that the content should just be a date in a standard encoding scheme: "Recommended best practice is to use an encoding scheme, such as the W3CDTF profile of ISO 8601" Cf. http://dublincore.org/documents/dces/

I think putting a date in dc:date that matches a standard encoding scheme would be less likely to cause problems with harvesters that make assumptions about data content based on best practice recommendations (many of the standard OAI tools do this unfortunately).  If you're only using dc:date for publication dates (is it currently used for anything else?) then people who harvest will be able to make that assumption without relying on the "Published in" text.

Thanks for considering this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3043116&group_id=248804

[Treebase-guts] [ treebase-Bugs-3039881 ] TreeBASE not consistent with PhyloWS

[SourceForge.net <no...@so...>]

Bugs item #3039881, was opened at 2010-08-05 08:46
Message generated for change (Tracker Item Submitted) made by rdmpage
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3039881&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: ui
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Roderic Page (rdmpage)
Assigned to: Nobody/Anonymous (nobody)
Summary: TreeBASE not consistent with PhyloWS

Initial Comment:
The TreeBASE implementation of PhyloWS searching doesn't match the "standard" and is internally inconsistent. According to https://www.nescent.org/wg_evoinfo/PhyloWS/REST the URL to search for a taxon is of the form http://purl.org/phylo/treebase/phylows/find/taxon?query= (note /find/taxon) whereas the TreeBASE API (https://sourceforge.net/apps/mediawiki/treebase/index.php?title=API#Searching) uses http://purl.org/phylo/treebase/phylows/taxon/find?query= (note taxon/find, the reverse order to the TreeBASE URL).

The RSS feed returned by a TreeBASE search uses the correct PhyloWS URI for <channel rdf:about>, not the actual URL used to perform the search. This means we can't, for example, query a triple store for a taxon using the same URI as we'd use to undertake a search. 

The solutiion I'd prefer is to make the TreeBASE search API follow PhyloWS, which would also make the search URI and results URI the same.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3039881&group_id=248804

[Treebase-guts] [ treebase-Bugs-3037233 ] NexML character matrix contains {}

[SourceForge.net <no...@so...>]

Bugs item #3037233, was opened at 2010-07-30 19:20
Message generated for change (Tracker Item Submitted) made by thasso
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3037233&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Thasso Griebel (thasso)
Assigned to: Nobody/Anonymous (nobody)
Summary: NexML character matrix contains {}

Initial Comment:
The matrix rows in nexml files contain nexus style ambiguities. For example {AT} instead of W. I am not sure if this is allowed, but the validator reports the files as invalid. Also note that there is a bug in the nexml library ( https://sourceforge.net/tracker/?func=detail&aid=3037220&group_id=209571&atid=1010249 ), so the ambiguity states for Y and H are wron in the resulting nexml file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3037233&group_id=248804

[Treebase-guts] [ treebase-Bugs-3037070 ] Invalid NexML output

[SourceForge.net <no...@so...>]

Bugs item #3037070, was opened at 2010-07-30 14:24
Message generated for change (Comment added) made by thasso
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3037070&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Thasso Griebel (thasso)
Assigned to: Nobody/Anonymous (nobody)
Summary: Invalid NexML output

Initial Comment:
Downloading a Study as NexML can result in an invalid XML file. 

The Study I tried was 1144 ( http://www.treebase.org/treebase-web/search/study/summary.html?id=1144 ).

I tried the online validator and the Java parser and both report the file as invalid. The problem seems to be the OTUs groups and the the tree blocks referencing the OTUs. The file for 1144 contains two OTUs (otus304719 and otus304819) and two tree blocks. The first tree block ( trees306715 ) references the OTUs otus304819, but contains a node that references an OTU ( otu304721 ) from the other OTUs group (otus304719). It should istead reference to otu304821.

The essential parts:
<nex:nexml>
  <otus about="#otus304719" id="otus304719" label="TaxonLabelSet11490">
    <otu about="#otu304721" id="otu304721" label="Megadyptes antipodes"></otu>
  </otus>
  <otus about="#otus304819" id="otus304819" label="TaxonLabelSet11489">
    <otu about="#otu304821" id="otu304821" label="Megadyptes antipodes"></otu>
  </otus>
  <trees about="#trees306715" id="trees306715" label="Tb6842" otus="otus304819">
    <tree>
      <node about="#node306728" id="node306728" label="Megadyptes antipodes" otu="otu304721"></node>
    </tree>
  </trees>
  <trees about="#trees306934" id="trees306934" label="Tb6843" otus="otus304719">
    <tree>
      <node about="#node306947" id="node306947" label="Megadyptes antipodes" otu="otu304721"></node>
    </tree>
  </trees>
</nex:nexml>

----------------------------------------------------------------------

Comment By: Thasso Griebel (thasso)
Date: 2010-07-30 16:44

Message:
I uploaded a patch as a first suggestion. Looks like the treeblock gets its
OTUs reference but the treecreation was actually searching all OTUs when
looking for taxon references. I just passed on the TreesBlock OTUs to the
tree creation and looking for references only in that OTUs.

Note that the whole project compiles, but the patch is untested. Is there
an easy way to get a test database running ?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3037070&group_id=248804

[Treebase-guts] [ treebase-Bugs-3037070 ] Invalid NexML output

[SourceForge.net <no...@so...>]

Bugs item #3037070, was opened at 2010-07-30 14:24
Message generated for change (Tracker Item Submitted) made by thasso
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3037070&group_id=248804

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Thasso Griebel (thasso)
Assigned to: Nobody/Anonymous (nobody)
Summary: Invalid NexML output

Initial Comment:
Downloading a Study as NexML can result in an invalid XML file. 

The Study I tried was 1144 ( http://www.treebase.org/treebase-web/search/study/summary.html?id=1144 ).

I tried the online validator and the Java parser and both report the file as invalid. The problem seems to be the OTUs groups and the the tree blocks referencing the OTUs. The file for 1144 contains two OTUs (otus304719 and otus304819) and two tree blocks. The first tree block ( trees306715 ) references the OTUs otus304819, but contains a node that references an OTU ( otu304721 ) from the other OTUs group (otus304719). It should istead reference to otu304821.

The essential parts:
<nex:nexml>
  <otus about="#otus304719" id="otus304719" label="TaxonLabelSet11490">
    <otu about="#otu304721" id="otu304721" label="Megadyptes antipodes"></otu>
  </otus>
  <otus about="#otus304819" id="otus304819" label="TaxonLabelSet11489">
    <otu about="#otu304821" id="otu304821" label="Megadyptes antipodes"></otu>
  </otus>
  <trees about="#trees306715" id="trees306715" label="Tb6842" otus="otus304819">
    <tree>
      <node about="#node306728" id="node306728" label="Megadyptes antipodes" otu="otu304721"></node>
    </tree>
  </trees>
  <trees about="#trees306934" id="trees306934" label="Tb6843" otus="otus304719">
    <tree>
      <node about="#node306947" id="node306947" label="Megadyptes antipodes" otu="otu304721"></node>
    </tree>
  </trees>
</nex:nexml>

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=1126676&aid=3037070&group_id=248804