treebase-devel — TreeBASE developer mailing list

[Treebase-devel] TreeBASE migration

[Rutger V. <rut...@gm...>]

Hi all,

the Naturalis ICT dept. are currently migrating the TreeBASE code base from
sourceforge to github. To this end they would like to set the sf.net repo
to read-only. The question now is whether there might still be people with
uncommitted code. I suspect this is not the case but I'd like to propose a
short grace period till Friday for any outstanding commits to be made until
we lock down the sf.net repo. If I am missing anyone that you know of that
should be alerted can you please let them know that this is happening?

Thanks!

Rutger

Re: [Treebase-devel] TreeBASE hosting at Naturalis

[Hilmar L. <hl...@du...>]

On 10/24/14, 10:19 AM, Rutger Vos wrote:
>
>>     - testing, i.e. do bug fixes and enhancements work as intended on the
>>     staging server (presumably this is shared, e.g. they deploy, others
>>     test & verify)
>
>     Congruent with NESCent hosting, except that we redeploy staging and
>     testing automatically upon commits.
>
>
> Is this Jenkins? I expect the ICT dept will just want to copy this setup.

Yes, that's Jenkins. -hilmar

-- 
Hilmar Lapp -:- genome.duke.edu -:- lappland.io

Re: [Treebase-devel] TreeBASE hosting at Naturalis

[Rutger V. <rut...@gm...>]

>
> Mostly ditto. However, TreeBASE still stores passwords in the clear, which
> is a major security flaw and vulnerability. There may be others waiting to
> be discovered. The code, to the extent I know, has never been
> security-audited. There is currently apparently zero funding for code
> maintenance, and so time will only reveal more security issues, not less,
> including issues caused by reliance on end-of-support versions of Java,
> Tomcat, etc.
>

Mmmm... yes, we never did get around to hashing the passwords, did we? I
think I will have to discuss this with them because they are quite big on
practicing due diligence on stuff like this.


>  - testing, i.e. do bug fixes and enhancements work as intended on the
> staging server (presumably this is shared, e.g. they deploy, others test &
> verify)
>
>
> Congruent with NESCent hosting, except that we redeploy staging and
> testing automatically upon commits.
>

Is this Jenkins? I expect the ICT dept will just want to copy this setup.


> - system/db administration, i.e. checking performance and tuning it (to
> the extent that this doesn't involve code modifications they would do this)
>
>
> Again, this would be congruent with NESCent hosting.
>

Cool, thanks!

Rutger

Re: [Treebase-devel] TreeBASE hosting at Naturalis

[Hilmar L. <hl...@ne...>]

On 10/23/14, 5:19 AM, Rutger Vos wrote:
> *they would like further clarification on a number of issues - so the
> following are important and require your feedback:*
> *- do they need to worry about copyright issues w.r.t. the abstracts in
> the database?*

No. They are covered under fair use.

> *- what exactly does the annual growth rate (60%/pa) refer to?*

Storage (flat file and database). (Which means it's not clear, and we don't
have good data, on how this growth translates to increased CPU and memory
needs.)

> *- what's in place in terms of automation (e.g. continuous integration)
> and how would they best be able to copy this?*

We use Jenkins for deployment. Staging and testing are auto-redeployed upon
each commit. Production is triggered manually.

  -hilmar
-- 
Hilmar Lapp -:- informatics.nescent.org/wiki -:- lappland.io

Re: [Treebase-devel] TreeBASE hosting at Naturalis

[Hilmar L. <hl...@ne...>]

On 10/23/14, 5:19 AM, Rutger Vos wrote:
> - the content, i.e. editing and reviewing submissions (presumably, they
> have nothing to do with that)

Congruent with NESCent hosting.

> - the code, i.e. bug fixes and feature additions (again, they don't plan
> to do anything with that)

Mostly ditto. However, TreeBASE still stores passwords in the clear, which
is a major security flaw and vulnerability. There may be others waiting to
be discovered. The code, to the extent I know, has never been
security-audited. There is currently apparently zero funding for code
maintenance, and so time will only reveal more security issues, not less,
including issues caused by reliance on end-of-support versions of Java,
Tomcat, etc.

> - testing, i.e. do bug fixes and enhancements work as intended on the
> staging server (presumably this is shared, e.g. they deploy, others test &
> verify)

Congruent with NESCent hosting, except that we redeploy staging and testing
automatically upon commits.

> - system/db administration, i.e. checking performance and tuning it (to
> the extent that this doesn't involve code modifications they would do this)

Again, this would be congruent with NESCent hosting.

  -hilmar

-- 
Hilmar Lapp -:- informatics.nescent.org/wiki -:- lappland.io

[Treebase-devel] TreeBASE hosting at Naturalis

[Rutger V. <rut...@gm...>]

Hi all,

here a quick status update on TreeBASE hosting at Naturalis, with some
pressing questions from our ICT dept at the bottom:

The "project bureau" of the Naturalis ICT department has considered hosting
TreeBASE and would be keen to do this, though they are aware that they are
not the only candidates for this so they are now developing a plan for what
hosting would mean and whether they can offer something that we can all
commit to with confidence.

The basic idea is that they would replicate as much as possible the
practices and configuration that were developed at NESCent, but folding it
a bit more into the ICT departments workflow and way of doing this. This
might include things such as:
- move code to github
- decide whether to adopt Jenkins (or something else)
- expand automated provisioning of dev/stage/prod servers using puppet

They want to make very, very specific agreements about who's responsible
for what. As far as they've been able to tell, this includes the following
topics, with they responsibility between parentheses:
- the content, i.e. editing and reviewing submissions (presumably, they
have nothing to do with that)
- the code, i.e. bug fixes and feature additions (again, they don't plan to
do anything with that)
- testing, i.e. do bug fixes and enhancements work as intended on the
staging server (presumably this is shared, e.g. they deploy, others test &
verify)
- system/db administration, i.e. checking performance and tuning it (to the
extent that this doesn't involve code modifications they would do this)

*And they would like further clarification on a number of issues - so the
following are important and require your feedback:*
*- do they need to worry about copyright issues w.r.t. the abstracts in the
database?*
*- what exactly does the annual growth rate (60%/pa) refer to?*
*- what's in place in terms of automation (e.g. continuous integration) and
how would they best be able to copy this?*

Thanks,

Rutger

Re: [Treebase-devel] TreeBase NeXML export bug: ids of otu elements are not unique

[William P. <wil...@ya...>]

In an ideal world, submitters would upload matrices and trees in the same Mesquite project, all sharing the same taxon block. This would prevent the proliferation of redundant taxon sets. In reality, matrices and trees are often uploaded separately, and so long as the set of taxon labels in the trees are an equal or smaller subset of the set in the matrices, the trees can be linked to the matrices by way of an analysis (a rule that is, regrettably, incompatible with *BEAST or Mesquite's taxon-association table). 

When users click the "download submission as NEXUS" any redundant taxon sets are consolidated into single taxon blocks that become shared by compatible trees and matrices. This does not happen when users click the "download submission as NeXML."

Some day it would be nice to express the analysis in NEXML -- i.e. the relationship among matrices and trees, such as which is a derivative of which, along with analysis metadata.  Given TreeBASE's rule that all analysis objects share the same taxon set, the analysis records could form the basis for shared taxon label elements. (course, this is all very kluge-like; ideally we would simply rely on a more direct object-element mapping). 

bp


On Jan 13, 2014, at 8:53 PM, Rutger Vos <rut...@gm...> wrote:

> For posterity and in the interest of completeness: 
> 
> When TreeBASE reads such a NEXUS file with namespaced taxon labels (TaxonLabel) it will scope the strings to the study, and will attempt to give them normalized references to at least the level of a commonly-occurring "variant" of a name (TaxonVariant), and ideally (most commonly) a known, true Taxon.
> 
> Subsequently, references to these TaxonLabel objects are collected into multiple TaxonLabelSet bins, which represent the encountered taxa blocks.
> 
> When NeXML is generated, the generic (not entirely thought-through) approach has been to generate all XML identifiers from the primary keys in the database. This might fail to generate unique XML IDs for otu elements in studies where the same TaxonLabel occurs in multiple TaxonLabelSet bins (which are serialized to multiple "otus" elements).
> 
> A possible fix would be to generate XML IDs for otu elements by combining the TaxonLabel primary key with the TaxonLabelSet PK. Unfortunately, this will be a bit messy to get right because the ID references (from tree node to otu, from sequence to otu) need to generate the correct string as well.

Re: [Treebase-devel] TreeBase NeXML export bug: ids of otu elements are not unique

[Rutger V. <rut...@gm...>]

For posterity and in the interest of completeness:

When TreeBASE reads such a NEXUS file with namespaced taxon labels
(TaxonLabel) it will scope the strings to the study, and will attempt to
give them normalized references to at least the level of a
commonly-occurring "variant" of a name (TaxonVariant), and ideally (most
commonly) a known, true Taxon.

Subsequently, references to these TaxonLabel objects are collected into
multiple TaxonLabelSet bins, which represent the encountered taxa blocks.

When NeXML is generated, the generic (not entirely thought-through)
approach has been to generate all XML identifiers from the primary keys in
the database. This might fail to generate unique XML IDs for otu elements
in studies where the same TaxonLabel occurs in multiple TaxonLabelSet bins
(which are serialized to multiple "otus" elements).

A possible fix would be to generate XML IDs for otu elements by combining
the TaxonLabel primary key with the TaxonLabelSet PK. Unfortunately, this
will be a bit messy to get right because the ID references (from tree node
to otu, from sequence to otu) need to generate the correct string as well.


On Thu, Jan 2, 2014 at 9:15 PM, Mark Holder <mth...@gm...> wrote:

> Hi,
> The example is the ID  Tl241048 in
>
> http://treebase.org/treebase-web/search/downloadAStudy.html?id=10000&format=nexml
>
> The nexus output has multiple taxa blocks and the taxon name
> Cardamine_alpina_DAE2_R325 (which is associated with the offending otu
> elements in the nexml form) occurs in multiple taxa blocks. In the
> (mesquitized) form of NEXUS that allows multiple TAXA blocks, the
> block TITLE acts like a namespace. So identical taxa labels are not a
> problem in NEXUS. But in nexml spec ids are supposed to be unique
> throughout the file.
>
> I suspect this difference in approaches to dealing with disambiguation
> may be the source of the bug.
>
>
> all the best,
> Mark
>
> --
> Mark Holder
>
> mth...@gm...
> mth...@ku...
> http://phylo.bio.ku.edu/mark-holder
>
> ==============================================
> Department of Ecology and Evolutionary Biology
> University of Kansas
> 6031 Haworth Hall
> 1200 Sunnyside Avenue
> Lawrence, Kansas 66045
>
> lab phone:  785.864.5789
> fax (shared): 785.864.5860
> ==============================================
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
> Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



-- 

Dr. Rutger A. Vos
Bioinformaticist
Naturalis Biodiversity Center
Visiting address: Office A109, Einsteinweg 2, 2333 CC, Leiden, the
Netherlands
Mailing address: Postbus 9517, 2300 RA, Leiden, the Netherlands
http://rutgervos.blogspot.com

Re: [Treebase-devel] nexml and treebase status

[Rutger V. <rut...@gm...>]

>         1544 files (43%) had errors associated with using repeated IDs. I
> think that in most of these cases the taxa with repeated IDs were just
> shallow copies of the same data.
>

Yes. TreeBASE studies can have multiple TaxonLabelSets (~taxa blocks), that
can (and do) reference the same taxon labels. I guess the IDs that are
generated from the taxon label objects should therefore become namespaced
to the containing taxon label set. This will get ugly, though, when
generating the OTU ID references in tree nodes and character rows.

        2 studies failed to emit a <char> element in the matrix
>

>From the web interface it also appears that there are NO characters in
these matrices. Not sure what's up with that.


>         6 studies used lower case in a DNA or AA matrix.
>

This ought to be trivially fixable.


>
> There were a fair number of studies (372) that look like rows from the
> NEXUS file were simply exported as the text of a NeXML seq element.
> This is not allowed if there is special NEXUS-specific syntax in the
> matrix. Also augmenting the DNA or AA type with additional character
> state symbols is not allowed.
>

Yes, this is a huge bug bear. TreeBASE character sequences are
de-normalized (i.e. just opaque strings), because proper normalization of
ambiguous character states is nowhere near scalable. It is conceivable that
the nexml generator classes could parse those strings and replace the
ambiguity tokens (e.g. () {}) with IUPAC symbols but I'm guessing this will
only be non-painful, performance-wise, if the generated nexml is
subsequently cached by the web application.


> The remaining 1 % of files look like more serious errors in TreeBase
> code (empty studies or stack traces) or corrupt data in the db.
>

There are some studies where the web application times out (too much data),
and then there are some where the data appears to be corrupted somehow.
Unfortunately.

[Treebase-devel] nexml and treebase status

[Mark H. <mth...@gm...>]

Hi,
I'm crossposting this to NeXML and TreeBASE lists because I think that
TreeBASE could be the largest supplier of NeXML files.

In the process of working on a roundrip test for a NeXML->JSON tool
for the open tree project, I've been doing some more characterizations
of the nexml-validity of the files in Rutger's supertreebase repo (
https://github.com/rvosa/supertreebase ). My understanding is that
these were all produced via export from TreeBase. In a couple of cases
I re-downloaded the files from the TreeBase API to verify that they
were the same as the versions that Rutger committed, but I mainly used
his cached versions in the git repo to avoid hammering the treebase
servers.

The validation was done against a version of the nexml schema which
would be generated if someone with nexml-repo admin privileges were to
accept my recent pull request (https://github.com/nexml/nexml/pull/8
).

The tool used for validation was the xml-validator-1.0 tool (available
from http://code.google.com/p/xml-validator/).

Results:
The bad news that only 1652 files (about 46%) validate successfully.

The good news is that many of the errors appear to be easy to fix:
they seem to just be related to how the NexML is composed rather than
any deeper problem in the data or TreeBase. In many cases, I think
that a lax NeXML parser will accept these errors and interpret the
data correctly. For example:
	1544 files (43%) had errors associated with using repeated IDs. I
think that in most of these cases the taxa with repeated IDs were just
shallow copies of the same data.
	2 studies failed to emit a <char> element in the matrix
	6 studies used lower case in a DNA or AA matrix.

There were a fair number of studies (372) that look like rows from the
NEXUS file were simply exported as the text of a NeXML seq element.
This is not allowed if there is special NEXUS-specific syntax in the
matrix. Also augmenting the DNA or AA type with additional character
state symbols is not allowed.

The remaining 1 % of files look like more serious errors in TreeBase
code (empty studies or stack traces) or corrupt data in the db.

Some more details can be found at:
http://phylo.bio.ku.edu/public/supertreebase.txt

I submitted treebase bug reports.

all the best,
Mark

-- 
Mark Holder

mth...@gm...
mth...@ku...
http://phylo.bio.ku.edu/mark-holder

==============================================
Department of Ecology and Evolutionary Biology
University of Kansas
6031 Haworth Hall
1200 Sunnyside Avenue
Lawrence, Kansas 66045

lab phone:  785.864.5789
fax (shared): 785.864.5860
==============================================

[Treebase-devel] TreeBase NeXML export bug: ids of otu elements are not unique

[Mark H. <mth...@gm...>]

Hi,
The example is the ID  Tl241048 in
http://treebase.org/treebase-web/search/downloadAStudy.html?id=10000&format=nexml

The nexus output has multiple taxa blocks and the taxon name
Cardamine_alpina_DAE2_R325 (which is associated with the offending otu
elements in the nexml form) occurs in multiple taxa blocks. In the
(mesquitized) form of NEXUS that allows multiple TAXA blocks, the
block TITLE acts like a namespace. So identical taxa labels are not a
problem in NEXUS. But in nexml spec ids are supposed to be unique
throughout the file.

I suspect this difference in approaches to dealing with disambiguation
may be the source of the bug.


all the best,
Mark

-- 
Mark Holder

mth...@gm...
mth...@ku...
http://phylo.bio.ku.edu/mark-holder

==============================================
Department of Ecology and Evolutionary Biology
University of Kansas
6031 Haworth Hall
1200 Sunnyside Avenue
Lawrence, Kansas 66045

lab phone:  785.864.5789
fax (shared): 785.864.5860
==============================================

[Treebase-devel] Fwd: SourceForge Project Upgrade - Code Repo Complete

[Hilmar L. <hl...@dr...>]

I finally got around to verify we have the project stats and to upgrade the repository to SourceForge's new hosting platform.

This means that anyone who has a checkout of the repository will need to do a fresh checkout, as per below.

Mattison - this may also mean that we need to change the checkout URLs for the respective Jenkins deployment tasks, although according to the below the old URL is still OK for read-only checkout.

	-hilmar

Begin forwarded message:

> From: SourceForge.net <nor...@in...>
> Subject: SourceForge Project Upgrade - Code Repo Complete
> Date: March 24, 2013 9:25:00 PM EDT
> To: no...@in...
> Reply-To: no...@in...
> 
> Your code repository in upgraded project treebase is now ready for use.
> 
> Old repository url: http://treebase.svn.sourceforge.net/svnroot/treebase 
> 
> New repository checkout command: svn checkout --username=hlapp svn+ssh://hl...@sv.../p/treebase/code/trunk treebase-code 
> 
> You should do a checkout using the new repository location.  The old repository is read-only now.
> 
> For more detailed instructions on migrating to your new repo, please see https://sourceforge.net/p/forge/community-docs/Repository%20Upgrade%20FAQ/
> 
> -- 
> SourceForge.net has sent this mailing to you as a registered user of
> the SourceForge.net site to convey important information regarding
> your SourceForge.net account or your use of SourceForge.net services.
> If you have concerns about this mailing please contact our Support
> team per: http://sourceforge.net/support

-- 
===========================================================
: Hilmar Lapp -:- Durham, NC -:- hlapp at drycafe dot net :
===========================================================

Re: [Treebase-devel] time to update TreeBase contributor page?

[Mattison W. <mat...@ne...>]

It built ok and is deployed now on treebase-dev.

-Mattison

On Tue, Jan 22, 2013 at 8:07 AM, Hilmar Lapp <hl...@ne...> wrote:
> I have now copied the main trunk to branches/simple-search-UI, undone the
> commits 1084 through 1091 on the main trunk, and undone commits 1092 through
> 1095 on the simple-search-UI branch (so that the problematics series of
> commits are the only ones on that branch after branching off).
>
> I can't test whether the main trunk builds correctly now - could someone
> check that?
>
> -hilmar
>
> On Jan 16, 2013, at 4:28 AM, Rutger Vos wrote:
>
> If they can be forked off that would be a good thing - I apologize for the
> mess, but it is actually good enough to keep in a branch somewhere so that
> some day, somebody might return to it. The idea was to make the search
> interfaces a lot simpler.
>
>
> On Tue, Jan 15, 2013 at 8:29 PM, Hilmar Lapp <hl...@ne...> wrote:
>>
>>
>> On Jan 14, 2013, at 6:03 PM, William Piel wrote:
>>
>> Rutger's additions (revisions 1085-1093) need to be forked off and
>> reintroduced later when some other problems that they introduce can be
>> fixed, and the current state of the code rewound to revision 1084.
>>
>>
>> In the svn log I can see Rutger's changes only go to r1091:
>>
>> ------------------------------------------------------------------------
>> r1084 | rvos | 2012-02-21 07:52:13 -0500 (Tue, 21 Feb 2012) | 1 line
>>
>> Made link to plus icon relative. Trivial commit.
>> ------------------------------------------------------------------------
>> r1085 | rvos | 2012-02-21 17:38:31 -0500 (Tue, 21 Feb 2012) | 1 line
>>
>> Defensive code against NPEs
>> ------------------------------------------------------------------------
>> r1086 | rvos | 2012-02-22 14:24:32 -0500 (Wed, 22 Feb 2012) | 1 line
>>
>> Deploying simple search box for all searches.
>> ------------------------------------------------------------------------
>> r1087 | rvos | 2012-02-23 10:25:03 -0500 (Thu, 23 Feb 2012) | 1 line
>>
>> Removing code dealing with old-style search forms
>> ------------------------------------------------------------------------
>> r1088 | rvos | 2012-02-23 10:43:14 -0500 (Thu, 23 Feb 2012) | 1 line
>>
>> Added test to ensure we don't get NullPointerExceptions
>> ------------------------------------------------------------------------
>> r1089 | rvos | 2012-02-23 13:59:11 -0500 (Thu, 23 Feb 2012) | 1 line
>>
>> Correcting a bug I introduced: onSubmit() is also visited when buttons
>> that modify the results set are pressed (i.e. there is no CQL query to run).
>> In such cases handling the request should be delegated to super.onSubmit().
>> ------------------------------------------------------------------------
>> r1090 | rvos | 2012-02-23 14:07:30 -0500 (Thu, 23 Feb 2012) | 1 line
>>
>> Actually, we should probably check if query is an empty string (but not
>> null).
>> ------------------------------------------------------------------------
>> r1091 | rvos | 2012-02-23 15:03:22 -0500 (Thu, 23 Feb 2012) | 1 line
>>
>> This commit should fix the NPE we were getting at
>> SearchResultsFrozen.java:41 because the result set should now no longer
>> contain null values (it could be an empty set, though). To ensure that this
>> is so and everything behaves as it should I am removing the cludge around
>> line 176.
>> ------------------------------------------------------------------------
>> r1092 | sfrgpiel | 2012-04-03 10:24:26 -0400 (Tue, 03 Apr 2012) | 1 line
>>
>> Fixing broken links, fixing spelling, etc.
>> ------------------------------------------------------------------------
>> r1093 | hshyket | 2012-05-25 11:25:44 -0400 (Fri, 25 May 2012) | 1 line
>>
>> Removing Caching
>> ------------------------------------------------------------------------
>> r1094 | hlapp | 2012-06-11 14:10:38 -0400 (Mon, 11 Jun 2012) | 2 lines
>>
>> Fixed spelling error reported by Michael Jones.
>>
>> ------------------------------------------------------------------------
>> r1095 | hshyket | 2012-06-15 11:19:11 -0400 (Fri, 15 Jun 2012) | 1 line
>>
>> Updating PostgreSQL dependency to 8.4.  Excluding geronimo-servlet and
>> servlet-api from dependecies.
>> ------------------------------------------------------------------------
>>
>> Should r1085-r1091 be excised?
>>
>> -hilmar
>>
>> --
>> ===========================================================
>> : Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
>> ===========================================================
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS
>> and more. Get SQL Server skills now (including 2012) with LearnDevNow -
>> 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
>> SALE $99.99 this month only - learn more at:
>> http://p.sf.net/sfu/learnmore_122512
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>>
>
>
>
> --
> Dr. Rutger A. Vos
> Bioinformaticist
> NCB Naturalis
> Visiting address: Office A109, Einsteinweg 2, 2333 CC, Leiden, the
> Netherlands
> Mailing address: Postbus 9517, 2300 RA, Leiden, the Netherlands
> http://rutgervos.blogspot.com
>
>
> --
> ===========================================================
> : Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
> ===========================================================
>
>
>
>
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. ON SALE this month only -- learn more at:
> http://p.sf.net/sfu/learnnow-d2d
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



--
Mattison Ward
NESCent at Duke University
2024 W. Main Street, Suite A200
Durham, NC 27705-4667
919-668-4585 (desk)
919-668-4551 (alternate)
919-668-9198 (fax)

Re: [Treebase-devel] time to update TreeBase contributor page?

[Hilmar L. <hl...@ne...>]

I have now copied the main trunk to branches/simple-search-UI, undone the commits 1084 through 1091 on the main trunk, and undone commits 1092 through 1095 on the simple-search-UI branch (so that the problematics series of commits are the only ones on that branch after branching off).

I can't test whether the main trunk builds correctly now - could someone check that?

	-hilmar

On Jan 16, 2013, at 4:28 AM, Rutger Vos wrote:

> If they can be forked off that would be a good thing - I apologize for the mess, but it is actually good enough to keep in a branch somewhere so that some day, somebody might return to it. The idea was to make the search interfaces a lot simpler.
> 
> 
> On Tue, Jan 15, 2013 at 8:29 PM, Hilmar Lapp <hl...@ne...> wrote:
> 
> On Jan 14, 2013, at 6:03 PM, William Piel wrote:
> 
>> Rutger's additions (revisions 1085-1093) need to be forked off and reintroduced later when some other problems that they introduce can be fixed, and the current state of the code rewound to revision 1084. 
> 
> 
> In the svn log I can see Rutger's changes only go to r1091:
> 
> ------------------------------------------------------------------------
> r1084 | rvos | 2012-02-21 07:52:13 -0500 (Tue, 21 Feb 2012) | 1 line
> 
> Made link to plus icon relative. Trivial commit.
> ------------------------------------------------------------------------
> r1085 | rvos | 2012-02-21 17:38:31 -0500 (Tue, 21 Feb 2012) | 1 line
> 
> Defensive code against NPEs
> ------------------------------------------------------------------------
> r1086 | rvos | 2012-02-22 14:24:32 -0500 (Wed, 22 Feb 2012) | 1 line
> 
> Deploying simple search box for all searches.
> ------------------------------------------------------------------------
> r1087 | rvos | 2012-02-23 10:25:03 -0500 (Thu, 23 Feb 2012) | 1 line
> 
> Removing code dealing with old-style search forms
> ------------------------------------------------------------------------
> r1088 | rvos | 2012-02-23 10:43:14 -0500 (Thu, 23 Feb 2012) | 1 line
> 
> Added test to ensure we don't get NullPointerExceptions
> ------------------------------------------------------------------------
> r1089 | rvos | 2012-02-23 13:59:11 -0500 (Thu, 23 Feb 2012) | 1 line
> 
> Correcting a bug I introduced: onSubmit() is also visited when buttons that modify the results set are pressed (i.e. there is no CQL query to run). In such cases handling the request should be delegated to super.onSubmit().
> ------------------------------------------------------------------------
> r1090 | rvos | 2012-02-23 14:07:30 -0500 (Thu, 23 Feb 2012) | 1 line
> 
> Actually, we should probably check if query is an empty string (but not null).
> ------------------------------------------------------------------------
> r1091 | rvos | 2012-02-23 15:03:22 -0500 (Thu, 23 Feb 2012) | 1 line
> 
> This commit should fix the NPE we were getting at SearchResultsFrozen.java:41 because the result set should now no longer contain null values (it could be an empty set, though). To ensure that this is so and everything behaves as it should I am removing the cludge around line 176.
> ------------------------------------------------------------------------
> r1092 | sfrgpiel | 2012-04-03 10:24:26 -0400 (Tue, 03 Apr 2012) | 1 line
> 
> Fixing broken links, fixing spelling, etc. 
> ------------------------------------------------------------------------
> r1093 | hshyket | 2012-05-25 11:25:44 -0400 (Fri, 25 May 2012) | 1 line
> 
> Removing Caching
> ------------------------------------------------------------------------
> r1094 | hlapp | 2012-06-11 14:10:38 -0400 (Mon, 11 Jun 2012) | 2 lines
> 
> Fixed spelling error reported by Michael Jones.
> 
> ------------------------------------------------------------------------
> r1095 | hshyket | 2012-06-15 11:19:11 -0400 (Fri, 15 Jun 2012) | 1 line
> 
> Updating PostgreSQL dependency to 8.4.  Excluding geronimo-servlet and servlet-api from dependecies.
> ------------------------------------------------------------------------
> 
> Should r1085-r1091 be excised?
> 
> 	-hilmar
> 
> -- 
> ===========================================================
> : Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
> ===========================================================
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS
> and more. Get SQL Server skills now (including 2012) with LearnDevNow -
> 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
> SALE $99.99 this month only - learn more at:
> http://p.sf.net/sfu/learnmore_122512
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
> 
> 
> 
> 
> -- 
> Dr. Rutger A. Vos
> Bioinformaticist
> NCB Naturalis
> Visiting address: Office A109, Einsteinweg 2, 2333 CC, Leiden, the Netherlands
> Mailing address: Postbus 9517, 2300 RA, Leiden, the Netherlands
> http://rutgervos.blogspot.com

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] time to update TreeBase contributor page?

[Rutger V. <rut...@gm...>]

If they can be forked off that would be a good thing - I apologize for the
mess, but it is actually good enough to keep in a branch somewhere so that
some day, somebody might return to it. The idea was to make the search
interfaces a lot simpler.


On Tue, Jan 15, 2013 at 8:29 PM, Hilmar Lapp <hl...@ne...> wrote:

>
> On Jan 14, 2013, at 6:03 PM, William Piel wrote:
>
> Rutger's additions (revisions 1085-1093) need to be forked off and
> reintroduced later when some other problems that they introduce can be
> fixed, and the current state of the code rewound to revision 1084.
>
>
> In the svn log I can see Rutger's changes only go to r1091:
>
> ------------------------------------------------------------------------
> r1084 | rvos | 2012-02-21 07:52:13 -0500 (Tue, 21 Feb 2012) | 1 line
>
> Made link to plus icon relative. Trivial commit.
> ------------------------------------------------------------------------
> r1085 | rvos | 2012-02-21 17:38:31 -0500 (Tue, 21 Feb 2012) | 1 line
>
> Defensive code against NPEs
> ------------------------------------------------------------------------
> r1086 | rvos | 2012-02-22 14:24:32 -0500 (Wed, 22 Feb 2012) | 1 line
>
> Deploying simple search box for all searches.
> ------------------------------------------------------------------------
> r1087 | rvos | 2012-02-23 10:25:03 -0500 (Thu, 23 Feb 2012) | 1 line
>
> Removing code dealing with old-style search forms
> ------------------------------------------------------------------------
> r1088 | rvos | 2012-02-23 10:43:14 -0500 (Thu, 23 Feb 2012) | 1 line
>
> Added test to ensure we don't get NullPointerExceptions
> ------------------------------------------------------------------------
> r1089 | rvos | 2012-02-23 13:59:11 -0500 (Thu, 23 Feb 2012) | 1 line
>
> Correcting a bug I introduced: onSubmit() is also visited when buttons
> that modify the results set are pressed (i.e. there is no CQL query to
> run). In such cases handling the request should be delegated to
> super.onSubmit().
> ------------------------------------------------------------------------
> r1090 | rvos | 2012-02-23 14:07:30 -0500 (Thu, 23 Feb 2012) | 1 line
>
> Actually, we should probably check if query is an empty string (but not
> null).
> ------------------------------------------------------------------------
> r1091 | rvos | 2012-02-23 15:03:22 -0500 (Thu, 23 Feb 2012) | 1 line
>
> This commit should fix the NPE we were getting at
> SearchResultsFrozen.java:41 because the result set should now no longer
> contain null values (it could be an empty set, though). To ensure that this
> is so and everything behaves as it should I am removing the cludge around
> line 176.
> ------------------------------------------------------------------------
> r1092 | sfrgpiel | 2012-04-03 10:24:26 -0400 (Tue, 03 Apr 2012) | 1 line
>
> Fixing broken links, fixing spelling, etc.
> ------------------------------------------------------------------------
> r1093 | hshyket | 2012-05-25 11:25:44 -0400 (Fri, 25 May 2012) | 1 line
>
> Removing Caching
> ------------------------------------------------------------------------
> r1094 | hlapp | 2012-06-11 14:10:38 -0400 (Mon, 11 Jun 2012) | 2 lines
>
> Fixed spelling error reported by Michael Jones.
>
> ------------------------------------------------------------------------
> r1095 | hshyket | 2012-06-15 11:19:11 -0400 (Fri, 15 Jun 2012) | 1 line
>
> Updating PostgreSQL dependency to 8.4.  Excluding geronimo-servlet and
> servlet-api from dependecies.
> ------------------------------------------------------------------------
>
> Should r1085-r1091 be excised?
>
> -hilmar
>
> --
> ===========================================================
> : Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
> ===========================================================
>
>
>
>
>
> ------------------------------------------------------------------------------
> Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS
> and more. Get SQL Server skills now (including 2012) with LearnDevNow -
> 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
> SALE $99.99 this month only - learn more at:
> http://p.sf.net/sfu/learnmore_122512
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
>


-- 
Dr. Rutger A. Vos
Bioinformaticist
NCB Naturalis
Visiting address: Office A109, Einsteinweg 2, 2333 CC, Leiden, the
Netherlands
Mailing address: Postbus 9517, 2300 RA, Leiden, the Netherlands
http://rutgervos.blogspot.com

Re: [Treebase-devel] time to update TreeBase contributor page?

[Hilmar L. <hl...@ne...>]

On Jan 14, 2013, at 6:03 PM, William Piel wrote:

> Rutger's additions (revisions 1085-1093) need to be forked off and reintroduced later when some other problems that they introduce can be fixed, and the current state of the code rewound to revision 1084. 


In the svn log I can see Rutger's changes only go to r1091:

------------------------------------------------------------------------
r1084 | rvos | 2012-02-21 07:52:13 -0500 (Tue, 21 Feb 2012) | 1 line

Made link to plus icon relative. Trivial commit.
------------------------------------------------------------------------
r1085 | rvos | 2012-02-21 17:38:31 -0500 (Tue, 21 Feb 2012) | 1 line

Defensive code against NPEs
------------------------------------------------------------------------
r1086 | rvos | 2012-02-22 14:24:32 -0500 (Wed, 22 Feb 2012) | 1 line

Deploying simple search box for all searches.
------------------------------------------------------------------------
r1087 | rvos | 2012-02-23 10:25:03 -0500 (Thu, 23 Feb 2012) | 1 line

Removing code dealing with old-style search forms
------------------------------------------------------------------------
r1088 | rvos | 2012-02-23 10:43:14 -0500 (Thu, 23 Feb 2012) | 1 line

Added test to ensure we don't get NullPointerExceptions
------------------------------------------------------------------------
r1089 | rvos | 2012-02-23 13:59:11 -0500 (Thu, 23 Feb 2012) | 1 line

Correcting a bug I introduced: onSubmit() is also visited when buttons that modify the results set are pressed (i.e. there is no CQL query to run). In such cases handling the request should be delegated to super.onSubmit().
------------------------------------------------------------------------
r1090 | rvos | 2012-02-23 14:07:30 -0500 (Thu, 23 Feb 2012) | 1 line

Actually, we should probably check if query is an empty string (but not null).
------------------------------------------------------------------------
r1091 | rvos | 2012-02-23 15:03:22 -0500 (Thu, 23 Feb 2012) | 1 line

This commit should fix the NPE we were getting at SearchResultsFrozen.java:41 because the result set should now no longer contain null values (it could be an empty set, though). To ensure that this is so and everything behaves as it should I am removing the cludge around line 176.
------------------------------------------------------------------------
r1092 | sfrgpiel | 2012-04-03 10:24:26 -0400 (Tue, 03 Apr 2012) | 1 line

Fixing broken links, fixing spelling, etc. 
------------------------------------------------------------------------
r1093 | hshyket | 2012-05-25 11:25:44 -0400 (Fri, 25 May 2012) | 1 line

Removing Caching
------------------------------------------------------------------------
r1094 | hlapp | 2012-06-11 14:10:38 -0400 (Mon, 11 Jun 2012) | 2 lines

Fixed spelling error reported by Michael Jones.

------------------------------------------------------------------------
r1095 | hshyket | 2012-06-15 11:19:11 -0400 (Fri, 15 Jun 2012) | 1 line

Updating PostgreSQL dependency to 8.4.  Excluding geronimo-servlet and servlet-api from dependecies.
------------------------------------------------------------------------

Should r1085-r1091 be excised?

	-hilmar

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] time to update TreeBase contributor page?

[William P. <wil...@ya...>]

On Jan 15, 2013, at 2:47 AM, Vision, Todd J wrote:

> Hi all,
> 
> Just looking at the contributor page on TreeBASE and noticed that some of these people moved on a while ago, while others have since joined:
> http://treebase.org/treebase-web/people.html
> 
> cheers,
> Todd

Yes, this requires an update. 

Before any new commits can be made, someone needs to do a bit of surgery to the code: Rutger's additions (revisions 1085-1093) need to be forked off and reintroduced later when some other problems that they introduce can be fixed, and the current state of the code rewound to revision 1084. 

bp

Re: [Treebase-devel] time to update TreeBase contributor page?

[Hilmar L. <hl...@ne...>]

And there seem to be a lot of past volunteers that have yet to sign up :)

On Jan 14, 2013, at 12:47 PM, Vision, Todd J wrote:

> Hi all,
> 
> Just looking at the contributor page on TreeBASE and noticed that some of these people moved on a while ago, while others have since joined:
> http://treebase.org/treebase-web/people.html
> 
> cheers,
> Todd
> 
> Todd Vision
> Associate Prof of Biology, UNC Chapel Hill
> Associate Director of Informatics, NESCent
> http://visionlab.unc.edu
> 
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> MVPs and experts. SALE $99.99 this month only -- learn more at:
> http://p.sf.net/sfu/learnmore_122412_______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

[Treebase-devel] time to update TreeBase contributor page?

[Vision, T. J <tj...@bi...>]

Hi all,

Just looking at the contributor page on TreeBASE and noticed that some of these people moved on a while ago, while others have since joined:
http://treebase.org/treebase-web/people.html

cheers,
Todd

Todd Vision
Associate Prof of Biology, UNC Chapel Hill
Associate Director of Informatics, NESCent
http://visionlab.unc.edu

Re: [Treebase-devel] Reviewer access broken

[William P. <wil...@ya...>]

On Jan 7, 2013, at 12:34 PM, Mattison Ward wrote:

> I disabled the rule and it seems to be working for me now.

great! thanks.

bp

Re: [Treebase-devel] Reviewer access broken

[William P. <wil...@ya...>]

On Dec 19, 2012, at 4:12 PM, Mattison Ward wrote:

> I can disable load balancing for download requests where the
> querystring includes x-access-code.   Is that worth a try?
> 
> The other option would be to load balance everything rather than just
> downloads.  I can direct requests to the backend tomcat instances
> based on JSESSIONID.  I think it would be fairly easy to isolate
> submissions from Dryad and make sure they always go to a primary
> tomcat instance if that is the only impediment.
> 
> Regards,
> 
> Mattison

Hi Mattison,

I just noticed that the auto-complete feature when entering a journal name in the citation section does not produce a pop-up list as it should. Instead it generates a "Session Error" warning dialog with each letter that I press.  Might this new problem be related to the load balancing changes? 

Attached are screen shots for treebase-dev and treebase, both entering a journal name in the exact same submission record. In the case of treebase-dev it works; in the case of production treebase it generates the Session Error. 

The way to test this is to log in, create a new submission, click on the "citation" item in the toolbox on the right, and then try typing something in the "Journal Name" field. 

regards,

Bill



TreeBASE Dev:



Public TreeBASE:

Re: [Treebase-devel] treebase.org redirecting to dev!

[Mattison W. <mat...@ne...>]

This was an error on my part, but only lasted a few minutes.  I will
check the treebase-dev logs for that time and let Bill know if anyone
submitted anything to the dev system during that time.

Regards,

Mattison

On Fri, Dec 21, 2012 at 9:24 AM, William Piel <wil...@ya...> wrote:
>
> This is a serious problem: When I set my browser to treebase.org it gets redirected to http://treebase-dev.nescent.org
>
> People will start submitted to our dev site, which is really bad. Please revert back to hitting our normal server (or if that's out of order, we should disable login on dev to prevent people from submitting).
>
> Bill
>
>
>

[Treebase-devel] treebase.org redirecting to dev!

[William P. <wil...@ya...>]

This is a serious problem: When I set my browser to treebase.org it gets redirected to http://treebase-dev.nescent.org 

People will start submitted to our dev site, which is really bad. Please revert back to hitting our normal server (or if that's out of order, we should disable login on dev to prevent people from submitting). 

Bill

Re: [Treebase-devel] Reviewer access broken

[Shyket, H. <har...@ya...>]

Hi Mattison,

I agree with Bill.  As for the second suggestion I would say let's hold off on doing that.  We should revisit it if we are still having other issues.  Ideally, we should refactor the code to handle items differently to work in a single or multi machine environment.  Unfortunately,  I don't think we have the time and resources to do so.

Thanks,

Harry Shyket
Digital Media Specialist
Yale University Peabody Museum
ph. 203-436-9428
har...@ya...

From: William Piel [mailto:wil...@ya...]
Sent: Wednesday, December 19, 2012 4:33 PM
To: Mattison Ward
Cc: Shyket, Harry; TreeBASE devel
Subject: Re: Reviewer access broken


On Dec 19, 2012, at 4:12 PM, Mattison Ward wrote:
I can disable load balancing for download requests where the
querystring includes x-access-code.   Is that worth a try?

That sounds like a good solution seeing as x-access-code requests are quite rare.

bp

Re: [Treebase-devel] Reviewer access broken

[William P. <wil...@ya...>]

On Dec 19, 2012, at 4:12 PM, Mattison Ward wrote:
> I can disable load balancing for download requests where the
> querystring includes x-access-code.   Is that worth a try?

That sounds like a good solution seeing as x-access-code requests are quite rare. 

bp