treebase-devel — TreeBASE developer mailing list

Re: [Treebase-devel] Some other oddities

[Vladimir G. <vla...@du...>]

On Mar 19, 2010, at 5:52 PM, Hilmar Lapp wrote:

>
> On Mar 19, 2010, at 4:21 PM, William Piel wrote:
>
>> 2. After deleting the duplicates in (1), we have a total of 2417
>> studies in TreeBASE2. That includes two new ones that were never in
>> TreeBASE1. TreeBASE1, however, has 2427 studies -- so that means we
>> have 12 missing studies. Not a show-stopper. After release, we will
>> identify these and re-enter them manually.
>
> At the risk of stating the obvious, can you add that to the bug
> tracker, priority 8, so it doesn't get lost?

Bill, I'll take care of that.
--VG 

Re: [Treebase-devel] Some other oddities

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 4:21 PM, William Piel wrote:

> 2. After deleting the duplicates in (1), we have a total of 2417  
> studies in TreeBASE2. That includes two new ones that were never in  
> TreeBASE1. TreeBASE1, however, has 2427 studies -- so that means we  
> have 12 missing studies. Not a show-stopper. After release, we will  
> identify these and re-enter them manually.

At the risk of stating the obvious, can you add that to the bug  
tracker, priority 8, so it doesn't get lost?

	-hilmar

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Dummy studies in TaxonLabelSet

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 4:46 PM, Vladimir Gapeyev wrote:

> I don't quite grasp what is the end result we are looking for...


The end result we are looking for is a series of SQL scripts that,  
once applied, provide for a state of the database that we are  
comfortable releasing to the public.

	-hilmar
-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Dummy studies in TaxonLabelSet

[Vladimir G. <vla...@du...>]

On Mar 19, 2010, at 4:44 PM, William Piel wrote:

> I see that not all taxonlabelsets got changed from id 22 to whatever  
> they should be -- the taxonlabelsets created by trees blocks did not  
> get changed.
>

Here are the counts of the dummy studies that still show up in  
TaxonLabelSet:

select study_id, count(*)
from taxonlabelset where study_id in (22, 2264, 10215)
group by study_id

  study_id | count
----------+-------
     10215 |   285
      2264 |   690
        22 |  8615

So, there are quite a few.  Unlike we did with matrices, they do not  
seem to be reachable from legitimate studies via the  Study ->  
Phylotree -> Treeblock -> Taxonlabelset chain, i.e.

select s.study_id, pt.phylotree_id, tb.treeblock_id, tls.*
from study s, phylotree pt, treeblock tb, taxonlabelset tls
where s.study_id = pt.study_id and  pt.treeblock_id = tb.treeblock_id  
and tb.taxonlabelset_id = tls.taxonlabelset_id
   and s.study_id <> tls.study_id
order by s.study_id

returns zero results.

I can continue poking in this, but I admit I don't quite grasp what is  
the end result we are looking for...

Please keep me updated on how I could be useful with data cleaning.   
If you think a phone call could be useful, please call -- I am here  
till about 6 - 6:10 pm

Scripts that I collect for Jon to apply to production are in SVN in  
treebase-core/db/cleaning/pre-release_hotfixes

--Vladimir 

Re: [Treebase-devel] Some other oddities

[William P. <wil...@ya...>]

1. I noticed that somehow we generated a lot of duplicates here:

http://purl.org/phylo/treebase/phylows/study/find?query=tb.identifier.study=66+or+tb.identifier.study=67+or+tb.identifier.study=68+or+tb.identifier.study=69+or+tb.identifier.study=70+or+tb.identifier.study=71+or+tb.identifier.study=72+or+tb.identifier.study=73&recordSchema=study

These should all be deleted because they are copies of study_id 84 (which is the only one to have the legacy id number properly written). Not a show-stopper. 

2. After deleting the duplicates in (1), we have a total of 2417 studies in TreeBASE2. That includes two new ones that were never in TreeBASE1. TreeBASE1, however, has 2427 studies -- so that means we have 12 missing studies. Not a show-stopper. After release, we will identify these and re-enter them manually. 

bp

Re: [Treebase-devel] Some results of testing with a user

[William P. <wil...@ya...>]

On Mar 19, 2010, at 4:00 PM, Vladimir Gapeyev wrote:

> I see Bill has already run a fix on the 2264 TLSs.  Should we do it for 22-TLSs as well?  

I just did that. Unfortunately, it doesn't quite solve the problem. 

If I search on S2377 and then click on the Taxa tab, I get nothing listed.
If I search on S2377 and then click on Matrices and then click on Taxa, I get a long list.
If I search on S2377 and then click on Tree and then click on Taxa, I get a long list. 

With this query:

select taxonlabelset_id, title, study_id from taxonlabelset_taxonlabel join taxonlabelset using (taxonlabelset_id) where taxonlabel_id = 72150;

I see that not all records with the same taxonlabel_id got 

 taxonlabelset_id |         title          | study_id 
------------------+------------------------+----------
             3105 | M3562                  |     2377
             8626 | Untitled Block of Taxa |       22
(2 rows)

I see that not all taxonlabelsets got changed from id 22 to whatever they should be -- the taxonlabelsets created by trees blocks did not get changed. 

This is for a misbehaving taxonlabel. Suppose I compare this with a behaving one from an earlier migration:

select taxonlabelset_id, title, study_id from taxonlabelset_taxonlabel join taxonlabelset using (taxonlabelset_id) where taxonlabel_id = 95894;                                                                                                                                                          

taxonlabelset_id |         title          | study_id 
------------------+------------------------+----------
             3841 | M713                   |       22
             7509 | Untitled Block of Taxa |       22
            10400 | TaxonLabelSet10400     |      654
(3 rows)

... there's an extra row, with the two older ones still mapping to the dud study_id 22.  This new row looks like it was created by TreeBASE2, probably for the explicit purpose of having both trees and matrices map to the same taxonlabelset. Both matrix and treeblock tables have records with taxonlabelset_id = 10400, which has the correct study_id of 654. That seems to be missing for some of our more recently migrated data. 

This looks complicated to fix, but it may only affect a few studies. If so, perhaps we could just delete and re-enter these. 

I guess I'll look to see how many are affected in this way.

bp

Re: [Treebase-devel] Deployment schedule

[Hilmar L. <hl...@ne...>]

Sounds good. Looks like I'll be online here in Chicago most of the  
day, so I'll make the call.

	-hilmar

On Mar 19, 2010, at 3:03 PM, Jon Auman wrote:

> Bill,
>
> I think this seems like a reasonable operating procedure:
>
> 1) When youjun has finished with his changes, email the list
> 2) Vladimir will checkout the code from subversion and rebuild the  
> war file
> 3) After Vladimir notifies me, I will change the application to  
> point towards production and deploy the new war file
> 4) Once that has been done, I will email the list
> 5) Once the powers that be give me their blessings, I will change  
> DNS (unless you'd rather handle the editing of the A record, Bill?)
>
> As there are several folks involved, there may be somewhat of a  
> delay between each step, but I think that's OK.
>
> Does this sound reasonable to everyone?
>
> -Jon
>
>
> On Mar 19, 2010, at 3:40 PM, William Piel wrote:
>
>>
>> I'm optimistic that this last sql update statement will solve the  
>> show-stopper bug, and that can be done today.
>>
>> Youjun tells me that with the new pages, CSS, etc, that he's  
>> working on, he predicts that he will be done by tomorrow at noon.
>>
>> So...  Jon --  is it possible to get a rebuild on Saturday  
>> afternoon (this time with treebase.nescent.org pointing using the  
>> production database)?  That would imply that after a last read- 
>> through of the web pages, I put in for a DNS change Saturday  
>> evening or Sunday. I think I have credentials to change the PURL  
>> records too -- else Hilmar can take care of that.
>>
>> bp
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download Intel® Parallel Studio Eval
>> Try the new software tools for yourself. Speed compiling, find bugs
>> proactively, and fine-tune applications for parallel performance.
>> See why Intel Parallel Studio got high marks during beta.
>> http://p.sf.net/sfu/intel-sw-dev
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
> -------------------------------------------------------
> Jon Auman
> Systems Administrator
> National Evolutionary Synthesis Center
> Duke University
> http:www.nescent.org
> jon...@ne...
> ------------------------------------------------------
>
>
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev_______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Some results of testing with a user

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 3:06 PM, Jon Auman wrote:

> He commits these changes to subversion


Great. I was going to point that out, hoping that it is stating the  
obvious, and an glad to hear that indeed it is.

	-hilmar
-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Some results of testing with a user

[Jon A. <jon...@ne...>]

After all these discrepancies have been worked out, Vladimir will send me the sql to make the changes on production. Vladimir should be the gatekeeper to all changes made on the production database and application. He commits these changes to subversion and notifies me that its time to update.

-Jon
 
On Mar 19, 2010, at 3:58 PM, William Piel wrote:

> 
> On Mar 19, 2010, at 3:48 PM, William Piel wrote:
> 
>> Unless Vladimir has other placeholder study ids used in the migration scripts, this fixes it. 
> 
> Oops... just noticed that S2377 has taxon labels that don't map. There must be some other placeholder ids.  Vladimir?
> 
> bp
> 
> 
> 
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel

-------------------------------------------------------
Jon Auman
Systems Administrator
National Evolutionary Synthesis Center
Duke University
http:www.nescent.org
jon...@ne...
------------------------------------------------------

Re: [Treebase-devel] Deployment schedule

[Jon A. <jon...@ne...>]

Bill, 

I think this seems like a reasonable operating procedure:

1) When youjun has finished with his changes, email the list
2) Vladimir will checkout the code from subversion and rebuild the war file
3) After Vladimir notifies me, I will change the application to point towards production and deploy the new war file
4) Once that has been done, I will email the list
5) Once the powers that be give me their blessings, I will change DNS (unless you'd rather handle the editing of the A record, Bill?)

As there are several folks involved, there may be somewhat of a delay between each step, but I think that's OK.

Does this sound reasonable to everyone?

-Jon


On Mar 19, 2010, at 3:40 PM, William Piel wrote:

> 
> I'm optimistic that this last sql update statement will solve the show-stopper bug, and that can be done today. 
> 
> Youjun tells me that with the new pages, CSS, etc, that he's working on, he predicts that he will be done by tomorrow at noon. 
> 
> So...  Jon --  is it possible to get a rebuild on Saturday afternoon (this time with treebase.nescent.org pointing using the production database)?  That would imply that after a last read-through of the web pages, I put in for a DNS change Saturday evening or Sunday. I think I have credentials to change the PURL records too -- else Hilmar can take care of that.  
> 
> bp
> 
> 
> 
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel

-------------------------------------------------------
Jon Auman
Systems Administrator
National Evolutionary Synthesis Center
Duke University
http:www.nescent.org
jon...@ne...
------------------------------------------------------

Re: [Treebase-devel] Some results of testing with a user

[Vladimir G. <vla...@du...>]

On Mar 19, 2010, at 3:25 PM, William Piel wrote:

>
> On Mar 19, 2010, at 12:07 PM, William Piel wrote:
>
>>
>> On Mar 19, 2010, at 11:44 AM, Hilmar Lapp wrote:
>>
>>> On Mar 18, 2010, at 8:06 PM, Vladimir Gapeyev wrote:
>>>
>>>> (1) Unexpected different results on the Taxa tab -- a feature or  
>>>> a bug? E.g., find a single study, e.g. 10051.
>>>>     -- Click the study (which goes to Citation tab), then to Taxa  
>>>> tab ==> "Nothing to display"
>>>>     -- Go to Matrices tab; click on "View Taxa" in the table ==>  
>>>> it goes back to the Taxa tab, showing lots of stuff
>>>>     -- Go back to Citation tab; then Taxa tab  ==> "Nothing found  
>>>> to display"
>>>>    -- Go to Trees tab; click on "View Taxa" in the table ==> it  
>>>> goes back to the Taxa tab, showing lots of stuff.
>>>
>>> This is for Bill to judge in terms of what kind of problem it may  
>>> signal, and hence how severe it is.
>>
>> This may be severe -- let's at least understand why it is  
>> happening. It is affecting a lot of studies.
>
> I think I've figured out what's going on here. I took one taxon  
> label that maps nicely and compared it with another taxon label that  
> does not map nicely, and then I ran queries on tables related to  
> these taxon labels. The main difference is that in the good case the  
> taxonlabelset table has the correct study_id, while in the bad case  
> the taxonlabelset has "2264" as study_id instead of "10053", which  
> is what it should be.
>
> Last weekend I had solved this problem using an update query that  
> assumed that study_id 10215 was the only "placeholder" for migrated  
> records. Turns out that there seems to be another one: 2264.  
> Vladimir, can you confirm that study_id 2264 is another placeholder?  
> And if so, can you let me know if there are any other placeholder  
> study_ids that I'm not aware of.
>
> If this is the problem as I think it is, the solution is the  
> following update:
>
> UPDATE taxonlabelset SET study_id = mx.study_id
> FROM matrix mx JOIN taxonlabelset tls USING (taxonlabelset_id)
> WHERE tls.study_id = 2264
> AND mx.study_id <> 2264
> AND taxonlabelset.taxonlabelset_id = tls.taxonlabelset_id

I followed a similar path and was reaching a similar conclusion. Here  
is a query that shows all TaxonLabelSets whose tls.study_id is not the  
same as the study reachable from the TLS via a matrix:

select s.study_id, m.matrix_id, tls.*
from study s, taxonlabelset tls, matrix m
where s.study_id = m.study_id and tls.taxonlabelset_id =  
m.taxonlabelset_id
   and s.study_id <> tls.study_id
order by s.study_id

The only Study IDs showing up as tls.study_id  in the result are  22  
and 2264.

#2264 was picked up by the migration scripts during my 1st migration  
-- not something I could have known ahead of time, since the scripts  
were creating a fresh dummy study in my testing daabase! By the time  
of the 2nd migration batch I realized it was going on and created the  
10215 study.

I see Bill has already run a fix on the 2264 TLSs.  Should we do it  
for 22-TLSs as well?


> The other difference is that the matrix associated with the bad one  
> has version 0, while the other is version 1. Is there any rhyme or  
> reason for the version fields that appear in lots of tables?

My guess is that version fields are an artifact of Hibernate.   
Apparently, each time a data object is updated, the version is  
incremented.

Re: [Treebase-devel] Some results of testing with a user

[William P. <wil...@ya...>]

On Mar 19, 2010, at 3:48 PM, William Piel wrote:

> Unless Vladimir has other placeholder study ids used in the migration scripts, this fixes it. 

Oops... just noticed that S2377 has taxon labels that don't map. There must be some other placeholder ids.  Vladimir?

bp

Re: [Treebase-devel] Some results of testing with a user

[William P. <wil...@ya...>]

On Mar 19, 2010, at 3:40 PM, Hilmar Lapp wrote:

> Sounds good to me.


Okay, I just did it. I checked back with study_id 10051, and it seems to have fixed the problem. 

Unless Vladimir has other placeholder study ids used in the migration scripts, this fixes it. 

Below is the query for Jon to run on production. 

bp




begin work;

UPDATE taxonlabelset SET study_id = mx.study_id
FROM matrix mx JOIN taxonlabelset tls USING (taxonlabelset_id)
WHERE tls.study_id = 2264
AND mx.study_id <> 2264
AND taxonlabelset.taxonlabelset_id = tls.taxonlabelset_id

-- should result in 570 changes

commit;

[Treebase-devel] Deployment schedule

[William P. <wil...@ya...>]

I'm optimistic that this last sql update statement will solve the show-stopper bug, and that can be done today. 

Youjun tells me that with the new pages, CSS, etc, that he's working on, he predicts that he will be done by tomorrow at noon. 

So...  Jon --  is it possible to get a rebuild on Saturday afternoon (this time with treebase.nescent.org pointing using the production database)?  That would imply that after a last read-through of the web pages, I put in for a DNS change Saturday evening or Sunday. I think I have credentials to change the PURL records too -- else Hilmar can take care of that.  

bp

Re: [Treebase-devel] Some results of testing with a user

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 2:25 PM, William Piel wrote:

> I vote that we run this on stage -- if that solves the problem, then  
> we hand it off to Jon to run it on production.  Presumably Jon still  
> has the data dump used in the transfer of data to production, so if  
> anything screws up with can repopulate production.
>
> What do people think?

Sounds good to me.

	-hilmar

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Some results of testing with a user

[William P. <wil...@ya...>]

On Mar 19, 2010, at 12:07 PM, William Piel wrote:

> 
> On Mar 19, 2010, at 11:44 AM, Hilmar Lapp wrote:
> 
>> On Mar 18, 2010, at 8:06 PM, Vladimir Gapeyev wrote:
>> 
>>> (1) Unexpected different results on the Taxa tab -- a feature or a bug? E.g., find a single study, e.g. 10051. 
>>>     -- Click the study (which goes to Citation tab), then to Taxa tab ==> "Nothing to display"
>>>     -- Go to Matrices tab; click on "View Taxa" in the table ==> it goes back to the Taxa tab, showing lots of stuff
>>>     -- Go back to Citation tab; then Taxa tab  ==> "Nothing found to display" 
>>>    -- Go to Trees tab; click on "View Taxa" in the table ==> it goes back to the Taxa tab, showing lots of stuff.
>> 
>> This is for Bill to judge in terms of what kind of problem it may signal, and hence how severe it is.
> 
> This may be severe -- let's at least understand why it is happening. It is affecting a lot of studies.

I think I've figured out what's going on here. I took one taxon label that maps nicely and compared it with another taxon label that does not map nicely, and then I ran queries on tables related to these taxon labels. The main difference is that in the good case the taxonlabelset table has the correct study_id, while in the bad case the taxonlabelset has "2264" as study_id instead of "10053", which is what it should be. 

Last weekend I had solved this problem using an update query that assumed that study_id 10215 was the only "placeholder" for migrated records. Turns out that there seems to be another one: 2264. Vladimir, can you confirm that study_id 2264 is another placeholder? And if so, can you let me know if there are any other placeholder study_ids that I'm not aware of. 

If this is the problem as I think it is, the solution is the following update:

UPDATE taxonlabelset SET study_id = mx.study_id
FROM matrix mx JOIN taxonlabelset tls USING (taxonlabelset_id)
WHERE tls.study_id = 2264
AND mx.study_id <> 2264
AND taxonlabelset.taxonlabelset_id = tls.taxonlabelset_id

I vote that we run this on stage -- if that solves the problem, then we hand it off to Jon to run it on production.  Presumably Jon still has the data dump used in the transfer of data to production, so if anything screws up with can repopulate production. 

What do people think? 

The other difference is that the matrix associated with the bad one has version 0, while the other is version 1. Is there any rhyme or reason for the version fields that appear in lots of tables? 


Bill

Re: [Treebase-devel] Some results of testing with a user

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 11:07 AM, William Piel wrote:

> On Mar 19, 2010, at 11:44 AM, Hilmar Lapp wrote:
>
>> On Mar 18, 2010, at 8:06 PM, Vladimir Gapeyev wrote:
>>
>>> (1) Unexpected different results on the Taxa tab -- a feature or a  
>>> bug? E.g., find a single study, e.g. 10051.
>>>     -- Click the study (which goes to Citation tab), then to Taxa  
>>> tab ==> "Nothing to display"
>>>     -- Go to Matrices tab; click on "View Taxa" in the table ==>  
>>> it goes back to the Taxa tab, showing lots of stuff
>>>     -- Go back to Citation tab; then Taxa tab  ==> "Nothing found  
>>> to display"
>>>    -- Go to Trees tab; click on "View Taxa" in the table ==> it  
>>> goes back to the Taxa tab, showing lots of stuff.
>>
>> This is for Bill to judge in terms of what kind of problem it may  
>> signal, and hence how severe it is.
>
> This may be severe -- let's at least understand why it is happening.  
> It is affecting a lot of studies.

OK - I am assuming that this is being investigated then right now.  
Vladimir or Bill - can you please keep me posted as to whether we are  
calling off or moving ahead with releasing tonight.

	-hilmar

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Some results of testing with a user

[William P. <wil...@ya...>]

On Mar 19, 2010, at 11:44 AM, Hilmar Lapp wrote:

> On Mar 18, 2010, at 8:06 PM, Vladimir Gapeyev wrote:
> 
>> (1) Unexpected different results on the Taxa tab -- a feature or a bug? E.g., find a single study, e.g. 10051. 
>>     -- Click the study (which goes to Citation tab), then to Taxa tab ==> "Nothing to display"
>>     -- Go to Matrices tab; click on "View Taxa" in the table ==> it goes back to the Taxa tab, showing lots of stuff
>>     -- Go back to Citation tab; then Taxa tab  ==> "Nothing found to display" 
>>    -- Go to Trees tab; click on "View Taxa" in the table ==> it goes back to the Taxa tab, showing lots of stuff.
> 
> This is for Bill to judge in terms of what kind of problem it may signal, and hence how severe it is.

This may be severe -- let's at least understand why it is happening. It is affecting a lot of studies.

> 
>> (2) An ominous message when Phylowidget loads: "An applet requires access to your computer.  The digital signature could not be verified."
> 
> The digital signature warning is because it is custom issued rather than by a trusted authority. What I wasn't sure of myself though when this dialog presented itself to me is what the applet wants to do that would require this seemingly unqualified access to the computer. Is this indeed necessary? If yes, we should add a dialog or a message somewhere that explains to the user what the applet will be doing that requires this permission to be granted.  

yup, but not a show-stopper.

>> (3) Havoc from purls.
> 
> I think we've exhausted that discussion at this point. If you do want to subject these offending PURL links to testing, I appreciate that and have therefore created the additional purl.org base URLs - just use those when you deploy to dev and stage.
> 
> Personally I would be willing to go public without this additional testing step, if you run out of time to perform it.

Let's just leave the situation as-is for now. It's mostly us who do beta testing on stage and dev, and we know to edit these links accordingly. 


>> 
>> (4)  We noticed that some many tree and matrix entries had original files missing. It seemed like a big data problem to me initially, but the more I look into it, the less I understand how things are intended to be, so I'll just enumerate the bothersome issues. 
> 
> It sounds too here like this is for Bill to judge in terms of kind of issue and severity it may signal. 

Not a show-stopper, IMO. We can come up with a plant to repopulate the missing files after release. 

>> 
>> If it were up to me, I'd prefer the purls issue resolved before Treebase goes live.
> 
> I don't see any harm with going live with the contentious ones. The only shortcoming from the perspective of release seems to be that it's nearly impossible to fully test them. I've enabled that now (see above), so if time permits we can do it. 

yeah, this is not an artifact that public users of treebase.nescent.org will encounter. Let's let this dog lie, for now. 

bp

Re: [Treebase-devel] A stop-gap proposal for the purls issue

[Vladimir G. <vga...@ne...>]

On Mar 19, 2010, at 11:47 AM, Hilmar Lapp wrote:

>
> On Mar 19, 2010, at 10:40 AM, Vladimir Gapeyev wrote:
>
>> On Mar 19, 2010, at 10:34 AM, Hilmar Lapp wrote:
>>> So instead I created the following alternative base URLs, with
>>> obvious redirection:
>>> http://purl.org/phylo/treebase/dev/phylows/
>>> http://purl.org/phylo/treebase/stage/phylows/
>>
>> This will relieve us from the need to change purls in application
>> links to relative urls.
>>
>> Can we settle on this for the time being?

> Yes, please, and if we do then why do we need your extra steps re:  
> treebase.nescent.org if then we don't jump from treebase-dev to  
> production anymore?

What extra steps?

> Let's just keep it simple and effective - any reason why using the  
> above two base URLs for PURLs on dev and stage don't fully solve the  
> issue for now?

To make this work right now, I'd need to do 3 different builds with  
each update, and I am already going nuts.  If I do feel I have a free  
hour today, I'll try to go ahead and set up JNDI to pass the purl  
prefix to the app, but it may take longer than that.

--VG 

Re: [Treebase-devel] A stop-gap proposal for the purls issue

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 10:40 AM, Vladimir Gapeyev wrote:

> On Mar 19, 2010, at 10:34 AM, Hilmar Lapp wrote:
>> So instead I created the following alternative base URLs, with
>> obvious redirection:
>> http://purl.org/phylo/treebase/dev/phylows/
>> http://purl.org/phylo/treebase/stage/phylows/
>
> This will relieve us from the need to change purls in application
> links to relative urls.
>
> Can we settle on this for the time being?


Yes, please, and if we do then why do we need your extra steps re:  
treebase.nescent.org if then we don't jump from treebase-dev to  
production anymore?

Let's just keep it simple and effective - any reason why using the  
above two base URLs for PURLs on dev and stage don't fully solve the  
issue for now?

	-hilmar
-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Some results of testing with a user

[Hilmar L. <hl...@ne...>]

On Mar 18, 2010, at 8:06 PM, Vladimir Gapeyev wrote:

> (1) Unexpected different results on the Taxa tab -- a feature or a  
> bug? E.g., find a single study, e.g. 10051.
>     -- Click the study (which goes to Citation tab), then to Taxa  
> tab ==> "Nothing to display"
>     -- Go to Matrices tab; click on "View Taxa" in the table ==> it  
> goes back to the Taxa tab, showing lots of stuff
>     -- Go back to Citation tab; then Taxa tab  ==> "Nothing found to  
> display"
>    -- Go to Trees tab; click on "View Taxa" in the table ==> it goes  
> back to the Taxa tab, showing lots of stuff.

This is for Bill to judge in terms of what kind of problem it may  
signal, and hence how severe it is.

> (2) An ominous message when Phylowidget loads: "An applet requires  
> access to your computer.  The digital signature could not be  
> verified."

The digital signature warning is because it is custom issued rather  
than by a trusted authority. What I wasn't sure of myself though when  
this dialog presented itself to me is what the applet wants to do that  
would require this seemingly unqualified access to the computer. Is  
this indeed necessary? If yes, we should add a dialog or a message  
somewhere that explains to the user what the applet will be doing that  
requires this permission to be granted.

> (3) Havoc from purls.

I think we've exhausted that discussion at this point. If you do want  
to subject these offending PURL links to testing, I appreciate that  
and have therefore created the additional purl.org base URLs - just  
use those when you deploy to dev and stage.

Personally I would be willing to go public without this additional  
testing step, if you run out of time to perform it.

> (4)  We noticed that some many tree and matrix entries had original  
> files missing. It seemed like a big data problem to me initially,  
> but the more I look into it, the less I understand how things are  
> intended to be, so I'll just enumerate the bothersome issues.

It sounds too here like this is for Bill to judge in terms of kind of  
issue and severity it may signal.

> If it were up to me, I'd prefer the purls issue resolved before  
> Treebase goes live.

I don't see any harm with going live with the contentious ones. The  
only shortcoming from the perspective of release seems to be that it's  
nearly impossible to fully test them. I've enabled that now (see  
above), so if time permits we can do it.

	-hilmar
-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

[Treebase-devel] A stop-gap proposal for the purls issue

[Vladimir G. <vga...@ne...>]

It looks like the purls issue is as hot as before.  Until we can meet  
face-to-face, fully explain to each other our concerns,  and reach a  
consensus, here is a proposal for an interim hack.

Till the release, we just use [treebase.nescent.org pointing to a non- 
production db] as our primary development front-end.

Immediately after the release, if we need a development front end, we  
use treebase-dev, being mindful that some links jump to the production  
instance.

Soon after the release, I'll figure out how use JNDI to pass the purl  
prefix to the application, so that we can configure each of our  
instances to use the appropriate one of these:

On Mar 19, 2010, at 10:34 AM, Hilmar Lapp wrote:
> So instead I created the following alternative base URLs, with  
> obvious redirection:
> http://purl.org/phylo/treebase/dev/phylows/
> http://purl.org/phylo/treebase/stage/phylows/

This will relieve us from the need to change purls in application  
links to relative urls.

Can we settle on this for the time being?

--VG

Re: [Treebase-devel] Some results of testing with a user

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 10:11 AM, William Piel wrote:

> To prevent proliferation of synonymous /phyows/ links getting out  
> there, we decided to use purl.org links wherever /phylows/ links are  
> present.
>
> So the lazy "human user" benefits because right-click-copy is a  
> quick way for them to bookmark objects in TreeBASE without having to  
> edit the copied link to comply with our purl.org standard.

Note that this is a use-case that's distinctly different from  
canonical resolvable GUIDs for data objects to play with the semweb.  
The above is basically saying that we want bookmarks that people make  
in their web browser to be protected from a future change in how the  
TreeBASE webapp is implemented. For example, if we change to RoR,  
those bookmarks should still work.

This is a worthwhile use-case, but I think it's good to keep in mind  
that it ought to be far less in priority than getting canonical HTTP  
URIs communicated to machine clients.

	-hilmar

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================

Re: [Treebase-devel] Some results of testing with a user

[Rutger V. <rut...@gm...>]

> [1] Especially the Rod Pages of this world understand the problem well
> enough to not do this, I'd worry about those who aren't trained enough,
> which basically translates to a need for more training. Making application
> programming more intricate and less easy to test isn't the right solution to
> this, in my opinion. I think we need to architect for the future, and in the
> future the majority of people will be well trained in this b/c the semantic
> web will be omnipresent.

You're right, of course. I was going to make the caveat that these
would be people one step down from the Rod Pages of this world, but
you made my point for me already :)

Re: [Treebase-devel] Some results of testing with a user

[Hilmar L. <hl...@ne...>]

On Mar 19, 2010, at 10:04 AM, Rutger Vos wrote:

> My worry is that, by placing ugly webapp download links in the public
> GUI, the Rod Pages of this world will copy & paste those, distribute
> them, build hacks around them, etc.

I appreciate your point (even though I don't agree [1]). However, the  
current solution doesn't address it, because the URL that is easiest  
to copy&paste, namely the one in the address bar, still is an  
application URL.

	-hilmar

[1] Especially the Rod Pages of this world understand the problem well  
enough to not do this, I'd worry about those who aren't trained  
enough, which basically translates to a need for more training. Making  
application programming more intricate and less easy to test isn't the  
right solution to this, in my opinion. I think we need to architect  
for the future, and in the future the majority of people will be well  
trained in this b/c the semantic web will be omnipresent.

-- 
===========================================================
: Hilmar Lapp  -:- Durham, NC -:- informatics.nescent.org :
===========================================================