#174 Dashboard sometimes can't find update center, can't find updates or finds updates from pre-6.0-updatecenter
The issue is that under still to be determined circumstances, the Dashboard apparently tries to go to an Update Center other than http://updates.biospice.org. When I first started on the Bio-SPICE project and installed the then current 6.0 version via the installer, I found modules via the update center which I later figured out came from http://updates.biospice.org/pre-6.0-updatecenter/.
In August, Chris Geib at Wright-Patterson Air Force Base had the same thing happen on several occasions, including once when I was on the phone with him. In conversation, Valerie and I determined the modules he was intermittently seeing wer in the pre-6.0-updatecenter directory. She set the permissions so people coming to the update center over the web would not be able to get to that directory. If their Dashboard tried to do that and couldn't find a biospice_updates.xml file, we expected them to get a message about not being able to connect to the update center.
In late August, Valerie created a new installer for 6.0.3, hoping to fix the problem by having people instal the new Dashboard with that instead of using an update NBM from the update center. The very first time I tested the new installer in Linux, it automatically tried to go to the update center and when it came up, there were no modules there. (There should have been some since it was a clean install.) I pressed the back button, then the forward button, and this time several modules were there. I tried repeatedly and could not re-produce this.
A couple weeks later, I returned to the problem, installing and tested versions 6.0 and 6.0.3 of the Dashboard from the Bio-SPICE web site and also building and testing both from source.
I believe that I have re-experienced -- I won't say consistently re-produced -- the problem on a Linux machine, though I am not sure it is the same problem. I even went so far as to install a new Linux distribution on a different machine and test there. (This was because the first time I ran the 6.0 installed version, I experienced the problem described above. Thereafter, I did not.)
In the later round of testing, I have seen a different _set_ of problems on the Linux boxes. Sometimes when it tries to go to the update center, it returns a message about not being able to contact it. This makes sense if in fact the Dahsboard could not find the biospice-setttings.xml file. Under a particular set of circumstances, I instead get a "No updates found" message.
There is an anomaly in the 'updatecenter' target of the 6.0 build.xml file that sets the desc tag to "biospice_beta_updates.xml." This is what it was set to in the beta-updatecanter of the 6.0 build.xml file and appears to have been a copy and paste problem that was rectified in the 6.0.3 build.xml file. It occurred to me that somehow this was getting saved and propagated. Yet from looking more closely, it seems that this target only produces the 'dist' directory under bsbuild which is then intended to be moved up to the web site. There is no other reference to this biospice_beta_updates.xml file on my system after using the updatecenter target. Of coures, I can not use the installer target since I do not have Install Shield installed, so I don't know what happens when it runs.
I verified that putting biospice_beta_updates.xml -- or any other non-existent file -- as the URL in the "Bio-SPICE Update Center " portion of the dashboard\ide\src\java\gov\lbl\biospice\ide\resource\analysis.xml file results in the same behavior I occasionally experience. This makes sense and means that some part of the system is working correctly :-) The same thing can be accomplished by changing the URL value in
Tools > IDE Configuration > Options > System > AutoupdateTypes > Bio-SPICE Update Center
These are the only ways I have consistently seen to create the problem.
It is interesting that the problem Chris Geib and I both experienced earlier involved somehow gaining access to the pre-6.0-updatecenter directory. The settings file in that directory is "biospice_updates.xml." The "biospice_beta_updates.xml" file only exists in the http://updates.biospice.org/beta/ subdirectory. The latter only contains a single nbm file, biospice-ide-2.0.nbm. I do not know whether I ever saw this when I first was experimenting with the Dashboard when I initially joined the project.
The only thing I can find that persists across user sessions is the Settings object in the dashboard/autoupdate/src/org/netbeans/modules/autoupdate directory. It is serialized to persist its values across user sessions. It is stored in the file
~/.netbeans/biospice3.6/system/Services/org-netbeans-modules-autoupdate-Settings.settings
(there's a mouthful). Of course, it is stored in an unreadable form, so it is hard to tell what is really there.
At one point in this exploration, the only time I had seen the problem was when the Dashboard automatically checked for updates. As I noted, I even installed a new version of Linux and installed the 6.0 Dashboard installer...but it refused to do an auto update and after some long period of time, I broke down and tried a manual one...which worked fine.
I was curious whether there was something in the machinery which determined whether to check for updates that somehow loaded a different URL. So I tried to get the Dashboard to check for updates every time instead on once a week, which is the default setting. Eventually I discovered -- not surprisingly -- that I could change this value in the
Tools > IDE Configuration > Options > System > AutoupdateTypes > Bio-SPICE Update Center
dialog. I guess that should have been obvious ;-), but I started out the hard way, which revealed some curious things about how the serialized Settings object is used. I changed the value where it is initialized in the Settings object itself. That made no difference whatsoever, so tramping through the debugger in a rather painful way, I determined that the update frequency value is set 3 different times on start up. One of these -- I believe the last -- is from the serialized settings object, which is how I eventually found the Tools...dialog.
Having solved one minor mystery, I continued testing. At this point, I was looking for either old modules in the update center or no modules whatsoever. Instead, several of the first few times it tried to go to the update center, it failed to connect. I wondered for awhile whether there was something wrong with the networking on my Linux box, but everything else seemed to work fine. At some point, I remembered that if the Dashboard can't find the biospice_updates.xml file (or whatever name it was told to look for), it would give a failure to connect error.
I dug back into the debugger and traced the autoupdate path through its torturous route. I kept doing this and the first ~10 times, the URL was fine (http://updates.biospice.org/biospice_updates.xml), but it reported a failure to connect. Intermittently, I would run without the debugger and it would ocnnect just fine. At this point, I thought there was a problem with the debugger somehow interfering with the process.
Then I changed the URL in the analysis.xml file to a non-existent value and, as expected, it failed to connect outside the debugger. Then I changed it back and it continued to fail to connect outside the debugger. After a few more iterations, it succeeded in cnnecting both when running through the debugger and outside. Then it began failing intermittently each way. Then I threw up my hands for awhile.
One possible issue with using the debugger, is that I set up the project in the IDE to include just the source of the autoupdate module, so at certain points the source code is not available. This happens for parts of the Netbeans framework, e.g., openide/util/src/org/openide/util/RequestProcessor.java.
So when the "Connecting" dialog is hanging, I am not sure that there isn't something going on in "Hidden Source Code" that I was not seeing. The reason I am suspicious of this is that while in Windows, it times out very quickly when it can't connect (less than a second), and outside the debugger in Linux it times out in a couple minutes. In contrast, there are times -- not every time -- when it never times out in the debugger and I have to manually kill it.
Here some other oddities that may reflect some unknown stored setting. I have verified these on multiple Linux distributions on different boxes (to rule out the network configuration being the cause). I did a clean build of the 6.0.3 code and deleted my testuserdir directory under dhasboard/nbbuild. When I ran the Dashboard, it went to the Update Center and found updates appropriately. Then, I editied the analysis.xml file and changed the URL to a non-existing settings file and re-compiled ("ant dashboard"). The first time, it went successfully to the Update Center. I shut the Dashboard down and started it again and it failed to connect.
After another clean build, I used the
Tools > IDE Configuration > Options > System > AutoupdateTypes > Bio-SPICE Update Center
dialog to change the URL to a non-existent site. I shut the Dashboard down, re-started it and, as expected, it presented a message about not beig able to connect to the update center. Then, before shutting down the Dashboard, I changed the URL back to http://updates.biospice.org. After re-starting, it automatically tried to go to the update center and reported "No updates.found." This happens after repeated re-startings and even after re-compiling, until the project is cleaned and then re-bult (e.g., "ant clean dashboard").
This implies that there is another serialized settings object somewhere, or that values are not getting stored as one would expect.
Below is an email exchange between Rick and Valerie on this topic.
Valerie Wagner wrote:
> Well, I'm glad you tested Rick. This is really bizarre--I've never heard reports of these kinds of problems until you and Chris uncovered them. However, I'm wondering how much of the confusion comes from deleting/not deleting your user dir before installing 6.0.3? We've had a "feature request" for the installer to automatically delete any existing user dir before installing the new Dashboard, or to at least do that during uninstall. We need to consider whether that's a viable/desirable idea or not.
>
> Also, remember that the installer is supposed to install a minimal set of modules (those included in the CVS repository and specified with user.build.properties) and the rest come from the Update Center.
>
> Clint, can you reproduce any of these problems?
>
> Rick, please start tracking down what is going on. I'd start by studying the update center until you feel you know how it works under the hood, then watching the Dashboard via a debugger to see what's actually happening. But do whatever works for you.
>
> Until now, we've treated the update center as a black-box component of NetBeans. I don't believe any of us have looked under the hood. The relevant code is in the dashboard/autoupdate module.
>
> Both installers were produced using a clean check-out of the source tree (or should have been). The 6.0 release was tagged with "dashboard-6-0". The 6.0.3 test installer was produced by checking out a copy of the tree using the "latest-release" tag. See dashboard/bsbuild/README-TAGS and Release.txt for a list of all tags and what modules they apply to.
>
> The list of modules included is governed by dashboard/bsbuild/etc/user.build.properties.
>
> The URL of the update center is specified in a properties or XML file somewhere, probably under the ide module. Search for it.
>
> The physical update center is located on athena.sri.com at /web/htdocs/biospice/updates/. Let me know if I need to change any permissions for you to access anything.
>
> The update center descriptor is generated by using the "ant updatecenter" command in bsbuild. However, this was really only done once; as now when we add new modules, we just cut the new info from the generated file and paste it into the biospice_updates.xml file on the server. If you need more detail on how this works, let me know.
>
> I have a few more comments in line below. Otherwise, let me know what you are able to figure out.
>
> Thanks,
> Valerie
>
> Rick Walsh wrote:
>
>> Valerie,
>>
>> I may have encountered the same problem Chris Geib was having.
>>
>> I installed 6.0.3 on Windows, where I already had 6.0. After installing, I had the follow analyzers installed:
>
> Did you delete/move away your biospice3.6 user dir first? If not, the newer Dashboard may have been confused--I'm not sure.
>
>> * BioMat Bridge
>> * Biodata Viewer
>> * Graphviz
>> * Model Builder Analyser
>> * PlotML Translator
>> * PtPlot
>> * Render Matlab simulation
>> * Render SBML
>> * SBML to Graphviz DOT
>> * SBML to Pathway Builder
>> * SBML to laid-out Graphviz DOT
>> * SBML-SCC to Pathway Builder
>> * Tab Delimited Text Converter
>> * TableView
>> * TimeSeries To ZipFile Converter
>>
>>
>>
>> I went to the download center and had these updates:
>>
>>
>>
>> I installed as root in /opt/Bio-SPICE... under Linux. I had fewer analyzers available when I first started the Dashboard after installing:
>
>
> This what really concerns me--analyzers should not disappear. Are you checking the list of installed modules under the Options menu? Were the modules really gone, or just disabled?
>
>> * BioMat Bridge
>> * PlotML Translator
>> * Tab Delimited Text Converter
>> * TimeSeries To ZipFile Converter
>> * Biodata Viewer
>> * PtPlot
>> * TableView
>>
>> The also resident 6.0 version has a much larger set, including some of the ones that appeared under the Windows 6.0.3 install:
>>
>> * 2D Grapher
>> * BioMat Bridge *
>> * Biodata Viewer *
>> * Model Builder Analyser *
>> * Oscill8 One Parameter
>> * Oscill8 Plotter
>> * PlotML Translator *
>> * PtPlot *
>> * 11 QIS analyzers
>> * Tab Delimited Text Converter *
>> * TableView *
>> * TimeSeries To ZipFile Converter *
>>
>> (The starred items above were present in the Windows 6.0.3 installl) This may simply be because I had more recently updated the Windows 6.0 version.
>
> Yes, this is true. So you are not comparing the set of analyzers included on a clean 6.0 install versus a clean 6.0.3 install?
>
>>
>> The following were missing under the Linux 6.0.3 install:
>>
>> * Graphviz
>> * Render Matlab simulation
>> * Render SBML
>> * SBML to Graphviz DOT
>> * SBML to Pathway Builder
>> * SBML to laid-out Graphviz DOT
>> * SBML-SCC to Pathway Builder
>>
> None of these should be included with the installers...These are all modules produced by biospice contributors that probably are/were avaialble via the update center.
>
>> *
>>
>>
>> When I first started the Dashboard, it offered to go to the Update Center. Initially,there were no updates whatsoever available. Confused, I pressed the Back button on the dialog, then the Next button again, and I got a large list of updates -- the same ones I got under Windows.
>
> This is bad too.
>
>>
>> I have not been able to re-produce this behavior,
>
> Yikes, we need to reproduce this behavior.
>
>> despite uninstalling and re-installing on both machine. I suspect the Dashboard was trying to go to the now inaccessible archive directory. Can we check the web log and see if I tried to access that directory tonight at around 6:30? Under Linux, it offered to go to the Update Center.
>
> Check with Derek.
>
>> Under Windows and subsequent visits on Linux, I manually went. Might that make a difference?
>
> Who knows? Find out.
>
>> Is that handled by a different part of the code? I recall that Chris Geib had said that the Dashboard had offered to check for updates on at least one of the occasions when he had the problem.
>
>>
>> Rick
>>
>> Valerie Wagner wrote:
>>
>>> Oops, I accidentally sent this to biospice-sri@sri.com instead of biospice-sri@biospice.org. No wonder no one got back to me. Please see below.
>>>
>>> Valerie
>>>
>>> -------- Original Message --------
>>> Subject: Need to test Bio-SPICE 6.0.3 installer
>>> Date: Fri, 12 Aug 2005 10:18:52 -0600
>>> From: Valerie Wagner <valerie.wagner@sri.com>
>>> CC: biospice-sri@sri.com
>>>
>>>
>>> Clint, Rick, Angela, and whoever else has time:
>>>
>>> I've posted installers for Dashboard 6.0.3 here:
>>>
>>> http://www.mt.sri.com/~valerie/biospice/
>>>
>>> Dashboard-SetupLinux-6.0.3.bin
>>> Dashboard-SetupWin32-6.0.3.exe
>>>
>>> These need to be thoroughly tested before we release them. Please try
>>> out both versions. Check that it connects to the correct Update Center,
>>> check that the correct set of modules are installed, check that
>>> installing modules from the Update Center doesn't cause problems, check
>>> that the installation directory set looks ok. Try installing both with
>>> and without an existing 6.0 installation. In summary: test and look at
>>> anything you can think of.
>>>
>>> I'd like to get these out mid-next week.
>>>
>>> Thanks,
>>> Valerie
>>>
>>>
>>> _______________________________________________
>>> Biospice-sri mailing list
>>> Biospice-sri@biospice.org
>>> https://biospice.org/mailman/listinfo/biospice-sri
>>>
>
