gmod-ajax — JBrowse list

Re: [Gmod-ajax] [Gmod-architecture] AJAX, WebService, Middleware, Chode

[Ian H. <ih...@be...>]

Now that we have a full-time gmod-ajax person (in the form of Mitch) 
it's probably about time to kick off a discussion on how to move towards 
our goals of community annotation (and what this means exactly) and full 
browser functionality including search, rich display, etc (and to what 
extent this should be GBrowse-like).

Chris, Andrew and I had a tete-a-tete a couple of months ago in which we 
scribbled some notes on the back of an envelope (or coffeeshop napkin). 
Andrew, do you have that napkin anywhere? Could we start by reprising 
those? I am running out the door right now but could dredge up my memory 
of the meeting if necessary.

One thing is that Mitch is away for a week or so. So probably the 
discussion will have to wait a little while for him to get back, before 
it gets really productive.

I.



Chris Mungall wrote:
> Hi Xianhua
> 
> Note there is now a gmod-ajax list too.
> 
> I think a RESTFUL architecture makes a lot of sense for an AJAX  
> application.
> 
> By the way, I think you mean "Chado". (try a google search for  
> "define: chode" - I do hope chode doesn't catch on as another Chado  
> mispronounciation!).
> 
> On Jan 18, 2007, at 7:47 AM, Xianhua Liu wrote:
> 
>> I am trying to build web interface to query and view data from  
>> Chode with extension schema for bio-diversity data. I am planning  
>> to use a 4-lawyer architecture as following:
>>
>> 1. AJAX: front-end
>> 2. WebService: web lawyer
>> 3. XXX: Middleware
>> 4. Chode: back-end
>>
>> Why WebService?
>> 1. independency of front-end on middleware.
>> 2. loose coupling
>> 3. easy data integration over web
>> 4. ...
>>
>> Challenges:
>> 1. standardization of web service interface
>> 2. federalization of data representation model exchanged through  
>> the web service (XML Schema, Ontology, ...?)
>> 3. Performance?
>>
>> RESTful or SOAP:
>> I prefer to choose RESTful web service, because it is easy, direct  
>> and light.
>>
>> If someone have already tried the same and similar thing, please  
>> share your experience and lessons and tech choises.
>>
>> Welcome every one to send comments about the idea, and suggestions  
>> about techniques, software packages and any other sources.
>>
>> Thanks,
>>
>> Xianhua
>>
>>
>> ---------------------------------------------------------------------- 
>> ---
>> Take Surveys. Earn Cash. Influence the Future of IT
>> Join SourceForge.net's Techsay panel and you'll get the chance to  
>> share your
>> opinions on IT & business topics through brief surveys - and earn cash
>> http://www.techsay.com/default.php? 
>> page=join.php&p=sourceforge&CID=DEVDEV________________________________ 
>> _______________
>> Gmod-architecture mailing list
>> Gmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/gmod-architecture
> 
> 
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] [Gmod-architecture] AJAX, WebService, Middleware, Chode

[Chris M. <cj...@fr...>]

Hi Xianhua

Note there is now a gmod-ajax list too.

I think a RESTFUL architecture makes a lot of sense for an AJAX  
application.

By the way, I think you mean "Chado". (try a google search for  
"define: chode" - I do hope chode doesn't catch on as another Chado  
mispronounciation!).

On Jan 18, 2007, at 7:47 AM, Xianhua Liu wrote:

> I am trying to build web interface to query and view data from  
> Chode with extension schema for bio-diversity data. I am planning  
> to use a 4-lawyer architecture as following:
>
> 1. AJAX: front-end
> 2. WebService: web lawyer
> 3. XXX: Middleware
> 4. Chode: back-end
>
> Why WebService?
> 1. independency of front-end on middleware.
> 2. loose coupling
> 3. easy data integration over web
> 4. ...
>
> Challenges:
> 1. standardization of web service interface
> 2. federalization of data representation model exchanged through  
> the web service (XML Schema, Ontology, ...?)
> 3. Performance?
>
> RESTful or SOAP:
> I prefer to choose RESTful web service, because it is easy, direct  
> and light.
>
> If someone have already tried the same and similar thing, please  
> share your experience and lessons and tech choises.
>
> Welcome every one to send comments about the idea, and suggestions  
> about techniques, software packages and any other sources.
>
> Thanks,
>
> Xianhua
>
>
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV________________________________ 
> _______________
> Gmod-architecture mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-architecture

[Gmod-ajax] Ajax-GBrowse hire

[Ian H. <ih...@be...>]

I'm pleased to announce that we have hired Mitch Skinner (CC'd), whom 
readers of the gmod-ajax list may recognize, as a full-time developer on 
the Ajax GBrowse project. He'll be working from Berkeley.

He will be starting some time around mid-January.

best wishes,
Ian Holmes

Re: [Gmod-ajax] storing primitives in memory

[Mitch S. <li...@ar...>]

I've implemented some of the things I talked about in my last message;
there are two patches attached, one for GD and one for gmod-ajax.  With
them both, my hardware can render Dmel chr. 4 mRNA (with the transcript
glyph) at zoom 1 in about five and a half minutes.

More inline:
On Mon, 2006-12-04 at 08:22 -0800, Mitchell Skinner wrote:
> * the TiledImage::AUTOLOAD routine was taking up a
> significant proportion of the time, especially on
> primitive-intensive tracks like the yeast_chr1
> translation tracks at full zoom.  I'm working on
> reimplementing the accessors using closures or
> Class::Struct, and reimplementing the GD primitive
> interception with closures

This is done.

> * on some tracks, GD::filledRectangle was taking the
> lion's share of the time (96% on one run that I
> measured for a track that was using the "transcript"
> glyph).  I looked at the gd c code, and
> gdImageFilledRectangle calls the retail 
> gdImageSetPixel function (which is non-trivial) in
> a nested loop.

The attached GD patch adds a special case to gdImageFilledRectangle that
uses memset on non-truecolor images where the rectangle is being filled
with a regular color.  If no one here sees any problems with it, I'll be
sending it on upstream.

The optimal case for the memset version is very wide (say, tens of
thousands of pixels) and short rectangles, just like the ones we're
doing at full zoom.  On rectangles like that, this patch makes
gdImageFilledRectangle up to 40x faster on my system.

> * for the less primitive-intensive tracks (like gene
> or mRNA) at high zoom levels, the gridlines are big
> memory users (for the in-memory primitive storage
> approach).  Rendering Dmel chrom 4 mRNA at zoom 1
> without gridlines reduced peak memory usage (RSS as
> measured by ps) from 653 megabytes to 169 megabytes.
> If we can find a way to avoid rendering the gridlines
> (like using PNG transparency as mentioned on the
> wiki) or if we can find a way to avoid storing the
> gridline primitives then I don't think memory usage
> will be a problem for these kinds of tracks.

I thought about trying to take the gridline code from TiledImagePanel
and copying it into TiledImage, but that would mean that TiledImage
would have to know about a lot more stuff than it does now; plus
blurring the line betweeen TiledImagePanel and the rest of the
pre-rendering code works against the goal of merging TiledImagePanel
back into BioPerl proper.

Instead, I took an approach that's a bit less efficient but IMHO cleaner
layering-wise: trying to represent all of the gridline information
without explicitly storing all of it.  In other words, the fact that the
gridline line primitives are so regular means that we can store the
gridline information a little more compactly.  I wrote a class that
compactly stores sequences of integers where the numbers are mostly
increasing by the same interval or mostly the same, and I used it to
store the arguments for vertical line primitives.  See CompactList.pm
and TiledImage::line in the patch.  This reduces memory usage on Dmel
chr. 4 mRNA at zoom 1 from 653 megabytes to 225 megabytes, and I think
it will scale fairly well to larger chromosomes.  The runtime cost is
several percent; I think it's a good tradeoff.

> * for the primitive-intensive tracks like the
> translation and dna tracks, I'm hoping that rendering
> smaller tile ranges will make the jobs fit in RAM.
> I'm not sure if we can avoid generating all
> primitives when we're not rendering the whole track,
> but in any case we should be able to avoid storing
> all those primitives.

I haven't done anything here yet, but I believe we can do something
similar the gridlines to store those primitives more compactly.


I think this all suggests that pre-rendering large chromosomes will
work; when I get a chance I'll be loading up and trying out the other
Drosophila chromosomes.  In the attached patch I've gone ahead and taken
out the code related to the primitive database.  If people are cool with
the patch, then I'll go ahead and commit; if not, I'd appreciate any
feedback.

Mitch

Re: [Gmod-ajax] storing primitives in memory

[Mitchell S. <mi...@ar...>]

Well, I've been spending some quality time with D.
melanogaster chromosome 4 and perl's DProf, and
I've learned a few things:

* the TiledImage::AUTOLOAD routine was taking up a
significant proportion of the time, especially on
primitive-intensive tracks like the yeast_chr1
translation tracks at full zoom.  I'm working on
reimplementing the accessors using closures or
Class::Struct, and reimplementing the GD primitive
interception with closures like this (in
TiledImage):

    foreach my $sub (keys %intercept) {
        no strict "refs";
        *$sub = sub  {
            my ($self, @args) = @_;
            my @originalArgs = @args;

            # get bounding box
            my @bb;
            @bb =  $self->getBoundingBox ($sub, @args);
            ...

* on some tracks, GD::filledRectangle was taking the
lion's share of the time (96% on one run that I
measured for a track that was using the "transcript"
glyph).  I looked at the gd c code, and
gdImageFilledRectangle calls the retail 
gdImageSetPixel function (which is non-trivial) in
a nested loop.  Since AFAIK we're not doing any
antialiasing or other compositing, it seems like
this could be improved a lot.  For the immediate
future the advice is to avoid rectangle based glyphs.

* for the less primitive-intensive tracks (like gene
or mRNA) at high zoom levels, the gridlines are big
memory users (for the in-memory primitive storage
approach).  Rendering Dmel chrom 4 mRNA at zoom 1
without gridlines reduced peak memory usage (RSS as
measured by ps) from 653 megabytes to 169 megabytes.
If we can find a way to avoid rendering the gridlines
(like using PNG transparency as mentioned on the
wiki) or if we can find a way to avoid storing the
gridline primitives then I don't think memory usage
will be a problem for these kinds of tracks.

* for the primitive-intensive tracks like the
translation and dna tracks, I'm hoping that rendering
smaller tile ranges will make the jobs fit in RAM.
I'm not sure if we can avoid generating all
primitives when we're not rendering the whole track,
but in any case we should be able to avoid storing
all those primitives.

If we can make all those improvements (not a huge if
IMO; call it a medium-small if) then I think
pre-rendering is the way to go.

Unless there are other reasons than speed and memory
usage to do render-on-demand?  I've been assuming
that rendering on demand would be a nontrivial amount
of work, not to mention that it's yet another CGI
sitting on top of a database (not sure how well it'll
scale), but admittedly I haven't thought all the way
through it.

If we don't need to render on demand, then AFAICT
we don't need the primitive database (?), and then
I'll feel comfortable producing a patch that takes
out the database bits.  It's not like I have some
kind of vendetta against the database bits, it's
just that making database/memory a run-time option
is that much extra work.

Mitch

[Gmod-ajax] Fwd: Open AJAX Alliance

[Ian H. <ih...@be...>]

The Open Ajax Alliance might be something to keep an eye on:

Begin forwarded message:

> Resent-From: <ih...@ca...>
> From: Jon Ferraiolo <jf...@us...>
> Date: June 7, 2006 5:30:03 PM PDT
> To: Ian Holmes <ih...@be...>
> Subject: Re: Open AJAX Alliance
>
> Hi Ian,
> Thanks for your interest! Sorry I was slow in responding.
>
> As you can tell, the OAA is still in the process of bootstrapping =20
> the web site. For the time being, here is some summary information =20
> in text form:
>
> ------
> The Open Ajax Alliance (OAA) has approximately 35 member =20
> organizations at this point and is growing. The chief goal is to =20
> accelerate customer success with Ajax by promoting a customer's =20
> ability to mix and match solutions from Ajax technology providers =20
> and by helping to drive the future of the Ajax ecosystem. Among the =20=

> organizations who have joined so far: IBM, Yahoo, Google, Mozilla, =20
> Opera, Adobe, Oracle, SAP, BEA, TIBCO, SoftwareAG, Eclipse =20
> Foundation, Intel, Novell, RedHat, Borland, Dojo Foundation, Zimbra =20=

> (leaders beyind the Kabuki toolkit), Zend (the PHP company), =20
> Backbase, Jackbe, Icesoft, Laszlo, and Nexaweb. Here are some links =20=

> to past articles on "Open Ajax":
>
> http://www-03.ibm.com/press/us/en/pressrelease/19187.wss - "Open =20
> Ajax" launched with 15 original members
> http://www-03.ibm.com/press/us/en/pressrelease/19623.wss - "Open =20
> Ajax" gains 13 additional members
> http://www.infoworld.com/article/06/05/22/78577_HNajaxforge_1.html =20
> - Press report on first Open Ajax Alliance meeting
>
> The OAA will provide value to the community on both the marketing =20
> communications and technical fronts. At this point, we have three =20
> working committees. The Marketing committee has responsibility for =20
> all of OAA's outbound activities, including what goes on the web =20
> site. A key activity in the short term is a whitepaper that defines =20=

> the Open Ajax vocabulary and architectural blueprint. The =20
> Interoperability committee has a short-term focus on defining =20
> integration specifications and conformance requirements that will =20
> enable customers to mix and match different Ajax toolkits within =20
> the same Ajax solution. The Declarative Markup committee works =20
> closely with the Interoperability committee to ensure the ability =20
> to mix XML markup from different Ajax toolkits. The two technical =20
> committees will pursue simple technical approaches that provide =20
> high levels of customer benefit on an accelerated schedule. The =20
> committees have phone calls every two weeks for one hour and =20
> collaborate on email and a wiki in between meetings. Face-to-face =20
> meetings happen approximately every 3 months. It is likely that a =20
> fourth committee on Mobile Ajax will start sometime soon.
>
> The primary reasons for joining the OAA would be to contribute to =20
> the committees or to gain early access to the content from the =20
> committee discussions. Joining is simple: (1) Have a quick phone =20
> call with me so that we can ask each other questions. Just propose =20
> a time when it would be convenient to talk , (2) Agree on an =20
> interim good-faith basis to a royalty-free, unencumbered IP policy =20
> for any contributions to and discussions within the alliance (a =20
> formal IP agreement is under development). (3) Send me an email =20
> saying that you want to join, along with the names and emails of =20
> participants and which committees people will join. We might get =20
> more formal in the future, but right now we are just a bunch of =20
> people who simply meet regularly acting in good faith and =20
> ultimately post information on a web site. There is no formal OAA =20
> organization and no bylaws at this point.
> ------
>
> If you think it is a good match to become a member and participate =20
> in the OAA, propose a time so we can talk on the phone. Until then,
>
> Best regards,
> Jon
>
> Jon Ferraiolo <jf...@us...>
> Web Architect, Emerging Technologies
> IBM, Menlo Park, CA
> Mobile: +1-650-464-7817
=EF=BF=BCIan Holmes <ih...@be...>
>
>
> Ian Holmes <ih...@be...>
> 06/01/2006 09:44 PM
>
=EF=BF=BC
> To
=EF=BF=BC
> Jon Ferraiolo/Menlo Park/IBM@IBMUS
=EF=BF=BC
> cc
=EF=BF=BC
=EF=BF=BC
> Subject
=EF=BF=BC
> Open AJAX Alliance
=EF=BF=BC=EF=BF=BC
> Hello Jon,
>
> I'm a computational biologist developing bioinformatics software in
> academia (I'm an assistant prof at UC Berkeley).
>
> One of our lab projects involves an AJAX-ified genome browser for
> viewing genome annotations. A prototype is at http://=20
> genome.biowiki.org/
> Our collaborator is Lincoln Stein, who developed two Perl modules
> (CGI.pm and GD.pm) that have been pretty influential in the
> development of web apps.
>
> I'm very interested in joining the Open AJAX Alliance, which I read
> about over at ajaxian.com. Can you tell me more about it?
>
> We'll be developing a lot more bioinformatics stuff in AJAX...
> looking forward to seeing how this tech progresses.
>
> Best wishes,
> Ian Holmes
>
>

Re: [Gmod-ajax] ViewerComponent.js cursor

[Mitch S. <li...@ar...>]

On Mon, 2006-11-27 at 13:17 -0800, Andrew Uzilov wrote:
> I checked your cursor patch and it looks good; I'm going to check over
> the two others, apply them and commit them shortly.

Don't feel too rushed; I'm not sure if the memory storage patches are
ready to go in.  My main goal with them was to make something simple
that could be compared with the database version.  If you like the idea
then there are still some decisions to make:

If we want to make memory/database a run-time option, then some more
reorganization is needed.  Maybe DatabasePrimStorage and
MemoryPrimStorage classes?  Anyway, the patch as it is certainly can't
switch at run-time.

If we want to give up on the database version and take it out, then a
different reorganization would be better IMO.  My first impulse would be
to merge BatchTiledImage with TiledImage, but I'm not sure how well the
users of TiledImage (other than BatchTiledImage) would work with that.

If we're not sure yet, then I think it's fine to leave the patch out of
the tree for a while.

My hope was that memory or database would be the clear winner and that
would save us from carrying the extra code.  As I see things, it depends
on whether or not the memory version uses too much ram on large/complex
tracks, and whether or not the memory version can be made fast enough to
make render-on-demand less interesting.

I'm not sure how much work you've done on render-on-demand.  Also, are
there reasons for it other than pre-rendering taking too long?

If no one beats me to it, I'll probably get the chance to make some
measurements before next week.  Hopefully that will answer the speed and
memory usage questions.  I'm also hopeful that it's possible to improve
on those things a bit from the current version.  I've been trying to
render without -r, and it seems to be taking much more memory than any
individual track/zoom render, which surprised me because I thought each
track/zoom combo was a different BatchTiledImage and I expected them to
get garbage collected after they got used.  Clearly I'm missing
something at this point, and it needs some more work.

Mitch

Re: [Gmod-ajax] ViewerComponent.js cursor

[Andrew U. <and...@gm...>]

Hi Mitch....

Sorry for taking so long to reply (this goes out to everyone,
actually), I've been away for a bit, among other things... but now I'm
trying to catch up.

I checked your cursor patch and it looks good; I'm going to check over
the two others, apply them and commit them shortly.

You should actually be registered as a developer on the SourceForge
GMOD project so you can commit the changes directly... I'm not an
admin, so I can't do it, but you should e-mail your SF login to Scott
Cain (ca...@cs...) who can add you to the project.

Thanks for your help!


Andrew


On 11/16/06, Mitch Skinner <li...@ar...> wrote:
> I just randomly came across this as I was working my way through reading
> the code, and I happened to have dealt with this recently.
>
> If the patch doesn't make it onto the list as an attachment, I'll try
> again and paste it into the body of the message.
>
> It would be nice if the cursor was the open hand (-moz-grab) while over
> a draggable area and the closed hand (-moz-grabbing) while actually
> dragging.  We could make dragDiv*Style.cursor="-moz-grab" in
> VewerComponent.js to start with, but I imagine we'd want the cursor to
> be something other than -moz-grab when it's over an actual feature, and
> I can't think of how to do that off the top of my head.  Can you attach
> styles to imagemap regions?
>
> I haven't tested this in context, as I haven't gotten the full
> ajax-gbrowse setup going on my machine yet, but I believe it's correct,
> or at least close.
>
> Mitch
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>
>
>

Re: [Gmod-ajax] storing primitives in memory

[Ian H. <ih...@be...>]

Brilliant, thanks Mitch! This is very promising.

It might also be interesting to investigate (i) how many features the  
in-memory approach can handle (e.g. on Drosophila, human chromosomes)  
and (ii) the performance compared to the render-on-demand approach  
(which fills the database with primitives, but does not render any  
tiles until someone views that particular tile).

Ian

On Nov 26, 2006, at 7:38 AM, Mitch Skinner wrote:

> Hello,
>
> Reading the wiki, it looks like in-memory was the original  
> approach, but
> it was too slow because it was rendering all the primitives for each
> tile.  As I understand it, storing the primitives in a database  
> buys you
> the ability to query for just those primitives that overlap the  
> current
> tile.
>
> After reading Andrew's message that said that database access was the
> bottleneck, I wanted to try another take on the in-memory approach.
> Instead of storing all the primitives in one big array, I'm keeping an
> array of arrays of primitives, with one array of primitives per
> rendering tile.  This version of GDRecordPrimitives adds its primitive
> to the primitive array of each of the rendering tiles that overlap  
> with
> the primitive.  There's also a separate global primitive array.
>
> On the yeast_chr1 that comes with gbrowse, rendering track 3 (named
> genes) at zoom level 1 with the in-memory patch takes about a third of
> the time that the db version does (see below).  I haven't gotten chado
> going on my machine yet, so I'd be interested in seeing comparisons
> with/without the patches from anyone who wants to do one on their data
> (particularly with larger/more complex tracks).
>
> I've attached two patches (against CVS HEAD); the first one
> (ti-prim-api.patch) changes the primitive storage api in TiledImage a
> little so that I could cleanly override those functions in
> BatchTiledImage.  Basically, it moves some work between callers and
> callees so that the in-memory version doesn't have to do the
> serialization work.
>
> The second patch (bti-memstorage.patch, which depends on the first
> patch) changes BatchTiledImage to override the primitive storage  
> methods
> of TiledImage.  I put this stuff in BatchTiledImage because
> BatchTiledImage is the class that knows about the rendering tile
> dimensions, which I wanted to use.  I thought this was the minimally
> invasive way to do it; I wanted to make it easy to see what I was  
> trying
> to do by reading the patch.  If there's consensus that this is the way
> to go, then more reorganization would probably be a good idea.
>
> Regards,
> Mitch
>
> This is on a 2.2 GHz Athlon 64 -
>
> [mitch@firebolt server]$ patch < ti-prim-api.patch
> patching file TiledImage.pm
> [mitch@firebolt server]$ mkdir testdb
> [mitch@firebolt server]$ time ./generate-tiles.pl -c
> ~/apache/conf/gbrowse.conf/ -o testdb/ -s yeast_chr1 -m 0 -v 1 --no- 
> xml
> --print-tile-nums --render-gridlines -l I -r t3z1r1-2303 &> withdb.out
>
> real    4m10.798s
> user    2m16.425s
> sys     0m6.968s
> [mitch@firebolt server]$ patch < bti-memstorage.patch
> patching file BatchTiledImage.pm
> [mitch@firebolt server]$ mkdir testmem
> [mitch@firebolt server]$ time ./generate-tiles.pl -c
> ~/apache/conf/gbrowse.conf/ -o testmem/ -s yeast_chr1 -m 0 -v 1 -- 
> no-xml
> --print-tile-nums --render-gridlines -l I -r t3z1r1-2303 &>  
> withmem.out
>
> real    1m23.400s
> user    1m18.229s
> sys     0m2.572s
> [mitch@firebolt server]$ diff -r testdb testmem
> [mitch@firebolt server]$
> <bti-memstorage.patch>
> <ti-prim-api.patch>
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV________________________________ 
> _______________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax

[Gmod-ajax] storing primitives in memory

[Mitch S. <li...@ar...>]

Hello,

Reading the wiki, it looks like in-memory was the original approach, but
it was too slow because it was rendering all the primitives for each
tile.  As I understand it, storing the primitives in a database buys you
the ability to query for just those primitives that overlap the current
tile.

After reading Andrew's message that said that database access was the
bottleneck, I wanted to try another take on the in-memory approach.
Instead of storing all the primitives in one big array, I'm keeping an
array of arrays of primitives, with one array of primitives per
rendering tile.  This version of GDRecordPrimitives adds its primitive
to the primitive array of each of the rendering tiles that overlap with
the primitive.  There's also a separate global primitive array.

On the yeast_chr1 that comes with gbrowse, rendering track 3 (named
genes) at zoom level 1 with the in-memory patch takes about a third of
the time that the db version does (see below).  I haven't gotten chado
going on my machine yet, so I'd be interested in seeing comparisons
with/without the patches from anyone who wants to do one on their data
(particularly with larger/more complex tracks).

I've attached two patches (against CVS HEAD); the first one
(ti-prim-api.patch) changes the primitive storage api in TiledImage a
little so that I could cleanly override those functions in
BatchTiledImage.  Basically, it moves some work between callers and
callees so that the in-memory version doesn't have to do the
serialization work.

The second patch (bti-memstorage.patch, which depends on the first
patch) changes BatchTiledImage to override the primitive storage methods
of TiledImage.  I put this stuff in BatchTiledImage because
BatchTiledImage is the class that knows about the rendering tile
dimensions, which I wanted to use.  I thought this was the minimally
invasive way to do it; I wanted to make it easy to see what I was trying
to do by reading the patch.  If there's consensus that this is the way
to go, then more reorganization would probably be a good idea.  

Regards,
Mitch

This is on a 2.2 GHz Athlon 64 -

[mitch@firebolt server]$ patch < ti-prim-api.patch
patching file TiledImage.pm
[mitch@firebolt server]$ mkdir testdb
[mitch@firebolt server]$ time ./generate-tiles.pl -c
~/apache/conf/gbrowse.conf/ -o testdb/ -s yeast_chr1 -m 0 -v 1 --no-xml
--print-tile-nums --render-gridlines -l I -r t3z1r1-2303 &> withdb.out

real    4m10.798s
user    2m16.425s
sys     0m6.968s
[mitch@firebolt server]$ patch < bti-memstorage.patch
patching file BatchTiledImage.pm
[mitch@firebolt server]$ mkdir testmem
[mitch@firebolt server]$ time ./generate-tiles.pl -c
~/apache/conf/gbrowse.conf/ -o testmem/ -s yeast_chr1 -m 0 -v 1 --no-xml
--print-tile-nums --render-gridlines -l I -r t3z1r1-2303 &> withmem.out

real    1m23.400s
user    1m18.229s
sys     0m2.572s
[mitch@firebolt server]$ diff -r testdb testmem
[mitch@firebolt server]$

Re: [Gmod-ajax] Feature Mark Up

[Steve T. <st...@mo...>]

Hi,

Been away for a few days but I have committed the code now. To get the required mark up you will have to run generate-tiles.pl which will generate html files containing the image map data in the same 
dir as the .png files. Currently, all the mouse over events are also stored in the html too. Yuk. This is a pretty low tech way of doing things and not very desirable in the long run esp. when we 
start populating it with real data... I tried to set mouseovers the on the fly using Ajax.Updater/onSuccess but had some problems that were related to caching (I think). I am sure Andrew would be able 
to fix this extremely quickly.

I have added a new var called htmlFiles in ViewerComponent.js. Setting this to 1 causes the browser to update the divs by pulling in the html, setting it to 0 means it uses the old style of specifying 
the path of the local image src.  htmlFiles default is 1 so if you do a cvs update and haven't run generate-tiles.pl remember to change it to 0.

> Actually, what I wrote before isn't quite correct; while the gbrowse cgi
> doesn't server up raw information, other cgis in the gbrowse
> distribution can.  You can get both DAS1 and biomoby, though I don't
> know that much the details of what the streams look like (DAS2 is XML, I
> don't think DAS1 is).  It is possible that javascript client-side code
> is already written for these services, I just don't know about it.
> 

Gbrowse Moby certainly looks like it could be a viable solution. In theory it seems to have in its favour that it should be straightforward to GET the data via a URL...

Cheers,

Steve


> Scott
> 
> 
> On Thu, 2006-11-16 at 09:54 -0800, Ian Holmes wrote:
> 
>>Chris Mungall mentioned something a while back about some sort of  
>>universal markup language he & Seth had specced out for GO. He also  
>>suggested some kind of protocol for publishing feature annotations in  
>>a sort of universal ontology framework.
>>
>>Chris, are you on this list? it'd be good to get these & related  
>>ideas in more concrete form...
>>
>>As well as rich pop-up info, we need to develop client-server  
>>protocols for (i) search and (ii) annotation upload.
>>
>>
>>On Nov 16, 2006, at 8:46 AM, Scott Cain wrote:
>>
>>
>>>On Wed, 2006-11-15 at 20:47 +0000, Stephen Taylor wrote:
>>>
>>>>Hi Andrew,
>>>>
>>>>>Next comes the figuring out how to populate the pop up menu with
>>>>>relevant info...
>>>>>
>>>>>
>>>>
>>>>Yep. This is more tricky... I think we should try and use as many
>>>>'official' GBrowse calls as possible to ensure future  
>>>>compatibility. I
>>>>wonder if Lincoln or Scott (if they are reading this) could offer any
>>>>advice?
>>>>
>>>
>>>The GBrowse cgi doesn't expose much in terms of fundamentals that one
>>>could use for populating popup menus with data.  Currently, people who
>>>use popups with gbrowse create all of the data up front and encode  
>>>it in
>>>javascript elements in the imagemap.  I suppose that works fine from a
>>>cgi point of view, but is not as appropriate for an ajax  
>>>application (is
>>>it?).  So if we want to make it work in an 'ajaxy' way, we'll need
>>>either to expand the api of the gbrowse cgi, or create another cgi for
>>>serving up xml for requests like that.
>>>
>>>If you want to see an example of how people are currently generating
>>>popups, you can look at this example:
>>>
>>>http://www.plasmodb.org/cgi-bin/gbrowse/plasmodb/? 
>>>start=101357;stop=121356;ref=MAL12;width=800;version=100;label=Annotat 
>>>edGenes-SyntenySpansVivaxMC- 
>>>SyntenyGenesVivaxMC;id=9cf36ae32368b7ec3e3c516c08c358d4
>>>
>>>which is done with this conf file:
>>>
>>>http://gmod.cvs.sourceforge.net/gmod/Generic-Genome-Browser/contrib/ 
>>>SynView/gbrowse.conf/plasmodb.conf? 
>>>revision=1.1.2.1&view=markup&pathrev=gbrowse-session
>>>
>>>Scott
>>>
>>>-- 
>>>---------------------------------------------------------------------- 
>>>--
>>>Scott Cain, Ph. D.                                          
>>>ca...@cs...
>>>GMOD Coordinator (http://www.gmod.org/)                      
>>>216-392-3087
>>>Cold Spring Harbor Laboratory
>>>---------------------------------------------------------------------- 

Re: [Gmod-ajax] [personal] Re: [unclassified] javascript client code for BioMoby?

[Scott C. <cai...@gm...>]

(I cc'ed to the GBrowse-AJAX mailing list, since you've offered code
help :-)

Mark,

I agree that I (or anyone else) shouldn't be writing another cgi to sit
on top of a database, I just thought it was funny that my first instinct
was to write something new without thinking too much about alternatives.

Hopefully, BioMoby or DAS javascript clients will provide the types of
data they are after.

Scott


On Thu, 2006-11-16 at 11:07 -0800, Mark Wilkinson wrote:
> On Thu, 2006-11-16 at 13:58 -0500, Scott Cain wrote:
>=20
> > into the database and the funny thing is that I thought I would need to
> > write a cgi from scratch to serve up the type of data they would
> > probably like.  It took me a few hours to remember that GBrowse serves
> > up both biomoby and das without too much work.
>=20
> The LAST thing the world needs is another from-scratch CGI program
> sitting on top of a database!  Good Lord!  Lincoln wrote his
> bioinformatics nation paper years ago and nobody seems to be listening
> to him...  even the people in his own lab ;-)
>=20
> Please Please Please use BioMoby and/or DAS to achieve what you are
> trying to achieve!  Eddie and I will offer whatever help and/or code you
> need with big smiles on our faces :-)
>=20
> Cheers!
>=20
> M
>=20
>=20
>=20
--=20
------------------------------------------------------------------------
Scott Cain, Ph. D.                                   cai...@gm...
GMOD Coordinator (http://www.gmod.org/)                     216-392-3087
Cold Spring Harbor Laboratory

Re: [Gmod-ajax] Feature Mark Up

[Scott C. <ca...@cs...>]

Actually, what I wrote before isn't quite correct; while the gbrowse cgi
doesn't server up raw information, other cgis in the gbrowse
distribution can.  You can get both DAS1 and biomoby, though I don't
know that much the details of what the streams look like (DAS2 is XML, I
don't think DAS1 is).  It is possible that javascript client-side code
is already written for these services, I just don't know about it.

Scott


On Thu, 2006-11-16 at 09:54 -0800, Ian Holmes wrote:
> Chris Mungall mentioned something a while back about some sort of =20
> universal markup language he & Seth had specced out for GO. He also =20
> suggested some kind of protocol for publishing feature annotations in =20
> a sort of universal ontology framework.
>=20
> Chris, are you on this list? it'd be good to get these & related =20
> ideas in more concrete form...
>=20
> As well as rich pop-up info, we need to develop client-server =20
> protocols for (i) search and (ii) annotation upload.
>=20
>=20
> On Nov 16, 2006, at 8:46 AM, Scott Cain wrote:
>=20
> > On Wed, 2006-11-15 at 20:47 +0000, Stephen Taylor wrote:
> >> Hi Andrew,
> >>> Next comes the figuring out how to populate the pop up menu with
> >>> relevant info...
> >>>
> >>>
> >> Yep. This is more tricky... I think we should try and use as many
> >> 'official' GBrowse calls as possible to ensure future =20
> >> compatibility. I
> >> wonder if Lincoln or Scott (if they are reading this) could offer any
> >> advice?
> >>
> > The GBrowse cgi doesn't expose much in terms of fundamentals that one
> > could use for populating popup menus with data.  Currently, people who
> > use popups with gbrowse create all of the data up front and encode =20
> > it in
> > javascript elements in the imagemap.  I suppose that works fine from a
> > cgi point of view, but is not as appropriate for an ajax =20
> > application (is
> > it?).  So if we want to make it work in an 'ajaxy' way, we'll need
> > either to expand the api of the gbrowse cgi, or create another cgi for
> > serving up xml for requests like that.
> >
> > If you want to see an example of how people are currently generating
> > popups, you can look at this example:
> >
> > http://www.plasmodb.org/cgi-bin/gbrowse/plasmodb/?=20
> > start=3D101357;stop=3D121356;ref=3DMAL12;width=3D800;version=3D100;labe=
l=3DAnnotat=20
> > edGenes-SyntenySpansVivaxMC-=20
> > SyntenyGenesVivaxMC;id=3D9cf36ae32368b7ec3e3c516c08c358d4
> >
> > which is done with this conf file:
> >
> > http://gmod.cvs.sourceforge.net/gmod/Generic-Genome-Browser/contrib/=20
> > SynView/gbrowse.conf/plasmodb.conf?=20
> > revision=3D1.1.2.1&view=3Dmarkup&pathrev=3Dgbrowse-session
> >
> > Scott
> >
> > --=20
> > ----------------------------------------------------------------------=20
> > --
> > Scott Cain, Ph. D.                                         =20
> > ca...@cs...
> > GMOD Coordinator (http://www.gmod.org/)                     =20
> > 216-392-3087
> > Cold Spring Harbor Laboratory
> > ----------------------------------------------------------------------=20
> > ---
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to =20
> > share your
> > opinions on IT & business topics through brief surveys - and earn cash
> > http://www.techsay.com/default.php?=20
> > page=3Djoin.php&p=3Dsourceforge&CID=3DDEVDEV___________________________=
_____=20
> > _______________
> > Gmod-ajax mailing list
> > Gmo...@li...
> > https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>=20
>=20
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share y=
our
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
--=20
------------------------------------------------------------------------
Scott Cain, Ph. D.                                         ca...@cs...
GMOD Coordinator (http://www.gmod.org/)                     216-392-3087
Cold Spring Harbor Laboratory

Re: [Gmod-ajax] Feature Mark Up

[Ian H. <ih...@be...>]

Chris Mungall mentioned something a while back about some sort of  
universal markup language he & Seth had specced out for GO. He also  
suggested some kind of protocol for publishing feature annotations in  
a sort of universal ontology framework.

Chris, are you on this list? it'd be good to get these & related  
ideas in more concrete form...

As well as rich pop-up info, we need to develop client-server  
protocols for (i) search and (ii) annotation upload.


On Nov 16, 2006, at 8:46 AM, Scott Cain wrote:

> On Wed, 2006-11-15 at 20:47 +0000, Stephen Taylor wrote:
>> Hi Andrew,
>>> Next comes the figuring out how to populate the pop up menu with
>>> relevant info...
>>>
>>>
>> Yep. This is more tricky... I think we should try and use as many
>> 'official' GBrowse calls as possible to ensure future  
>> compatibility. I
>> wonder if Lincoln or Scott (if they are reading this) could offer any
>> advice?
>>
> The GBrowse cgi doesn't expose much in terms of fundamentals that one
> could use for populating popup menus with data.  Currently, people who
> use popups with gbrowse create all of the data up front and encode  
> it in
> javascript elements in the imagemap.  I suppose that works fine from a
> cgi point of view, but is not as appropriate for an ajax  
> application (is
> it?).  So if we want to make it work in an 'ajaxy' way, we'll need
> either to expand the api of the gbrowse cgi, or create another cgi for
> serving up xml for requests like that.
>
> If you want to see an example of how people are currently generating
> popups, you can look at this example:
>
> http://www.plasmodb.org/cgi-bin/gbrowse/plasmodb/? 
> start=101357;stop=121356;ref=MAL12;width=800;version=100;label=Annotat 
> edGenes-SyntenySpansVivaxMC- 
> SyntenyGenesVivaxMC;id=9cf36ae32368b7ec3e3c516c08c358d4
>
> which is done with this conf file:
>
> http://gmod.cvs.sourceforge.net/gmod/Generic-Genome-Browser/contrib/ 
> SynView/gbrowse.conf/plasmodb.conf? 
> revision=1.1.2.1&view=markup&pathrev=gbrowse-session
>
> Scott
>
> -- 
> ---------------------------------------------------------------------- 
> --
> Scott Cain, Ph. D.                                          
> ca...@cs...
> GMOD Coordinator (http://www.gmod.org/)                      
> 216-392-3087
> Cold Spring Harbor Laboratory
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV________________________________ 
> _______________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] Feature Mark Up

[Scott C. <ca...@cs...>]

On Wed, 2006-11-15 at 20:47 +0000, Stephen Taylor wrote:
> Hi Andrew,
> >Next comes the figuring out how to populate the pop up menu with
> >relevant info...
> > =20
> >
> Yep. This is more tricky... I think we should try and use as many=20
> 'official' GBrowse calls as possible to ensure future compatibility. I=20
> wonder if Lincoln or Scott (if they are reading this) could offer any=20
> advice?
>=20
The GBrowse cgi doesn't expose much in terms of fundamentals that one
could use for populating popup menus with data.  Currently, people who
use popups with gbrowse create all of the data up front and encode it in
javascript elements in the imagemap.  I suppose that works fine from a
cgi point of view, but is not as appropriate for an ajax application (is
it?).  So if we want to make it work in an 'ajaxy' way, we'll need
either to expand the api of the gbrowse cgi, or create another cgi for
serving up xml for requests like that.

If you want to see an example of how people are currently generating
popups, you can look at this example:

http://www.plasmodb.org/cgi-bin/gbrowse/plasmodb/?start=3D101357;stop=3D121=
356;ref=3DMAL12;width=3D800;version=3D100;label=3DAnnotatedGenes-SyntenySpa=
nsVivaxMC-SyntenyGenesVivaxMC;id=3D9cf36ae32368b7ec3e3c516c08c358d4

which is done with this conf file:

http://gmod.cvs.sourceforge.net/gmod/Generic-Genome-Browser/contrib/SynView=
/gbrowse.conf/plasmodb.conf?revision=3D1.1.2.1&view=3Dmarkup&pathrev=3Dgbro=
wse-session

Scott

--=20
------------------------------------------------------------------------
Scott Cain, Ph. D.                                         ca...@cs...
GMOD Coordinator (http://www.gmod.org/)                     216-392-3087
Cold Spring Harbor Laboratory

[Gmod-ajax] ViewerComponent.js cursor

[Mitch S. <li...@ar...>]

I just randomly came across this as I was working my way through reading
the code, and I happened to have dealt with this recently.

If the patch doesn't make it onto the list as an attachment, I'll try
again and paste it into the body of the message.

It would be nice if the cursor was the open hand (-moz-grab) while over
a draggable area and the closed hand (-moz-grabbing) while actually
dragging.  We could make dragDiv*Style.cursor="-moz-grab" in
VewerComponent.js to start with, but I imagine we'd want the cursor to
be something other than -moz-grab when it's over an actual feature, and
I can't think of how to do that off the top of my head.  Can you attach
styles to imagemap regions?

I haven't tested this in context, as I haven't gotten the full
ajax-gbrowse setup going on my machine yet, but I believe it's correct,
or at least close.

Mitch

Re: [Gmod-ajax] Feature Mark Up

[Mitch S. <li...@ar...>]

On Wed, 2006-11-15 at 20:47 +0000, Stephen Taylor wrote:
> I had it set on mouseover as you described and it felt not quite right 
> for some reason but having a pop up on a hover is how we use the cgi 
> GBrowse and it works great. We can try out a few options and see what's 
> best and I think you are right about keeping the click events free. A 
> rubber banding mechanism might be a good idea in the future to zoom in, 
> for example.

Clearly I'm coming into this discussion half-way, so maybe you've
already talked about this, but -

The pop up menu seems pretty much like a regular context menu that would
be shown on a right-click.  I know dojo has a widget for this; not sure
about prototype or prototype-based js libs.  It sounds fairly
straightforward to roll this bit of functionality on your own, though:
http://lab.artlung.com/oncontextmenu/
The Dojo version does do some extra work to support Opera and Konqueror
which apparently don't have this natively.  Of course, this leaves the
left-click open for dragging or selection.

Maybe people aren't terribly used to web apps providing their own
context menus yet, but I've seen this in Yahoo's new mail app (with
accelerator keys, even!), and it fits pretty well with general desktop
UI habits.

Mitch

Re: [Gmod-ajax] Feature Mark Up

[Stephen T. <st...@mo...>]

Hi Andrew,

>
>Yep, it works and looks good, you should check it in.  The only issue
>I see is that the window scrolls back to top whenever the pop up menu
>fires... maybe it has to do with its positioning in the DOM, I'm not
>sure... it should be a child of outerDivMain, I think, not the entire
>document.  But I'd have to play with it to figure it out.
>
>  
>
I hadn't noticed that. I'll have a look.

>We were thinking about having the pop up menu behave a little
>differently - it shoud pop up after hovering over a feature for a
>fraction of a second or so, then disappear when dragging starts.  The
>reason for this is that single-clicking should be reserved for other
>things, such as dragging or maybe drawing a little box or something to
>select features, etc. (this will come later, it's not well-defined at
>the moment).  Also, double-clicking to center should be unhindered by
>a menu popping up after the first click, but I think that can be taken
>care of by cancelling the results of a single-click event if a
>double-click comes in... somehow.
>  
>
I had it set on mouseover as you described and it felt not quite right 
for some reason but having a pop up on a hover is how we use the cgi 
GBrowse and it works great. We can try out a few options and see what's 
best and I think you are right about keeping the click events free. A 
rubber banding mechanism might be a good idea in the future to zoom in, 
for example.

>I also see that double-clicking to center (on the main view cell, not
>the ruler cell) is broken now, but I think I know why and I can fix it
>when the code is in CVS.
>
>  
>

Ok will check it in.

>Next comes the figuring out how to populate the pop up menu with
>relevant info...
>  
>
Yep. This is more tricky... I think we should try and use as many 
'official' GBrowse calls as possible to ensure future compatibility. I 
wonder if Lincoln or Scott (if they are reading this) could offer any 
advice?
Cheers,

Steve

>>
>>    
>>
>>>I found the problem.  Everywhere, including "prototype_gbrowse.html",
>>>your URLs should be "https", not "http"... on a whim I portscanned
>>>{slave,gbrowse}.molbiol.ox.ac.uk and noticed port 80 is closed, but
>>>443 is open... or at least for U.S.-originating packets.
>>>
>>>So this one:
>>>
>>>https://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000
>>>
>>>works just fine.  This one:
>>>
>>>https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>>
>>>seems to load the XHTML, but nothing else.  The problem seems to be
>>>that the XHTML has "http" instead of "https" for the JavaScript
>>>library URLs, so I can't load the JavaScript libraries.  If the URLs
>>>for libraries were changed to something like:
>>>
>>>https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/client/View.js
>>>
>>>then they would load.
>>>
>>>I guess there is some firewall config causing this problem... although
>>>I'm puzzled why it works for someone from Australia but not the U.S.
>>>
>>>      
>>>
>>We have worked out what the problem is and it is down to some configuration issues set up by the Oxford IT service which I won't go into since it's so bizarre (unless you're *really* interested :-)).
>>http will work in a day or so but in the meantime have a look on our test server. This *should* work...
>>
>>http://jenna.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>
>>I have also added a js pop-up based on one that was originally developed by Sylvain Gaillard and Gaëtan Droc.
>>
>>If it all works your end I will check it in.
>>
>>Cheers,
>>
>>Steve
>>    
>>
>>>On 11/14/06, Steve Taylor <st...@mo...> wrote:
>>>
>>>      
>>>
>>>>Hi Andrew,
>>>>
>>>>        
>>>>
>>>>>This sounds like excellent news, thanks!  But, I'm getting timed-out
>>>>>loading both pages you linked to... can you let us know when they're
>>>>>back up?  I would love to take a look.
>>>>>
>>>>>          
>>>>>
>>>>Hmmm. I have tested both here and they work reasonably quickly although are more slow to load than the original AjaxGBrowse...I also got a got someone to test in Australia and he said it loaded ok
>>>>though slightly more slowly.
>>>>
>>>>Can you try again please? Has anyone else on the list had time out problems? Again the URLs to try are:
>>>>
>>>>Ajax Gbrowse
>>>>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>>>
>>>>Original Gbrowse
>>>>http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000
>>>>
>>>>
>>>>        
>>>>
>>>>>You should commit the code to CVS... I've also made a few changes
>>>>>across the server-side code (including generate-tiles) to test
>>>>>on-demand tile rendering, which will be useful for large genomes, and
>>>>>to fix some stupid bugs left in TiledImage, etc. from back in the day
>>>>>(e.g. database connections not hanging up), which you probably noticed
>>>>>and fixed by now also.  Once your code is in, I'll merge in my
>>>>>changes.
>>>>>
>>>>>Just to let everyone know about the "render on demand" stuff... the
>>>>>idea is that we still fill a database with primitives, but only render
>>>>>tiles as they are request by user (they are also saved, so no need to
>>>>>re-render unless the underlying data changes, so we'll have some sort
>>>>>of "dirty tile" flag or some such).
>>>>>
>>>>>This will enable people to put up large genomes without pre-rendering
>>>>>all the tiles.  Currently, the rendering step is the major limiting
>>>>>factor (my benchmarks show around 90% of the actual generate-tiles.pl
>>>>>time is spent on database lookup of primitives).  For example, Dmel
>>>>>arm 4 takes a full day to render on our 24-CPU cluster, but only an
>>>>>hour of that time is spent filling the database.
>>>>>
>>>>>I'm trying a few things to optimize this right now, including the
>>>>>on-demand rendering, and I'll keep the list updated once I get
>>>>>somewhere (the things I'm trying can be found on
>>>>>http://biowiki.org/view/GBrowse/WishList under the "rendering
>>>>>optimizations" sublist).  I think we could seriously reduce the
>>>>>fill/access time, but until then, on-demand rendering will at least
>>>>>make large genomes feasible.
>>>>>
>>>>>          
>>>>>
>>>>Sounds good. Look forward to seeing that!
>>>>
>>>>Steve
>>>>
>>>>        
>>>>
>>>>>On 11/13/06, Steve Taylor <st...@mo...> wrote:
>>>>>
>>>>>
>>>>>          
>>>>>
>>>>>>Hi,
>>>>>>
>>>>>>I thought I'd send this to open up the discussions on the new list!:-)
>>>>>>
>>>>>>I haven't checked in this code yet but I have modified generate-tiles.pm and BatchTilesImage.pm to allow marking up of the features in each of the tiles using client side image maps. We may want to
>>>>>>change this later to do CSS maps but that would be only a small code modification. It basically generates an html file per tile with an inlined image and the appropriate map coordinates. The browser
>>>>>>XMLHttpRequests the data over when it needs to using Ajax.Updater (from the js library Prototype).
>>>>>>
>>>>>>You can see what it looks like on:
>>>>>>
>>>>>>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>>>>>
>>>>>>Try dragging and clicking on features.
>>>>>>
>>>>>>Advantages of current implementation
>>>>>>-----------------------------------
>>>>>>- should work on all browsers and doesn't require any fancy javascript
>>>>>>- will support polygons, ellipses etc
>>>>>>- allows distribution on different servers for tiles
>>>>>>
>>>>>>Disadvantages of implementation
>>>>>>-------------------------------
>>>>>>- all html per tile currently generated server side (not sure if this is an advantage or disadvantage at this stage) but may be a pain to do updates
>>>>>>
>>>>>>Obviously where javascript:alert('Active image map') you should have the relevant details of the feature and a nicer pop up. The js alerts are pretty annoying at the moment when dragging! I intend to
>>>>>>put js pop ups similar to the ones we have at http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000.
>>>>>>
>>>>>>Let me know what you think. I will check it in if people think it is worthwhile at this stage.
>>>>>>
>>>>>>Regards,
>>>>>>
>>>>>>Steve
>>>>>>------------------------------------------------------------------
>>>>>>Head of Computational Biology Research Group
>>>>>>Medical Sciences Division
>>>>>>Weatherall Institute of Molecular Medicine/Sir William Dunn School
>>>>>>Oxford University
>>>>>>Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
>>>>>>Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
>>>>>>Web: http://www.compbio.ox.ac.uk
>>>>>>
>>>>>>            
>>>>>>
>>-------------------------------------------------------------------------
>>Take Surveys. Earn Cash. Influence the Future of IT
>>Join SourceForge.net's Techsay panel and you'll get the chance to share your
>>opinions on IT & business topics through brief surveys - and earn cash
>>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>>_______________________________________________
>>Gmod-ajax mailing list
>>Gmo...@li...
>>https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>>
>>    
>>
>
>-------------------------------------------------------------------------
>Take Surveys. Earn Cash. Influence the Future of IT
>Join SourceForge.net's Techsay panel and you'll get the chance to share your
>opinions on IT & business topics through brief surveys - and earn cash
>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>_______________________________________________
>Gmod-ajax mailing list
>Gmo...@li...
>https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>  
>

Re: [Gmod-ajax] Feature Mark Up

[Andrew U. <and...@gm...>]

Hi Steve...

Yep, it works and looks good, you should check it in.  The only issue
I see is that the window scrolls back to top whenever the pop up menu
fires... maybe it has to do with its positioning in the DOM, I'm not
sure... it should be a child of outerDivMain, I think, not the entire
document.  But I'd have to play with it to figure it out.

We were thinking about having the pop up menu behave a little
differently - it shoud pop up after hovering over a feature for a
fraction of a second or so, then disappear when dragging starts.  The
reason for this is that single-clicking should be reserved for other
things, such as dragging or maybe drawing a little box or something to
select features, etc. (this will come later, it's not well-defined at
the moment).  Also, double-clicking to center should be unhindered by
a menu popping up after the first click, but I think that can be taken
care of by cancelling the results of a single-click event if a
double-click comes in... somehow.

I also see that double-clicking to center (on the main view cell, not
the ruler cell) is broken now, but I think I know why and I can fix it
when the code is in CVS.

Next comes the figuring out how to populate the pop up menu with
relevant info...

Thanks!


Andrew


On 11/15/06, Steve Taylor <st...@mo...> wrote:
> Hi Andrew,
>
> > I found the problem.  Everywhere, including "prototype_gbrowse.html",
> > your URLs should be "https", not "http"... on a whim I portscanned
> > {slave,gbrowse}.molbiol.ox.ac.uk and noticed port 80 is closed, but
> > 443 is open... or at least for U.S.-originating packets.
> >
> > So this one:
> >
> > https://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=3DU000=
96%3A1..10000
> >
> > works just fine.  This one:
> >
> > https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrow=
se.html
> >
> > seems to load the XHTML, but nothing else.  The problem seems to be
> > that the XHTML has "http" instead of "https" for the JavaScript
> > library URLs, so I can't load the JavaScript libraries.  If the URLs
> > for libraries were changed to something like:
> >
> > https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/client/View.js
> >
> > then they would load.
> >
> > I guess there is some firewall config causing this problem... although
> > I'm puzzled why it works for someone from Australia but not the U.S.
> >
>
> We have worked out what the problem is and it is down to some configurati=
on issues set up by the Oxford IT service which I won't go into since it's =
so bizarre (unless you're *really* interested :-)).
> http will work in a day or so but in the meantime have a look on our test=
 server. This *should* work...
>
> http://jenna.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.=
html
>
> I have also added a js pop-up based on one that was originally developed =
by Sylvain Gaillard and Ga=EBtan Droc.
>
> If it all works your end I will check it in.
>
> Cheers,
>
> Steve
> >
> >
> > On 11/14/06, Steve Taylor <st...@mo...> wrote:
> >
> >>Hi Andrew,
> >>
> >>>This sounds like excellent news, thanks!  But, I'm getting timed-out
> >>>loading both pages you linked to... can you let us know when they're
> >>>back up?  I would love to take a look.
> >>>
> >>
> >>Hmmm. I have tested both here and they work reasonably quickly although=
 are more slow to load than the original AjaxGBrowse...I also got a got som=
eone to test in Australia and he said it loaded ok
> >>though slightly more slowly.
> >>
> >>Can you try again please? Has anyone else on the list had time out prob=
lems? Again the URLs to try are:
> >>
> >>Ajax Gbrowse
> >>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrows=
e.html
> >>
> >>Original Gbrowse
> >>http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=3DU0009=
6%3A1..10000
> >>
> >>
> >>>You should commit the code to CVS... I've also made a few changes
> >>>across the server-side code (including generate-tiles) to test
> >>>on-demand tile rendering, which will be useful for large genomes, and
> >>>to fix some stupid bugs left in TiledImage, etc. from back in the day
> >>>(e.g. database connections not hanging up), which you probably noticed
> >>>and fixed by now also.  Once your code is in, I'll merge in my
> >>>changes.
> >>>
> >>>Just to let everyone know about the "render on demand" stuff... the
> >>>idea is that we still fill a database with primitives, but only render
> >>>tiles as they are request by user (they are also saved, so no need to
> >>>re-render unless the underlying data changes, so we'll have some sort
> >>>of "dirty tile" flag or some such).
> >>>
> >>>This will enable people to put up large genomes without pre-rendering
> >>>all the tiles.  Currently, the rendering step is the major limiting
> >>>factor (my benchmarks show around 90% of the actual generate-tiles.pl
> >>>time is spent on database lookup of primitives).  For example, Dmel
> >>>arm 4 takes a full day to render on our 24-CPU cluster, but only an
> >>>hour of that time is spent filling the database.
> >>>
> >>>I'm trying a few things to optimize this right now, including the
> >>>on-demand rendering, and I'll keep the list updated once I get
> >>>somewhere (the things I'm trying can be found on
> >>>http://biowiki.org/view/GBrowse/WishList under the "rendering
> >>>optimizations" sublist).  I think we could seriously reduce the
> >>>fill/access time, but until then, on-demand rendering will at least
> >>>make large genomes feasible.
> >>>
> >>
> >>Sounds good. Look forward to seeing that!
> >>
> >>Steve
> >>
> >>>
> >>>On 11/13/06, Steve Taylor <st...@mo...> wrote:
> >>>
> >>>
> >>>>Hi,
> >>>>
> >>>>I thought I'd send this to open up the discussions on the new list!:-=
)
> >>>>
> >>>>I haven't checked in this code yet but I have modified generate-tiles=
.pm and BatchTilesImage.pm to allow marking up of the features in each of t=
he tiles using client side image maps. We may want to
> >>>>change this later to do CSS maps but that would be only a small code =
modification. It basically generates an html file per tile with an inlined =
image and the appropriate map coordinates. The browser
> >>>>XMLHttpRequests the data over when it needs to using Ajax.Updater (fr=
om the js library Prototype).
> >>>>
> >>>>You can see what it looks like on:
> >>>>
> >>>>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbro=
wse.html
> >>>>
> >>>>Try dragging and clicking on features.
> >>>>
> >>>>Advantages of current implementation
> >>>>-----------------------------------
> >>>>- should work on all browsers and doesn't require any fancy javascrip=
t
> >>>>- will support polygons, ellipses etc
> >>>>- allows distribution on different servers for tiles
> >>>>
> >>>>Disadvantages of implementation
> >>>>-------------------------------
> >>>>- all html per tile currently generated server side (not sure if this=
 is an advantage or disadvantage at this stage) but may be a pain to do upd=
ates
> >>>>
> >>>>Obviously where javascript:alert('Active image map') you should have =
the relevant details of the feature and a nicer pop up. The js alerts are p=
retty annoying at the moment when dragging! I intend to
> >>>>put js pop ups similar to the ones we have at http://gbrowse.molbiol.=
ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=3DU00096%3A1..10000.
> >>>>
> >>>>Let me know what you think. I will check it in if people think it is =
worthwhile at this stage.
> >>>>
> >>>>Regards,
> >>>>
> >>>>Steve
> >>>>------------------------------------------------------------------
> >>>>Head of Computational Biology Research Group
> >>>>Medical Sciences Division
> >>>>Weatherall Institute of Molecular Medicine/Sir William Dunn School
> >>>>Oxford University
> >>>>Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
> >>>>Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
> >>>>Web: http://www.compbio.ox.ac.uk
> >>>>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share y=
our
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

Re: [Gmod-ajax] Feature Mark Up

[Steve T. <st...@mo...>]

Hi Andrew,

> I found the problem.  Everywhere, including "prototype_gbrowse.html",
> your URLs should be "https", not "http"... on a whim I portscanned
> {slave,gbrowse}.molbiol.ox.ac.uk and noticed port 80 is closed, but
> 443 is open... or at least for U.S.-originating packets.
> 
> So this one:
> 
> https://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000
> 
> works just fine.  This one:
> 
> https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
> 
> seems to load the XHTML, but nothing else.  The problem seems to be
> that the XHTML has "http" instead of "https" for the JavaScript
> library URLs, so I can't load the JavaScript libraries.  If the URLs
> for libraries were changed to something like:
> 
> https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/client/View.js
> 
> then they would load.
> 
> I guess there is some firewall config causing this problem... although
> I'm puzzled why it works for someone from Australia but not the U.S.
> 

We have worked out what the problem is and it is down to some configuration issues set up by the Oxford IT service which I won't go into since it's so bizarre (unless you're *really* interested :-)).
http will work in a day or so but in the meantime have a look on our test server. This *should* work...

http://jenna.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html

I have also added a js pop-up based on one that was originally developed by Sylvain Gaillard and Gaëtan Droc.

If it all works your end I will check it in.

Cheers,

Steve
> 
> 
> On 11/14/06, Steve Taylor <st...@mo...> wrote:
> 
>>Hi Andrew,
>>
>>>This sounds like excellent news, thanks!  But, I'm getting timed-out
>>>loading both pages you linked to... can you let us know when they're
>>>back up?  I would love to take a look.
>>>
>>
>>Hmmm. I have tested both here and they work reasonably quickly although are more slow to load than the original AjaxGBrowse...I also got a got someone to test in Australia and he said it loaded ok
>>though slightly more slowly.
>>
>>Can you try again please? Has anyone else on the list had time out problems? Again the URLs to try are:
>>
>>Ajax Gbrowse
>>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>
>>Original Gbrowse
>>http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000
>>
>>
>>>You should commit the code to CVS... I've also made a few changes
>>>across the server-side code (including generate-tiles) to test
>>>on-demand tile rendering, which will be useful for large genomes, and
>>>to fix some stupid bugs left in TiledImage, etc. from back in the day
>>>(e.g. database connections not hanging up), which you probably noticed
>>>and fixed by now also.  Once your code is in, I'll merge in my
>>>changes.
>>>
>>>Just to let everyone know about the "render on demand" stuff... the
>>>idea is that we still fill a database with primitives, but only render
>>>tiles as they are request by user (they are also saved, so no need to
>>>re-render unless the underlying data changes, so we'll have some sort
>>>of "dirty tile" flag or some such).
>>>
>>>This will enable people to put up large genomes without pre-rendering
>>>all the tiles.  Currently, the rendering step is the major limiting
>>>factor (my benchmarks show around 90% of the actual generate-tiles.pl
>>>time is spent on database lookup of primitives).  For example, Dmel
>>>arm 4 takes a full day to render on our 24-CPU cluster, but only an
>>>hour of that time is spent filling the database.
>>>
>>>I'm trying a few things to optimize this right now, including the
>>>on-demand rendering, and I'll keep the list updated once I get
>>>somewhere (the things I'm trying can be found on
>>>http://biowiki.org/view/GBrowse/WishList under the "rendering
>>>optimizations" sublist).  I think we could seriously reduce the
>>>fill/access time, but until then, on-demand rendering will at least
>>>make large genomes feasible.
>>>
>>
>>Sounds good. Look forward to seeing that!
>>
>>Steve
>>
>>>
>>>On 11/13/06, Steve Taylor <st...@mo...> wrote:
>>>
>>>
>>>>Hi,
>>>>
>>>>I thought I'd send this to open up the discussions on the new list!:-)
>>>>
>>>>I haven't checked in this code yet but I have modified generate-tiles.pm and BatchTilesImage.pm to allow marking up of the features in each of the tiles using client side image maps. We may want to
>>>>change this later to do CSS maps but that would be only a small code modification. It basically generates an html file per tile with an inlined image and the appropriate map coordinates. The browser
>>>>XMLHttpRequests the data over when it needs to using Ajax.Updater (from the js library Prototype).
>>>>
>>>>You can see what it looks like on:
>>>>
>>>>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>>>
>>>>Try dragging and clicking on features.
>>>>
>>>>Advantages of current implementation
>>>>-----------------------------------
>>>>- should work on all browsers and doesn't require any fancy javascript
>>>>- will support polygons, ellipses etc
>>>>- allows distribution on different servers for tiles
>>>>
>>>>Disadvantages of implementation
>>>>-------------------------------
>>>>- all html per tile currently generated server side (not sure if this is an advantage or disadvantage at this stage) but may be a pain to do updates
>>>>
>>>>Obviously where javascript:alert('Active image map') you should have the relevant details of the feature and a nicer pop up. The js alerts are pretty annoying at the moment when dragging! I intend to
>>>>put js pop ups similar to the ones we have at http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000.
>>>>
>>>>Let me know what you think. I will check it in if people think it is worthwhile at this stage.
>>>>
>>>>Regards,
>>>>
>>>>Steve
>>>>------------------------------------------------------------------
>>>>Head of Computational Biology Research Group
>>>>Medical Sciences Division
>>>>Weatherall Institute of Molecular Medicine/Sir William Dunn School
>>>>Oxford University
>>>>Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
>>>>Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
>>>>Web: http://www.compbio.ox.ac.uk
>>>>

Re: [Gmod-ajax] Feature Mark Up

[Andrew U. <and...@gm...>]

Hi Steve...

I found the problem.  Everywhere, including "prototype_gbrowse.html",
your URLs should be "https", not "http"... on a whim I portscanned
{slave,gbrowse}.molbiol.ox.ac.uk and noticed port 80 is closed, but
443 is open... or at least for U.S.-originating packets.

So this one:

https://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000

works just fine.  This one:

https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html

seems to load the XHTML, but nothing else.  The problem seems to be
that the XHTML has "http" instead of "https" for the JavaScript
library URLs, so I can't load the JavaScript libraries.  If the URLs
for libraries were changed to something like:

https://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/client/View.js

then they would load.

I guess there is some firewall config causing this problem... although
I'm puzzled why it works for someone from Australia but not the U.S.

Cheers,


Andrew


On 11/14/06, Steve Taylor <st...@mo...> wrote:
> Hi Andrew,
> >
> > This sounds like excellent news, thanks!  But, I'm getting timed-out
> > loading both pages you linked to... can you let us know when they're
> > back up?  I would love to take a look.
> >
>
> Hmmm. I have tested both here and they work reasonably quickly although are more slow to load than the original AjaxGBrowse...I also got a got someone to test in Australia and he said it loaded ok
> though slightly more slowly.
>
> Can you try again please? Has anyone else on the list had time out problems? Again the URLs to try are:
>
> Ajax Gbrowse
> http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>
> Original Gbrowse
> http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000
>
> > You should commit the code to CVS... I've also made a few changes
> > across the server-side code (including generate-tiles) to test
> > on-demand tile rendering, which will be useful for large genomes, and
> > to fix some stupid bugs left in TiledImage, etc. from back in the day
> > (e.g. database connections not hanging up), which you probably noticed
> > and fixed by now also.  Once your code is in, I'll merge in my
> > changes.
> >
> > Just to let everyone know about the "render on demand" stuff... the
> > idea is that we still fill a database with primitives, but only render
> > tiles as they are request by user (they are also saved, so no need to
> > re-render unless the underlying data changes, so we'll have some sort
> > of "dirty tile" flag or some such).
> >
> > This will enable people to put up large genomes without pre-rendering
> > all the tiles.  Currently, the rendering step is the major limiting
> > factor (my benchmarks show around 90% of the actual generate-tiles.pl
> > time is spent on database lookup of primitives).  For example, Dmel
> > arm 4 takes a full day to render on our 24-CPU cluster, but only an
> > hour of that time is spent filling the database.
> >
> > I'm trying a few things to optimize this right now, including the
> > on-demand rendering, and I'll keep the list updated once I get
> > somewhere (the things I'm trying can be found on
> > http://biowiki.org/view/GBrowse/WishList under the "rendering
> > optimizations" sublist).  I think we could seriously reduce the
> > fill/access time, but until then, on-demand rendering will at least
> > make large genomes feasible.
> >
>
> Sounds good. Look forward to seeing that!
>
> Steve
> >
> >
> > On 11/13/06, Steve Taylor <st...@mo...> wrote:
> >
> >>Hi,
> >>
> >>I thought I'd send this to open up the discussions on the new list!:-)
> >>
> >>I haven't checked in this code yet but I have modified generate-tiles.pm and BatchTilesImage.pm to allow marking up of the features in each of the tiles using client side image maps. We may want to
> >>change this later to do CSS maps but that would be only a small code modification. It basically generates an html file per tile with an inlined image and the appropriate map coordinates. The browser
> >>XMLHttpRequests the data over when it needs to using Ajax.Updater (from the js library Prototype).
> >>
> >>You can see what it looks like on:
> >>
> >>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
> >>
> >>Try dragging and clicking on features.
> >>
> >>Advantages of current implementation
> >>-----------------------------------
> >>- should work on all browsers and doesn't require any fancy javascript
> >>- will support polygons, ellipses etc
> >>- allows distribution on different servers for tiles
> >>
> >>Disadvantages of implementation
> >>-------------------------------
> >>- all html per tile currently generated server side (not sure if this is an advantage or disadvantage at this stage) but may be a pain to do updates
> >>
> >>Obviously where javascript:alert('Active image map') you should have the relevant details of the feature and a nicer pop up. The js alerts are pretty annoying at the moment when dragging! I intend to
> >>put js pop ups similar to the ones we have at http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000.
> >>
> >>Let me know what you think. I will check it in if people think it is worthwhile at this stage.
> >>
> >>Regards,
> >>
> >>Steve
> >>------------------------------------------------------------------
> >>Head of Computational Biology Research Group
> >>Medical Sciences Division
> >>Weatherall Institute of Molecular Medicine/Sir William Dunn School
> >>Oxford University
> >>Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
> >>Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
> >>Web: http://www.compbio.ox.ac.uk
> >>
> >>-------------------------------------------------------------------------
> >>Using Tomcat but need to do more? Need to support web services, security?
> >>Get stuff done quickly with pre-integrated technology to make your job easier
> >>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> >>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >>_______________________________________________
> >>Gmod-ajax mailing list
> >>Gmo...@li...
> >>https://lists.sourceforge.net/lists/listinfo/gmod-ajax
> >>
> >
> >
> > -------------------------------------------------------------------------
> > Using Tomcat but need to do more? Need to support web services, security?
> > Get stuff done quickly with pre-integrated technology to make your job easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Gmod-ajax mailing list
> > Gmo...@li...
> > https://lists.sourceforge.net/lists/listinfo/gmod-ajax
> >
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

Re: [Gmod-ajax] Feature Mark Up

[Steve T. <st...@mo...>]

Hi Andrew,
> 
> This sounds like excellent news, thanks!  But, I'm getting timed-out
> loading both pages you linked to... can you let us know when they're
> back up?  I would love to take a look.
> 

Hmmm. I have tested both here and they work reasonably quickly although are more slow to load than the original AjaxGBrowse...I also got a got someone to test in Australia and he said it loaded ok 
though slightly more slowly.

Can you try again please? Has anyone else on the list had time out problems? Again the URLs to try are:

Ajax Gbrowse
http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html

Original Gbrowse
http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000

> You should commit the code to CVS... I've also made a few changes
> across the server-side code (including generate-tiles) to test
> on-demand tile rendering, which will be useful for large genomes, and
> to fix some stupid bugs left in TiledImage, etc. from back in the day
> (e.g. database connections not hanging up), which you probably noticed
> and fixed by now also.  Once your code is in, I'll merge in my
> changes.
> 
> Just to let everyone know about the "render on demand" stuff... the
> idea is that we still fill a database with primitives, but only render
> tiles as they are request by user (they are also saved, so no need to
> re-render unless the underlying data changes, so we'll have some sort
> of "dirty tile" flag or some such).
> 
> This will enable people to put up large genomes without pre-rendering
> all the tiles.  Currently, the rendering step is the major limiting
> factor (my benchmarks show around 90% of the actual generate-tiles.pl
> time is spent on database lookup of primitives).  For example, Dmel
> arm 4 takes a full day to render on our 24-CPU cluster, but only an
> hour of that time is spent filling the database.
> 
> I'm trying a few things to optimize this right now, including the
> on-demand rendering, and I'll keep the list updated once I get
> somewhere (the things I'm trying can be found on
> http://biowiki.org/view/GBrowse/WishList under the "rendering
> optimizations" sublist).  I think we could seriously reduce the
> fill/access time, but until then, on-demand rendering will at least
> make large genomes feasible.
> 

Sounds good. Look forward to seeing that!

Steve
> 
> 
> On 11/13/06, Steve Taylor <st...@mo...> wrote:
> 
>>Hi,
>>
>>I thought I'd send this to open up the discussions on the new list!:-)
>>
>>I haven't checked in this code yet but I have modified generate-tiles.pm and BatchTilesImage.pm to allow marking up of the features in each of the tiles using client side image maps. We may want to
>>change this later to do CSS maps but that would be only a small code modification. It basically generates an html file per tile with an inlined image and the appropriate map coordinates. The browser
>>XMLHttpRequests the data over when it needs to using Ajax.Updater (from the js library Prototype).
>>
>>You can see what it looks like on:
>>
>>http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>>
>>Try dragging and clicking on features.
>>
>>Advantages of current implementation
>>-----------------------------------
>>- should work on all browsers and doesn't require any fancy javascript
>>- will support polygons, ellipses etc
>>- allows distribution on different servers for tiles
>>
>>Disadvantages of implementation
>>-------------------------------
>>- all html per tile currently generated server side (not sure if this is an advantage or disadvantage at this stage) but may be a pain to do updates
>>
>>Obviously where javascript:alert('Active image map') you should have the relevant details of the feature and a nicer pop up. The js alerts are pretty annoying at the moment when dragging! I intend to
>>put js pop ups similar to the ones we have at http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000.
>>
>>Let me know what you think. I will check it in if people think it is worthwhile at this stage.
>>
>>Regards,
>>
>>Steve
>>------------------------------------------------------------------
>>Head of Computational Biology Research Group
>>Medical Sciences Division
>>Weatherall Institute of Molecular Medicine/Sir William Dunn School
>>Oxford University
>>Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
>>Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
>>Web: http://www.compbio.ox.ac.uk
>>
>>-------------------------------------------------------------------------
>>Using Tomcat but need to do more? Need to support web services, security?
>>Get stuff done quickly with pre-integrated technology to make your job easier
>>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>_______________________________________________
>>Gmod-ajax mailing list
>>Gmo...@li...
>>https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>>
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
> 

Re: [Gmod-ajax] Feature Mark Up

[Andrew U. <and...@gm...>]

Hi Steve (et al)...

This sounds like excellent news, thanks!  But, I'm getting timed-out
loading both pages you linked to... can you let us know when they're
back up?  I would love to take a look.

You should commit the code to CVS... I've also made a few changes
across the server-side code (including generate-tiles) to test
on-demand tile rendering, which will be useful for large genomes, and
to fix some stupid bugs left in TiledImage, etc. from back in the day
(e.g. database connections not hanging up), which you probably noticed
and fixed by now also.  Once your code is in, I'll merge in my
changes.

Just to let everyone know about the "render on demand" stuff... the
idea is that we still fill a database with primitives, but only render
tiles as they are request by user (they are also saved, so no need to
re-render unless the underlying data changes, so we'll have some sort
of "dirty tile" flag or some such).

This will enable people to put up large genomes without pre-rendering
all the tiles.  Currently, the rendering step is the major limiting
factor (my benchmarks show around 90% of the actual generate-tiles.pl
time is spent on database lookup of primitives).  For example, Dmel
arm 4 takes a full day to render on our 24-CPU cluster, but only an
hour of that time is spent filling the database.

I'm trying a few things to optimize this right now, including the
on-demand rendering, and I'll keep the list updated once I get
somewhere (the things I'm trying can be found on
http://biowiki.org/view/GBrowse/WishList under the "rendering
optimizations" sublist).  I think we could seriously reduce the
fill/access time, but until then, on-demand rendering will at least
make large genomes feasible.

Cheers,


Andrew


On 11/13/06, Steve Taylor <st...@mo...> wrote:
> Hi,
>
> I thought I'd send this to open up the discussions on the new list!:-)
>
> I haven't checked in this code yet but I have modified generate-tiles.pm and BatchTilesImage.pm to allow marking up of the features in each of the tiles using client side image maps. We may want to
> change this later to do CSS maps but that would be only a small code modification. It basically generates an html file per tile with an inlined image and the appropriate map coordinates. The browser
> XMLHttpRequests the data over when it needs to using Ajax.Updater (from the js library Prototype).
>
> You can see what it looks like on:
>
> http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html
>
> Try dragging and clicking on features.
>
> Advantages of current implementation
> -----------------------------------
> - should work on all browsers and doesn't require any fancy javascript
> - will support polygons, ellipses etc
> - allows distribution on different servers for tiles
>
> Disadvantages of implementation
> -------------------------------
> - all html per tile currently generated server side (not sure if this is an advantage or disadvantage at this stage) but may be a pain to do updates
>
> Obviously where javascript:alert('Active image map') you should have the relevant details of the feature and a nicer pop up. The js alerts are pretty annoying at the moment when dragging! I intend to
> put js pop ups similar to the ones we have at http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000.
>
> Let me know what you think. I will check it in if people think it is worthwhile at this stage.
>
> Regards,
>
> Steve
> ------------------------------------------------------------------
> Head of Computational Biology Research Group
> Medical Sciences Division
> Weatherall Institute of Molecular Medicine/Sir William Dunn School
> Oxford University
> Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
> Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
> Web: http://www.compbio.ox.ac.uk
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

[Gmod-ajax] Feature Mark Up

[Steve T. <st...@mo...>]

Hi,

I thought I'd send this to open up the discussions on the new list!:-)

I haven't checked in this code yet but I have modified generate-tiles.pm and BatchTilesImage.pm to allow marking up of the features in each of the tiles using client side image maps. We may want to 
change this later to do CSS maps but that would be only a small code modification. It basically generates an html file per tile with an inlined image and the appropriate map coordinates. The browser 
XMLHttpRequests the data over when it needs to using Ajax.Updater (from the js library Prototype).

You can see what it looks like on:

http://slave.molbiol.ox.ac.uk/gbrowse/TiledImage/ecoli/prototype_gbrowse.html

Try dragging and clicking on features.

Advantages of current implementation
-----------------------------------
- should work on all browsers and doesn't require any fancy javascript
- will support polygons, ellipses etc
- allows distribution on different servers for tiles

Disadvantages of implementation
-------------------------------
- all html per tile currently generated server side (not sure if this is an advantage or disadvantage at this stage) but may be a pain to do updates

Obviously where javascript:alert('Active image map') you should have the relevant details of the feature and a nicer pop up. The js alerts are pretty annoying at the moment when dragging! I intend to 
put js pop ups similar to the ones we have at http://gbrowse.molbiol.ox.ac.uk/cgi-bin/gbrowse/coli_demo/?name=U00096%3A1..10000.

Let me know what you think. I will check it in if people think it is worthwhile at this stage.

Regards,

Steve
------------------------------------------------------------------
Head of Computational Biology Research Group
Medical Sciences Division
Weatherall Institute of Molecular Medicine/Sir William Dunn School
Oxford University
Tel: +44 (0)1865 (2)22640 (WIMM - Monday to Wednesday)
Tel: +44 (0)1865 (2)85732 (Dunn - Thursday to Friday)
Web: http://www.compbio.ox.ac.uk

[Gmod-ajax] welcome and general announcements

[Andrew U. <and...@gm...>]

Greetings!

I would like to welcome everyone to the new AJAX GBrowse mailing list,
and thank you for your interest, participation, support, and so on...
this is very exciting!

Those of you that aren't on the list are being CC'ed and should subscribe here:

https://lists.sourceforge.net/lists/listinfo/gmod-ajax

This list is for announcements, development discussion, and pretty
much anything related to this project... so, let me get the ball
rolling.

0. DEMO

Just so everyone knows, the demo of what we've done so far can be found here:

http://genome.biowiki.org/

1. CVS ACCESS AND SOURCEFORGE STUFF

We are now part of the Generic-Genome-Browser module of the GMOD
SourceForge CVS repository (see the "ajax" subdir).

Developers can check the code out like this:

cvs -z3 -d:ext:YOU...@gm...:/cvsroot/gmod co
-P Generic-Genome-Browser

If you want to be a developer, please get a SF account (at sf.net) and
e-mail your login to Scott Cain (ca...@cs...).

We also have bugtracking through SF (which I haven't taken a look at,
but will in a moment) and a CVS commits mailing list, which you can
subscribe to here:

https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse-ajax-commits

Scott, thanks a million for setting all this up!

2. LIST ADMINISTRIVIA

Currently, if you hit "reply" to a list message, it replies to the
person who originally sent the message.  While this is nice for
announcements and keeps the traffic down, it's not great for
discussion, and discussion is the main reason why we have this list.

I want to change the settings so that hitting "reply" will reply back
to everyone on the list... anyone for or against this?  I'll set this
up if I don't hear otherwise.

3. OTHER RESOURCES

We will be using the GBrowse web of biowiki.org (Ian Holmes' lab wiki)
for posting info about all aspects of the project.  The web is rooted
here:

http://biowiki.org/view/GBrowse/WebHome

In my opinion, the two most important items are our more-or-less
development outline:

http://biowiki.org/view/GBrowse/WishList

and this "documentation dump" I started up the other day (so, it's
woefully incomplete... but evolving):

http://biowiki.org/view/GBrowse/AJAXGBrowseDocs

The documentation dump is a good place to start if you're new to this.

4. JOB ANNOUNCEMENT

We are looking for a full-time Ajax developer for this project.
Spread the word!  More details here:

http://biowiki.org/AjaxProgrammerJob

5. QUESTIONS? REQUESTS? COMMENTS?

Please post to the list or e-mail me.

Cheers,


Andrew Uzilov
Holmes Lab, Dept. of Bioengineering, UC-Berkeley