Re: [Gmod-ajax] AJAX GBrowse poster feedback from RECOMB

[Ian H. <ih...@be...>]

Hi Andrew, thanks for passing on all this feedback.

> less-than-reusable fashion.  A certain Bay Area company (should I
> mention who?  I don't know what the etiquette rules are...)

Unless they asked you not to mention them (which would be odd at an open 
poster session), I don't see why not.

> "Pragmatic Ajax" book)... then they invited us to demo our version for
> them sometime.

Can you pass on contact details and I'll try to arrange a date?

> learned if there's a good reason for it.  But why not Java applets?
> The only two reasons we came up with are (1) long load time, and (2)
> need to install Java, but that's not hard...

Interesting. I'd like to know what others think of this.

My answer to this question would be "of course we will welcome your Java 
code contributions", delivered with a polite grin.

In all seriousness, I think we already have an emerging dream of a 
language-agnostic message bus, certainly on the the server side.

I've been asked about standalone clients before. Google Earth often 
comes up in these discussions. Sticking with the analogy, I usually 
point out that (A) Google Maps came first and (B) the two applications 
are designed to be compatible, but they occupy distinct niches in the 
ecosystem.

One can argue language merits endlessly. Java has clear advantages.

Still, there ARE good reasons for scripting the web browser, rather than 
budding off an applet, mainly to do with the interface. "Long load time" 
and "no plugin" are part of it, but there are a whole lot of subtle 
issues that go along with this: look & feel, web-browser integration, 
compatibility with other components.

When language discussions go negative it's all over, but having said 
that, if you DO get backed up against a wall and need a good reason 
AGAINST Java, go and take a look at some of the Java genome browsers out 
there. They just don't feel as light on their feet. I'm sure their 
codebase is more stable, but they're not as fun.

I definitely DON'T want to block Java-heads from hitching a ride on our 
message bus, so we probably need to avoid using JSON or other 
Javascript-only tricks, and keep casual eye on compatibility. But I 
don't want to switch to Java ;)


> - I got a couple of questions about whether our approach will scale.
> One gentleman did point out that storage requirements for e.g. a human
> genome would be ridiculous, even if we get rid of storing feature info
> in HTML.

So we definitely need HARD FACTS AND FIGURES at our disposal. 
Anticipating such questions is due diligence we should be doing.

Do we have an actual estimate for human track size (fully-rendered)? A 
gigabyte?

Even before having done the back-of-envelope calculation, I am going to 
(perhaps unwisely) stick my neck out: I don't think the human genome 
requirements are ridiculous when you stop to think about what you 
actually have to achieve.

Specifically, I think the problem mostly melts away with a few simple 
tricks, e.g.:

(1) Most tiles are whitespace, and you only really need to store one 
image of an empty tile; all the other empty tiles can be symlinks back 
to this one (this was/is the idea behind my MD5-filename hack for tile 
images)

(2) Most close-up tiles won't ever get viewed, so rendering on demand 
saves a lot of space;

(3) If the annotation upload facility takes off, you might not need more 
than a few human genome installations at first, and the large 
storage/cluster costs become centralized and benefit from economy of 
scale (c.f. UCSC);

(4) Client-side rendering has to show up in this list of tricks; but, to 
me, reimplementing parts of GBrowse in javascript is just another rabbit 
we can pull out of the hat to address problems of scale, not a goal in 
itself.

(5) Maybe we can even dip in and out of classic (CGI) GBrowse when we 
need to, though Mitch has pointed out that this might complicate the 
user experience somewhat.


> Related to this, I got many questions/suggestions that we
> should shift layout/rendering to the client side completely.  There
> were quite a few people in favor of that idea and seem to think even
> JavaScript can do what we need.

I am gradually being persuaded that this is practical. Mitch has been 
trying to get me on board his boatload of divs for some time now. I 
realize it makes mashups easier too.

I still see it as just one thing in our bag of tricks for providing a 
smooth experience, rather than an end in itself ("wow! porting code is 
so much fun! why don't you rewrite the entire thing in F# and run it on 
the Mono VM! Go on, I'll watch" </snark>)

Mitch has done quite well nudging forward Javascript rendering on the 
sly. I'll probably get very excited about it as soon as it's 
demonstrably the quickest route to a significant milestone, such as a 
functioning prototype of a human genome wiki.

Our methodology so far has been to develop fluid, functional prototypes. 
I hope we can keep doing that, and that this will lead us to the human 
genome by small steps, each yielding an intermediate product of 
practical interest.

Short answer: "Yes, clientside rendering offers many benefits and is 
likely to help with scaling issues, like storage & rendering time. It is 
definitely on our list."


 > A lot of people asked "can I do this or that track" or "this or that
 > species"... it appears people have an impression that genome browsers
 > are very restricted to the underlying data

I think this really makes it worth emphasizing the "wiki farm" idea -- 
that anyone will eventually be able to create their own new species via 
the web interface, a la Blogger or Wordpress.

To a Unix geek who can checkout & build GBrowse, that's an almost 
trivial piece of functionality to expose via the web interface, but 
clearly there are a lot of people who would benefit from it.

I.