gmod-ajax — JBrowse list

[Gmod-ajax] Problems With bam-to-json.pl

[David C. <hil...@gm...>]

Hi again,

I was actually able to resolve this problem by changing the CFLAGS in the
MAKEFILE in samtools.  I will post the solution for others.

The new line in the make file needs to be adjusted to look like the one
below.
CFLAGS= -g -Wall -O2 -fPIC #-m64 #-arch ppc

Once you change this line, running "make clean" and "make" again will
resolve the issue.

I would still like some information on the wig track visualization problem I
mentioned in my last post, but I didn't want anyone to spend time responding
to something I was able to figure out.

Thanks!

[Gmod-ajax] Problems With bam-to-json.pl

[David C. <hil...@gm...>]

Hi,

I work in Ann Loraine's lab, and we are currently setting up an instance of
Jbrowse for testing and I'm having a little trouble with installing the
Bio::DB::Sam perl module, and I was hoping someone might be able to suggest
a solution.  It seems the problem is coming from my compiled samtools files,
but I have tried about 6 different versions of samtools with no resolution.
One other problem I am having is getting my .wig files to display. I can
prepare them with no errors, and they show up in my jbrowse instance, but
with no track visualization.  Could you also suggest a fix for this problem?
 Thanks very much for your help!

Here is the error I am getting from cpan.

Creating new 'Build' script for 'Bio-SamTools' version '1.28'
Could not read '/home/david/.cpan/build/Bio-SamTools-1.28-xqGh82/META.yml'.
Falling back to other methods to determine prerequisites
CPAN: Module::Build loaded ok (v0.340201)
Copying lib/Bio/DB/Bam/Pileup.pm -> blib/lib/Bio/DB/Bam/Pileup.pm
Copying lib/Bio/DB/Bam/AlignWrapper.pm ->
blib/lib/Bio/DB/Bam/AlignWrapper.pm
Copying lib/Bio/DB/Sam.pm -> blib/lib/Bio/DB/Sam.pm
Copying lib/Bio/DB/Sam/Constants.pm -> blib/lib/Bio/DB/Sam/Constants.pm
Copying lib/Bio/DB/Sam/Segment.pm -> blib/lib/Bio/DB/Sam/Segment.pm
Copying lib/Bio/DB/Bam/PileupWrapper.pm ->
blib/lib/Bio/DB/Bam/PileupWrapper.pm
Copying lib/Bio/DB/Bam/Alignment.pm -> blib/lib/Bio/DB/Bam/Alignment.pm
Copying lib/Bio/DB/Bam/Target.pm -> blib/lib/Bio/DB/Bam/Target.pm
Copying lib/Bio/DB/Bam/ReadIterator.pm ->
blib/lib/Bio/DB/Bam/ReadIterator.pm
Copying lib/Bio/DB/Bam/Query.pm -> blib/lib/Bio/DB/Bam/Query.pm
Copying lib/Bio/DB/Bam/FetchIterator.pm ->
blib/lib/Bio/DB/Bam/FetchIterator.pm
Copying lib/Bio/DB/Sam/SamToGBrowse.pm ->
blib/lib/Bio/DB/Sam/SamToGBrowse.pm
lib/Bio/DB/Sam.xs -> lib/Bio/DB/Sam.c
cc -I/home/david/samtools -I/usr/lib/perl/5.10/CORE -DXS_VERSION="1.28"
-DVERSION="1.28" -fPIC -D_IOLIB=2 -D_FILE_OFFSET_BITS=64 -Wformat=0 -c
-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -O2 -g -o lib/Bio/DB/Sam.o lib/Bio/DB/Sam.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/Bio/DB/Sam/Sam.bs')
cc -shared -O2 -g -L/usr/local/lib -fstack-protector -o
blib/arch/auto/Bio/DB/Sam/Sam.so lib/Bio/DB/Sam.o -L/home/david/samtools
-lbam -lz
/usr/bin/ld: /home/david/samtools/libbam.a(bgzf.o): relocation R_X86_64_32S
against `.rodata.str1.1' can not be used when making a shared object;
recompile with -fPIC
/home/david/samtools/libbam.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error building blib/arch/auto/Bio/DB/Sam/Sam.so from lib/Bio/DB/Sam.o at
/usr/share/perl/5.10/ExtUtils/CBuilder/Base.pm line 227.
  LDS/Bio-SamTools-1.28.tar.gz
  ./Build -- NOT OK
Warning (usually harmless): 'YAML' not installed, will not store persistent
state
Running Build test
  Can't test without successful make
Running Build install
  Make had returned bad status, install seems impossible
Failed during this command:
 LDS/Bio-SamTools-1.28.tar.gz                 : make NO



-David Norris

Re: [Gmod-ajax] using JBrowse

[Philippe L. <phi...@ob...>]

Hi,

We we're able to setup JBrowse on a database of MPS reads and genomes. It
generates tracks and histograms at runtime.

I had to reverse engineer most of the json format though since it's not
documented. I had planned to contribute this documentation here, but have
not found the time to do so yet...

It's possible, but not trivial. Also, you need to be careful when building
the nested containment lists. Reading the paper helped doing this
properly...

Hope it helps,
Philippe

On Wed, Apr 6, 2011 at 21:15, Ian Holmes <ih...@be...> wrote:

> Hi Andrea, yes there is nothing in principle stopping you from
> converting your data into these formats. It should be pretty easy to get
> at least a coarse representation of most annotation data in these
> formats. There are a lot of conversion scripts e.g. in BioPerl if you're
> using another standard format. Ian
>
> On 4/6/11 10:59 AM, Andrea Edwards wrote:
> > Hello
> >
> > I am considering using JBrowse but it seems from the pre requisites page
> > that the underlying data should be in  GFF/Bed files or a genomic
> > annotation database such as chado, Bio::DB::SeqFeature::Store, or
> > Bio::DB::GFF.
> >
> > Could you still use JBrowse if your data was not in this format? I
> > presume there is nothing stopping you from generating BED files of your
> > annotations from your custom database and then generating the JSON
> > tracks from these files?
> >
> > Thanks a lot
> >
> >
> >
> ------------------------------------------------------------------------------
> > Xperia(TM) PLAY
> > It's a major breakthrough. An authentic gaming
> > smartphone on the nation's most reliable network.
> > And it wants your games.
> > http://p.sf.net/sfu/verizon-sfdev
> > _______________________________________________
> > Gmod-ajax mailing list
> > Gmo...@li...
> > https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>
> ------------------------------------------------------------------------------
> Xperia(TM) PLAY
> It's a major breakthrough. An authentic gaming
> smartphone on the nation's most reliable network.
> And it wants your games.
> http://p.sf.net/sfu/verizon-sfdev
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

[Gmod-ajax] JBrowse is hiring!

[Ian H. <ih...@be...>]

The lead developer on the JBrowse team is moving on to pastures new: 
Mitch Skinner has accepted an offer to work at an exciting new medical 
genomics startup (which will, for the moment, remain nameless, but keep 
an eye out for their launch sometime in the next year).

Mitch hopes to continue contributing to JBrowse, but this also leaves an 
open position in the core team. Accordingly, we are now hiring.

Full details of the position can be found here:
http://biowiki.org/JBrowseJob

Re: [Gmod-ajax] using JBrowse

[Ian H. <ih...@be...>]

Hi Andrea, yes there is nothing in principle stopping you from 
converting your data into these formats. It should be pretty easy to get 
at least a coarse representation of most annotation data in these 
formats. There are a lot of conversion scripts e.g. in BioPerl if you're 
using another standard format. Ian

On 4/6/11 10:59 AM, Andrea Edwards wrote:
> Hello
>
> I am considering using JBrowse but it seems from the pre requisites page
> that the underlying data should be in  GFF/Bed files or a genomic
> annotation database such as chado, Bio::DB::SeqFeature::Store, or
> Bio::DB::GFF.
>
> Could you still use JBrowse if your data was not in this format? I
> presume there is nothing stopping you from generating BED files of your
> annotations from your custom database and then generating the JSON
> tracks from these files?
>
> Thanks a lot
>
>
> ------------------------------------------------------------------------------
> Xperia(TM) PLAY
> It's a major breakthrough. An authentic gaming
> smartphone on the nation's most reliable network.
> And it wants your games.
> http://p.sf.net/sfu/verizon-sfdev
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] SNPS tracks

[Ian H. <ih...@be...>]

Hi Michael, these tracks were generated from the UCSC hg19 database 
using the ucsc-to-json.pl script which you can find in the jbrowse git 
repository. Ian

On 4/6/11 10:41 AM, Lankerovich, Michael wrote:
> On your main http://jbrowse.org/ website you display a picture of JBrowse.
> On this picture there is a "GWAS Catalog" track which shows SNPS, I believe.
>
> Can you tell how it was done, please?
>
> Thanks,
> Michael Lankerovich.
> ------------------------------------------------------------------------------
> Xperia(TM) PLAY
> It's a major breakthrough. An authentic gaming
> smartphone on the nation's most reliable network.
> And it wants your games.
> http://p.sf.net/sfu/verizon-sfdev
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax

[Gmod-ajax] using JBrowse

[Andrea E. <edw...@cs...>]

Hello

I am considering using JBrowse but it seems from the pre requisites page 
that the underlying data should be in  GFF/Bed files or a genomic 
annotation database such as chado, Bio::DB::SeqFeature::Store, or 
Bio::DB::GFF.

Could you still use JBrowse if your data was not in this format? I 
presume there is nothing stopping you from generating BED files of your 
annotations from your custom database and then generating the JSON 
tracks from these files?

Thanks a lot

[Gmod-ajax] SNPS tracks

[Lankerovich, M. <Mic...@sw...>]

On your main http://jbrowse.org/ website you display a picture of JBrowse.
On this picture there is a "GWAS Catalog" track which shows SNPS, I believe.

Can you tell how it was done, please?

Thanks,
Michael Lankerovich.

[Gmod-ajax] y-axis labels on wig tracks

[David B. <dav...@gm...>]

Hi,

It seems this issue goes back for as far as I can remember.
Will this trivial feature be implemented?

Thanks,
Dave

[Gmod-ajax] JBrowse 1.2.1 release

[Mitch S. <mit...@be...>]

Just tagged the 1.2.1 release, with fixes for the bugs reported in 
version 1.2.

It's available here:

http://jbrowse.org/releases/jbrowse-1.2.1.zip

Release Notes:

version 1.2.1, March 2011

Fixed bugs in release 1.2:

- problems with BED handling reported by Gregg Helt and Brenton Graveley

- performance regression reported by Chris Childers

- incorrect handling of names in ucsc-to-json.pl

- wig2png slowdown reported by Brenton Graveley

Re: [Gmod-ajax] [apollo-dev] JBrowse flatfiles off-by-one bug

[Mitch S. <mit...@be...>]

Bleh, should be fixed now in HEAD.  Thanks for the report.

Mitch

On 03/08/2011 06:29 AM, Gregg Helt wrote:
> As near as I can tell (by checking for splice site GT/AG consensus) 
> all of the subfeatures in JSON data files being generated from BED 
> files via flatfile-to-json.pl <http://flatfile-to-json.pl> have both 
> their start and end properties offset by -1 from what they should be.  
> This doesn't appear to be a new bug -- I've checked both the current 
> version of JBrowse and a version from last July, and it has this bug too.
>
> Gregg
>

[Gmod-ajax] JBrowse flatfiles off-by-one bug

[Gregg H. <gre...@gm...>]

As near as I can tell (by checking for splice site GT/AG consensus) all of
the subfeatures in JSON data files being generated from BED files via
flatfile-to-json.pl have both their start and end properties offset by -1
from what they should be.  This doesn't appear to be a new bug -- I've
checked both the current version of JBrowse and a version from last July,
and it has this bug too.

Gregg

Re: [Gmod-ajax] Please test JBrowse git master

[Gregg H. <gre...@gm...>]

Trying now with JBrowse master HEAD.  Preliminary testing with refseq
bed file, flatfile-to-json.pl is working well.

Thanks Mitch!

Gregg

On Sat, Feb 26, 2011 at 4:30 AM, Mitch Skinner
<mit...@be...> wrote:
> Thanks for all the bug reports; several of them should be fixed in the HEAD
> version of the JBrowse master branch now.
>
> Specifically, this should address:
>
> Gregg's two reports, the segfault and the error message from Storable about
> storing CODE items.  The segfault wasn't technically a JBrowse bug, but
> JBrowse now avoids doing the thing that triggers the bug.
>
> Brenton's first report about "cmp" not being implemented on
> Bio::Annotation::SimpleValue.  I haven't seen that error myself, but I fixed
> what I believe to be the problem; please test.
>
> Chris' report about biodb-to-json.pl being slow.  With my test data set
> (smaller than Chris'), on one track that I've been benchmarking, doing one
> refseq took 34 seconds with JBrowse 1.1, 148 seconds with JBrowse 1.2, and
> 39 seconds with git HEAD.
>
> We're still looking at the other reports, including Brenton's one about
> wig-to-json.pl being much slower.
>
> Mitch
>
>
>

[Gmod-ajax] Please test JBrowse git master

[Mitch S. <mit...@be...>]

Thanks for all the bug reports; several of them should be fixed in the 
HEAD version of the JBrowse master branch now.

Specifically, this should address:

Gregg's two reports, the segfault and the error message from Storable 
about storing CODE items.  The segfault wasn't technically a JBrowse 
bug, but JBrowse now avoids doing the thing that triggers the bug.

Brenton's first report about "cmp" not being implemented on 
Bio::Annotation::SimpleValue.  I haven't seen that error myself, but I 
fixed what I believe to be the problem; please test.

Chris' report about biodb-to-json.pl being slow.  With my test data set 
(smaller than Chris'), on one track that I've been benchmarking, doing 
one refseq took 34 seconds with JBrowse 1.1, 148 seconds with JBrowse 
1.2, and 39 seconds with git HEAD.

We're still looking at the other reports, including Brenton's one about 
wig-to-json.pl being much slower.

Mitch

Re: [Gmod-ajax] extraData

[Mariette <jma...@to...>]

ok I use "alias" instead of "load_id" (which was used in a jbrowse exemple)
and it's working just fine !

thx,
Jerome





Mariette a écrit :
> Hi everyone,
> I have a jbrowse track where features have a urlTemplate,
> so my config file looks like this :
>
> {
>   "description": "data",
>   "db_adaptor": "Bio::DB::SeqFeature::Store",
>   "db_args": { "-adaptor": "memory",
>                "-dir": "/path/" },
>
>   "TRACK DEFAULTS": {
>     "class": "feature",
>     "autocomplete": "all"
>   },
>
>   "tracks": [
>     {
>       "track": "ncRNA",
>       "key": "ncRNA",
>       "feature": ["ncRNA"],
>       "class": "feature5",
>       "urlTemplate": "http://url/index?id={name}",
>     }
>   ]
> }
> what is working with a gff looking like this :
> sample    tRNAscan-SE    ncRNA    16995    17070    91.83    +    .    
> Name=000002
> sample    tRNAscan-SE    ncRNA    17079    17163    67.63    +    .    
> Name=000003
> ...
>
> however, I'd like the display name is different than the link value ! So 
> I change for this :
>
>   "tracks": [
>     {
>       "track": "ncRNA",
>       "key": "ncRNA",
>       "feature": ["ncRNA"],
>       "class": "feature5",
>       "urlTemplate": "http://url/index?id={load_id}",
>       "extraData": {"load_id": "sub {shift->attributes(\"load_id\");}"}
>     }
> with this gff :
> sample    tRNAscan-SE    ncRNA    16995    17070    91.83    +    .    
> Name=test1;load_id=000002
> sample    tRNAscan-SE    ncRNA    17079    17163    67.63    +    .    
> Name=test2;load_id=000003
> ...
>
>
> In the trackData.json, I do have the header looking like this :
> "headers":["start","end","strand","id","name","load_id"]
>
> but the values doesn't integrate the load_id ! I'm guessing the probleme 
> is coming from the sub but I have no idea how to fixe this.
> any help,
> thx,
>
> Jerome
>
>
> ------------------------------------------------------------------------------
> Free Software Download: Index, Search & Analyze Logs and other IT data in 
> Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
> generated by your applications, servers and devices whether physical, virtual
> or in the cloud. Deliver compliance at lower cost and gain new business 
> insights. http://p.sf.net/sfu/splunk-dev2dev 
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>   

[Gmod-ajax] extraData

[Mariette <jma...@to...>]

Hi everyone,
I have a jbrowse track where features have a urlTemplate,
so my config file looks like this :

{
  "description": "data",
  "db_adaptor": "Bio::DB::SeqFeature::Store",
  "db_args": { "-adaptor": "memory",
               "-dir": "/path/" },

  "TRACK DEFAULTS": {
    "class": "feature",
    "autocomplete": "all"
  },

  "tracks": [
    {
      "track": "ncRNA",
      "key": "ncRNA",
      "feature": ["ncRNA"],
      "class": "feature5",
      "urlTemplate": "http://url/index?id={name}",
    }
  ]
}
what is working with a gff looking like this :
sample    tRNAscan-SE    ncRNA    16995    17070    91.83    +    .    
Name=000002
sample    tRNAscan-SE    ncRNA    17079    17163    67.63    +    .    
Name=000003
...

however, I'd like the display name is different than the link value ! So 
I change for this :

  "tracks": [
    {
      "track": "ncRNA",
      "key": "ncRNA",
      "feature": ["ncRNA"],
      "class": "feature5",
      "urlTemplate": "http://url/index?id={load_id}",
      "extraData": {"load_id": "sub {shift->attributes(\"load_id\");}"}
    }
with this gff :
sample    tRNAscan-SE    ncRNA    16995    17070    91.83    +    .    
Name=test1;load_id=000002
sample    tRNAscan-SE    ncRNA    17079    17163    67.63    +    .    
Name=test2;load_id=000003
...


In the trackData.json, I do have the header looking like this :
"headers":["start","end","strand","id","name","load_id"]

but the values doesn't integrate the load_id ! I'm guessing the probleme 
is coming from the sub but I have no idea how to fixe this.
any help,
thx,

Jerome

Re: [Gmod-ajax] [apollo-dev] Re: JBrowse 1.2 problem generating JSON data from bed files?

[Chris C. <gen...@gm...>]

The loading is not using swap memory.  The process is using ~11.2% of the
system resources.  That is up from ~10.5% around nine hours ago.  The
refseqs get very large.  Monodelphis has eight chromosomes, and chr1 is
massive (~720 Mbp), and the other ref seqs start ~550 Mbp, 500Mbp, ~450Mbp
and down from there.

Chrs

On Wed, Feb 23, 2011 at 2:45 AM, Mitch Skinner
<mit...@be...>wrote:

>  On 02/22/2011 06:02 AM, Chris Childers wrote:
>
> Hey all,
>
> I have also noticed a huge slowdown in the biodb-to-json.pl script with
> the new release.  This was the same Bio::SeqFeature::Store database that I
> was using with the older version, and I believe it loaded in less than an
> hour before.  With this release, the job I started on Feb 18 is still
> running.
>
>
> Is it swapping?  How much ram does the machine have, and how long are the
> refseqs?  Memory usage used to be a function of the number of features, and
> now it's a function of the size of the largest refseq.  That function could
> be tweaked, though.
>
> Mitch
>



-- 
Chris Childers
Postdoctoral Fellow
Elsik Computational Genomics Laboratory
Georgetown University
Department of Biology
406 Reiss Bldg
Washington, DC 20057
Phone 202-687-5855
Fax 202-687-5662

Re: [Gmod-ajax] [apollo-dev] Re: JBrowse 1.2 problem generating JSON data from bed files?

[Mitch S. <mit...@be...>]

On 02/22/2011 06:02 AM, Chris Childers wrote:
> Hey all,
>
> I have also noticed a huge slowdown in the biodb-to-json.pl 
> <http://biodb-to-json.pl> script with the new release.  This was the 
> same Bio::SeqFeature::Store database that I was using with the older 
> version, and I believe it loaded in less than an hour before.  With 
> this release, the job I started on Feb 18 is still running.

Is it swapping?  How much ram does the machine have, and how long are 
the refseqs?  Memory usage used to be a function of the number of 
features, and now it's a function of the size of the largest refseq.  
That function could be tweaked, though.

Mitch

Re: [Gmod-ajax] JBrowse 1.2 problem generating JSON data from bed files?

[Mitch S. <mit...@be...>]

Okay, I think I've diagnosed both of these things.

They're both triggered, in this case, by BED files that have subfeatures 
(e.g., exons).  The subfeatures that come out of the BED parser in 
JBrowse are represented using Bioperl Bio::SeqFeature::Generic objects.  
Those objects contain coderefs (in their '_root_cleanup_methods' property).

That has two effects, which correspond to your two error messages below.

The first one is that those coderefs tickle a bug in the Devel::Size 
CPAN module:

http://codenode.com/2010/04/21/devel-size-coderef-segfault-fix/

That's causing the segfault.

The second effect occurs when the BED file with subfeatures is large 
enough.  In that case, the external sorting mechanism tries to write 
those bioperl objects out to disk, and the serialization mechanism that 
does the writing doesn't like coderefs.

I have some ideas about how to address these two issues, but the right 
approach is non-obvious IMO.  One possibility is to rewrite JBrowse's 
BED-handling to avoid bioperl objects.  One advantage of doing that is 
that it would fix the second issue and also work around the first 
issue.  That would take a bit of time to do, though.  A quick fix for 
the first issue is for users to download the Devel::Size CPAN module:

http://search.cpan.org/dist/Devel-Size/

and patch it by hand and install it.  That fix is available now; 
however, it's more work for users, and it doesn't address the second 
issue that you reported.

I'm going to look at it some more and see if I can find a quick fix that 
addresses both of these issues.

Mitch

On 02/18/2011 08:23 AM, Gregg Helt wrote:
> With the updated JBrowse 1.2, I'm getting "Segmentation fault" errors 
> when trying to process bed files with flatfile-to-json.pl 
> <http://flatfile-to-json.pl>
>
> With minimal parameters I just get the seg fault message:
> bin/flatfile-to-json.pl <http://flatfile-to-json.pl> --bed refGene.bed 
> --tracklabel refGene
> Segmentation fault
>
> With a fuller set of parameters I'm getting an additional error message:
>
> bin/flatfile-to-json.pl <http://flatfile-to-json.pl> --bed 
> all_mrna.bed --tracklabel all_mrna --key "Drosophila mRNAs" --cssClass 
> transcript --subfeatureClasses '{"CDS": "transcript-CDS", "UTR": 
> "transcript-UTR"}' --arrowheadClass transcript-arrowhead --getLabel 
> --autoComplete label --getSubs
>
> Can't store CODE items at ../../lib/Storable.pm (autosplit into 
> ../../lib/auto/Storable/_store_fd.al <http://store_fd.al>) line 304, 
> <GEN1> line 33158, at 
> /Users/gregg/projects/webapollo/jbrowse/bin/../lib/ExternalSorter.pm 
> line 92
> Segmentation fault
>
> I tried whittling down the bed file to simplify, but even with a one 
> line bed file I still get the seg fault.   I've attached the one-line 
> bed file, can someone try to reproduce this problem to see if it's 
> specific to my setup or not?
>
> These are standard bed files dowloaded from UCSC.  I'm using a fresh 
> install of BioPerl, so to verify that the problem is specific to the 
> new JBrowse release I went back to an earlier version of JBrowse (from 
> September 2010) and tried processing the same bed files.  The earlier 
> version worked.
>
> When processing normal-sized bed files there's a significant delay 
> (proportional to the size of the bed file) before the seg fault.  
> During the delay memory usage climbs linearly, and is 10x to 20x 
> greater than the earlier JBrowse version.  I'm not hitting my overall 
> memory limits, but wondering if something is causing a recursion that 
> leads to exceeding my stack size limits?
>
> Gregg

[Gmod-ajax] Genome Visualization Thread on BioStar

[Dave C. <cle...@ne...>]

Hello all,

There's an interesting thread about genome visualization going on at
BioStar:

http://biostar.stackexchange.com/questions/363/what-tools-libraries-do-you-use-to-visualize-genomic-feature-data

Lots of options have come up, including GBrowse and JBrowse.

Dave C.

-- 
http://gmod.org/wiki/GMOD_Americas_2011
http://gmod.org/wiki/GMOD_News
http://usegalaxy.org/
http://getgalaxy.org/

Re: [Gmod-ajax] JBrowse 1.2 problem generating JSON data from bed files?

[Mitch S. <mit...@be...>]

Thanks for the reports, these are my top priority right now.  I'll 
follow up to the list when they're fixed.

Mitch

On 02/18/2011 08:32 AM, Brenton Graveley wrote:
> I am having issues with JBrowse 1.2 as well, though they seem different.
>
> First, when using bin/flatfile-to-json.pl on a bed file that worked 
> previously I get the following:
>
> bin/flatfile-to-json.pl --bed FlyBase_FBtr_r5_12_annotations.nochr.bed 
> --tracklabel FlyBase_5.12 --key "FlyBase_5.12" --getLabel 
> --autocomplete label --cssclass transcript --subfeatureClasses 
> '{"UTR": "transcript-UTR", "CDS": "transcript-CDS"}' --getSubs 
> --arrowheadClass transcript-arrowhead
> Operation "cmp": no method found,
> left argument in overloaded package Bio::Annotation::SimpleValue,
> right argument in overloaded package Bio::Annotation::SimpleValue at 
> bin/flatfile-to-json.pl line 169, <GEN1> line 22309.
>
> Second, when loading wiggle tracks in using wig-to-json.pl 
> <http://wig-to-json.pl>, the process takes very long (4 hours so far) 
> for a wiggle track that took about 20 minutes with the previous version.
>
> Brent
>
>
> On Feb 18, 2011, at 11:23 AM, Gregg Helt wrote:
>
>> With the updated JBrowse 1.2, I'm getting "Segmentation fault" errors 
>> when trying to process bed files with flatfile-to-json.pl 
>> <http://flatfile-to-json.pl/>
>>
>> With minimal parameters I just get the seg fault message:
>> bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --bed 
>> refGene.bed --tracklabel refGene
>> Segmentation fault
>>
>> With a fuller set of parameters I'm getting an additional error message:
>>
>> bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --bed 
>> all_mrna.bed --tracklabel all_mrna --key "Drosophila mRNAs" 
>> --cssClass transcript --subfeatureClasses '{"CDS": "transcript-CDS", 
>> "UTR": "transcript-UTR"}' --arrowheadClass transcript-arrowhead 
>> --getLabel --autoComplete label --getSubs
>>
>> Can't store CODE items at ../../lib/Storable.pm (autosplit into 
>> ../../lib/auto/Storable/_store_fd.al <http://store_fd.al/>) line 304, 
>> <GEN1> line 33158, at 
>> /Users/gregg/projects/webapollo/jbrowse/bin/../lib/ExternalSorter.pm 
>> line 92
>> Segmentation fault
>>
>> I tried whittling down the bed file to simplify, but even with a one 
>> line bed file I still get the seg fault.   I've attached the one-line 
>> bed file, can someone try to reproduce this problem to see if it's 
>> specific to my setup or not?
>>
>> These are standard bed files dowloaded from UCSC.  I'm using a fresh 
>> install of BioPerl, so to verify that the problem is specific to the 
>> new JBrowse release I went back to an earlier version of JBrowse 
>> (from September 2010) and tried processing the same bed files.  The 
>> earlier version worked.
>>
>> When processing normal-sized bed files there's a significant delay 
>> (proportional to the size of the bed file) before the seg fault.  
>> During the delay memory usage climbs linearly, and is 10x to 20x 
>> greater than the earlier JBrowse version.  I'm not hitting my overall 
>> memory limits, but wondering if something is causing a recursion that 
>> leads to exceeding my stack size limits?
>>
>> Gregg
>> <tiny.bed>------------------------------------------------------------------------------
>> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
>> Pinpoint memory and threading errors before they happen.
>> Find and fix more than 250 security defects in the development cycle.
>> Locate bottlenecks in serial and parallel code that limit performance.
>> http://p.sf.net/sfu/intel-dev2devfeb_______________________________________________
>> Gmod-ajax mailing list
>> Gmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

Re: [Gmod-ajax] JBrowse 1.2 problem generating JSON data from bed files?

[Brenton G. <brg...@gm...>]

I am having issues with JBrowse 1.2 as well, though they seem different.  

First, when using bin/flatfile-to-json.pl on a bed file that worked previously I get the following:

bin/flatfile-to-json.pl --bed FlyBase_FBtr_r5_12_annotations.nochr.bed --tracklabel FlyBase_5.12 --key "FlyBase_5.12" --getLabel --autocomplete label --cssclass transcript --subfeatureClasses '{"UTR": "transcript-UTR", "CDS": "transcript-CDS"}' --getSubs --arrowheadClass transcript-arrowhead
Operation "cmp": no method found,
	left argument in overloaded package Bio::Annotation::SimpleValue,
	right argument in overloaded package Bio::Annotation::SimpleValue at bin/flatfile-to-json.pl line 169, <GEN1> line 22309.

Second, when loading wiggle tracks in using wig-to-json.pl, the process takes very long (4 hours so far) for a wiggle track that took about 20 minutes with the previous version.  

Brent


On Feb 18, 2011, at 11:23 AM, Gregg Helt wrote:

> With the updated JBrowse 1.2, I'm getting "Segmentation fault" errors when trying to process bed files with flatfile-to-json.pl
> 
> With minimal parameters I just get the seg fault message:
> bin/flatfile-to-json.pl --bed refGene.bed --tracklabel refGene 
> Segmentation fault
> 
> With a fuller set of parameters I'm getting an additional error message:
> 
> bin/flatfile-to-json.pl --bed all_mrna.bed --tracklabel all_mrna --key "Drosophila mRNAs" --cssClass transcript --subfeatureClasses '{"CDS": "transcript-CDS", "UTR": "transcript-UTR"}' --arrowheadClass transcript-arrowhead --getLabel --autoComplete label --getSubs
> 
> Can't store CODE items at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_store_fd.al) line 304, <GEN1> line 33158, at /Users/gregg/projects/webapollo/jbrowse/bin/../lib/ExternalSorter.pm line 92
> Segmentation fault
> 
> I tried whittling down the bed file to simplify, but even with a one line bed file I still get the seg fault.   I've attached the one-line bed file, can someone try to reproduce this problem to see if it's specific to my setup or not?
> 
> These are standard bed files dowloaded from UCSC.  I'm using a fresh install of BioPerl, so to verify that the problem is specific to the new JBrowse release I went back to an earlier version of JBrowse (from September 2010) and tried processing the same bed files.  The earlier version worked.
> 
> When processing normal-sized bed files there's a significant delay (proportional to the size of the bed file) before the seg fault.  During the delay memory usage climbs linearly, and is 10x to 20x greater than the earlier JBrowse version.  I'm not hitting my overall memory limits, but wondering if something is causing a recursion that leads to exceeding my stack size limits?
> 
> Gregg
> <tiny.bed>------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb_______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] JBrowse 1.2 release

[Bernd J. <ber...@pa...>]

Hi Mitch,

just installed it without much of a problem (only thing being that I 
don't know how to update with git or how to set up my MAC to use the 
http_proxy system variable..., but downloading and installing worked 
without any glitch).

I have imported a sorted bam file and it worked. Unfortunately I am 
missing a scale for the histogram view of the bam files. Also, in the 
most zoomed-out view the bars seem to be off or I don't understand how 
these values are calculated (I would probably want to have a max value 
rather than a mean...) Is there a way to change this?

Great work, will send feedback as it comes...

Best,

Bernd

On 2/17/2011 10:16 AM, Mitch Skinner wrote:
> JBrowse 1.2 is released, and now I'll be catching up on my email
> backlog, including a number of gmod-ajax messages.
>
> The code is available here (with minified javascript, which speeds up
> initial page loads):
>
> http://jbrowse.org/releases/jbrowse-1.2.zip
>
> or from github (without the minified JS):
>
> https://github.com/jbrowse/jbrowse/tree/1.2-release
>
> Release notes:
>
> version 1.2, Febrary 2011
>
> These notes document changes since release 1.1 in September 2010.
>
> Most of the work in this release went into making JBrowse handle large
> amounts of feature data better.  Before, the amount of memory used
> when processing BAM files was more than 10 times the size of the file;
> now, the amount of memory required is fixed.
>
> Other new features in this release:
>
> - Import of UCSC database dumps.  A ucsc-to-json.pl script is now
>     provided for taking database dumps from UCSC and creating a JBrowse
>     instance using them.  The "genePred" and "bed" track types are
>     currently supported; "psl" tracks are not yet supported.
>
> - Touch.  Juan Aguilar's code for using JBrowse on an iOS device
>     (iPhone, iPod touch, iPad) is now integrated.  As of the current
>     release, users wanting to use JBrowse on those devices have to
>     navigate to a separate HTML page (touch.html) rather than the
>     default index.html; i.e. the code does not currently detect
>     touchscreen devices automatically.
>
> - Bug fixes. A number of bugs have also been fixed, including one that
>     restricted the placement of the "data" directory, and a bug in
>     wiggle rendering that caused spurious peaks or troughs at tile
>     boundaries.
>
> Known issues/limitations with this release:
>
> - Some additional CPAN modules are now required:
>
>       PerlIO::gzip
>       Heap::Simple
>       Devel::Size
>
> - No JSON-level backward compatibility. If you are upgrading from an
>     older version of JBrowse, you will have to regenerate all the JSON
>     files on your server. This means wiping your jbrowse/data directory
>     and re-running all server scripts (flatfile-to-json, biodb-to-json,
>     wig-to-json, etc.) to regenerate your data from the original
>     FASTA/GFF/BED/WIG files. We apologize for the inconvenience of this,
>     but it is inevitable sometimes; we do aim to minimize the number of
>     releases which are backwardly-incompatible in this way.
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>

[Gmod-ajax] JBrowse 1.2 release

[Mitch S. <mit...@be...>]

JBrowse 1.2 is released, and now I'll be catching up on my email 
backlog, including a number of gmod-ajax messages.

The code is available here (with minified javascript, which speeds up 
initial page loads):

http://jbrowse.org/releases/jbrowse-1.2.zip

or from github (without the minified JS):

https://github.com/jbrowse/jbrowse/tree/1.2-release

Release notes:

version 1.2, Febrary 2011

These notes document changes since release 1.1 in September 2010.

Most of the work in this release went into making JBrowse handle large
amounts of feature data better.  Before, the amount of memory used
when processing BAM files was more than 10 times the size of the file;
now, the amount of memory required is fixed.

Other new features in this release:

- Import of UCSC database dumps.  A ucsc-to-json.pl script is now
   provided for taking database dumps from UCSC and creating a JBrowse
   instance using them.  The "genePred" and "bed" track types are
   currently supported; "psl" tracks are not yet supported.

- Touch.  Juan Aguilar's code for using JBrowse on an iOS device
   (iPhone, iPod touch, iPad) is now integrated.  As of the current
   release, users wanting to use JBrowse on those devices have to
   navigate to a separate HTML page (touch.html) rather than the
   default index.html; i.e. the code does not currently detect
   touchscreen devices automatically.

- Bug fixes. A number of bugs have also been fixed, including one that
   restricted the placement of the "data" directory, and a bug in
   wiggle rendering that caused spurious peaks or troughs at tile
   boundaries.

Known issues/limitations with this release:

- Some additional CPAN modules are now required:

     PerlIO::gzip
     Heap::Simple
     Devel::Size

- No JSON-level backward compatibility. If you are upgrading from an
   older version of JBrowse, you will have to regenerate all the JSON
   files on your server. This means wiping your jbrowse/data directory
   and re-running all server scripts (flatfile-to-json, biodb-to-json,
   wig-to-json, etc.) to regenerate your data from the original
   FASTA/GFF/BED/WIG files. We apologize for the inconvenience of this,
   but it is inevitable sometimes; we do aim to minimize the number of
   releases which are backwardly-incompatible in this way.

[Gmod-ajax] GMOD Satellite Meetings, March 7, NESCent, Durham, NC

[Dave C. <cle...@ne...>]

Hello all,

Yesterday, we opened registration for the March 2011 GMOD Meeting (see
http://gmod.oicr.on.ca/wiki/March_2011_GMOD_Meeting).  *That meeting is part
of a larger event, GMOD Americas 2011, that also includes several Satellite
Meetings on March 7*, the day after the meeting ends.

Satellite meetings are smaller groups of people meeting to discuss a common
interest, or work on a common problem (think special interest groups /
birds-of-a-feather).  Unlike the GMOD Meeting, there is *no registration fee
* for the Satellites, and you don't even need to go to any other GMOD
Americas events to participate in the Satellites.  *If you are in the area,
or attending other GMOD Americas events, or are just very interested in the
topic, we strongly encourage you to attend.*

The current list of Satellites (see
http://gmod.oicr.on.ca/wiki/Satellite_Meetings_-_GMOD_Americas_2011)
includes:

   - *GMOD Evo Hackathon
Followup*<http://gmod.oicr.on.ca/wiki/GMOD_Evo_Hackathon#March_2011_Satellite>,
   organized by Duke Leto <http://gmod.oicr.on.ca/wiki/User:Dukeleto>.  A
   followup to the GMOD Evo
Hackathon<http://gmod.oicr.on.ca/wiki/GMOD_Evo_Hackathon>held at
NESCent in November 2011.  You didn't to participate in the original
   event to participate in this followup.  Also, if there is interest, this
   satellite can extend for more than one day.
   - *Customizing and Extending JBrowse<http://gmod.oicr.on.ca/wiki/JBrowse>
   *, organized by Mitch Skinner<http://gmod.oicr.on.ca/wiki/User:MitchSkinner>
   .  JBrowse <http://gmod.oicr.on.ca/wiki/JBrowse> has a few different
   extension points, but they're not (yet) well-documented or widely used. The
   GMOD meeting would be a good time to review those
APIs<http://gmod.oicr.on.ca/wiki/Glossary#API>,
   relate them to the things that people want to do with them, discuss any
   potential changes or new APIs to support specific use cases, and potentially
   start to implement an extension.
   - *GMOD Web services toolkit* <http://gmod.oicr.on.ca/wiki/GMOD_RPC_API>,
   organized by Josh Goodman <http://gmod.oicr.on.ca/wiki/User:Jogoodma>.  Come
   to work on or discuss the GMOD Web services API and the
toolkit<http://gmod.oicr.on.ca/wiki/GMOD_RPC_API>
   .
   - *GMOD in the Sequencing
Center*<http://gmod.oicr.on.ca/wiki/GMOD_in_the_Sequencing_Center>,
   organized by  Chris Hemmerich <http://gmod.oicr.on.ca/wiki/User:Chemmeri>,
   Dave Clements <http://gmod.oicr.on.ca/wiki/User:Clements>.  Sequencing
   centers have tremendous bioinformatics needs that GMOD can help address.
   Attend this satellite to find out what other sequencing centers are doing
   with GMOD, and how GMOD can help you help your researchers.

If you are interested in participating in these, please contact the
organizers, and/or add your name to the satellite's participants list on the
wiki.

Satellites can be organized by anyone.  If you have a topic you would like
to cover, please add it to the list and announce it the appropriate mailing
lists.  Several previous satellites are written up on the GMOD wiki, if you
want an idea of what happens at a satellite.

Finally, please let me and Scott know if you have any questions.

Thanks, and hope to see you in March!

Dave C.
-- 
http://gmod.org/wiki/GMOD_Americas_2011
http://gmod.org/wiki/GMOD_News
http://usegalaxy.org/