gmod-ajax — JBrowse list

Re: [Gmod-ajax] [apollo] ways to implement cross-species links in apollo/jbrowse?

[Andrew F. <ad...@nc...>]

Hi Michael-
sure, we'd be happy to chat with you further, either on-list or off, 
just let me know what seems like
the best mechanism and I can try to set something up with the right 
people on my end involved.

And we'd be more than happy also to help get you set up with GCV on your 
data. The current implementation
of the services layer is using a m chado-based backend with a Django 
layer, but we have an alternative
implementation in progress using redis and a "microservices"-oriented 
approach (primarily to help with scaling
some of the heavier back-end operations).

looking forward to further discussion

Andrew


On 1/23/19 3:05 PM, Michael Poeschla wrote:
> Hi Andrew, thanks for your response. Both of these methods look pretty 
> promising, and it'd be great to try the first way first, and the GCV 
> looks really cool and I'd love to try it too at some point as you guys 
> develop it.
>
> I'm not too familiar with AJAX so it'd be really great to chat further 
> with you or someone over there and see how you guys implemented that.
>
> I'm also wondering, if anyone knows, how easy it is to generate a 
> bunch of links in apollo based on genome locations. I believe this 
> would be scriptable in jbrowse.. would the url structure let me do 
> this easily in apollo?
>
>
> Thanks so much,
> Michael Poeschla
> ------------------------------------------------------------------------
> *From: *"Andrew Farmer" <ad...@nc...>
> *To: *"apollo" <ap...@li...>, "Michael Poeschla" 
> <Mic...@ag...>, "gmod-ajax" 
> <gmo...@li...>
> *Sent: *Tuesday, January 22, 2019 7:31:35 AM
> *Subject: *Re: [apollo] ways to implement cross-species links in 
> apollo/jbrowse?
>
> Hi Michael-
> we have similar comparative genomics interests within our project 
> (which happens to deal with more or less closely related plant 
> species); and we've approached it in a couple of different ways, one 
> of which might be able to work for you too.
>
> The first way is simplest and perhaps appropriate for your case since 
> you already have precomputed orthology; we've simply injected an AJAX 
> service callout into the gene details dialog in JBrowse, and the 
> implementation of the service is responsible for providing links to 
> other resources for the gene (which in your case could be a set of 
> links to each of the orthologs).
> You can see it in action here:
> http://www.medicagohapmap2.org/jbrowse/?data=medtr%2FA17_HM341.gnm4&loc=medtr.A17_HM341.gnm4.ann2.Medtr5g089110&tracks=A17_HM341.gnm4.ann2
> (after you click on the glyph for the gene model, there might be a 
> slight pause before the links appear while the AJAX call is being 
> handled; we should probably have an indicator of sorts there)
>
> The second way is having JBrowse working in conjunction with a 
> multi-genome micro/macro synteny comparison app we've been developing 
> (the Genome Context Viewer (GCV): 
> https://github.com/legumeinfo/lis_context_viewer/wiki/Examples);
> this allows on-the-fly parameterized determination of syntelogs, which 
> can be communicated to drive JBrowse using messaging via a 
> BroadcastChannel; although we're still pretty much just exploring this 
> mechanism for effecting inter-application communication it so far 
> seems promising. But actually, even without this event-based approach, 
> the mechanism in the first option I described above can also just 
> linkout to the GCV app (as we've done among our links in that example, 
> just see the one described as "similar genomic contexts"). One 
> possible merit of this approach is that it can take you to related 
> regions of genomes via synteny even when an ortholog for the gene in 
> question appears to be missing (which is sometimes just an artifact of 
> assembly/annotation).
>
> If any of this seems to merit further discussion, I'd happy to discuss 
> further (and/or point you to someone from my group who can give more 
> implementation details).
>
> hope it helps, in any case
> Andrew
>
> On 1/21/19 9:48 AM, Michael Poeschla wrote:
>
>     Hi all,
>
>     I want to implement something with our apollo browser and was hoping that someone could point me in the right direction to get started, or let me know if a solution already exists. We have a browser loaded with closely related teleost species. I also have a table with orthologous genes between the species and their genomic locations that I want to use as the basis for a new feature, where I could click on an annotation in apollo (or jbrowse track) and then be able to click through to orthologous sites in other species. Not sure if this is a jbrowse or apollo issue so posting it to both lists per the developer's advice (thanks Nathan!).
>
>     Any advice/suggestions would be appreciated.
>
>     Thanks in advance!
>     Michael
>
>
>     This list is for the Apollo Annotation Editing Tool. Info athttp://genomearchitect.org/
>     If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to...@li...  | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.
>
>

Re: [Gmod-ajax] [apollo] ways to implement cross-species links in apollo/jbrowse?

[Cook, M. <ME...@st...>]

Lurking here, but, mightn’t a perhaps useful interface to navigating such syntelogs/orthologues be to (somehow) extend the mafviewer<https://github.com/cmdcolin/mafviewer> plugin to allow linking to precomputed multiple alignments…???


From: apo...@li... <apo...@li...> On Behalf Of Andrew Farmer
Sent: Tuesday, January 22, 2019 12:32 AM
To: ap...@li...; Michael Poeschla <Mic...@ag...>; gmo...@li...
Subject: Re: [apollo] ways to implement cross-species links in apollo/jbrowse?

**CAUTION: Non-Stowers email**
Hi Michael-
we have similar comparative genomics interests within our project (which happens to deal with more or less closely related plant species); and we've approached it in a couple of different ways, one of which might be able to work for you too.

The first way is simplest and perhaps appropriate for your case since you already have precomputed orthology; we've simply injected an AJAX service callout into the gene details dialog in JBrowse, and the implementation of the service is responsible for providing links to other resources for the gene (which in your case could be a set of links to each of the orthologs).
You can see it in action here:
http://www.medicagohapmap2.org/jbrowse/?data=medtr%2FA17_HM341.gnm4&loc=medtr.A17_HM341.gnm4.ann2.Medtr5g089110&tracks=A17_HM341.gnm4.ann2
(after you click on the glyph for the gene model, there might be a slight pause before the links appear while the AJAX call is being handled; we should probably have an indicator of sorts there)

The second way is having JBrowse working in conjunction with a multi-genome micro/macro synteny comparison app we've been developing (the Genome Context Viewer (GCV): https://github.com/legumeinfo/lis_context_viewer/wiki/Examples);
this allows on-the-fly parameterized determination of syntelogs, which can be communicated to drive JBrowse using messaging via a BroadcastChannel; although we're still pretty much just exploring this mechanism for effecting inter-application communication it so far seems promising. But actually, even without this event-based approach, the mechanism in the first option I described above can also just linkout to the GCV app (as we've done among our links in that example, just see the one described as "similar genomic contexts"). One possible merit of this approach is that it can take you to related regions of genomes via synteny even when an ortholog for the gene in question appears to be missing (which is sometimes just an artifact of assembly/annotation).

If any of this seems to merit further discussion, I'd happy to discuss further (and/or point you to someone from my group who can give more implementation details).

hope it helps, in any case
Andrew
On 1/21/19 9:48 AM, Michael Poeschla wrote:

Hi all,



I want to implement something with our apollo browser and was hoping that someone could point me in the right direction to get started, or let me know if a solution already exists. We have a browser loaded with closely related teleost species. I also have a table with orthologous genes between the species and their genomic locations that I want to use as the basis for a new feature, where I could click on an annotation in apollo (or jbrowse track) and then be able to click through to orthologous sites in other species. Not sure if this is a jbrowse or apollo issue so posting it to both lists per the developer's advice (thanks Nathan!).



Any advice/suggestions would be appreciated.



Thanks in advance!

Michael









This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/

If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li...<mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.

Re: [Gmod-ajax] [apollo] ways to implement cross-species links in apollo/jbrowse?

[Michael P. <Mic...@ag...>]

Hi Andrew, thanks for your response. Both of these methods look pretty promising, and it'd be great to try the first way first, and the GCV looks really cool and I'd love to try it too at some point as you guys develop it. 

I'm not too familiar with AJAX so it'd be really great to chat further with you or someone over there and see how you guys implemented that. 

I'm also wondering, if anyone knows, how easy it is to generate a bunch of links in apollo based on genome locations. I believe this would be scriptable in jbrowse.. would the url structure let me do this easily in apollo? 


Thanks so much, 
Michael Poeschla 
From: "Andrew Farmer" <ad...@nc...> 
To: "apollo" <ap...@li...>, "Michael Poeschla" <Mic...@ag...>, "gmod-ajax" <gmo...@li...> 
Sent: Tuesday, January 22, 2019 7:31:35 AM 
Subject: Re: [apollo] ways to implement cross-species links in apollo/jbrowse? 

Hi Michael- 
we have similar comparative genomics interests within our project (which happens to deal with more or less closely related plant species); and we've approached it in a couple of different ways, one of which might be able to work for you too. 

The first way is simplest and perhaps appropriate for your case since you already have precomputed orthology; we've simply injected an AJAX service callout into the gene details dialog in JBrowse, and the implementation of the service is responsible for providing links to other resources for the gene (which in your case could be a set of links to each of the orthologs). 
You can see it in action here: 
[ http://www.medicagohapmap2.org/jbrowse/?data=medtr%2FA17_HM341.gnm4&loc=medtr.A17_HM341.gnm4.ann2.Medtr5g089110&tracks=A17_HM341.gnm4.ann2 | http://www.medicagohapmap2.org/jbrowse/?data=medtr%2FA17_HM341.gnm4&loc=medtr.A17_HM341.gnm4.ann2.Medtr5g089110&tracks=A17_HM341.gnm4.ann2 ] 
(after you click on the glyph for the gene model, there might be a slight pause before the links appear while the AJAX call is being handled; we should probably have an indicator of sorts there) 

The second way is having JBrowse working in conjunction with a multi-genome micro/macro synteny comparison app we've been developing (the Genome Context Viewer (GCV): [ https://github.com/legumeinfo/lis_context_viewer/wiki/Examples | https://github.com/legumeinfo/lis_context_viewer/wiki/Examples ] ); 
this allows on-the-fly parameterized determination of syntelogs, which can be communicated to drive JBrowse using messaging via a BroadcastChannel; although we're still pretty much just exploring this mechanism for effecting inter-application communication it so far seems promising. But actually, even without this event-based approach, the mechanism in the first option I described above can also just linkout to the GCV app (as we've done among our links in that example, just see the one described as "similar genomic contexts"). One possible merit of this approach is that it can take you to related regions of genomes via synteny even when an ortholog for the gene in question appears to be missing (which is sometimes just an artifact of assembly/annotation). 

If any of this seems to merit further discussion, I'd happy to discuss further (and/or point you to someone from my group who can give more implementation details). 

hope it helps, in any case 
Andrew 

On 1/21/19 9:48 AM, Michael Poeschla wrote: 



Hi all, 

I want to implement something with our apollo browser and was hoping that someone could point me in the right direction to get started, or let me know if a solution already exists. We have a browser loaded with closely related teleost species. I also have a table with orthologous genes between the species and their genomic locations that I want to use as the basis for a new feature, where I could click on an annotation in apollo (or jbrowse track) and then be able to click through to orthologous sites in other species. Not sure if this is a jbrowse or apollo issue so posting it to both lists per the developer's advice (thanks Nathan!). 

Any advice/suggestions would be appreciated.

Thanks in advance!
Michael 

This list is for the Apollo Annotation Editing Tool. Info at [ http://genomearchitect.org/ | http://genomearchitect.org/ ] If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to [ mailto:sy...@li... | sy...@li... ] | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 

[Gmod-ajax] Fwd: [GSoC 2019] [Open Genome Informatics] Google Summer of Code 2019 Participation

[Scott C. <sc...@sc...>]

---------- Forwarded message ---------
From: Robin Haw <Rob...@oi...>
Date: Wed, Jan 23, 2019 at 10:20 AM
Subject: [GSoC 2019] [Open Genome Informatics] Google Summer of Code 2019
Participation
To: gmo...@li... <gmo...@li...>
Cc: Dannon Baker <dan...@gm...>, Dave Clements <
cle...@ga...>, Marc Gillespie <gil...@st...>, Scott
Cain <sc...@sc...>


Dear All,

The Open Genome Informatics team serves as an “umbrella" organization to
support the efforts of many open access open-source bioinformatics projects
for Google Summer of Code (GSoC) <https://summerofcode.withgoogle.com/>.
Among this list of projects are GMOD and its software projects -- GBrowse,
JBrowse; Galaxy; Reactome; WormBase; DockStore; Bioconda; and others.

Call for 2019 Project Ideas and Mentors:  We are seeking project ideas to
post and attract talented students to this year’s Summer of Code
competition. If you have a project idea for which you would like to mentor
a student, please contact Robin Haw, Marc Gillespie, Dave Clements, Dannon
Baker, and Scott Cain (emails above). You can also submit your ideas here
<http://gmod.org/wiki/GSOC_Project_Ideas_2019> or this online form
<https://docs.google.com/forms/d/1uQlpsbPCnhNwO5OFKAB-IiY9DLWlmaHZATnaeP6KHhw>.


For more information please refer to the Open Genome Informatics page
<http://gmod.org/wiki/GSoC>.

The mentoring organization application deadline with GSoC is February 6th,
2019 at 3 pm EST. So, if you are interested in taking part with the team
please let us know as soon as possible.

Please forward this to others who might be interested in taking part.

If you have any questions please let us know.

Thanks,
Robin, Marc, Dave, Dannon, and Scott.


-- 
------------------------------------------------------------------------
Scott Cain, Ph. D.                                   scott at scottcain dot
net
GMOD Coordinator (http://gmod.org/)                     216-392-3087
Ontario Institute for Cancer Research

[Gmod-ajax] JBrowse: override metadata display in 'About this track' box

[Vaneet L. <van...@uc...>]

Hello,

I am using the ComboTrackSelector plugin to have both the Hierarchical and Faceted track selectors active at the same time currently in my JBrowse.  This is because I have metadata to display in the faceted track selector which comes from a CSV file.  What I notice is that every column of this same metadata is displayed in the pop up box when I click on the track label drop down bar and select 'About this track'.  I've tried adding additional metadata fields or overwriting the content of some of the fields there because I didn't like the format in which they are displayed but that doesn't work it seems locked in to displaying the metadata from the CSV file referenced in the tracklist2.json file.

Do you know if there's any way I can stop it from displaying the metadata show in the faceted track selector and specify my own metadata fields to be displayed in the 'About this track' box possibly via the original tracklist.json file?

Thanks,

Vaneet

Vaneet Lotay
Xenbase Bioinformatician
716 ICT Building - University of Calgary
2500 University Drive NW
Calgary, AB T2N 1N4
CANADA

Re: [Gmod-ajax] [apollo] ways to implement cross-species links in apollo/jbrowse?

[Andrew F. <ad...@nc...>]

Hi Michael-
we have similar comparative genomics interests within our project (which 
happens to deal with more or less closely related plant species); and 
we've approached it in a couple of different ways, one of which might be 
able to work for you too.

The first way is simplest and perhaps appropriate for your case since 
you already have precomputed orthology; we've simply injected an AJAX 
service callout into the gene details dialog in JBrowse, and the 
implementation of the service is responsible for providing links to 
other resources for the gene (which in your case could be a set of links 
to each of the orthologs).
You can see it in action here:
http://www.medicagohapmap2.org/jbrowse/?data=medtr%2FA17_HM341.gnm4&loc=medtr.A17_HM341.gnm4.ann2.Medtr5g089110&tracks=A17_HM341.gnm4.ann2
(after you click on the glyph for the gene model, there might be a 
slight pause before the links appear while the AJAX call is being 
handled; we should probably have an indicator of sorts there)

The second way is having JBrowse working in conjunction with a 
multi-genome micro/macro synteny comparison app we've been developing 
(the Genome Context Viewer (GCV): 
https://github.com/legumeinfo/lis_context_viewer/wiki/Examples);
this allows on-the-fly parameterized determination of syntelogs, which 
can be communicated to drive JBrowse using messaging via a 
BroadcastChannel; although we're still pretty much just exploring this 
mechanism for effecting inter-application communication it so far seems 
promising. But actually, even without this event-based approach, the 
mechanism in the first option I described above can also just linkout to 
the GCV app (as we've done among our links in that example, just see the 
one described as "similar genomic contexts"). One possible merit of this 
approach is that it can take you to related regions of genomes via 
synteny even when an ortholog for the gene in question appears to be 
missing (which is sometimes just an artifact of assembly/annotation).

If any of this seems to merit further discussion, I'd happy to discuss 
further (and/or point you to someone from my group who can give more 
implementation details).

hope it helps, in any case
Andrew

On 1/21/19 9:48 AM, Michael Poeschla wrote:
> Hi all,
>
> I want to implement something with our apollo browser and was hoping that someone could point me in the right direction to get started, or let me know if a solution already exists. We have a browser loaded with closely related teleost species. I also have a table with orthologous genes between the species and their genomic locations that I want to use as the basis for a new feature, where I could click on an annotation in apollo (or jbrowse track) and then be able to click through to orthologous sites in other species. Not sure if this is a jbrowse or apollo issue so posting it to both lists per the developer's advice (thanks Nathan!).
>
> Any advice/suggestions would be appreciated.
>
> Thanks in advance!
> Michael
>
>
>
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.
>

[Gmod-ajax] ways to implement cross-species links in apollo/jbrowse?

[Michael P. <Mic...@ag...>]

Hi all, 

I want to implement something with our apollo browser and was hoping that someone could point me in the right direction to get started, or let me know if a solution already exists. We have a browser loaded with closely related teleost species. I also have a table with orthologous genes between the species and their genomic locations that I want to use as the basis for a new feature, where I could click on an annotation in apollo (or jbrowse track) and then be able to click through to orthologous sites in other species. Not sure if this is a jbrowse or apollo issue so posting it to both lists per the developer's advice (thanks Nathan!). 

Any advice/suggestions would be appreciated.

Thanks in advance!
Michael 

Re: [Gmod-ajax] jbrowse rangeerror: trying to access beyond buffer length

[Colin <col...@gm...>]

Maybe this email is out of date (caught in moderation queue) but I think
you had also asked this here https://github.com/GMOD/jbrowse/issues/1302 :)

Let me know if there are any other issues!


-Colin

On Fri, Jan 18, 2019 at 1:25 PM Jhon Trujillo <jht...@ce...>
wrote:

> Hi,
>
> I got the following error whit a bam file:
>
> jbrowse rangeerror: trying to access beyond buffer length
>
>
>
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Colin <col...@gm...>]

Awesome, I am glad that helped!


-Colin

On Thu, Jan 17, 2019 at 4:55 AM Douwe van der Leest <
dou...@gm...> wrote:

> Amazing!! This solved the entire issue (also the one of bed tracks not
> decompressing correctly).
>
> Thank you very much Colin and Rob for helping me with this issue, you’re
> life savers!
>
> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>
> *Van:* Colin [mailto:col...@gm...]
> *Verzonden:* woensdag 16 januari 2019 17:04
> *Aan:* Douwe van der Leest <dou...@gm...>
> *CC:* Robert Buels <rb...@gm...>; gmod-ajax <
> gmo...@li...>
> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
> appearing in newer versions of JBrowse, with same configuration
>
>
>
> Thanks for the details here. There have been changes to how CORS resources
> are accessed, specifically using some specific HTTP headers which affects
> CORS.
>
>
>
> One thing you might want to try is this modify the Amazon S3 bucket
> configuration for CORS, I haven't tested this but I recommended using
> something like this for someone before and they were able to get it working
>
>
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
>
>     <CORSRule>
>
>         <AllowedOrigin>*</AllowedOrigin>
>
>         <AllowedMethod>GET</AllowedMethod>
>
>         <AllowedHeader>Range</AllowedHeader>
>
>         <MaxAgeSeconds>3000</MaxAgeSeconds>
>
>         <ExposeHeader>Accept-Ranges</ExposeHeader>
>
>         <ExposeHeader>Content-Range</ExposeHeader>
>
>         <ExposeHeader>Content-Encoding</ExposeHeader>
>
>         <ExposeHeader>Content-Length</ExposeHeader>
>
>         <AllowedHeader>Authorization</AllowedHeader>
>
>     </CORSRule>
>
> </CORSConfiguration>
>
>
>
> Let me know if that helps,
>
>
>
> -Colin
>
>
>
> On Wed, Jan 16, 2019 at 6:58 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
> Colin, your interpretation is exactly what the document intended to show.
> Below you can see the actual filesizes and metadata regarding the three
> files in the Excel file.
>
> Rob, I have had truncated BAM files a few times, but each time viewing it
> with samtools view and subsequently tailing the end of the file alwas
> lead to an “BGZF EOF marker is missing” error (or something similar for
> SAMs).
>
>
>
> [root@cloud]# ls -lisan
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam
>
>
>
> 23485 869 -rw-r--r-- 1 1001 1001 888859 Jan  9 15:56
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam
>
> 22880 237 -rwxrwxr-x 1 1001 1001 242125 Dec 13 08:43
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam
>
> 23490 160 -rwxrwxr-x 1 1001 1001 163169 Jan  9 16:18
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>
> As shown by the metadata the requests in row 2 are acurate. You mention
> that “In the next request when it asks for bytes=262144-524287 this is
> beyond the file size.” How is this possible though, were there any
> modifications in the way JBrowse requests for data between these versions?
> Also the error messages vary over different versions, has anyone worked on
> this part as well, or is the way that data is requested varying over these
> versions such that different errors show up?
>
> Perhaps totally unrelated I noticed that my BED track for primers provides
> an error too. To show the different error messages for both the BED and the
> two BAM tracks over all versions between JBrowse-1.14.2-dev and
> JBrowse-1.16.1-release I have transferred some screenshots to the following
> we transfer link.
>
> https://we.tl/t-siBfz4F8JG
>
> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>
>
>
> *Van:* Robert Buels [mailto:rb...@gm... <rb...@gm...>]
> *Verzonden:* dinsdag 15 januari 2019 08:54
> *Aan:* Colin <col...@gm...>
> *CC:* Douwe van der Leest <dou...@gm...>; gmod-ajax <
> gmo...@li...>
> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
> appearing in newer versions of JBrowse, with same configuration
>
>
>
> Hmm, could that BAM file have been truncated somehow? Perhaps an
> incomplete copy onto the web server?
>
>
>
> On Mon, Jan 14, 2019 at 8:12 AM Colin <col...@gm...> wrote:
>
> I checked out this spreadsheet and hopefully i'm interpreting right but it
> looks like there are three different columns representing three different
> files as well as three different rows
>
>
>
> In the rows, I think you have essentially
>
>
>
> row 1) request for bai file
>
> row 2) request for bam header starting at byte 0
>
> row 3) request for slice of bam starting elsewhere in the file
>
>
>
> As it looks, the row 3 returns the error in the first two columns. From
> what I can tell, it might be accurate in doing so?
>
>
>
> For example, the content-range in the first column is given as bytes
> 0-163168/163169 which indicates the file is actually just 163169 bytes. In
> the next request when it asks for bytes=262144-524287 this is beyond the
> file size. Maybe you can check what the actual file size of that is?
>
>
>
>
>
> -Colin
>
>
>
>
>
>
>
> On Mon, Jan 14, 2019 at 3:52 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
> Hey Rob and Colin,
>
>
>
> Thank you for replying so quickly. You can find the headers enclosed in
> the Excel file. I have also changed the location of the files to see if it
> was a “location on bucket” related issue but this didn’t have any effect.
> Unfortunately, the data is confidential, so I have to verify (with my boss)
> whether I am allowed to send the file(s) to you. I will try rerunning the
> analysis to see if this somehow resolves the issue. I thought is might be
> related to metadata that was perhaps incorrect, but this seems file as
> well. Is there anything else I can do, which doesn’t include sending you
> the file, to help you narrow down the cause?
>
>
>
> Thanks a lot in advance.
>
> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>
> PS: Part of the Excel file is in Dutch, though I think the relevant
> information from my Firefox browser is in there.
>
>  *Douwe van der Leest*
>  Bioinformatician
>  Limgroup
>
> ✉
>
> dvd...@li...
>
> ☏
>
> +31 (0)77 397 99 00
>
> 🌍
>
> www.limgroup.eu
>
>
>
> *Van:* Robert Buels [mailto:rb...@gm...]
> *Verzonden:* vrijdag 11 januari 2019 19:14
> *Aan:* Colin <col...@gm...>
> *CC:* Douwe van der Leest <dou...@gm...>; gmod-ajax <
> gmo...@li...>
> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
> appearing in newer versions of JBrowse, with same configuration
>
>
>
> Hi Douwe,
>
>
>
> Could you look in the dev tools and get the request and response HTTP
> headers that the browser is sending and receiving? That would help us
> figure out what's going on here.
>
>
>
> Rob
>
>
>
> On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm...> wrote:
>
> Both these issues are very weird. I cannot think exactly what might be
> going on, it might be helpful to have data to test on.
>
>
>
> The Amazon S3 URLs linked are access denied for me but something like that
> would be good for testing
>
>
>
>
>
> -Colin
>
>
>
> On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
> Hey there,
>
> On advance, thanks for taking the time for reading and perhaps answering
> my question. I am updating my Jbrowse version from JBrowse-1.12.5; to
> JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and
> JBrowse-1.15.0-dev for troubleshooting some issues.
>
> I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in #1091 <https://github.com/GMOD/jbrowse/issues/1091> because it works in certain versions of JBrowse, but not in all.
>
>
>
> [image: jbrowse v1 14 2]
> <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png>
> [image: jbrowse v1 15 0]
> <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png>
> [image: jbrowse v1 16 1]
> <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png>
>
> Can someone please explain how to solve this?
>
> PS: some extra info:
> The CORS configuration of the bucket in which the data is stored:
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
>
> <CORSRule>
>
>     <AllowedOrigin>*</AllowedOrigin>
>
>     <AllowedMethod>GET</AllowedMethod>
>
>     <MaxAgeSeconds>3000</MaxAgeSeconds>
>
>     <AllowedHeader>*</AllowedHeader>
>
> </CORSRule>
>
> </CORSConfiguration>
>
> I am using apache:
>
> > rpm -q httpd
>
> httpd-2.4.6-80.el7.centos.1.x86_64
>
> The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think
> is meant for the configuration of my apache server
>
> Alias /jbrowse /var/www/jbrowse/default
>
> <Directory "/var/www/jbrowse/default/">
>
> Options Indexes FollowSymLinks ExecCGI
>
> Require all granted
>
> AllowOverride All
>
>  <IfModule mod_gzip.c>
>
>    mod_gzip_item_exclude "(\.jsonz|\.txtz)"
>
>  </IfModule>
>
>  <IfModule setenvif.c>
>
>    SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
>
>  </IfModule>
>
>  <IfModule mod_headers.c>
>
>    <FilesMatch "(\.jsonz|\.txtz)">
>
>      Header onsuccess set Content-Encoding gzip
>
>    </FilesMatch>
>
>  </IfModule>
>
> </Directory>
>
>
>
> Using curl –I on the urls for the bam and the bai file returns the following, respectively:
>
> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>
> HTTP/1.1 200 OK
>
> x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
>
> x-amz-request-id: 28B9E9172181C705
>
> Date: Fri, 11 Jan 2019 11:36:24 GMT
>
> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>
> ETag: "02aef1647f185c451ec55cf69731f939"
>
> x-amz-meta-mode: 33188
>
> x-amz-meta-gid: 0
>
> x-amz-meta-uid: 0
>
> x-amz-meta-mtime: 1547047128
>
> x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
>
> Accept-Ranges: bytes
>
> Content-Type: application/octet-stream
>
> Content-Length: 163169
>
> Server: AmazonS3
>
>
>
> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
>
> HTTP/1.1 200 OK
>
> x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
>
> x-amz-request-id: 9C523CCFAA1B2301
>
> Date: Fri, 11 Jan 2019 11:36:27 GMT
>
> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>
> ETag: "ac2c06762fc282602aa0f1db2d562b19"
>
> x-amz-meta-mode: 33188
>
> x-amz-meta-gid: 0
>
> x-amz-meta-uid: 0
>
> x-amz-meta-mtime: 1547047128
>
> x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
>
> Accept-Ranges: bytes
>
> Content-Type: application/octet-stream
>
> Content-Length: 502496
>
> Server: AmazonS3
>
>
>
> Please feel free to mail me if you need additional data.
>
> Thanks a bunch in advance,
>
>
>
> Cheers,
>
> Douwe
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Douwe v. d. L. <dou...@gm...>]

Amazing!! This solved the entire issue (also the one of bed tracks not decompressing correctly).

Thank you very much Colin and Rob for helping me with this issue, you’re life savers! 

Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,

Van: Colin [mailto:col...@gm...] 
Verzonden: woensdag 16 januari 2019 17:04
Aan: Douwe van der Leest <dou...@gm...>
CC: Robert Buels <rb...@gm...>; gmod-ajax <gmo...@li...>
Onderwerp: Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

 

Thanks for the details here. There have been changes to how CORS resources are accessed, specifically using some specific HTTP headers which affects CORS.

 

One thing you might want to try is this modify the Amazon S3 bucket configuration for CORS, I haven't tested this but I recommended using something like this for someone before and they were able to get it working

 

<?xml version="1.0" encoding="UTF-8"?>

<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">

    <CORSRule>

        <AllowedOrigin>*</AllowedOrigin>

        <AllowedMethod>GET</AllowedMethod>

        <AllowedHeader>Range</AllowedHeader>

        <MaxAgeSeconds>3000</MaxAgeSeconds>

        <ExposeHeader>Accept-Ranges</ExposeHeader>

        <ExposeHeader>Content-Range</ExposeHeader>

        <ExposeHeader>Content-Encoding</ExposeHeader>

        <ExposeHeader>Content-Length</ExposeHeader>

        <AllowedHeader>Authorization</AllowedHeader>

    </CORSRule>

</CORSConfiguration>

 

Let me know if that helps,

 

-Colin

 

On Wed, Jan 16, 2019 at 6:58 AM Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> > wrote:

Colin, your interpretation is exactly what the document intended to show. Below you can see the actual filesizes and metadata regarding the three files in the Excel file.

Rob, I have had truncated BAM files a few times, but each time viewing it with samtools view and subsequently tailing the end of the file alwas lead to an “BGZF EOF marker is missing” error (or something similar for SAMs).

 

[root@cloud]# ls -lisan /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam

 

23485 869 -rw-r--r-- 1 1001 1001 888859 Jan  9 15:56 /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam

22880 237 -rwxrwxr-x 1 1001 1001 242125 Dec 13 08:43 /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam

23490 160 -rwxrwxr-x 1 1001 1001 163169 Jan  9 16:18 /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam

As shown by the metadata the requests in row 2 are acurate. You mention that “In the next request when it asks for bytes=262144-524287 this is beyond the file size.” How is this possible though, were there any modifications in the way JBrowse requests for data between these versions? Also the error messages vary over different versions, has anyone worked on this part as well, or is the way that data is requested varying over these versions such that different errors show up?

Perhaps totally unrelated I noticed that my BED track for primers provides an error too. To show the different error messages for both the BED and the two BAM tracks over all versions between JBrowse-1.14.2-dev and JBrowse-1.16.1-release I have transferred some screenshots to the following we transfer link.

https://we.tl/t-siBfz4F8JG

Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,

 

Van: Robert Buels [ <mailto:rb...@gm...> mailto:rb...@gm...] 
Verzonden: dinsdag 15 januari 2019 08:54
Aan: Colin < <mailto:col...@gm...> col...@gm...>
CC: Douwe van der Leest < <mailto:dou...@gm...> dou...@gm...>; gmod-ajax < <mailto:gmo...@li...> gmo...@li...>
Onderwerp: Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

 

Hmm, could that BAM file have been truncated somehow? Perhaps an incomplete copy onto the web server?

 

On Mon, Jan 14, 2019 at 8:12 AM Colin <col...@gm... <mailto:col...@gm...> > wrote:

I checked out this spreadsheet and hopefully i'm interpreting right but it looks like there are three different columns representing three different files as well as three different rows

 

In the rows, I think you have essentially

 

row 1) request for bai file

row 2) request for bam header starting at byte 0

row 3) request for slice of bam starting elsewhere in the file

 

As it looks, the row 3 returns the error in the first two columns. From what I can tell, it might be accurate in doing so?

 

For example, the content-range in the first column is given as bytes 0-163168/163169 which indicates the file is actually just 163169 bytes. In the next request when it asks for bytes=262144-524287 this is beyond the file size. Maybe you can check what the actual file size of that is?

 

 

-Colin

 

 

 

On Mon, Jan 14, 2019 at 3:52 AM Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> > wrote:

Hey Rob and Colin,

 

Thank you for replying so quickly. You can find the headers enclosed in the Excel file. I have also changed the location of the files to see if it was a “location on bucket” related issue but this didn’t have any effect. Unfortunately, the data is confidential, so I have to verify (with my boss) whether I am allowed to send the file(s) to you. I will try rerunning the analysis to see if this somehow resolves the issue. I thought is might be related to metadata that was perhaps incorrect, but this seems file as well. Is there anything else I can do, which doesn’t include sending you the file, to help you narrow down the cause?

 

Thanks a lot in advance.

Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,

PS: Part of the Excel file is in Dutch, though I think the relevant information from my Firefox browser is in there.


 Douwe van der Leest 
 Bioinformatician
 Limgroup


✉ 

dvd...@li... <mailto:dvd...@li...> 


☏  

+31 (0)77 397 99 00


🌍 

www.limgroup.eu <http://www.limgroup.eu> 

 

Van: Robert Buels [mailto:rb...@gm... <mailto:rb...@gm...> ] 
Verzonden: vrijdag 11 januari 2019 19:14
Aan: Colin <col...@gm... <mailto:col...@gm...> >
CC: Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> >; gmod-ajax <gmo...@li... <mailto:gmo...@li...> >
Onderwerp: Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

 

Hi Douwe,

 

Could you look in the dev tools and get the request and response HTTP headers that the browser is sending and receiving? That would help us figure out what's going on here.

 

Rob

 

On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm... <mailto:col...@gm...> > wrote:

Both these issues are very weird. I cannot think exactly what might be going on, it might be helpful to have data to test on.

 

The Amazon S3 URLs linked are access denied for me but something like that would be good for testing

 

 

-Colin

 

On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> > wrote:

Hey there,

On advance, thanks for taking the time for reading and perhaps answering my question. I am updating my Jbrowse version from JBrowse-1.12.5; to JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and JBrowse-1.15.0-dev for troubleshooting some issues.

I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in  <https://github.com/GMOD/jbrowse/issues/1091> #1091 because it works in certain versions of JBrowse, but not in all.

 

 <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png> 
 <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png> 
 <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png> 

Can someone please explain how to solve this?

PS: some extra info:
The CORS configuration of the bucket in which the data is stored:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

I am using apache:

> rpm -q httpd
httpd-2.4.6-80.el7.centos.1.x86_64

The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think is meant for the configuration of my apache server

Alias /jbrowse /var/www/jbrowse/default
<Directory "/var/www/jbrowse/default/">
Options Indexes FollowSymLinks ExecCGI
Require all granted
AllowOverride All
 <IfModule mod_gzip.c>
   mod_gzip_item_exclude "(\.jsonz|\.txtz)"
 </IfModule>
 <IfModule setenvif.c>
   SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
 </IfModule>
 <IfModule mod_headers.c>
   <FilesMatch "(\.jsonz|\.txtz)">
     Header onsuccess set Content-Encoding gzip
   </FilesMatch>
 </IfModule>
</Directory>
 
Using curl –I on the urls for the bam and the bai file returns the following, respectively:
[root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
HTTP/1.1 200 OK
x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
x-amz-request-id: 28B9E9172181C705
Date: Fri, 11 Jan 2019 11:36:24 GMT
Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
ETag: "02aef1647f185c451ec55cf69731f939"
x-amz-meta-mode: 33188
x-amz-meta-gid: 0
x-amz-meta-uid: 0
x-amz-meta-mtime: 1547047128
x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 163169
Server: AmazonS3
 
[root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
HTTP/1.1 200 OK
x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
x-amz-request-id: 9C523CCFAA1B2301
Date: Fri, 11 Jan 2019 11:36:27 GMT
Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
ETag: "ac2c06762fc282602aa0f1db2d562b19"
x-amz-meta-mode: 33188
x-amz-meta-gid: 0
x-amz-meta-uid: 0
x-amz-meta-mtime: 1547047128
x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 502496
Server: AmazonS3
 

Please feel free to mail me if you need additional data.

Thanks a bunch in advance,

 

Cheers,

Douwe

_______________________________________________
Gmod-ajax mailing list
Gmo...@li... <mailto:Gmo...@li...> 
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

_______________________________________________
Gmod-ajax mailing list
Gmo...@li... <mailto:Gmo...@li...> 
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Colin <col...@gm...>]

Thanks for the details here. There have been changes to how CORS resources
are accessed, specifically using some specific HTTP headers which affects
CORS.

One thing you might want to try is this modify the Amazon S3 bucket
configuration for CORS, I haven't tested this but I recommended using
something like this for someone before and they were able to get it working

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>*</AllowedOrigin>
        <AllowedMethod>GET</AllowedMethod>
        <AllowedHeader>Range</AllowedHeader>
        <MaxAgeSeconds>3000</MaxAgeSeconds>
        <ExposeHeader>Accept-Ranges</ExposeHeader>
        <ExposeHeader>Content-Range</ExposeHeader>
        <ExposeHeader>Content-Encoding</ExposeHeader>
        <ExposeHeader>Content-Length</ExposeHeader>
        <AllowedHeader>Authorization</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

Let me know if that helps,

-Colin

On Wed, Jan 16, 2019 at 6:58 AM Douwe van der Leest <
dou...@gm...> wrote:

> Colin, your interpretation is exactly what the document intended to show.
> Below you can see the actual filesizes and metadata regarding the three
> files in the Excel file.
>
> Rob, I have had truncated BAM files a few times, but each time viewing it
> with samtools view and subsequently tailing the end of the file alwas
> lead to an “BGZF EOF marker is missing” error (or something similar for
> SAMs).
>
>
>
> [root@cloud]# ls -lisan
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam
>
>
>
> 23485 869 -rw-r--r-- 1 1001 1001 888859 Jan  9 15:56
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam
>
> 22880 237 -rwxrwxr-x 1 1001 1001 242125 Dec 13 08:43
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam
>
> 23490 160 -rwxrwxr-x 1 1001 1001 163169 Jan  9 16:18
> /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>
> As shown by the metadata the requests in row 2 are acurate. You mention
> that “In the next request when it asks for bytes=262144-524287 this is
> beyond the file size.” How is this possible though, were there any
> modifications in the way JBrowse requests for data between these versions?
> Also the error messages vary over different versions, has anyone worked on
> this part as well, or is the way that data is requested varying over these
> versions such that different errors show up?
>
> Perhaps totally unrelated I noticed that my BED track for primers provides
> an error too. To show the different error messages for both the BED and the
> two BAM tracks over all versions between JBrowse-1.14.2-dev and
> JBrowse-1.16.1-release I have transferred some screenshots to the following
> we transfer link.
>
> https://we.tl/t-siBfz4F8JG
>
> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>
>
>
> *Van:* Robert Buels [mailto:rb...@gm... <rb...@gm...>]
> *Verzonden:* dinsdag 15 januari 2019 08:54
> *Aan:* Colin <col...@gm...>
> *CC:* Douwe van der Leest <dou...@gm...>; gmod-ajax <
> gmo...@li...>
> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
> appearing in newer versions of JBrowse, with same configuration
>
>
>
> Hmm, could that BAM file have been truncated somehow? Perhaps an
> incomplete copy onto the web server?
>
>
>
> On Mon, Jan 14, 2019 at 8:12 AM Colin <col...@gm...> wrote:
>
> I checked out this spreadsheet and hopefully i'm interpreting right but it
> looks like there are three different columns representing three different
> files as well as three different rows
>
>
>
> In the rows, I think you have essentially
>
>
>
> row 1) request for bai file
>
> row 2) request for bam header starting at byte 0
>
> row 3) request for slice of bam starting elsewhere in the file
>
>
>
> As it looks, the row 3 returns the error in the first two columns. From
> what I can tell, it might be accurate in doing so?
>
>
>
> For example, the content-range in the first column is given as bytes
> 0-163168/163169 which indicates the file is actually just 163169 bytes. In
> the next request when it asks for bytes=262144-524287 this is beyond the
> file size. Maybe you can check what the actual file size of that is?
>
>
>
>
>
> -Colin
>
>
>
>
>
>
>
> On Mon, Jan 14, 2019 at 3:52 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
> Hey Rob and Colin,
>
>
>
> Thank you for replying so quickly. You can find the headers enclosed in
> the Excel file. I have also changed the location of the files to see if it
> was a “location on bucket” related issue but this didn’t have any effect.
> Unfortunately, the data is confidential, so I have to verify (with my boss)
> whether I am allowed to send the file(s) to you. I will try rerunning the
> analysis to see if this somehow resolves the issue. I thought is might be
> related to metadata that was perhaps incorrect, but this seems file as
> well. Is there anything else I can do, which doesn’t include sending you
> the file, to help you narrow down the cause?
>
>
>
> Thanks a lot in advance.
>
> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>
> PS: Part of the Excel file is in Dutch, though I think the relevant
> information from my Firefox browser is in there.
>
>  *Douwe van der Leest*
>  Bioinformatician
>  Limgroup
>
> ✉
>
> dvd...@li...
>
> ☏
>
> +31 (0)77 397 99 00
>
> 🌍
>
> www.limgroup.eu
>
>
>
> *Van:* Robert Buels [mailto:rb...@gm...]
> *Verzonden:* vrijdag 11 januari 2019 19:14
> *Aan:* Colin <col...@gm...>
> *CC:* Douwe van der Leest <dou...@gm...>; gmod-ajax <
> gmo...@li...>
> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
> appearing in newer versions of JBrowse, with same configuration
>
>
>
> Hi Douwe,
>
>
>
> Could you look in the dev tools and get the request and response HTTP
> headers that the browser is sending and receiving? That would help us
> figure out what's going on here.
>
>
>
> Rob
>
>
>
> On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm...> wrote:
>
> Both these issues are very weird. I cannot think exactly what might be
> going on, it might be helpful to have data to test on.
>
>
>
> The Amazon S3 URLs linked are access denied for me but something like that
> would be good for testing
>
>
>
>
>
> -Colin
>
>
>
> On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
> Hey there,
>
> On advance, thanks for taking the time for reading and perhaps answering
> my question. I am updating my Jbrowse version from JBrowse-1.12.5; to
> JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and
> JBrowse-1.15.0-dev for troubleshooting some issues.
>
> I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in #1091 <https://github.com/GMOD/jbrowse/issues/1091> because it works in certain versions of JBrowse, but not in all.
>
>
>
> [image: jbrowse v1 14 2]
> <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png>
> [image: jbrowse v1 15 0]
> <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png>
> [image: jbrowse v1 16 1]
> <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png>
>
> Can someone please explain how to solve this?
>
> PS: some extra info:
> The CORS configuration of the bucket in which the data is stored:
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
>
> <CORSRule>
>
>     <AllowedOrigin>*</AllowedOrigin>
>
>     <AllowedMethod>GET</AllowedMethod>
>
>     <MaxAgeSeconds>3000</MaxAgeSeconds>
>
>     <AllowedHeader>*</AllowedHeader>
>
> </CORSRule>
>
> </CORSConfiguration>
>
> I am using apache:
>
> > rpm -q httpd
>
> httpd-2.4.6-80.el7.centos.1.x86_64
>
> The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think
> is meant for the configuration of my apache server
>
> Alias /jbrowse /var/www/jbrowse/default
>
> <Directory "/var/www/jbrowse/default/">
>
> Options Indexes FollowSymLinks ExecCGI
>
> Require all granted
>
> AllowOverride All
>
>  <IfModule mod_gzip.c>
>
>    mod_gzip_item_exclude "(\.jsonz|\.txtz)"
>
>  </IfModule>
>
>  <IfModule setenvif.c>
>
>    SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
>
>  </IfModule>
>
>  <IfModule mod_headers.c>
>
>    <FilesMatch "(\.jsonz|\.txtz)">
>
>      Header onsuccess set Content-Encoding gzip
>
>    </FilesMatch>
>
>  </IfModule>
>
> </Directory>
>
>
>
> Using curl –I on the urls for the bam and the bai file returns the following, respectively:
>
> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>
> HTTP/1.1 200 OK
>
> x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
>
> x-amz-request-id: 28B9E9172181C705
>
> Date: Fri, 11 Jan 2019 11:36:24 GMT
>
> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>
> ETag: "02aef1647f185c451ec55cf69731f939"
>
> x-amz-meta-mode: 33188
>
> x-amz-meta-gid: 0
>
> x-amz-meta-uid: 0
>
> x-amz-meta-mtime: 1547047128
>
> x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
>
> Accept-Ranges: bytes
>
> Content-Type: application/octet-stream
>
> Content-Length: 163169
>
> Server: AmazonS3
>
>
>
> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
>
> HTTP/1.1 200 OK
>
> x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
>
> x-amz-request-id: 9C523CCFAA1B2301
>
> Date: Fri, 11 Jan 2019 11:36:27 GMT
>
> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>
> ETag: "ac2c06762fc282602aa0f1db2d562b19"
>
> x-amz-meta-mode: 33188
>
> x-amz-meta-gid: 0
>
> x-amz-meta-uid: 0
>
> x-amz-meta-mtime: 1547047128
>
> x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
>
> Accept-Ranges: bytes
>
> Content-Type: application/octet-stream
>
> Content-Length: 502496
>
> Server: AmazonS3
>
>
>
> Please feel free to mail me if you need additional data.
>
> Thanks a bunch in advance,
>
>
>
> Cheers,
>
> Douwe
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>

[Gmod-ajax] Link to JBrowse, highlighting a specific feature?

[Ben B. <bb...@gm...>]

Hello,

We often link to a jbrowse instance from other pages.  one example is that
we have a table of variants, and we have a link for 'view in jbrowse'.
this loads jbrowse, centered on that genomic region.  even though the
window is centered on the feature of interest, it is easy for people to get
confused as to which exact feature was selected among neighboring
features.  is there any existing functionality to highlight or emphasize a
specific feature within one track, and if so can this be passed on the
URL?

Thanks,
Ben

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Douwe v. d. L. <dou...@gm...>]

Colin, your interpretation is exactly what the document intended to show. Below you can see the actual filesizes and metadata regarding the three files in the Excel file.

Rob, I have had truncated BAM files a few times, but each time viewing it with samtools view and subsequently tailing the end of the file alwas lead to an “BGZF EOF marker is missing” error (or something similar for SAMs).

 

[root@cloud]# ls -lisan /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam

 

23485 869 -rw-r--r-- 1 1001 1001 888859 Jan  9 15:56 /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/ABCtoA.minimap2_assembly-AsparagusCHR_V1.1.sorted.bam

22880 237 -rwxrwxr-x 1 1001 1001 242125 Dec 13 08:43 /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.bam

23490 160 -rwxrwxr-x 1 1001 1001 163169 Jan  9 16:18 /mnt/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam

As shown by the metadata the requests in row 2 are acurate. You mention that “In the next request when it asks for bytes=262144-524287 this is beyond the file size.” How is this possible though, were there any modifications in the way JBrowse requests for data between these versions? Also the error messages vary over different versions, has anyone worked on this part as well, or is the way that data is requested varying over these versions such that different errors show up?

Perhaps totally unrelated I noticed that my BED track for primers provides an error too. To show the different error messages for both the BED and the two BAM tracks over all versions between JBrowse-1.14.2-dev and JBrowse-1.16.1-release I have transferred some screenshots to the following we transfer link.

https://we.tl/t-siBfz4F8JG

Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,

 

Van: Robert Buels [ <mailto:rb...@gm...> mailto:rb...@gm...] 
Verzonden: dinsdag 15 januari 2019 08:54
Aan: Colin < <mailto:col...@gm...> col...@gm...>
CC: Douwe van der Leest < <mailto:dou...@gm...> dou...@gm...>; gmod-ajax < <mailto:gmo...@li...> gmo...@li...>
Onderwerp: Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

 

Hmm, could that BAM file have been truncated somehow? Perhaps an incomplete copy onto the web server?

 

On Mon, Jan 14, 2019 at 8:12 AM Colin <col...@gm... <mailto:col...@gm...> > wrote:

I checked out this spreadsheet and hopefully i'm interpreting right but it looks like there are three different columns representing three different files as well as three different rows

 

In the rows, I think you have essentially

 

row 1) request for bai file

row 2) request for bam header starting at byte 0

row 3) request for slice of bam starting elsewhere in the file

 

As it looks, the row 3 returns the error in the first two columns. From what I can tell, it might be accurate in doing so?

 

For example, the content-range in the first column is given as bytes 0-163168/163169 which indicates the file is actually just 163169 bytes. In the next request when it asks for bytes=262144-524287 this is beyond the file size. Maybe you can check what the actual file size of that is?

 

 

-Colin

 

 

 

On Mon, Jan 14, 2019 at 3:52 AM Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> > wrote:

Hey Rob and Colin,

 

Thank you for replying so quickly. You can find the headers enclosed in the Excel file. I have also changed the location of the files to see if it was a “location on bucket” related issue but this didn’t have any effect. Unfortunately, the data is confidential, so I have to verify (with my boss) whether I am allowed to send the file(s) to you. I will try rerunning the analysis to see if this somehow resolves the issue. I thought is might be related to metadata that was perhaps incorrect, but this seems file as well. Is there anything else I can do, which doesn’t include sending you the file, to help you narrow down the cause?

 

Thanks a lot in advance.

Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,

PS: Part of the Excel file is in Dutch, though I think the relevant information from my Firefox browser is in there.


 Douwe van der Leest 
 Bioinformatician
 Limgroup


✉ 

dvd...@li... <mailto:dvd...@li...> 


☏  

+31 (0)77 397 99 00


🌍 

www.limgroup.eu <http://www.limgroup.eu> 

 

Van: Robert Buels [mailto:rb...@gm... <mailto:rb...@gm...> ] 
Verzonden: vrijdag 11 januari 2019 19:14
Aan: Colin <col...@gm... <mailto:col...@gm...> >
CC: Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> >; gmod-ajax <gmo...@li... <mailto:gmo...@li...> >
Onderwerp: Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

 

Hi Douwe,

 

Could you look in the dev tools and get the request and response HTTP headers that the browser is sending and receiving? That would help us figure out what's going on here.

 

Rob

 

On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm... <mailto:col...@gm...> > wrote:

Both these issues are very weird. I cannot think exactly what might be going on, it might be helpful to have data to test on.

 

The Amazon S3 URLs linked are access denied for me but something like that would be good for testing

 

 

-Colin

 

On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> > wrote:

Hey there,

On advance, thanks for taking the time for reading and perhaps answering my question. I am updating my Jbrowse version from JBrowse-1.12.5; to JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and JBrowse-1.15.0-dev for troubleshooting some issues.

I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in  <https://github.com/GMOD/jbrowse/issues/1091> #1091 because it works in certain versions of JBrowse, but not in all.

 

 <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png> 
 <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png> 
 <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png> 

Can someone please explain how to solve this?

PS: some extra info:
The CORS configuration of the bucket in which the data is stored:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

I am using apache:

> rpm -q httpd
httpd-2.4.6-80.el7.centos.1.x86_64

The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think is meant for the configuration of my apache server

Alias /jbrowse /var/www/jbrowse/default
<Directory "/var/www/jbrowse/default/">
Options Indexes FollowSymLinks ExecCGI
Require all granted
AllowOverride All
 <IfModule mod_gzip.c>
   mod_gzip_item_exclude "(\.jsonz|\.txtz)"
 </IfModule>
 <IfModule setenvif.c>
   SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
 </IfModule>
 <IfModule mod_headers.c>
   <FilesMatch "(\.jsonz|\.txtz)">
     Header onsuccess set Content-Encoding gzip
   </FilesMatch>
 </IfModule>
</Directory>
 
Using curl –I on the urls for the bam and the bai file returns the following, respectively:
[root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
HTTP/1.1 200 OK
x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
x-amz-request-id: 28B9E9172181C705
Date: Fri, 11 Jan 2019 11:36:24 GMT
Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
ETag: "02aef1647f185c451ec55cf69731f939"
x-amz-meta-mode: 33188
x-amz-meta-gid: 0
x-amz-meta-uid: 0
x-amz-meta-mtime: 1547047128
x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 163169
Server: AmazonS3
 
[root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
HTTP/1.1 200 OK
x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
x-amz-request-id: 9C523CCFAA1B2301
Date: Fri, 11 Jan 2019 11:36:27 GMT
Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
ETag: "ac2c06762fc282602aa0f1db2d562b19"
x-amz-meta-mode: 33188
x-amz-meta-gid: 0
x-amz-meta-uid: 0
x-amz-meta-mtime: 1547047128
x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 502496
Server: AmazonS3
 

Please feel free to mail me if you need additional data.

Thanks a bunch in advance,

 

Cheers,

Douwe

_______________________________________________
Gmod-ajax mailing list
Gmo...@li... <mailto:Gmo...@li...> 
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

_______________________________________________
Gmod-ajax mailing list
Gmo...@li... <mailto:Gmo...@li...> 
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] [apollo] add new organism

[Nathan D. <nat...@lb...>]

You’ll need to share your trackList.json and add a screen shot. 

Nathan


> On Jan 15, 2019, at 12:18 PM, Quanwei Zhang <qwz...@gm...> wrote:
> 
> Dear Loeaine and Nathan:
> 
> Thank you for your help. Now it works. 
> But I have a question about the track of BAM file (read alingments). Now it seems it shows individual reads (with line connecting paired reads that far from each other). Is that correct?
> But in some regions there may be thousands of reads mapped, and it seems only  few reads were shown. Can I make it show all the reads in the track? Besides, is there a way to show the pile up coverage of RNA-seq at each positions? 
> 
> Thanks
> 
> Best
> Quanwei
> 
> Quanwei Zhang <qwz...@gm... <mailto:qwz...@gm...>> 于2019年1月15日周二 上午11:09写道：
> Thank you all. I will try it!
> 
> Best
> Quanwei
> 
> Loraine Guéguen <lor...@sb... <mailto:lor...@sb...>> 于2019年1月15日周二 上午10:44写道：
> Dear Nathan and Quanwei,
> 
> Having a quick look at your posts, I see that you try to load a GTF file (and not a GFF file).
> See http://jbrowse.org/docs/faq.html#how-do-i-convert-gtf-to-gff <http://jbrowse.org/docs/faq.html#how-do-i-convert-gtf-to-gff>
> 
> My 2 cents.
> 
> Loraine 
> 
> 
> 
> Le 15/01/2019 16:26, Nathan Dunn a écrit :
>> 
>> It should produce a tracks directory with that label in the directory provided (and added to the trackList.json). 
>> 
>> That being said, at a minimum you should have transcript data in your GFF3.  Typically I would use type of mRNA or transcript to process the GFF3. 
>> 
>> Also, there is a maker2jbrowse command in JBrowse (I have not tried it), that might simplify this, as well.   
>> 
>> 
>> 
>> But depending on what you want, you are now more in the JBrowse realm and I am going to cross-post you there.  
>> 
>> Might be worth perusing the FAQ and a few other docs:
>> 
>> http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F <http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F>
>> http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html <http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html>
>> 
>> As well as the tutorial:  http://jbrowse.org/docs/tutorial_classic.html <http://jbrowse.org/docs/tutorial_classic.html> 
>> http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files <http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files>
>> 
>> Hope this helps,
>> 
>> Nathan
>> 
>> 
>>> On Jan 15, 2019, at 7:08 AM, Quanwei Zhang < <mailto:qwz...@gm...>qwz...@gm... <mailto:qwz...@gm...>> wrote:
>>> 
>>> Dear Nathan:
>>> 
>>> Thanks. Now I can upload some data. But it seems there may be still some error. Please see the attached figures for "Console" and "Network". 
>>> 
>>> Before I "add the new organism", I used the following command to prepare the data. Then I provide the directory of data.apollo when I "add the new organism". But I can only see the sequence file, no gene structure information from GFF. Isn't this something abnormal? Data under ./data.apollo include "seq        trackList.json    tracks        tracks.conf".
>>> 
>>> jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --fasta  ./data.input/Aldh1a1.folded.fasta --out ./data.apollo 
>>> jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --indexed_fasta ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>> jbrowse/bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --gff ./data.input/Aldh1a1.geneName.gtf --type exon --trackLabel MAKER --out ./data.apollo
>>> 
>>> 
>>> ##the GFF file are in the following format including information for exons and CDS
>>> Contig2667      maker   exon    11929   11954   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    12150   12251   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    12872   12979   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    13499   13569   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    14822   14955   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    15404   15498   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    21086   21187   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    21808   21915   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    22435   22505   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    23706   23839   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    24286   24380   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    29579   29680   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    30301   30408   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    30928   31036   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     11929   11954   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     12150   12251   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     12872   12979   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     13499   13569   .       -       1       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     14822   14955   .       -       0       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     15404   15498   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     21086   21187   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     21808   21915   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     22435   22505   .       -       1       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     23706   23839   .       -       0       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     24286   24380   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     29579   29680   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     30301   30408   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   CDS     30928   31036   .       -       0       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>>> Contig2667      maker   exon    154799  154971  .       +       .       transcript_id "UNKNOWN_1066"; gene_id "UNKNOWN_1066"; gene_name "UNKNOWN_1066";
>>> 
>>> 
>>> Thanks
>>> 
>>> Best
>>> Quanwei
>>> 
>>> 
>>> Nathan Dunn < <mailto:nat...@lb...>nat...@lb... <mailto:nat...@lb...>> 于2019年1月14日周一 下午7:53写道：
>>> 
>>> That’s exactly right.  It looks like its getting unparseable data (just a guess).  If you click on the tab you can see what the query is and click on the query preview (and copy and paste the                     response, as well) and hopefully we can see what’s wrong. 
>>> 
>>> Nathan
>>> 
>>> 
>>> <Edited.png>
>>> 
>>> <Screen Shot 2019-01-14 at 4.51.18 PM.png><Screen Shot 2019-01-14 at 4.51.22 PM.png>
>>> 
>>> 
>>> 
>>>> On Jan 14, 2019, at 12:40 PM, Quanwei Zhang <qwz...@gm... <mailto:qwz...@gm...>> wrote:
>>>> 
>>>> Dear Nathan:
>>>> 
>>>> Thank you for your reply. 
>>>> (1) In my example, I have only two sequences (with length 80145943bp,10970156bp) to upload. 
>>>> (2) I did not get new information in catalina.out. 
>>>> (3) For "javascript console" did you mean what shown in the attached file? In this example, I just add the sequence (no gff).
>>>> (4) Would you please explain what do you mean by "network output tabs"?
>>>> 
>>>> Besides the information I gave, do I need to provide information for "Search database"? 
>>>> 
>>>> Many thanks
>>>> 
>>>> Best
>>>> Quanwei
>>>> 
>>>> Nathan Dunn < <mailto:nat...@lb...>nat...@lb... <mailto:nat...@lb...>> 于2019年1月14日周一 下午1:15写道：
>>>> 
>>>> Looks like everything you are doing SHOULD work.  
>>>> 
>>>> Unless you have a lot of scaffolds (>50K) it shouldn’t take longer than a minute.  5K scaffolds takes me roughly 20 seconds to add, so I agree that something is wrong. 
>>>> 
>>>> 
>>>> If you don’t see anything produced in your catalina.out (as below) can you send me your javascript console and network output tabs? 
>>>> 
>>>> I am surprised you aren’t seeing any type of error on the backend when you click on “add organism” though. 
>>>> 
>>>> 
>>>> I think the directory name seems fine assuming you have read access to it.  
>>>> 
>>>> Nathan
>>>> 
>>>> 
>>>>> On Jan 14, 2019, at 7:51 AM, Quanwei Zhang < <mailto:qwz...@gm...>qwz...@gm... <mailto:qwz...@gm...>> wrote:
>>>>> 
>>>>> Hello: 
>>>>> 
>>>>> Thank again for your previous help. 
>>>>> 
>>>>> After I run command "apollo run-local", I can open the web page in web browser.  But when I tried to add a new organism, it always reported "Error" or "Failed" message as shown below in blue.
>>>>> 
>>>>> The attached file shows the information I put and now it says "processing" for about 20 minutes, but I guess there must be something wrong, because I am just trying to upload 2 reads (with length 80145943bp,10970156bp). The attached figure shows the information I input to add the new organism. As shown in it, the data I generated are under "/Users/qwzhang/Documents/dataAnalysis/BeaverNew/data/Apollo/Apollo2/data.apollo" (with  "seq        trackList.json    tracks        tracks.conf"). I used the following commands to generate the files.
>>>>> 
>>>>> Apollo/jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --fasta  ./data.input/Aldh1a1.folded.fasta --out ./data.apollo 
>>>>> Apollo/jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --indexed_fasta ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>>>> Apollo/jbrowse/bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --gff ./data.input/Aldh1a1.geneName.gtf --type exon --trackLabel MAKER --out ./data.apollo
>>>>> 
>>>>> 
>>>>> 
>>>>> Metrics servlet injected into web.xml
>>>>> Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml
>>>>> ***
>>>>> | Running Grails application
>>>>> 
>>>>> Configuring Shiro ...
>>>>> 
>>>>> Shiro Configured
>>>>> Starting Quartz Scheduler in QuartzFactoryBean
>>>>> | Server running. Browse to  <http://localhost:8082/apollo>http://localhost:8082/apollo <http://localhost:8082/apollo>
>>>>> | Error 2019-01-14 10:22:25,901 [http-nio-8082-exec-9] ERROR apollo.UserController  - org.bbop.apollo.PermissionException: User does not have permission for any organisms.
>>>>> | Error 2019-01-14 10:22:26,174 [http-nio-8082-exec-4] ERROR apollo.UserController  - User does not have permission for any organisms.
>>>>> | Error 2019-01-14 10:22:26,209 [http-nio-8082-exec-3] ERROR apollo.AnnotatorService  - Error: org.bbop.apollo.PermissionException: User does not have permission for any organisms.
>>>>> | Error 2019-01-14 10:26:34,612 [http-nio-8082-exec-7] ERROR authenticator.UsernamePasswordAuthenticatorService  - Problem authenticating: java.lang.IllegalArgumentException: Method argumet (authentication token) cannot be null.
>>>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN  apollo.PermissionService  - Failed to authenticate user
>>>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN  apollo.SecurityFilters  - Authentication failed
>>>>> 
>>>>> Any suggestions. Thanks
>>>>> 
>>>>> Best
>>>>> Quanwei
>>>>> <addNewOrganism.pdf>
>>>>> 
>>>>> 
>>>>> This list is for the Apollo Annotation Editing Tool. Info at  <http://genomearchitect.org/>http://genomearchitect.org/ <http://genomearchitect.org/>
>>>>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to  <mailto:sy...@li...>sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> This list is for the Apollo Annotation Editing Tool. Info at  <http://genomearchitect.org/>http://genomearchitect.org/ <http://genomearchitect.org/>
>>>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to  <mailto:sy...@li...>sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>>>> 
>>>> <NewExample.pdf>
>>>> 
>>>> 
>>>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>>>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to  <mailto:sy...@li...>sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>>>> 
>>> 
>>> 
>>> 
>>> 
>>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>>> 
>>> <Console.pdf><Network.pdf>
>>> 
>>> 
>>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>>> 
>> 
>> 
>> 
>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>> 
> 
> -- 
> Loraine Guéguen
> 
> UMR8227 LBI2M (Laboratory of Integrative Biology of Marine Models)
> Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences)
> 
> Station Biologique de Roscoff - CNRS | Sorbonne Université
> Place Georges Teissier - CS 90074
> 29688 Roscoff cedex
> tel: ++33 (0)2 98 29 56 46 (interne : 415)
> http://www.sb-roscoff.fr/en/laboratory-integrative-biology-marine-models <http://www.sb-roscoff.fr/en/laboratory-integrative-biology-marine-models>
> http://abims.sb-roscoff.fr/ <http://abims.sb-roscoff.fr/>
> 
> 
> 
> 
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
> 
> 
> 
> 
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
> 

Re: [Gmod-ajax] [apollo] add new organism

[Quanwei Z. <qwz...@gm...>]

Dear Loeaine and Nathan:

Thank you for your help. Now it works.
But I have a question about the track of BAM file (read alingments). Now it
seems it shows individual reads (with line connecting paired reads that far
from each other). Is that correct?
But in some regions there may be thousands of reads mapped, and it seems
only  few reads were shown. Can I make it show all the reads in the track?
Besides, is there a way to show the pile up coverage of RNA-seq at each
positions?

Thanks

Best
Quanwei

Quanwei Zhang <qwz...@gm...> 于2019年1月15日周二 上午11:09写道：

> Thank you all. I will try it!
>
> Best
> Quanwei
>
> Loraine Guéguen <lor...@sb...> 于2019年1月15日周二 上午10:44写道：
>
>> Dear Nathan and Quanwei,
>>
>> Having a quick look at your posts, I see that you try to load a GTF file
>> (and not a GFF file).
>> See http://jbrowse.org/docs/faq.html#how-do-i-convert-gtf-to-gff
>>
>> My 2 cents.
>>
>> Loraine
>>
>>
>>
>> Le 15/01/2019 16:26, Nathan Dunn a écrit :
>>
>>
>> It should produce a tracks directory with that label in the directory
>> provided (and added to the trackList.json).
>>
>> That being said, at a minimum you should have transcript data in your
>> GFF3.  Typically I would use type of mRNA or transcript to process the
>> GFF3.
>>
>> Also, there is a maker2jbrowse command in JBrowse (I have not tried it),
>> that might simplify this, as well.
>>
>>
>>
>> But depending on what you want, you are now more in the JBrowse realm and
>> I am going to cross-post you there.
>>
>> Might be worth perusing the FAQ and a few other docs:
>>
>> http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F
>> http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html
>>
>> As well as the tutorial:  http://jbrowse.org/docs/tutorial_classic.html
>> http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files
>>
>> Hope this helps,
>>
>> Nathan
>>
>>
>> On Jan 15, 2019, at 7:08 AM, Quanwei Zhang < <qwz...@gm...>
>> qwz...@gm...> wrote:
>>
>> Dear Nathan:
>>
>> Thanks. Now I can upload some data. But it seems there may be still some
>> error. Please see the attached figures for "Console" and "Network".
>>
>> Before I "add the new organism", I used the following command to prepare
>> the data. Then I provide the directory of data.apollo when I "add the new
>> organism". But I can only see the sequence file, no gene structure
>> information from GFF. Isn't this something abnormal? Data under
>> ./data.apollo include "seq        trackList.json    tracks
>> tracks.conf".
>>
>> jbrowse/bin/prepare-refseqs.pl --fasta
>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>> jbrowse/bin/prepare-refseqs.pl --indexed_fasta
>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>> jbrowse/bin/flatfile-to-json.pl --gff ./data.input/Aldh1a1.geneName.gtf
>> --type *exon* --trackLabel MAKER --out ./data.apollo
>>
>>
>> ##the GFF file are in the following format including information for
>> exons and CDS
>> Contig2667      maker   exon    11929   11954   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    12150   12251   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    12872   12979   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    13499   13569   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    14822   14955   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    15404   15498   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    21086   21187   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    21808   21915   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    22435   22505   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    23706   23839   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    24286   24380   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    29579   29680   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    30301   30408   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    30928   31036   .       -       .
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     11929   11954   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     12150   12251   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     12872   12979   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     13499   13569   .       -       1
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     14822   14955   .       -       0
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     15404   15498   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     21086   21187   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     21808   21915   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     22435   22505   .       -       1
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     23706   23839   .       -       0
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     24286   24380   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     29579   29680   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     30301   30408   .       -       2
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     30928   31036   .       -       0
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    154799  154971  .       +       .
>> transcript_id "UNKNOWN_1066"; gene_id "UNKNOWN_1066"; gene_name
>> "UNKNOWN_1066";
>>
>>
>> Thanks
>>
>> Best
>> Quanwei
>>
>>
>> Nathan Dunn < <nat...@lb...>nat...@lb...> 于2019年1月14日周一
>> 下午7:53写道：
>>
>>>
>>> That’s exactly right.  It looks like its getting unparseable data (just
>>> a guess).  If you click on the tab you can see what the query is and click
>>> on the query preview (and copy and paste the response, as well) and
>>> hopefully we can see what’s wrong.
>>>
>>> Nathan
>>>
>>>
>>> <Edited.png>
>>>
>>> <Screen Shot 2019-01-14 at 4.51.18 PM.png><Screen Shot 2019-01-14 at
>>> 4.51.22 PM.png>
>>>
>>>
>>>
>>> On Jan 14, 2019, at 12:40 PM, Quanwei Zhang <qwz...@gm...>
>>> wrote:
>>>
>>> Dear Nathan:
>>>
>>> Thank you for your reply.
>>> (1) In my example, I have only two sequences (with length
>>> 80145943bp,10970156bp) to upload.
>>> (2) I did not get new information in catalina.out.
>>> (3) For "javascript console" did you mean what shown in the attached
>>> file? In this example, I just add the sequence (no gff).
>>> (4) Would you please explain what do you mean by "network output tabs"?
>>>
>>> Besides the information I gave, do I need to provide information for
>>> "Search database"?
>>>
>>> Many thanks
>>>
>>> Best
>>> Quanwei
>>>
>>> Nathan Dunn < <nat...@lb...>nat...@lb...> 于2019年1月14日周一
>>> 下午1:15写道：
>>>
>>>>
>>>> Looks like everything you are doing SHOULD work.
>>>>
>>>> Unless you have a lot of scaffolds (>50K) it shouldn’t take longer than
>>>> a minute.  5K scaffolds takes me roughly 20 seconds to add, so I agree that
>>>> something is wrong.
>>>>
>>>>
>>>> If you don’t see anything produced in your catalina.out (as below) can
>>>> you send me your javascript console and network output tabs?
>>>>
>>>> I am surprised you aren’t seeing any type of error on the backend when
>>>> you click on “add organism” though.
>>>>
>>>>
>>>> I think the directory name seems fine assuming you have read access to
>>>> it.
>>>>
>>>> Nathan
>>>>
>>>>
>>>> On Jan 14, 2019, at 7:51 AM, Quanwei Zhang < <qwz...@gm...>
>>>> qwz...@gm...> wrote:
>>>>
>>>> Hello:
>>>>
>>>> Thank again for your previous help.
>>>>
>>>> After I run command "apollo run-local", I can open the web page in web
>>>> browser.  But when I tried to add a new organism, it always reported
>>>> "Error" or "Failed" message as shown below in blue.
>>>>
>>>> The attached file shows the information I put and now it says
>>>> "processing" for about 20 minutes, but I guess there must be something
>>>> wrong, because I am just trying to upload 2 reads (with length
>>>> 80145943bp,10970156bp). The attached figure shows the information I input
>>>> to add the new organism. As shown in it, the data I generated are under
>>>> "/Users/qwzhang/Documents/dataAnalysis/BeaverNew/data/Apollo/Apollo2/data.apollo"
>>>> (with  "seq        trackList.json    tracks        tracks.conf"). I used
>>>> the following commands to generate the files.
>>>>
>>>> Apollo/jbrowse/bin/prepare-refseqs.pl --fasta
>>>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>>> Apollo/jbrowse/bin/prepare-refseqs.pl --indexed_fasta
>>>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>>> Apollo/jbrowse/bin/flatfile-to-json.pl --gff
>>>> ./data.input/Aldh1a1.geneName.gtf --type *exon* --trackLabel MAKER
>>>> --out ./data.apollo
>>>>
>>>>
>>>>
>>>> Metrics servlet injected into web.xml
>>>> Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml
>>>> ***
>>>> | Running Grails application
>>>>
>>>> Configuring Shiro ...
>>>>
>>>> Shiro Configured
>>>> Starting Quartz Scheduler in QuartzFactoryBean
>>>> | Server running. Browse to <http://localhost:8082/apollo>
>>>> http://localhost:8082/apollo
>>>> | Error 2019-01-14 10:22:25,901 [http-nio-8082-exec-9] ERROR
>>>> apollo.UserController  - org.bbop.apollo.PermissionException: User does not
>>>> have permission for any organisms.
>>>> | Error 2019-01-14 10:22:26,174 [http-nio-8082-exec-4] ERROR
>>>> apollo.UserController  - User does not have permission for any organisms.
>>>> | Error 2019-01-14 10:22:26,209 [http-nio-8082-exec-3] ERROR
>>>> apollo.AnnotatorService  - Error: org.bbop.apollo.PermissionException: User
>>>> does not have permission for any organisms.
>>>> | Error 2019-01-14 10:26:34,612 [http-nio-8082-exec-7] ERROR
>>>> authenticator.UsernamePasswordAuthenticatorService  - Problem
>>>> authenticating: java.lang.IllegalArgumentException: Method argumet
>>>> (authentication token) cannot be null.
>>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN
>>>> apollo.PermissionService  - Failed to authenticate user
>>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN
>>>> apollo.SecurityFilters  - Authentication failed
>>>>
>>>> Any suggestions. Thanks
>>>>
>>>> Best
>>>> Quanwei
>>>> <addNewOrganism.pdf>
>>>>
>>>>
>>>> This list is for the Apollo Annotation Editing Tool. Info at
>>>> <http://genomearchitect.org/>http://genomearchitect.org/
>>>> If you wish to unsubscribe from the Apollo List: 1. From the address
>>>> with which you subscribed to the list, send a message to
>>>> <sy...@li...>sy...@li... | 2. In the subject line of
>>>> your email type: unsubscribe apollo | 3. Leave the message body blank.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> This list is for the Apollo Annotation Editing Tool. Info at
>>>> <http://genomearchitect.org/>http://genomearchitect.org/
>>>> If you wish to unsubscribe from the Apollo List: 1. From the address
>>>> with which you subscribed to the list, send a message to
>>>> <sy...@li...>sy...@li... | 2. In the subject line of
>>>> your email type: unsubscribe apollo | 3. Leave the message body blank.
>>>>
>>>> <NewExample.pdf>
>>>
>>>
>>> This list is for the Apollo Annotation Editing Tool. Info at
>>> http://genomearchitect.org/
>>> If you wish to unsubscribe from the Apollo List: 1. From the address
>>> with which you subscribed to the list, send a message to
>>> <sy...@li...>sy...@li... | 2. In the subject line of
>>> your email type: unsubscribe apollo | 3. Leave the message body blank.
>>>
>>>
>>>
>>>
>>>
>>> This list is for the Apollo Annotation Editing Tool. Info at
>>> http://genomearchitect.org/
>>> If you wish to unsubscribe from the Apollo List: 1. From the address
>>> with which you subscribed to the list, send a message to
>>> sy...@li... | 2. In the subject line of your email type:
>>> unsubscribe apollo | 3. Leave the message body blank.
>>>
>>> <Console.pdf><Network.pdf>
>>
>>
>> This list is for the Apollo Annotation Editing Tool. Info at
>> http://genomearchitect.org/
>> If you wish to unsubscribe from the Apollo List: 1. From the address with
>> which you subscribed to the list, send a message to sy...@li...
>> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave
>> the message body blank.
>>
>>
>>
>>
>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.
>>
>>
>>
>> --
>> Loraine Guéguen
>>
>> UMR8227 LBI2M (Laboratory of Integrative Biology of Marine Models)
>> Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences)
>>
>> Station Biologique de Roscoff - CNRS | Sorbonne Université
>> Place Georges Teissier - CS 90074
>> 29688 Roscoff cedex
>> tel: ++33 (0)2 98 29 56 46 (interne : 415)http://www.sb-roscoff.fr/en/laboratory-integrative-biology-marine-modelshttp://abims.sb-roscoff.fr/
>>
>>
>>
>>
>> This list is for the Apollo Annotation Editing Tool. Info at
>> http://genomearchitect.org/
>> If you wish to unsubscribe from the Apollo List: 1. From the address with
>> which you subscribed to the list, send a message to sy...@li...
>> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave
>> the message body blank.
>>
>>

Re: [Gmod-ajax] [apollo] add new organism

[Quanwei Z. <qwz...@gm...>]

Thank you all. I will try it!

Best
Quanwei

Loraine Guéguen <lor...@sb...> 于2019年1月15日周二 上午10:44写道：

> Dear Nathan and Quanwei,
>
> Having a quick look at your posts, I see that you try to load a GTF file
> (and not a GFF file).
> See http://jbrowse.org/docs/faq.html#how-do-i-convert-gtf-to-gff
>
> My 2 cents.
>
> Loraine
>
>
>
> Le 15/01/2019 16:26, Nathan Dunn a écrit :
>
>
> It should produce a tracks directory with that label in the directory
> provided (and added to the trackList.json).
>
> That being said, at a minimum you should have transcript data in your
> GFF3.  Typically I would use type of mRNA or transcript to process the
> GFF3.
>
> Also, there is a maker2jbrowse command in JBrowse (I have not tried it),
> that might simplify this, as well.
>
>
>
> But depending on what you want, you are now more in the JBrowse realm and
> I am going to cross-post you there.
>
> Might be worth perusing the FAQ and a few other docs:
>
> http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F
> http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html
>
> As well as the tutorial:  http://jbrowse.org/docs/tutorial_classic.html
> http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files
>
> Hope this helps,
>
> Nathan
>
>
> On Jan 15, 2019, at 7:08 AM, Quanwei Zhang < <qwz...@gm...>
> qwz...@gm...> wrote:
>
> Dear Nathan:
>
> Thanks. Now I can upload some data. But it seems there may be still some
> error. Please see the attached figures for "Console" and "Network".
>
> Before I "add the new organism", I used the following command to prepare
> the data. Then I provide the directory of data.apollo when I "add the new
> organism". But I can only see the sequence file, no gene structure
> information from GFF. Isn't this something abnormal? Data under
> ./data.apollo include "seq        trackList.json    tracks
> tracks.conf".
>
> jbrowse/bin/prepare-refseqs.pl --fasta  ./data.input/Aldh1a1.folded.fasta
> --out ./data.apollo
> jbrowse/bin/prepare-refseqs.pl --indexed_fasta
> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
> jbrowse/bin/flatfile-to-json.pl --gff ./data.input/Aldh1a1.geneName.gtf
> --type *exon* --trackLabel MAKER --out ./data.apollo
>
>
> ##the GFF file are in the following format including information for exons
> and CDS
> Contig2667      maker   exon    11929   11954   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    12150   12251   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    12872   12979   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    13499   13569   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    14822   14955   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    15404   15498   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    21086   21187   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    21808   21915   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    22435   22505   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    23706   23839   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    24286   24380   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    29579   29680   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    30301   30408   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    30928   31036   .       -       .
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     11929   11954   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     12150   12251   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     12872   12979   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     13499   13569   .       -       1
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     14822   14955   .       -       0
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     15404   15498   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     21086   21187   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     21808   21915   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     22435   22505   .       -       1
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     23706   23839   .       -       0
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     24286   24380   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     29579   29680   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     30301   30408   .       -       2
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     30928   31036   .       -       0
> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    154799  154971  .       +       .
> transcript_id "UNKNOWN_1066"; gene_id "UNKNOWN_1066"; gene_name
> "UNKNOWN_1066";
>
>
> Thanks
>
> Best
> Quanwei
>
>
> Nathan Dunn < <nat...@lb...>nat...@lb...> 于2019年1月14日周一
> 下午7:53写道：
>
>>
>> That’s exactly right.  It looks like its getting unparseable data (just a
>> guess).  If you click on the tab you can see what the query is and click on
>> the query preview (and copy and paste the response, as well) and hopefully
>> we can see what’s wrong.
>>
>> Nathan
>>
>>
>> <Edited.png>
>>
>> <Screen Shot 2019-01-14 at 4.51.18 PM.png><Screen Shot 2019-01-14 at
>> 4.51.22 PM.png>
>>
>>
>>
>> On Jan 14, 2019, at 12:40 PM, Quanwei Zhang <qwz...@gm...>
>> wrote:
>>
>> Dear Nathan:
>>
>> Thank you for your reply.
>> (1) In my example, I have only two sequences (with length
>> 80145943bp,10970156bp) to upload.
>> (2) I did not get new information in catalina.out.
>> (3) For "javascript console" did you mean what shown in the attached
>> file? In this example, I just add the sequence (no gff).
>> (4) Would you please explain what do you mean by "network output tabs"?
>>
>> Besides the information I gave, do I need to provide information for
>> "Search database"?
>>
>> Many thanks
>>
>> Best
>> Quanwei
>>
>> Nathan Dunn < <nat...@lb...>nat...@lb...> 于2019年1月14日周一
>> 下午1:15写道：
>>
>>>
>>> Looks like everything you are doing SHOULD work.
>>>
>>> Unless you have a lot of scaffolds (>50K) it shouldn’t take longer than
>>> a minute.  5K scaffolds takes me roughly 20 seconds to add, so I agree that
>>> something is wrong.
>>>
>>>
>>> If you don’t see anything produced in your catalina.out (as below) can
>>> you send me your javascript console and network output tabs?
>>>
>>> I am surprised you aren’t seeing any type of error on the backend when
>>> you click on “add organism” though.
>>>
>>>
>>> I think the directory name seems fine assuming you have read access to
>>> it.
>>>
>>> Nathan
>>>
>>>
>>> On Jan 14, 2019, at 7:51 AM, Quanwei Zhang < <qwz...@gm...>
>>> qwz...@gm...> wrote:
>>>
>>> Hello:
>>>
>>> Thank again for your previous help.
>>>
>>> After I run command "apollo run-local", I can open the web page in web
>>> browser.  But when I tried to add a new organism, it always reported
>>> "Error" or "Failed" message as shown below in blue.
>>>
>>> The attached file shows the information I put and now it says
>>> "processing" for about 20 minutes, but I guess there must be something
>>> wrong, because I am just trying to upload 2 reads (with length
>>> 80145943bp,10970156bp). The attached figure shows the information I input
>>> to add the new organism. As shown in it, the data I generated are under
>>> "/Users/qwzhang/Documents/dataAnalysis/BeaverNew/data/Apollo/Apollo2/data.apollo"
>>> (with  "seq        trackList.json    tracks        tracks.conf"). I used
>>> the following commands to generate the files.
>>>
>>> Apollo/jbrowse/bin/prepare-refseqs.pl --fasta
>>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>> Apollo/jbrowse/bin/prepare-refseqs.pl --indexed_fasta
>>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>> Apollo/jbrowse/bin/flatfile-to-json.pl --gff
>>> ./data.input/Aldh1a1.geneName.gtf --type *exon* --trackLabel MAKER
>>> --out ./data.apollo
>>>
>>>
>>>
>>> Metrics servlet injected into web.xml
>>> Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml
>>> ***
>>> | Running Grails application
>>>
>>> Configuring Shiro ...
>>>
>>> Shiro Configured
>>> Starting Quartz Scheduler in QuartzFactoryBean
>>> | Server running. Browse to <http://localhost:8082/apollo>
>>> http://localhost:8082/apollo
>>> | Error 2019-01-14 10:22:25,901 [http-nio-8082-exec-9] ERROR
>>> apollo.UserController  - org.bbop.apollo.PermissionException: User does not
>>> have permission for any organisms.
>>> | Error 2019-01-14 10:22:26,174 [http-nio-8082-exec-4] ERROR
>>> apollo.UserController  - User does not have permission for any organisms.
>>> | Error 2019-01-14 10:22:26,209 [http-nio-8082-exec-3] ERROR
>>> apollo.AnnotatorService  - Error: org.bbop.apollo.PermissionException: User
>>> does not have permission for any organisms.
>>> | Error 2019-01-14 10:26:34,612 [http-nio-8082-exec-7] ERROR
>>> authenticator.UsernamePasswordAuthenticatorService  - Problem
>>> authenticating: java.lang.IllegalArgumentException: Method argumet
>>> (authentication token) cannot be null.
>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN
>>> apollo.PermissionService  - Failed to authenticate user
>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN
>>> apollo.SecurityFilters  - Authentication failed
>>>
>>> Any suggestions. Thanks
>>>
>>> Best
>>> Quanwei
>>> <addNewOrganism.pdf>
>>>
>>>
>>> This list is for the Apollo Annotation Editing Tool. Info at
>>> <http://genomearchitect.org/>http://genomearchitect.org/
>>> If you wish to unsubscribe from the Apollo List: 1. From the address
>>> with which you subscribed to the list, send a message to
>>> <sy...@li...>sy...@li... | 2. In the subject line of
>>> your email type: unsubscribe apollo | 3. Leave the message body blank.
>>>
>>>
>>>
>>>
>>>
>>> This list is for the Apollo Annotation Editing Tool. Info at
>>> <http://genomearchitect.org/>http://genomearchitect.org/
>>> If you wish to unsubscribe from the Apollo List: 1. From the address
>>> with which you subscribed to the list, send a message to
>>> <sy...@li...>sy...@li... | 2. In the subject line of
>>> your email type: unsubscribe apollo | 3. Leave the message body blank.
>>>
>>> <NewExample.pdf>
>>
>>
>> This list is for the Apollo Annotation Editing Tool. Info at
>> http://genomearchitect.org/
>> If you wish to unsubscribe from the Apollo List: 1. From the address with
>> which you subscribed to the list, send a message to <sy...@li...>
>> sy...@li... | 2. In the subject line of your email type:
>> unsubscribe apollo | 3. Leave the message body blank.
>>
>>
>>
>>
>>
>> This list is for the Apollo Annotation Editing Tool. Info at
>> http://genomearchitect.org/
>> If you wish to unsubscribe from the Apollo List: 1. From the address with
>> which you subscribed to the list, send a message to sy...@li...
>> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave
>> the message body blank.
>>
>> <Console.pdf><Network.pdf>
>
>
> This list is for the Apollo Annotation Editing Tool. Info at
> http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with
> which you subscribed to the list, send a message to sy...@li... |
> 2. In the subject line of your email type: unsubscribe apollo | 3. Leave
> the message body blank.
>
>
>
>
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.
>
>
>
> --
> Loraine Guéguen
>
> UMR8227 LBI2M (Laboratory of Integrative Biology of Marine Models)
> Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences)
>
> Station Biologique de Roscoff - CNRS | Sorbonne Université
> Place Georges Teissier - CS 90074
> 29688 Roscoff cedex
> tel: ++33 (0)2 98 29 56 46 (interne : 415)http://www.sb-roscoff.fr/en/laboratory-integrative-biology-marine-modelshttp://abims.sb-roscoff.fr/
>
>
>
>
> This list is for the Apollo Annotation Editing Tool. Info at
> http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with
> which you subscribed to the list, send a message to sy...@li... |
> 2. In the subject line of your email type: unsubscribe apollo | 3. Leave
> the message body blank.
>
>

Re: [Gmod-ajax] [apollo] add new organism

[Loraine G. <lor...@sb...>]

Dear Nathan and Quanwei,

Having a quick look at your posts, I see that you try to load a GTF file 
(and not a GFF file).
See http://jbrowse.org/docs/faq.html#how-do-i-convert-gtf-to-gff

My 2 cents.

Loraine



Le 15/01/2019 16:26, Nathan Dunn a écrit :
>
> It should produce a tracks directory with that label in the directory 
> provided (and added to the trackList.json).
>
> That being said, at a minimum you should have transcript data in your 
> GFF3.  Typically I would use type of mRNA or transcript to process the 
> GFF3.
>
> Also, there is a maker2jbrowse command in JBrowse (I have not tried 
> it), that might simplify this, as well.
>
>
>
> But depending on what you want, you are now more in the JBrowse realm 
> and I am going to cross-post you there.
>
> Might be worth perusing the FAQ and a few other docs:
>
> http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F
> http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html
>
> As well as the tutorial: http://jbrowse.org/docs/tutorial_classic.html
> http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files
>
> Hope this helps,
>
> Nathan
>
>
>> On Jan 15, 2019, at 7:08 AM, Quanwei Zhang <qwz...@gm... 
>> <mailto:qwz...@gm...>> wrote:
>>
>> Dear Nathan:
>>
>> Thanks. Now I can upload some data. But it seems there may be still 
>> some error. Please see the attached figures for "Console" and "Network".
>>
>> Before I "add the new organism", I used the following command to 
>> prepare the data. Then I provide the directory of data.apollo when I 
>> "add the new organism". But I can only see the sequence file, no gene 
>> structure information from GFF. Isn't this something abnormal? Data 
>> under ./data.apollo include "seq trackList.json    tracks        
>> tracks.conf".
>>
>> jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --fasta  
>> ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>> jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> 
>> --indexed_fasta ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>> jbrowse/bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --gff 
>> ./data.input/Aldh1a1.geneName.gtf --type *exon* --trackLabel MAKER 
>> --out ./data.apollo
>>
>>
>> ##the GFF file are in the following format including information for 
>> exons and CDS
>> Contig2667      maker   exon    11929 11954   .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    12150   12251 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    12872   12979 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    13499   13569 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    14822   14955 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    15404   15498 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    21086   21187 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    21808   21915 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    22435   22505 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    23706   23839 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    24286   24380 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    29579   29680 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    30301   30408 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    30928   31036 .       -       .       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     11929   11954 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     12150   12251 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     12872   12979 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     13499   13569 .       -       1       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     14822   14955 .       -       0       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     15404   15498 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     21086   21187 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     21808   21915 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     22435   22505 .       -       1       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     23706   23839 .       -       0       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     24286   24380 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     29579   29680 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     30301   30408 .       -       2       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   CDS     30928   31036 .       -       0       
>> transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
>> Contig2667      maker   exon    154799  154971 .       +       
>> .       transcript_id "UNKNOWN_1066"; gene_id "UNKNOWN_1066"; 
>> gene_name "UNKNOWN_1066";
>>
>>
>> Thanks
>>
>> Best
>> Quanwei
>>
>>
>> Nathan Dunn <nat...@lb... <mailto:nat...@lb...>> 于2019年 
>> 1月14日周一 下午7:53写道：
>>
>>
>>     That’s exactly right.  It looks like its getting unparseable data
>>     (just a guess).  If you click on the tab you can see what the
>>     query is and click on the query preview (and copy and paste the
>>     response, as well) and hopefully we can see what’s wrong.
>>
>>     Nathan
>>
>>
>>     <Edited.png>
>>
>>     <Screen Shot 2019-01-14 at 4.51.18 PM.png><Screen Shot 2019-01-14
>>     at 4.51.22 PM.png>
>>
>>
>>
>>>     On Jan 14, 2019, at 12:40 PM, Quanwei Zhang
>>>     <qwz...@gm... <mailto:qwz...@gm...>> wrote:
>>>
>>>     Dear Nathan:
>>>
>>>     Thank you for your reply.
>>>     (1) In my example, I have only two sequences (with length
>>>     80145943bp,10970156bp) to upload.
>>>     (2) I did not get new information in catalina.out.
>>>     (3) For "javascript console" did you mean what shown in the
>>>     attached file? In this example, I just add the sequence (no gff).
>>>     (4) Would you please explain what do you mean by "network output
>>>     tabs"?
>>>
>>>     Besides the information I gave, do I need to provide information
>>>     for "Search database"?
>>>
>>>     Many thanks
>>>
>>>     Best
>>>     Quanwei
>>>
>>>     Nathan Dunn <nat...@lb... <mailto:nat...@lb...>> 于
>>>     2019年1月14日周一 下午1:15写道：
>>>
>>>
>>>         Looks like everything you are doing SHOULD work.
>>>
>>>         Unless you have a lot of scaffolds (>50K) it shouldn’t take
>>>         longer than a minute.  5K scaffolds takes me roughly 20
>>>         seconds to add, so I agree that something is wrong.
>>>
>>>
>>>         If you don’t see anything produced in your catalina.out (as
>>>         below) can you send me your javascript console and network
>>>         output tabs?
>>>
>>>         I am surprised you aren’t seeing any type of error on the
>>>         backend when you click on “add organism” though.
>>>
>>>
>>>         I think the directory name seems fine assuming you have read
>>>         access to it.
>>>
>>>         Nathan
>>>
>>>
>>>>         On Jan 14, 2019, at 7:51 AM, Quanwei Zhang
>>>>         <qwz...@gm... <mailto:qwz...@gm...>> wrote:
>>>>
>>>>         Hello:
>>>>
>>>>         Thank again for your previous help.
>>>>
>>>>         After I run command "apollo run-local", I can open the web
>>>>         page in web browser.  But when I tried to add a new
>>>>         organism, it always reported "Error" or "Failed" message as
>>>>         shown below in blue.
>>>>
>>>>         The attached file shows the information I put and now it
>>>>         says "processing" for about 20 minutes, but I guess there
>>>>         must be something wrong, because I am just trying to upload
>>>>         2 reads (with length 80145943bp,10970156bp). The attached
>>>>         figure shows the information I input to add the new
>>>>         organism. As shown in it, the data I generated are under
>>>>         "/Users/qwzhang/Documents/dataAnalysis/BeaverNew/data/Apollo/Apollo2/data.apollo"
>>>>         (with  "seq trackList.json tracks tracks.conf"). I used the
>>>>         following commands to generate the files.
>>>>
>>>>         Apollo/jbrowse/bin/prepare-refseqs.pl
>>>>         <http://prepare-refseqs.pl/> --fasta
>>>>         ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>>>         Apollo/jbrowse/bin/prepare-refseqs.pl
>>>>         <http://prepare-refseqs.pl/> --indexed_fasta
>>>>         ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>>>         Apollo/jbrowse/bin/flatfile-to-json.pl
>>>>         <http://flatfile-to-json.pl/> --gff
>>>>         ./data.input/Aldh1a1.geneName.gtf --type *exon*
>>>>         --trackLabel MAKER --out ./data.apollo
>>>>
>>>>
>>>>
>>>>         Metrics servlet injected into web.xml
>>>>         Metrics Admin servlet-mapping (for /metrics/*) injected
>>>>         into web.xml
>>>>         ***
>>>>         | Running Grails application
>>>>
>>>>         Configuring Shiro ...
>>>>
>>>>         Shiro Configured
>>>>         Starting Quartz Scheduler in QuartzFactoryBean
>>>>         | Server running. Browse to http://localhost:8082/apollo
>>>>         | Error 2019-01-14 10:22:25,901 [http-nio-8082-exec-9]
>>>>         ERROR apollo.UserController -
>>>>         org.bbop.apollo.PermissionException: User does not have
>>>>         permission for any organisms.
>>>>         | Error 2019-01-14 10:22:26,174 [http-nio-8082-exec-4]
>>>>         ERROR apollo.UserController - User does not have permission
>>>>         for any organisms.
>>>>         | Error 2019-01-14 10:22:26,209 [http-nio-8082-exec-3]
>>>>         ERROR apollo.AnnotatorService - Error:
>>>>         org.bbop.apollo.PermissionException: User does not have
>>>>         permission for any organisms.
>>>>         | Error 2019-01-14 10:26:34,612 [http-nio-8082-exec-7]
>>>>         ERROR authenticator.UsernamePasswordAuthenticatorService -
>>>>         Problem authenticating: java.lang.IllegalArgumentException:
>>>>         Method argumet (authentication token) cannot be null.
>>>>         2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN
>>>>         apollo.PermissionService - Failed to authenticate user
>>>>         2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN
>>>>         apollo.SecurityFilters - Authentication failed
>>>>
>>>>         Any suggestions. Thanks
>>>>
>>>>         Best
>>>>         Quanwei
>>>>         <addNewOrganism.pdf>
>>>>
>>>>
>>>>         This list is for the Apollo Annotation Editing Tool. Info
>>>>         at http://genomearchitect.org/
>>>>         If you wish to unsubscribe from the Apollo List: 1. From
>>>>         the address with which you subscribed to the list, send a
>>>>         message to sy...@li... <mailto:sy...@li...>
>>>>         | 2. In the subject line of your email type: unsubscribe
>>>>         apollo | 3. Leave the message body blank.
>>>>
>>>
>>>
>>>
>>>
>>>         This list is for the Apollo Annotation Editing Tool. Info at
>>>         http://genomearchitect.org/
>>>         If you wish to unsubscribe from the Apollo List: 1. From the
>>>         address with which you subscribed to the list, send a
>>>         message to sy...@li... <mailto:sy...@li...>
>>>         | 2. In the subject line of your email type: unsubscribe
>>>         apollo | 3. Leave the message body blank.
>>>
>>>     <NewExample.pdf>
>>>
>>>
>>>     This list is for the Apollo Annotation Editing Tool. Info at
>>>     http://genomearchitect.org/
>>>     If you wish to unsubscribe from the Apollo List: 1. From the
>>>     address with which you subscribed to the list, send a message to
>>>     sy...@li... <mailto:sy...@li...> | 2. In the
>>>     subject line of your email type: unsubscribe apollo | 3. Leave
>>>     the message body blank.
>>>
>>
>>
>>
>>
>>     This list is for the Apollo Annotation Editing Tool. Info at
>>     http://genomearchitect.org/
>>     If you wish to unsubscribe from the Apollo List: 1. From the
>>     address with which you subscribed to the list, send a message to
>>     sy...@li... <mailto:sy...@li...> | 2. In the
>>     subject line of your email type: unsubscribe apollo | 3. Leave
>>     the message body blank.
>>
>> <Console.pdf><Network.pdf>
>>
>>
>> This list is for the Apollo Annotation Editing Tool. Info at 
>> http://genomearchitect.org/
>> If you wish to unsubscribe from the Apollo List: 1. From the address 
>> with which you subscribed to the list, send a message to 
>> sy...@li... <mailto:sy...@li...> | 2. In the subject 
>> line of your email type: unsubscribe apollo | 3. Leave the message 
>> body blank.
>>
>
>
>
>
>
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank.
>

-- 
Loraine Guéguen

UMR8227 LBI2M (Laboratory of Integrative Biology of Marine Models)
Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences)

Station Biologique de Roscoff - CNRS | Sorbonne Université
Place Georges Teissier - CS 90074
29688 Roscoff cedex
tel: ++33 (0)2 98 29 56 46 (interne : 415)
http://www.sb-roscoff.fr/en/laboratory-integrative-biology-marine-models
http://abims.sb-roscoff.fr/

Re: [Gmod-ajax] [apollo] add new organism

[Nathan D. <nat...@lb...>]

It should produce a tracks directory with that label in the directory provided (and added to the trackList.json). 

That being said, at a minimum you should have transcript data in your GFF3.  Typically I would use type of mRNA or transcript to process the GFF3. 

Also, there is a maker2jbrowse command in JBrowse (I have not tried it), that might simplify this, as well.   



But depending on what you want, you are now more in the JBrowse realm and I am going to cross-post you there.  

Might be worth perusing the FAQ and a few other docs:

http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F <http://gmod.org/wiki/JBrowse_FAQ#How_do_I_setup_a_GFF_track.3F>
http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html <http://gmod.827538.n3.nabble.com/JBrowse-Support-f815920.html>

As well as the tutorial:  http://jbrowse.org/docs/tutorial_classic.html <http://jbrowse.org/docs/tutorial_classic.html> 
http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files <http://gmod.org/wiki/JBrowse_Tutorial#Features_from_GFF3_or_BED_files>

Hope this helps,

Nathan


> On Jan 15, 2019, at 7:08 AM, Quanwei Zhang <qwz...@gm...> wrote:
> 
> Dear Nathan:
> 
> Thanks. Now I can upload some data. But it seems there may be still some error. Please see the attached figures for "Console" and "Network". 
> 
> Before I "add the new organism", I used the following command to prepare the data. Then I provide the directory of data.apollo when I "add the new organism". But I can only see the sequence file, no gene structure information from GFF. Isn't this something abnormal? Data under ./data.apollo include "seq        trackList.json    tracks        tracks.conf".
> 
> jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --fasta  ./data.input/Aldh1a1.folded.fasta --out ./data.apollo 
> jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --indexed_fasta ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
> jbrowse/bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --gff ./data.input/Aldh1a1.geneName.gtf --type exon --trackLabel MAKER --out ./data.apollo
> 
> 
> ##the GFF file are in the following format including information for exons and CDS
> Contig2667      maker   exon    11929   11954   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    12150   12251   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    12872   12979   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    13499   13569   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    14822   14955   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    15404   15498   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    21086   21187   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    21808   21915   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    22435   22505   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    23706   23839   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    24286   24380   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    29579   29680   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    30301   30408   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    30928   31036   .       -       .       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     11929   11954   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     12150   12251   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     12872   12979   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     13499   13569   .       -       1       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     14822   14955   .       -       0       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     15404   15498   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     21086   21187   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     21808   21915   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     22435   22505   .       -       1       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     23706   23839   .       -       0       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     24286   24380   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     29579   29680   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     30301   30408   .       -       2       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   CDS     30928   31036   .       -       0       transcript_id "VEGP1"; gene_id "VEGP1"; gene_name "VEGP1";
> Contig2667      maker   exon    154799  154971  .       +       .       transcript_id "UNKNOWN_1066"; gene_id "UNKNOWN_1066"; gene_name "UNKNOWN_1066";
> 
> 
> Thanks
> 
> Best
> Quanwei
> 
> 
> Nathan Dunn <nat...@lb... <mailto:nat...@lb...>> 于2019年1月14日周一 下午7:53写道：
> 
> That’s exactly right.  It looks like its getting unparseable data (just a guess).  If you click on the tab you can see what the query is and click on the query preview (and copy and paste the response, as well) and hopefully we can see what’s wrong. 
> 
> Nathan
> 
> 
> <Edited.png>
> 
> <Screen Shot 2019-01-14 at 4.51.18 PM.png><Screen Shot 2019-01-14 at 4.51.22 PM.png>
> 
> 
> 
>> On Jan 14, 2019, at 12:40 PM, Quanwei Zhang <qwz...@gm... <mailto:qwz...@gm...>> wrote:
>> 
>> Dear Nathan:
>> 
>> Thank you for your reply. 
>> (1) In my example, I have only two sequences (with length 80145943bp,10970156bp) to upload. 
>> (2) I did not get new information in catalina.out. 
>> (3) For "javascript console" did you mean what shown in the attached file? In this example, I just add the sequence (no gff).
>> (4) Would you please explain what do you mean by "network output tabs"?
>> 
>> Besides the information I gave, do I need to provide information for "Search database"? 
>> 
>> Many thanks
>> 
>> Best
>> Quanwei
>> 
>> Nathan Dunn <nat...@lb... <mailto:nat...@lb...>> 于2019年1月14日周一 下午1:15写道：
>> 
>> Looks like everything you are doing SHOULD work.  
>> 
>> Unless you have a lot of scaffolds (>50K) it shouldn’t take longer than a minute.  5K scaffolds takes me roughly 20 seconds to add, so I agree that something is wrong. 
>> 
>> 
>> If you don’t see anything produced in your catalina.out (as below) can you send me your javascript console and network output tabs? 
>> 
>> I am surprised you aren’t seeing any type of error on the backend when you click on “add organism” though. 
>> 
>> 
>> I think the directory name seems fine assuming you have read access to it.  
>> 
>> Nathan
>> 
>> 
>>> On Jan 14, 2019, at 7:51 AM, Quanwei Zhang <qwz...@gm... <mailto:qwz...@gm...>> wrote:
>>> 
>>> Hello: 
>>> 
>>> Thank again for your previous help. 
>>> 
>>> After I run command "apollo run-local", I can open the web page in web browser.  But when I tried to add a new organism, it always reported "Error" or "Failed" message as shown below in blue.
>>> 
>>> The attached file shows the information I put and now it says "processing" for about 20 minutes, but I guess there must be something wrong, because I am just trying to upload 2 reads (with length 80145943bp,10970156bp). The attached figure shows the information I input to add the new organism. As shown in it, the data I generated are under "/Users/qwzhang/Documents/dataAnalysis/BeaverNew/data/Apollo/Apollo2/data.apollo" (with  "seq        trackList.json    tracks        tracks.conf"). I used the following commands to generate the files.
>>> 
>>> Apollo/jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --fasta  ./data.input/Aldh1a1.folded.fasta --out ./data.apollo 
>>> Apollo/jbrowse/bin/prepare-refseqs.pl <http://prepare-refseqs.pl/> --indexed_fasta ./data.input/Aldh1a1.folded.fasta --out ./data.apollo
>>> Apollo/jbrowse/bin/flatfile-to-json.pl <http://flatfile-to-json.pl/> --gff ./data.input/Aldh1a1.geneName.gtf --type exon --trackLabel MAKER --out ./data.apollo
>>> 
>>> 
>>> 
>>> Metrics servlet injected into web.xml
>>> Metrics Admin servlet-mapping (for /metrics/*) injected into web.xml
>>> ***
>>> | Running Grails application
>>> 
>>> Configuring Shiro ...
>>> 
>>> Shiro Configured
>>> Starting Quartz Scheduler in QuartzFactoryBean
>>> | Server running. Browse to http://localhost:8082/apollo <http://localhost:8082/apollo>
>>> | Error 2019-01-14 10:22:25,901 [http-nio-8082-exec-9] ERROR apollo.UserController  - org.bbop.apollo.PermissionException: User does not have permission for any organisms.
>>> | Error 2019-01-14 10:22:26,174 [http-nio-8082-exec-4] ERROR apollo.UserController  - User does not have permission for any organisms.
>>> | Error 2019-01-14 10:22:26,209 [http-nio-8082-exec-3] ERROR apollo.AnnotatorService  - Error: org.bbop.apollo.PermissionException: User does not have permission for any organisms.
>>> | Error 2019-01-14 10:26:34,612 [http-nio-8082-exec-7] ERROR authenticator.UsernamePasswordAuthenticatorService  - Problem authenticating: java.lang.IllegalArgumentException: Method argumet (authentication token) cannot be null.
>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN  apollo.PermissionService  - Failed to authenticate user
>>> 2019-01-14 10:26:34,613 [http-nio-8082-exec-7] WARN  apollo.SecurityFilters  - Authentication failed
>>> 
>>> Any suggestions. Thanks
>>> 
>>> Best
>>> Quanwei
>>> <addNewOrganism.pdf>
>>> 
>>> 
>>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>>> 
>> 
>> 
>> 
>> 
>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>> 
>> <NewExample.pdf>
>> 
>> 
>> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
>> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
>> 
> 
> 
> 
> 
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/ <http://genomearchitect.org/>
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... <mailto:sy...@li...> | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
> 
> <Console.pdf><Network.pdf>
> 
> 
> This list is for the Apollo Annotation Editing Tool. Info at http://genomearchitect.org/
> If you wish to unsubscribe from the Apollo List: 1. From the address with which you subscribed to the list, send a message to sy...@li... | 2. In the subject line of your email type: unsubscribe apollo | 3. Leave the message body blank. 
> 

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Robert B. <rb...@gm...>]

Hmm, could that BAM file have been truncated somehow? Perhaps an incomplete
copy onto the web server?

On Mon, Jan 14, 2019 at 8:12 AM Colin <col...@gm...> wrote:

> I checked out this spreadsheet and hopefully i'm interpreting right but it
> looks like there are three different columns representing three different
> files as well as three different rows
>
> In the rows, I think you have essentially
>
> row 1) request for bai file
> row 2) request for bam header starting at byte 0
> row 3) request for slice of bam starting elsewhere in the file
>
> As it looks, the row 3 returns the error in the first two columns. From
> what I can tell, it might be accurate in doing so?
>
> For example, the content-range in the first column is given as bytes
> 0-163168/163169 which indicates the file is actually just 163169 bytes. In
> the next request when it asks for bytes=262144-524287 this is beyond the
> file size. Maybe you can check what the actual file size of that is?
>
>
> -Colin
>
>
>
> On Mon, Jan 14, 2019 at 3:52 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
>> Hey Rob and Colin,
>>
>>
>>
>> Thank you for replying so quickly. You can find the headers enclosed in
>> the Excel file. I have also changed the location of the files to see if it
>> was a “location on bucket” related issue but this didn’t have any effect.
>> Unfortunately, the data is confidential, so I have to verify (with my boss)
>> whether I am allowed to send the file(s) to you. I will try rerunning the
>> analysis to see if this somehow resolves the issue. I thought is might be
>> related to metadata that was perhaps incorrect, but this seems file as
>> well. Is there anything else I can do, which doesn’t include sending you
>> the file, to help you narrow down the cause?
>>
>>
>>
>> Thanks a lot in advance.
>>
>> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>>
>> PS: Part of the Excel file is in Dutch, though I think the relevant
>> information from my Firefox browser is in there.
>>
>>  *Douwe van der Leest*
>>  Bioinformatician
>>  Limgroup
>>
>> ✉
>>
>> dvd...@li...
>>
>> ☏
>>
>> +31 (0)77 397 99 00
>>
>> 🌍
>>
>> www.limgroup.eu
>>
>>
>>
>> *Van:* Robert Buels [mailto:rb...@gm...]
>> *Verzonden:* vrijdag 11 januari 2019 19:14
>> *Aan:* Colin <col...@gm...>
>> *CC:* Douwe van der Leest <dou...@gm...>; gmod-ajax <
>> gmo...@li...>
>> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
>> appearing in newer versions of JBrowse, with same configuration
>>
>>
>>
>> Hi Douwe,
>>
>>
>>
>> Could you look in the dev tools and get the request and response HTTP
>> headers that the browser is sending and receiving? That would help us
>> figure out what's going on here.
>>
>>
>>
>> Rob
>>
>>
>>
>> On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm...> wrote:
>>
>> Both these issues are very weird. I cannot think exactly what might be
>> going on, it might be helpful to have data to test on.
>>
>>
>>
>> The Amazon S3 URLs linked are access denied for me but something like
>> that would be good for testing
>>
>>
>>
>>
>>
>> -Colin
>>
>>
>>
>> On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <
>> dou...@gm...> wrote:
>>
>> Hey there,
>>
>> On advance, thanks for taking the time for reading and perhaps answering
>> my question. I am updating my Jbrowse version from JBrowse-1.12.5; to
>> JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and
>> JBrowse-1.15.0-dev for troubleshooting some issues.
>>
>> I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in #1091 <https://github.com/GMOD/jbrowse/issues/1091> because it works in certain versions of JBrowse, but not in all.
>>
>>
>>
>> [image: jbrowse v1 14 2]
>> <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png>
>> [image: jbrowse v1 15 0]
>> <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png>
>> [image: jbrowse v1 16 1]
>> <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png>
>>
>> Can someone please explain how to solve this?
>>
>> PS: some extra info:
>> The CORS configuration of the bucket in which the data is stored:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>>
>> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
>>
>> <CORSRule>
>>
>>     <AllowedOrigin>*</AllowedOrigin>
>>
>>     <AllowedMethod>GET</AllowedMethod>
>>
>>     <MaxAgeSeconds>3000</MaxAgeSeconds>
>>
>>     <AllowedHeader>*</AllowedHeader>
>>
>> </CORSRule>
>>
>> </CORSConfiguration>
>>
>> I am using apache:
>>
>> > rpm -q httpd
>>
>> httpd-2.4.6-80.el7.centos.1.x86_64
>>
>> The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think
>> is meant for the configuration of my apache server
>>
>> Alias /jbrowse /var/www/jbrowse/default
>>
>> <Directory "/var/www/jbrowse/default/">
>>
>> Options Indexes FollowSymLinks ExecCGI
>>
>> Require all granted
>>
>> AllowOverride All
>>
>>  <IfModule mod_gzip.c>
>>
>>    mod_gzip_item_exclude "(\.jsonz|\.txtz)"
>>
>>  </IfModule>
>>
>>  <IfModule setenvif.c>
>>
>>    SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
>>
>>  </IfModule>
>>
>>  <IfModule mod_headers.c>
>>
>>    <FilesMatch "(\.jsonz|\.txtz)">
>>
>>      Header onsuccess set Content-Encoding gzip
>>
>>    </FilesMatch>
>>
>>  </IfModule>
>>
>> </Directory>
>>
>>
>>
>> Using curl –I on the urls for the bam and the bai file returns the following, respectively:
>>
>> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>>
>> HTTP/1.1 200 OK
>>
>> x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
>>
>> x-amz-request-id: 28B9E9172181C705
>>
>> Date: Fri, 11 Jan 2019 11:36:24 GMT
>>
>> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>>
>> ETag: "02aef1647f185c451ec55cf69731f939"
>>
>> x-amz-meta-mode: 33188
>>
>> x-amz-meta-gid: 0
>>
>> x-amz-meta-uid: 0
>>
>> x-amz-meta-mtime: 1547047128
>>
>> x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
>>
>> Accept-Ranges: bytes
>>
>> Content-Type: application/octet-stream
>>
>> Content-Length: 163169
>>
>> Server: AmazonS3
>>
>>
>>
>> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
>>
>> HTTP/1.1 200 OK
>>
>> x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
>>
>> x-amz-request-id: 9C523CCFAA1B2301
>>
>> Date: Fri, 11 Jan 2019 11:36:27 GMT
>>
>> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>>
>> ETag: "ac2c06762fc282602aa0f1db2d562b19"
>>
>> x-amz-meta-mode: 33188
>>
>> x-amz-meta-gid: 0
>>
>> x-amz-meta-uid: 0
>>
>> x-amz-meta-mtime: 1547047128
>>
>> x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
>>
>> Accept-Ranges: bytes
>>
>> Content-Type: application/octet-stream
>>
>> Content-Length: 502496
>>
>> Server: AmazonS3
>>
>>
>>
>> Please feel free to mail me if you need additional data.
>>
>> Thanks a bunch in advance,
>>
>>
>>
>> Cheers,
>>
>> Douwe
>>
>> _______________________________________________
>> Gmod-ajax mailing list
>> Gmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>>
>> _______________________________________________
>> Gmod-ajax mailing list
>> Gmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>>
>>

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Colin <col...@gm...>]

I checked out this spreadsheet and hopefully i'm interpreting right but it
looks like there are three different columns representing three different
files as well as three different rows

In the rows, I think you have essentially

row 1) request for bai file
row 2) request for bam header starting at byte 0
row 3) request for slice of bam starting elsewhere in the file

As it looks, the row 3 returns the error in the first two columns. From
what I can tell, it might be accurate in doing so?

For example, the content-range in the first column is given as bytes
0-163168/163169 which indicates the file is actually just 163169 bytes. In
the next request when it asks for bytes=262144-524287 this is beyond the
file size. Maybe you can check what the actual file size of that is?


-Colin



On Mon, Jan 14, 2019 at 3:52 AM Douwe van der Leest <
dou...@gm...> wrote:

> Hey Rob and Colin,
>
>
>
> Thank you for replying so quickly. You can find the headers enclosed in
> the Excel file. I have also changed the location of the files to see if it
> was a “location on bucket” related issue but this didn’t have any effect.
> Unfortunately, the data is confidential, so I have to verify (with my boss)
> whether I am allowed to send the file(s) to you. I will try rerunning the
> analysis to see if this somehow resolves the issue. I thought is might be
> related to metadata that was perhaps incorrect, but this seems file as
> well. Is there anything else I can do, which doesn’t include sending you
> the file, to help you narrow down the cause?
>
>
>
> Thanks a lot in advance.
>
> Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,
>
> PS: Part of the Excel file is in Dutch, though I think the relevant
> information from my Firefox browser is in there.
>
>  *Douwe van der Leest*
>  Bioinformatician
>  Limgroup
>
> ✉
>
> dvd...@li...
>
> ☏
>
> +31 (0)77 397 99 00
>
> 🌍
>
> www.limgroup.eu
>
>
>
> *Van:* Robert Buels [mailto:rb...@gm...]
> *Verzonden:* vrijdag 11 januari 2019 19:14
> *Aan:* Colin <col...@gm...>
> *CC:* Douwe van der Leest <dou...@gm...>; gmod-ajax <
> gmo...@li...>
> *Onderwerp:* Re: [Gmod-ajax] 416 Requested range not satisfiable message
> appearing in newer versions of JBrowse, with same configuration
>
>
>
> Hi Douwe,
>
>
>
> Could you look in the dev tools and get the request and response HTTP
> headers that the browser is sending and receiving? That would help us
> figure out what's going on here.
>
>
>
> Rob
>
>
>
> On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm...> wrote:
>
> Both these issues are very weird. I cannot think exactly what might be
> going on, it might be helpful to have data to test on.
>
>
>
> The Amazon S3 URLs linked are access denied for me but something like that
> would be good for testing
>
>
>
>
>
> -Colin
>
>
>
> On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
> Hey there,
>
> On advance, thanks for taking the time for reading and perhaps answering
> my question. I am updating my Jbrowse version from JBrowse-1.12.5; to
> JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and
> JBrowse-1.15.0-dev for troubleshooting some issues.
>
> I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in #1091 <https://github.com/GMOD/jbrowse/issues/1091> because it works in certain versions of JBrowse, but not in all.
>
>
>
> [image: jbrowse v1 14 2]
> <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png>
> [image: jbrowse v1 15 0]
> <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png>
> [image: jbrowse v1 16 1]
> <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png>
>
> Can someone please explain how to solve this?
>
> PS: some extra info:
> The CORS configuration of the bucket in which the data is stored:
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
>
> <CORSRule>
>
>     <AllowedOrigin>*</AllowedOrigin>
>
>     <AllowedMethod>GET</AllowedMethod>
>
>     <MaxAgeSeconds>3000</MaxAgeSeconds>
>
>     <AllowedHeader>*</AllowedHeader>
>
> </CORSRule>
>
> </CORSConfiguration>
>
> I am using apache:
>
> > rpm -q httpd
>
> httpd-2.4.6-80.el7.centos.1.x86_64
>
> The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think
> is meant for the configuration of my apache server
>
> Alias /jbrowse /var/www/jbrowse/default
>
> <Directory "/var/www/jbrowse/default/">
>
> Options Indexes FollowSymLinks ExecCGI
>
> Require all granted
>
> AllowOverride All
>
>  <IfModule mod_gzip.c>
>
>    mod_gzip_item_exclude "(\.jsonz|\.txtz)"
>
>  </IfModule>
>
>  <IfModule setenvif.c>
>
>    SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
>
>  </IfModule>
>
>  <IfModule mod_headers.c>
>
>    <FilesMatch "(\.jsonz|\.txtz)">
>
>      Header onsuccess set Content-Encoding gzip
>
>    </FilesMatch>
>
>  </IfModule>
>
> </Directory>
>
>
>
> Using curl –I on the urls for the bam and the bai file returns the following, respectively:
>
> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>
> HTTP/1.1 200 OK
>
> x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
>
> x-amz-request-id: 28B9E9172181C705
>
> Date: Fri, 11 Jan 2019 11:36:24 GMT
>
> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>
> ETag: "02aef1647f185c451ec55cf69731f939"
>
> x-amz-meta-mode: 33188
>
> x-amz-meta-gid: 0
>
> x-amz-meta-uid: 0
>
> x-amz-meta-mtime: 1547047128
>
> x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
>
> Accept-Ranges: bytes
>
> Content-Type: application/octet-stream
>
> Content-Length: 163169
>
> Server: AmazonS3
>
>
>
> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
>
> HTTP/1.1 200 OK
>
> x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
>
> x-amz-request-id: 9C523CCFAA1B2301
>
> Date: Fri, 11 Jan 2019 11:36:27 GMT
>
> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>
> ETag: "ac2c06762fc282602aa0f1db2d562b19"
>
> x-amz-meta-mode: 33188
>
> x-amz-meta-gid: 0
>
> x-amz-meta-uid: 0
>
> x-amz-meta-mtime: 1547047128
>
> x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
>
> Accept-Ranges: bytes
>
> Content-Type: application/octet-stream
>
> Content-Length: 502496
>
> Server: AmazonS3
>
>
>
> Please feel free to mail me if you need additional data.
>
> Thanks a bunch in advance,
>
>
>
> Cheers,
>
> Douwe
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>

[Gmod-ajax] trackMetadata url: why not relative to data directory?

[Loraine G. <lor...@sb...>]

Hi,

I wonder why the URL for metadata sources is relative to the JBrowse 
directory and not to the directory where the configuration file 
(trackList.json) is located, as is the case for all other url parameters 
in trackList.json (urlTemplate) ? Is there a reason for that ? Maybe 
because one single metadata source was intended to be used for all data 
repositories ?

I have one different trackMetadata file for each of my ~100 data 
repositories (~100 strains) which I would like to put into the related 
data repository (next to trackList.json), and I do not find it wise for 
maintenance reason to have a piece of absolute path (between the JBrowse 
and the data repository) in my trackList.json.

I guess I can use symbolic links to avoid using the path into the 
trackList.json, but whatever, I will have to manage this path somewhere.


Extract from 
http://jbrowse.org/docs/track_metadata.html#track-metadata-options: 
"-url - a URL (relative to JBrowse root directory, or absolute URL)"

TrackList.json configuration exemple :

{
  "trackMetadata": {
      "sources": [
           { "type": "csv", "url": 
"data/private/species/this_particular_strain.csv" }
      ]
},
    "tracks" : [
       {
          "category" : "Reference sequence",
          "chunkSize" : 20000,
          "label" : "DNA",
          "seqType" : "dna",
          "storeClass" : "JBrowse/Store/Sequence/StaticChunked",
          "type" : "SequenceTrack",
          "urlTemplate" : "seq/{refseq_dirpath}/{refseq}-"
       },
       {
          "compress" : 0,
          "label" : "this_strain.gff",
          "storeClass" : "JBrowse/Store/SeqFeature/NCList",
          "style" : {
             "className" : "feature"
          },
          "trackType" : "CanvasFeatures",
          "type" : "CanvasFeatures",
          "urlTemplate" : "tracks/this_strain.gff/{refseq}/trackData.json"
       },
       {
          "autoscale" : "local",
          "bicolor_pivot" : "zero",
          "label" : "bwFilename.bw",
          "storeClass" : "JBrowse/Store/SeqFeature/BigWig",
          "type" : "JBrowse/View/Track/Wiggle/XYPlot",
          "urlTemplate" : 
"flat_data/RNAseq/coverage/RNAseq_this_strain_conditionX.bw"
       }
]
}



I would like to be able to write:

"trackMetadata": {
      "sources": [
           { "type": "csv", "url":  "this_particular_strain.csv" }
      ]
},


Thanks,

Loraine Guéguen

-- 
Loraine Guéguen

UMR8227 LBI2M (Laboratory of Integrative Biology of Marine Models)
Plateforme ABiMS (Analyses and Bioinformatics for Marine Sciences)

Station Biologique de Roscoff - CNRS | Sorbonne Université
Place Georges Teissier - CS 90074
29688 Roscoff cedex
tel: ++33 (0)2 98 29 56 46 (interne : 415)
http://www.sb-roscoff.fr/en/laboratory-integrative-biology-marine-models
http://abims.sb-roscoff.fr/

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Douwe v. d. L. <dou...@gm...>]

Hey Rob and Colin,

 

Thank you for replying so quickly. You can find the headers enclosed in the Excel file. I have also changed the location of the files to see if it was a “location on bucket” related issue but this didn’t have any effect. Unfortunately, the data is confidential, so I have to verify (with my boss) whether I am allowed to send the file(s) to you. I will try rerunning the analysis to see if this somehow resolves the issue. I thought is might be related to metadata that was perhaps incorrect, but this seems file as well. Is there anything else I can do, which doesn’t include sending you the file, to help you narrow down the cause?

 

Thanks a lot in advance.

Met vriendelijke groet, / Kind regards, / Mit freundlichen Grüßen,

PS: Part of the Excel file is in Dutch, though I think the relevant information from my Firefox browser is in there.


 Douwe van der Leest 
 Bioinformatician
 Limgroup


✉ 

dvd...@li...


☏  

+31 (0)77 397 99 00


🌍 

www.limgroup.eu

 

Van: Robert Buels [mailto:rb...@gm...] 
Verzonden: vrijdag 11 januari 2019 19:14
Aan: Colin <col...@gm...>
CC: Douwe van der Leest <dou...@gm...>; gmod-ajax <gmo...@li...>
Onderwerp: Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

 

Hi Douwe,

 

Could you look in the dev tools and get the request and response HTTP headers that the browser is sending and receiving? That would help us figure out what's going on here.

 

Rob

 

On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm... <mailto:col...@gm...> > wrote:

Both these issues are very weird. I cannot think exactly what might be going on, it might be helpful to have data to test on.

 

The Amazon S3 URLs linked are access denied for me but something like that would be good for testing

 

 

-Colin

 

On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <dou...@gm... <mailto:dou...@gm...> > wrote:

Hey there,

On advance, thanks for taking the time for reading and perhaps answering my question. I am updating my Jbrowse version from JBrowse-1.12.5; to JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and JBrowse-1.15.0-dev for troubleshooting some issues.

I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in  <https://github.com/GMOD/jbrowse/issues/1091> #1091 because it works in certain versions of JBrowse, but not in all.

 

 <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png> 
 <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png> 
 <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png> 

Can someone please explain how to solve this?

PS: some extra info:
The CORS configuration of the bucket in which the data is stored:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

I am using apache:

> rpm -q httpd
httpd-2.4.6-80.el7.centos.1.x86_64

The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think is meant for the configuration of my apache server

Alias /jbrowse /var/www/jbrowse/default
<Directory "/var/www/jbrowse/default/">
Options Indexes FollowSymLinks ExecCGI
Require all granted
AllowOverride All
 <IfModule mod_gzip.c>
   mod_gzip_item_exclude "(\.jsonz|\.txtz)"
 </IfModule>
 <IfModule setenvif.c>
   SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
 </IfModule>
 <IfModule mod_headers.c>
   <FilesMatch "(\.jsonz|\.txtz)">
     Header onsuccess set Content-Encoding gzip
   </FilesMatch>
 </IfModule>
</Directory>
 
Using curl –I on the urls for the bam and the bai file returns the following, respectively:
[root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
HTTP/1.1 200 OK
x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
x-amz-request-id: 28B9E9172181C705
Date: Fri, 11 Jan 2019 11:36:24 GMT
Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
ETag: "02aef1647f185c451ec55cf69731f939"
x-amz-meta-mode: 33188
x-amz-meta-gid: 0
x-amz-meta-uid: 0
x-amz-meta-mtime: 1547047128
x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 163169
Server: AmazonS3
 
[root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
HTTP/1.1 200 OK
x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
x-amz-request-id: 9C523CCFAA1B2301
Date: Fri, 11 Jan 2019 11:36:27 GMT
Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
ETag: "ac2c06762fc282602aa0f1db2d562b19"
x-amz-meta-mode: 33188
x-amz-meta-gid: 0
x-amz-meta-uid: 0
x-amz-meta-mtime: 1547047128
x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 502496
Server: AmazonS3
 

Please feel free to mail me if you need additional data.

Thanks a bunch in advance,

 

Cheers,

Douwe

_______________________________________________
Gmod-ajax mailing list
Gmo...@li... <mailto:Gmo...@li...> 
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

_______________________________________________
Gmod-ajax mailing list
Gmo...@li... <mailto:Gmo...@li...> 
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

Re: [Gmod-ajax] some trouble on MacOS

[Colin <col...@gm...>]

Hi there

It appears that three perl packages are having trouble being installed,
specifically Devel::Size, Heap::Simple::XS, JSON::XS

The XS modules in particular imply compiling C interface files

Therefore you might need to install command line development tools for Mac
e.g. https://developer.apple.com/xcode/ since the XS modules require
development tools

Let me know if that helps!

-Colin

On Fri, Jan 11, 2019 at 8:32 AM 黄玮 via Gmod-ajax <
gmo...@li...> wrote:

> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>

[Gmod-ajax] jbrowse rangeerror: trying to access beyond buffer length

[Jhon T. <jht...@ce...>]

Hi,

I got the following error whit a bam file:

jbrowse rangeerror: trying to access beyond buffer length

Re: [Gmod-ajax] 416 Requested range not satisfiable message appearing in newer versions of JBrowse, with same configuration

[Robert B. <rb...@gm...>]

Hi Douwe,

Could you look in the dev tools and get the request and response HTTP
headers that the browser is sending and receiving? That would help us
figure out what's going on here.

Rob

On Fri, Jan 11, 2019 at 9:19 AM Colin <col...@gm...> wrote:

> Both these issues are very weird. I cannot think exactly what might be
> going on, it might be helpful to have data to test on.
>
> The Amazon S3 URLs linked are access denied for me but something like that
> would be good for testing
>
>
> -Colin
>
> On Fri, Jan 11, 2019 at 8:32 AM Douwe van der Leest <
> dou...@gm...> wrote:
>
>> Hey there,
>>
>> On advance, thanks for taking the time for reading and perhaps answering
>> my question. I am updating my Jbrowse version from JBrowse-1.12.5; to
>> JBrowse-1.16.1-release with as intermediate versions JBrowse-1.14.2-dev and
>> JBrowse-1.15.0-dev for troubleshooting some issues.
>>
>> I use Firefox for viewing my data and am looking at two specific bam files that for some reason provide an error starting from version 1.15.0. Also, I noticed the peaks of some related files changing with the version number (all included in the screenshots below), should I be concerned about this difference in displaying the exact same files? I am not sure whether it is related to CORS configuration as proposed in #1091 <https://github.com/GMOD/jbrowse/issues/1091> because it works in certain versions of JBrowse, but not in all.
>>
>>
>>
>> [image: jbrowse v1 14 2]
>> <https://user-images.githubusercontent.com/38953127/51028366-80fdc000-1593-11e9-8c33-75fefeb3276f.png>
>> [image: jbrowse v1 15 0]
>> <https://user-images.githubusercontent.com/38953127/51028367-80fdc000-1593-11e9-9ccc-967804a3ed79.png>
>> [image: jbrowse v1 16 1]
>> <https://user-images.githubusercontent.com/38953127/51028368-80fdc000-1593-11e9-8517-82cde00315a3.png>
>>
>> Can someone please explain how to solve this?
>>
>> PS: some extra info:
>> The CORS configuration of the bucket in which the data is stored:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>>
>> <CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
>>
>> <CORSRule>
>>
>>     <AllowedOrigin>*</AllowedOrigin>
>>
>>     <AllowedMethod>GET</AllowedMethod>
>>
>>     <MaxAgeSeconds>3000</MaxAgeSeconds>
>>
>>     <AllowedHeader>*</AllowedHeader>
>>
>> </CORSRule>
>>
>> </CORSConfiguration>
>>
>> I am using apache:
>>
>> > rpm -q httpd
>>
>> httpd-2.4.6-80.el7.centos.1.x86_64
>>
>> The code in /etc/httpd/conf/httpd.conf, related to JBrowse; which I think
>> is meant for the configuration of my apache server
>>
>> Alias /jbrowse /var/www/jbrowse/default
>>
>> <Directory "/var/www/jbrowse/default/">
>>
>> Options Indexes FollowSymLinks ExecCGI
>>
>> Require all granted
>>
>> AllowOverride All
>>
>>  <IfModule mod_gzip.c>
>>
>>    mod_gzip_item_exclude "(\.jsonz|\.txtz)"
>>
>>  </IfModule>
>>
>>  <IfModule setenvif.c>
>>
>>    SetEnvIf Request_URI "(\.jsonz|\.txtz)" no-gzip dont-vary
>>
>>  </IfModule>
>>
>>  <IfModule mod_headers.c>
>>
>>    <FilesMatch "(\.jsonz|\.txtz)">
>>
>>      Header onsuccess set Content-Encoding gzip
>>
>>    </FilesMatch>
>>
>>  </IfModule>
>>
>> </Directory>
>>
>>
>>
>> Using curl –I on the urls for the bam and the bai file returns the following, respectively:
>>
>> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam
>>
>> HTTP/1.1 200 OK
>>
>> x-amz-id-2: 1JbjscRo+vjcUIXpIs+zFdwk9WxLS8mveBViQM+gJlrVubz4KTYywfdLx6LAh7cEw1PETP2Cn1U=
>>
>> x-amz-request-id: 28B9E9172181C705
>>
>> Date: Fri, 11 Jan 2019 11:36:24 GMT
>>
>> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>>
>> ETag: "02aef1647f185c451ec55cf69731f939"
>>
>> x-amz-meta-mode: 33188
>>
>> x-amz-meta-gid: 0
>>
>> x-amz-meta-uid: 0
>>
>> x-amz-meta-mtime: 1547047128
>>
>> x-amz-version-id: n96JSHBApAMOClTPUpJCOkoBJ5X_2DEe
>>
>> Accept-Ranges: bytes
>>
>> Content-Type: application/octet-stream
>>
>> Content-Length: 163169
>>
>> Server: AmazonS3
>>
>>
>>
>> [root@cloud jbrowse]# curl -I https://s3.eu-central-1.amazonaws.com/lim-asp-data/PacBio/WGS/K30-1/AsparagusMaleV1/bam/K30-1.PacBio_Ref_run1.contigs.fasta.sorted.minimap2.sorted.bam.bai
>>
>> HTTP/1.1 200 OK
>>
>> x-amz-id-2: kcUF8e4oClVu6INZ78npWcPgNLO3UJP3KQKsDCUDWHhsCjNfC5r4XB9IbB93LYdOwQLXRLi8EgU=
>>
>> x-amz-request-id: 9C523CCFAA1B2301
>>
>> Date: Fri, 11 Jan 2019 11:36:27 GMT
>>
>> Last-Modified: Wed, 09 Jan 2019 15:18:49 GMT
>>
>> ETag: "ac2c06762fc282602aa0f1db2d562b19"
>>
>> x-amz-meta-mode: 33188
>>
>> x-amz-meta-gid: 0
>>
>> x-amz-meta-uid: 0
>>
>> x-amz-meta-mtime: 1547047128
>>
>> x-amz-version-id: pD4RMlnV1r8qW7Q9cnwSRhaZW.dMI6dA
>>
>> Accept-Ranges: bytes
>>
>> Content-Type: application/octet-stream
>>
>> Content-Length: 502496
>>
>> Server: AmazonS3
>>
>>
>>
>> Please feel free to mail me if you need additional data.
>>
>> Thanks a bunch in advance,
>>
>>
>>
>> Cheers,
>>
>> Douwe
>> _______________________________________________
>> Gmod-ajax mailing list
>> Gmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>>
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>