Re: [Gmod-ajax] JBrowse remote files with CORS config

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi Vaneet,

I suspect the problem is with this:

         "storeClass" : "JBrowse/Store/SeqFeature/NCList",
         "urlTemplate" : "
https://ftp.xenbase.org/pub/Genomics/JGI/Xenla9.2/XENLA_9.2_Xenbase_longest.gff3
",

You give a url to a GFF file, but you specified the storeClass as NCList,
so the urlTemplate should look something like this:

   https://ftp.xenbase.org/pub/Genomics/JGI/Xenla9.2/
<https://ftp.xenbase.org/pub/Genomics/JGI/Xenla9.2/XENLA_9.2_Xenbase_longest.gff3>
{refseq}/trackData.json

So, the message about json and M is because you can't have an "M" in the
first character of a json file.  You either need to bgzip/tabix index this
GFF file and change the urlTemplate to put at that bgzip file, or use
flatfile_to_json.pl to convert the GFF file to NCList, which will give you
a set of directories with the trackData.json at the top of it and modify
the urlTemplate to point at it.

Scott

On Mon, May 2, 2022 at 3:08 PM Vaneet Lotay <van...@uc...>
wrote:

> Hey Scott,
>
> Sorry sending the trackList would be more helpful of course.
> Here's the stanza for the track I'm trying to load remotely:
>
>       {
>          "style" : {
>             "className" : "feature"
>          },
>          "menuTemplate" : [
>             {},
>             {},
>             {
>                "iconClass" : "dijitIconBookmark",
>                "content" : "function(track,feature){return
> track.browser.plugins.FeatureSequence.callFxn(track, feature); }",
>                "action" : "contentDialog",
>                "label" : "View Feature Sequence"
>             }
>          ],
>          "key" : "Xenopus laevis 9.2-Xenbase (longest)",
>          "impliedUTRs" : true,
>          "trackType" : "CanvasFeatures",
>          "storeClass" : "JBrowse/Store/SeqFeature/NCList",
>          "urlTemplate" : "
> https://ftp.xenbase.org/pub/Genomics/JGI/Xenla9.2/XENLA_9.2_Xenbase_longest.gff3
> ",
>          "compress" : 0,
>          "type" : "CanvasFeatures",
>          "category" : "Gene models",
>          "label" : "XL9_2_Xenbase_long"
>       },
>
> I can certainly send you a simplified complete JSON file after if that's
> better however I think I found a more helpful clue.  It doesn't seem to be
> a problem with my trackList JSON file, it's more with the file it's
> fetching.  The reason I think so is the syntax error it complains about is
> always the first character of the remote GFF3 file.  This syntax error
> complains about token 'M' since the file starts with the sequence ID 'MT'
> for mitochondrial.  If I tried another GFF3, it complains about token '#'
> since many GFF3 files like the second one I was trying start with commented
> lines in the header of the file to describe the GFF type and when the file
> was produced.
>
> Do you know why it would have trouble parsing the file like this?
> Otherwise if I'm on the wrong track I'll find a way to send you the
> complete trackList file, but it seems like it's not the source JSON that's
> the issue since if I change that one line back to a local URL, there are no
> errors.
>
> Thanks,
>
> Vaneet
> ------------------------------
> *From:* Scott Cain <sc...@sc...>
> *Sent:* May 2, 2022 3:28 PM
> *To:* Vaneet Lotay <van...@uc...>
> *Cc:* gmo...@li... <gmo...@li...>
> *Subject:* Re: [Gmod-ajax] JBrowse remote files with CORS config
>
> [△EXTERNAL]
>
> Hi Vaneet,
>
> It's a little hard to debug a syntax error without seeing the syntax :-)
>
> If you could share your trackList.json or even better, a simplified
> trackList.json that reproduces the error that would be great. It seems like
> the most likely cause is a typo.
>
> Scott
>
>
> On Mon, May 2, 2022 at 1:26 PM Vaneet Lotay <van...@uc...>
> wrote:
>
> Hey Scott,
>
> Thanks for the help.  Seems like I forgot to restart the apache server to
> make sure the changes had been compiled. I also made sure the headers
> module was loaded in apache.
>
> So now it seems that it doesn't work when I try and load the remote track
> in my trackList.json file but when I try to add it as a custom track via
> the 'open track file or URL' menu option in JBrowse and choose remote file
> then it works fine after loading.  Do you know why that would occur?
> Here's the error I get when I try loading the remote GFF3 file in my
> trackList.json:
>
> at XMLHttpRequest.onLoad
> (webpack:///./node_modules/dojo/request/xhr.js?:126:9) SyntaxError:
> Unexpected token M in JSON at position 0
>
> All I did was replace the urlTemplate field in my JSON file with the https
> address for my remote server, was there any additional configuration steps
> needed?
>
> Thanks,
>
> Vaneet
> ------------------------------
> *From:* Scott Cain <sc...@sc...>
> *Sent:* April 28, 2022 5:53 PM
> *To:* Vaneet Lotay <van...@uc...>
> *Cc:* gmo...@li... <gmo...@li...>
> *Subject:* Re: [Gmod-ajax] JBrowse remote files with CORS config
>
> [△EXTERNAL]
>
> Hi Vaneet,
>
> A few years ago I played with CORS and apache, and I used this, which I
> suspect worked, but I honestly don't remember:
>
> #CORS support (check with Todd if this is OK)
> <IfModule mod_headers.c>
>    SetEnvIf Origin ^(https?://.+\.wormbase.org(?::\d{1,5})?)$
> CORS_ALLOW_ORIGIN=$1
>    Header append Access-Control-Allow-Origin  %{CORS_ALLOW_ORIGIN}e
> env=CORS_ALLOW_ORIGIN
>    Header onsuccess set Access-Control-Allow-Headers X-Requested-With,Range
>    Header merge  Vary "Origin"
> </IfModule>
>
> I feel like there is extra junk in here, but certainly the Header lines
> are probably needed.
>
> Scott
>
>
> On Thu, Apr 28, 2022 at 3:00 PM Vaneet Lotay <van...@uc...>
> wrote:
>
> Hey Scott,
>
> Sorry I should have specified, I'm trying to test this in an Apache
> virtual machine first and then if it works I will try in an AWS bucket
> perhaps as we're toying with the idea of migrating some of our servers on
> the cloud.  However, this info is very useful when I get to that step,
> thank you!
>
> Do you know how this setup would work for Apache?  I was attempting to add
> the necessary configuration files to the httpd.conf file, does that makes
> sense?  I'm still getting an error in the browser:
>
> ...... has been blocked by CORS policy: No 'Access-Control-Allow-Origin'
> header is present on the requested resource. If an opaque response serves
> your needs, set the request's mode to 'no-cors' to fetch the resource with
> CORS disabled.
>
> I did add an Access-Control-Allow-Origin configuration line with '*' to
> allow all origins but perhaps I didn't write it correctly.
>
> Thanks,
>
> Vaneet
> ------------------------------
> *From:* Scott Cain <sc...@sc...>
> *Sent:* April 28, 2022 3:45 PM
> *To:* Vaneet Lotay <van...@uc...>
> *Cc:* gmo...@li... <gmo...@li...>
> *Subject:* Re: [Gmod-ajax] JBrowse remote files with CORS config
>
> [△EXTERNAL]
>
> Hi Vaneet,
>
> I remember seeing this email a few weeks ago but forgot to answer; thanks
> for giving me a poke.
>
> There is nothing that needs to be done for anything on the server that is
> serving up JBrowse, other than making sure you have absolute URLs instead
> of the more typical relative urls that point at a "local" (to the JBrowse
> installation) data directory.
>
> For setting up CORS for S3, I know how to do that via the aws command line
> tool.  Essentially, you can run `aws s3api get-bucket-cors --bucket <name>`
> which will tell you what the current status of CORS is for the bucket, and
> then `aws s3api put-bucket-cors --bucket <name> --cors-configuration
> <json.file>` where the json file will look something like this (which is
> the one I use for one of our WormBase buckets):
>
> {
>     "CORSRules": [
>         {
>             "AllowedHeaders": [
>                 "*"
>             ],
>             "AllowedMethods": [
>                 "GET"
>             ],
>             "AllowedOrigins": [
>                 "*"
>             ],
>             "ExposeHeaders": [
>                 "Accept-Ranges",
>                 "Content-Range",
>                 "Content-Encoding",
>                 "Content-Length"
>             ],
>             "MaxAgeSeconds": 3000
>         }
>     ]
> }
>
> The other item is that the data in the bucket has to be public readable,
> which means that when you upload the data, you should add `--acl
> public-read` to the command line `aws s3 cp` command, and I assume there is
> a similar thing you can do with the web interface, but I don't know how to
> do that.
>
> Please let me know if this makes sense or if you need any more help,
> Scott
>
>
> On Thu, Apr 14, 2022 at 9:14 AM Vaneet Lotay <van...@uc...>
> wrote:
>
> Hi all,
>
> I'm just wondering if I wanted to load tracks from a remote URL,
> specifically an HTTPS address, I understand from the documentation that
> CORS configuration must be setup on the remote server that I'm accessing
> let's say the bigWig tracks from.  What exactly is involved in that setup,
> do you simply add the CORS configuration file to the remote server?  If
> you're working with apache do you simply add it do it's configuration
> folder?  Are there any other steps involved or is it that simple?  I'm
> guessing putting a wildcard (*) in the Allowed Origin field allows any
> server to fetch files from the remote server, but I suppose you can
> specifiy a specific IP address too?
>
> I did see an example given for configuration related to an Amazon S3
> bucket, that would be relevant to me also as I'm contemplating this type of
> setup as well to store my tracks.
>
> Thanks,
>
> Vaneet
> _______________________________________________
> Gmod-ajax mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax
>
>
>
> --
> ------------------------------------------------------------------------
> Scott Cain, Ph. D.                                                  scott
> at scottcain dot net
> GMOD Project Manager (http://gmod.org/)
> 216-392-3087
> WormBase Developer (http://wormbase.org/)
> Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
> VirusSeq Project Manager (https://virusseq-dataportal.ca/)
> Human Cancer Models Initiative Project Manager (
> https://hcmi-searchable-catalog.nci.nih.gov/)
>
>
>
> --
> ------------------------------------------------------------------------
> Scott Cain, Ph. D.                                                  scott
> at scottcain dot net
> GMOD Project Manager (http://gmod.org/)
> 216-392-3087
> WormBase Developer (http://wormbase.org/)
> Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
> VirusSeq Project Manager (https://virusseq-dataportal.ca/)
> Human Cancer Models Initiative Project Manager (
> https://hcmi-searchable-catalog.nci.nih.gov/)
>
>
>
> --
> ------------------------------------------------------------------------
> Scott Cain, Ph. D.                                                  scott
> at scottcain dot net
> GMOD Project Manager (http://gmod.org/)
> 216-392-3087
> WormBase Developer (http://wormbase.org/)
> Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
> VirusSeq Project Manager (https://virusseq-dataportal.ca/)
> Human Cancer Models Initiative Project Manager (
> https://hcmi-searchable-catalog.nci.nih.gov/)
>

-- 
------------------------------------------------------------------------
Scott Cain, Ph. D.                                                  scott
at scottcain dot net
GMOD Project Manager (http://gmod.org/)
216-392-3087
WormBase Developer (http://wormbase.org/)
Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
VirusSeq Project Manager (https://virusseq-dataportal.ca/)
Human Cancer Models Initiative Project Manager (
https://hcmi-searchable-catalog.nci.nih.gov/)