Re: [Gmod-ajax] JBrowse remote files with CORS config

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hey Scott,

Sorry sending the trackList would be more helpful of course.
Here's the stanza for the track I'm trying to load remotely:

      {
         "style" : {
            "className" : "feature"
         },
         "menuTemplate" : [
            {},
            {},
            {
               "iconClass" : "dijitIconBookmark",
               "content" : "function(track,feature){return track.browser.plugins.FeatureSequence.callFxn(track, feature); }",
               "action" : "contentDialog",
               "label" : "View Feature Sequence"
            }
         ],
         "key" : "Xenopus laevis 9.2-Xenbase (longest)",
         "impliedUTRs" : true,
         "trackType" : "CanvasFeatures",
         "storeClass" : "JBrowse/Store/SeqFeature/NCList",
         "urlTemplate" : "https://ftp.xenbase.org/pub/Genomics/JGI/Xenla9.2/XENLA_9.2_Xenbase_longest.gff3",
         "compress" : 0,
         "type" : "CanvasFeatures",
         "category" : "Gene models",
         "label" : "XL9_2_Xenbase_long"
      },

I can certainly send you a simplified complete JSON file after if that's better however I think I found a more helpful clue.  It doesn't seem to be a problem with my trackList JSON file, it's more with the file it's fetching.  The reason I think so is the syntax error it complains about is always the first character of the remote GFF3 file.  This syntax error complains about token 'M' since the file starts with the sequence ID 'MT' for mitochondrial.  If I tried another GFF3, it complains about token '#' since many GFF3 files like the second one I was trying start with commented lines in the header of the file to describe the GFF type and when the file was produced.

Do you know why it would have trouble parsing the file like this?  Otherwise if I'm on the wrong track I'll find a way to send you the complete trackList file, but it seems like it's not the source JSON that's the issue since if I change that one line back to a local URL, there are no errors.

Thanks,

Vaneet
________________________________
From: Scott Cain <sc...@sc...>
Sent: May 2, 2022 3:28 PM
To: Vaneet Lotay <van...@uc...>
Cc: gmo...@li... <gmo...@li...>
Subject: Re: [Gmod-ajax] JBrowse remote files with CORS config

[△EXTERNAL]

Hi Vaneet,

It's a little hard to debug a syntax error without seeing the syntax :-)

If you could share your trackList.json or even better, a simplified trackList.json that reproduces the error that would be great. It seems like the most likely cause is a typo.

Scott

On Mon, May 2, 2022 at 1:26 PM Vaneet Lotay <van...@uc...<mailto:van...@uc...>> wrote:
Hey Scott,

Thanks for the help.  Seems like I forgot to restart the apache server to make sure the changes had been compiled. I also made sure the headers module was loaded in apache.

So now it seems that it doesn't work when I try and load the remote track in my trackList.json file but when I try to add it as a custom track via the 'open track file or URL' menu option in JBrowse and choose remote file then it works fine after loading.  Do you know why that would occur?  Here's the error I get when I try loading the remote GFF3 file in my trackList.json:

at XMLHttpRequest.onLoad (webpack:///./node_modules/dojo/request/xhr.js?:126:9) SyntaxError: Unexpected token M in JSON at position 0

All I did was replace the urlTemplate field in my JSON file with the https address for my remote server, was there any additional configuration steps needed?

Thanks,

Vaneet
________________________________
From: Scott Cain <sc...@sc...<mailto:sc...@sc...>>
Sent: April 28, 2022 5:53 PM
To: Vaneet Lotay <van...@uc...<mailto:van...@uc...>>
Cc: gmo...@li...<mailto:gmo...@li...> <gmo...@li...<mailto:gmo...@li...>>
Subject: Re: [Gmod-ajax] JBrowse remote files with CORS config

[△EXTERNAL]

Hi Vaneet,

A few years ago I played with CORS and apache, and I used this, which I suspect worked, but I honestly don't remember:

#CORS support (check with Todd if this is OK)
<IfModule mod_headers.c>
   SetEnvIf Origin ^(https?://.+\.wormbase.org<http://wormbase.org>(?::\d{1,5})?)$   CORS_ALLOW_ORIGIN=$1
   Header append Access-Control-Allow-Origin  %{CORS_ALLOW_ORIGIN}e   env=CORS_ALLOW_ORIGIN
   Header onsuccess set Access-Control-Allow-Headers X-Requested-With,Range
   Header merge  Vary "Origin"
</IfModule>

I feel like there is extra junk in here, but certainly the Header lines are probably needed.

Scott

On Thu, Apr 28, 2022 at 3:00 PM Vaneet Lotay <van...@uc...<mailto:van...@uc...>> wrote:
Hey Scott,

Sorry I should have specified, I'm trying to test this in an Apache virtual machine first and then if it works I will try in an AWS bucket perhaps as we're toying with the idea of migrating some of our servers on the cloud.  However, this info is very useful when I get to that step, thank you!

Do you know how this setup would work for Apache?  I was attempting to add the necessary configuration files to the httpd.conf file, does that makes sense?  I'm still getting an error in the browser:

...... has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

I did add an Access-Control-Allow-Origin configuration line with '*' to allow all origins but perhaps I didn't write it correctly.

Thanks,

Vaneet
________________________________
From: Scott Cain <sc...@sc...<mailto:sc...@sc...>>
Sent: April 28, 2022 3:45 PM
To: Vaneet Lotay <van...@uc...<mailto:van...@uc...>>
Cc: gmo...@li...<mailto:gmo...@li...> <gmo...@li...<mailto:gmo...@li...>>
Subject: Re: [Gmod-ajax] JBrowse remote files with CORS config

[△EXTERNAL]

Hi Vaneet,

I remember seeing this email a few weeks ago but forgot to answer; thanks for giving me a poke.

There is nothing that needs to be done for anything on the server that is serving up JBrowse, other than making sure you have absolute URLs instead of the more typical relative urls that point at a "local" (to the JBrowse installation) data directory.

For setting up CORS for S3, I know how to do that via the aws command line tool.  Essentially, you can run `aws s3api get-bucket-cors --bucket <name>` which will tell you what the current status of CORS is for the bucket, and then `aws s3api put-bucket-cors --bucket <name> --cors-configuration <json.file>` where the json file will look something like this (which is the one I use for one of our WormBase buckets):

{
    "CORSRules": [
        {
            "AllowedHeaders": [
                "*"
            ],
            "AllowedMethods": [
                "GET"
            ],
            "AllowedOrigins": [
                "*"
            ],
            "ExposeHeaders": [
                "Accept-Ranges",
                "Content-Range",
                "Content-Encoding",
                "Content-Length"
            ],
            "MaxAgeSeconds": 3000
        }
    ]
}

The other item is that the data in the bucket has to be public readable, which means that when you upload the data, you should add `--acl public-read` to the command line `aws s3 cp` command, and I assume there is a similar thing you can do with the web interface, but I don't know how to do that.

Please let me know if this makes sense or if you need any more help,
Scott

On Thu, Apr 14, 2022 at 9:14 AM Vaneet Lotay <van...@uc...<mailto:van...@uc...>> wrote:
Hi all,

I'm just wondering if I wanted to load tracks from a remote URL, specifically an HTTPS address, I understand from the documentation that CORS configuration must be setup on the remote server that I'm accessing let's say the bigWig tracks from.  What exactly is involved in that setup, do you simply add the CORS configuration file to the remote server?  If you're working with apache do you simply add it do it's configuration folder?  Are there any other steps involved or is it that simple?  I'm guessing putting a wildcard (*) in the Allowed Origin field allows any server to fetch files from the remote server, but I suppose you can specifiy a specific IP address too?

I did see an example given for configuration related to an Amazon S3 bucket, that would be relevant to me also as I'm contemplating this type of setup as well to store my tracks.

Thanks,

Vaneet
_______________________________________________
Gmod-ajax mailing list
Gmo...@li...<mailto:Gmo...@li...>
https://lists.sourceforge.net/lists/listinfo/gmod-ajax

--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                                  scott at scottcain dot net
GMOD Project Manager (http://gmod.org/)                            216-392-3087
WormBase Developer (http://wormbase.org/)
Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
VirusSeq Project Manager (https://virusseq-dataportal.ca/)
Human Cancer Models Initiative Project Manager (https://hcmi-searchable-catalog.nci.nih.gov/)

--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                                  scott at scottcain dot net
GMOD Project Manager (http://gmod.org/)                            216-392-3087
WormBase Developer (http://wormbase.org/)
Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
VirusSeq Project Manager (https://virusseq-dataportal.ca/)
Human Cancer Models Initiative Project Manager (https://hcmi-searchable-catalog.nci.nih.gov/)

--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                                  scott at scottcain dot net
GMOD Project Manager (http://gmod.org/)                            216-392-3087
WormBase Developer (http://wormbase.org/)
Alliance of Genome Resources Group Leader (http://alliancegenome.org/)
VirusSeq Project Manager (https://virusseq-dataportal.ca/)
Human Cancer Models Initiative Project Manager (https://hcmi-searchable-catalog.nci.nih.gov/)