Re: [Gmod-tripal] [Gmod-schema] Tripal/Chado Web Services Update

[Stephen F. <spf...@gm...>]

Hi Joe,

To provide an update as to where we are with web services and Chado...

Yes, the link in the earlier email below has been removed.   When we 
first started on the project we had intended to create RESTful web 
services with direct access to Chado.  The link that was in that 
original email from early last year was a demo of that work. 
Essentially, you could find genomic features by using the 'feature' 
resource, and the 'stock' resource for germplasm, etc.  However, in 
meetings that we had with the community of Chado users it was clear that 
there was a preference for a web services that were not Chado-centric.  
Folks weren't interested in requiring their users to understand Chado 
table structure in order to query for information. Some tables, such as 
the stock table, can be very confusing to users as it's not always clear 
what's stored in that table.  Additionally, some of the sites that use 
Tripal and Chado have a set of authentication and access rules that 
would be difficult to enforce via Chado-centric web services.  For 
example, some data is private and only available to specific users, and 
Chado itself has no account of that access information.  Tripal / Drupal 
does, so we wanted tighter integration with web services and 
Tripal/Drupal authentication support.   Third, it is increasingly common 
for some data to not be housed in Chado (i.e. BAM alignments, all info 
in a VCF file, images, etc.), and we wanted those non-Chado (linked to 
Chado data) to also be available via web services.  This ensures we have 
consistency between the data that is shown on a page (i.e. gene page) 
and the same data that is available via web services for the same content.

Our goal is to provide web services for data in Chado, but we also have 
the goal of providing web services for the "community database" as well, 
so we have these additional constraints that need to be built into the 
web services, and hence they necessitated a change from our original 
plans.  There was some interest in also having Chado-centric web 
services, but we just didn't have the time to more fully develop both 
more robust web services and the Chado-centric version at the same time.

To meet the requirements described above we redesigned how we are going 
to offer data in Chado via web services for Tripal v3 (coming soon).  It 
will offer data in a more intuitive manner (e.g. via data types: genes, 
germplasm, genetic_marker, etc.), and ensures privacy of data.  It does 
support Chado but incorporates non-Chado data as well.   So, long 
story.... we took down the link to the original Chado-based web services 
we were working on, and unfortunately that added confusion because we 
failed to replace it with additional information.

Our work is progressing and we are close to offering a meeting to 
provide a demo of our current implementation for web services and to 
solicit more feedback on our web services design.  That should occur in 
April, and I'll send out a Doodle poll by email soon to schedule a time 
for interested groups.

Stephen


On 3/10/2016 9:33 AM, Joe Carlson wrote:
>
> I haven't been aware of what - if anything - has been happening on the 
> chado API scene. (the web_services link in this mail no longer works. 
> I take this as not a good sign.)
>
> But I wanted to pass along to the group some info of a project that 
> I've just learned about in case it is useful. postgrest 
> (www.postgrest.com and https://github.com/begriffs/postgrest) is a 
> project that provides an one-stop REST service for a postgres db.
>
> The project is at an early stage and it looks that there much to be 
> done. (I don't know if I trust the authentication just yet.) And the 
> querying seems to be limited to single tables (or views) and stored 
> procedures. I realize there are other similar projects out there. This 
> is just one that I've tried out. (No other involvement in the project, 
> though.)
>
> launching is easy:
>
> ./postgrest postgres://<database role>:<password>@<dbserver>/<dbname> 
> -a <database role for anonymous connections>
>
> A couple examples. A query for a single table (or view):
>
> curl 'localhost:3000/organism?species=eq.sapiens'
> [{"organism_id":1,"abbreviation":"H.sapiens","genus":"Homo","species":"sapiens","common_name":"human","comment":null}] 
>
>
> A query on a table uses the endpoint /<table name>. Column constraints 
> on the query are given as "<column>=<operator>.<value>"; in this case 
> species='sapiens'
>
> And a stored procedure. genomic_slice is one that I have for returning 
> a segment of a chromosome sequence. ('proteome'  is a dbxref... don't 
> worry about that)
>
> curl -d 
> '{"proteome":"210","chromosome":"Chr01","fmin":"1000000","len":"10"}' 
> -X POST -H "Content-Type: application/json" 
> 'http://localhost:3000/rpc/genomic_slice'
> [{"genomic_slice":"TACCAAGGTA"}]
>
> (specifying the application/json in the POST headers is important!)
>
> I wanted to try the queries on the public flybase db. But that server 
> is running postgres 8.4 and this requires at least 9.0.3
>
> Part of the reason for sending the email is that there will be a 
> meetup with the developer in San Fran next week for those that are local.
>
> http://www.meetup.com/postgresql-1/events/229237351
>
> The talk will be video taped and available later, I've heard.
>
> Joe
>
> On 01/29/2015 08:57 AM, Stephen Ficklin wrote:
>> Dear Tripal/Chado Community,
>>
>> Some progress has been made on the "low-level" Chado API that will
>> provide access to data in a Chado-centric manner, but does not include a
>> design for specific use cases.   The current state of this web service
>> API can be found on the Tripal 2.x demo site:
>>
>> http://demo.tripal.info/2.x/web_services
>>
>> Currently, the web services are very simple.  As progress is made we
>> will update the web services on that page.  There is still much work to
>> do, so what you see is not finalized but is made available in an effort
>> to solicit feedback for suggested changes.  Please let us know what you
>> do or don't like so far.
>>
>> Also, on Saturday, Jan 10th at the Plant and Animal Genome Conference
>> (PAG) in San Deigo, several folks interested in the development of web
>> services for Tripal/Chado met for our second meeting to discuss design
>> of the web services as funded by the NSF DIBBs grant.   The goal of
>> these meetings is to solicit input in the design of web services such
>> that they can support the needs of a broad set of Tripal sites. Below
>> is a Google doc containing a summary of that meeting. The section
>> containing attendee comments was drawn from memory as we failed to
>> designate an official note taker.
>>
>> https://docs.google.com/document/d/1gdL7Xo-Xoq2ztBVY0FVAHWPv5kNz_ajyNtAlYjCzC4U/edit 
>>
>>
>> For anyone interested in web services for Tripal/Chado, please contact
>> me directly if you have comments as the way we have approached
>> soliciting input during the design phase (positive or negative-- I'm
>> happy to hear all) as we want to ensure stake-holders feel heard and the
>> design fits the broadest needs.
>>
>> Thanks much to all who are participating in this!
>> Stephen
>>
>>
>>
>> ------------------------------------------------------------------------------ 
>>
>> Dive into the World of Parallel Programming. The Go Parallel Website,
>> sponsored by Intel and developed in partnership with Slashdot Media, 
>> is your
>> hub for all things parallel software development, from weekly thought
>> leadership blogs to news, videos, case studies, tutorials and more. 
>> Take a
>> look and join the conversation now. http://goparallel.sourceforge.net/
>> _______________________________________________
>> Gmod-schema mailing list
>> Gmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/gmod-schema
>