Menu
▾
▴
virtuoso-devel — Discussion of Virtuoso development topics
You can subscribe to this list here.
| 2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(2) |
Sep
|
Oct
|
Nov
|
Dec
(1) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2007 |
Jan
|
Feb
(4) |
Mar
|
Apr
(9) |
May
(9) |
Jun
(2) |
Jul
(7) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2008 |
Jan
|
Feb
(36) |
Mar
(17) |
Apr
(14) |
May
|
Jun
|
Jul
|
Aug
(16) |
Sep
(3) |
Oct
|
Nov
(2) |
Dec
(7) |
| 2009 |
Jan
(4) |
Feb
(7) |
Mar
|
Apr
(12) |
May
(59) |
Jun
(32) |
Jul
(13) |
Aug
(13) |
Sep
(2) |
Oct
(4) |
Nov
(4) |
Dec
(3) |
| 2010 |
Jan
(2) |
Feb
(6) |
Mar
(20) |
Apr
(1) |
May
(4) |
Jun
(8) |
Jul
(6) |
Aug
(5) |
Sep
(3) |
Oct
(3) |
Nov
|
Dec
(1) |
| 2011 |
Jan
(7) |
Feb
(6) |
Mar
(10) |
Apr
|
May
|
Jun
(9) |
Jul
(17) |
Aug
(27) |
Sep
(12) |
Oct
(5) |
Nov
|
Dec
|
| 2012 |
Jan
|
Feb
(4) |
Mar
(14) |
Apr
(6) |
May
(2) |
Jun
(1) |
Jul
|
Aug
|
Sep
(10) |
Oct
|
Nov
(13) |
Dec
|
| 2013 |
Jan
(26) |
Feb
(8) |
Mar
(6) |
Apr
(13) |
May
(11) |
Jun
|
Jul
(10) |
Aug
(1) |
Sep
(5) |
Oct
(5) |
Nov
|
Dec
(1) |
| 2014 |
Jan
|
Feb
(17) |
Mar
(7) |
Apr
(23) |
May
(5) |
Jun
(10) |
Jul
(10) |
Aug
(7) |
Sep
|
Oct
(2) |
Nov
(15) |
Dec
(2) |
| 2015 |
Jan
|
Feb
(1) |
Mar
|
Apr
(2) |
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(6) |
| 2016 |
Jan
(2) |
Feb
|
Mar
(9) |
Apr
(22) |
May
(2) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(7) |
Nov
(1) |
Dec
(6) |
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(10) |
Dec
(5) |
| 2018 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
(3) |
Sep
|
Oct
|
Nov
|
Dec
|
| 2019 |
Jan
(1) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Sherman M. <sdm...@gm...> - 2019-01-03 16:16:18
|
Hi, I would like to retract this issue <https://github.com/openlink/virtuoso-opensource/issues/698>, but I was unable to delete it since I don't own the repository. The approach turned out to be nonviable. Thanks, -sherman On Fri, Dec 1, 2017 at 6:04 PM Sherman Monroe <sdm...@gm...> wrote: > Okay. > > On Fri, Dec 1, 2017 at 3:45 PM, Kingsley Idehen <ki...@op...> > wrote: > >> On 12/1/17 1:22 PM, Sherman Monroe wrote: >> >> As an example of how a disk write will work for the eos.io adaptor, the >> buffer used by disk.c will pass the write onset and the total_bytes to >> the eos blockchain and this transaction (and nothing more) will be recorded >> on the blockchain, respecting its index in the sequence of i/o >> transactions. One a read request, the buffer will pass the read onset >> and length to eos. To derive the current state of the file inside that >> char window, it smart contract will do the equivalent of running the >> transaction history from beginning to end (this will happens using the >> blockchain's built-in parallel processing), but as it runs each >> transaction/state transition, it will only consider (i.e. read and record) >> the bytes that changed within the requested window. It will then return the >> state/content in the window back to the adaptor which passes the content >> back to the buffer. >> >> In eos.io, only the messages (the inputs, which in this case are the >> file write/delete/insert requests) are stored on the chain, not the file >> itself (the output). The state of the state machine at any given moment is >> therefore only implicit (in the message/transaction history). With >> knowledge of the message history and a description of the state transition >> logic, one can derive the state on demand. I propose using the state >> machine (the blockchain) to model a file of infinite length. >> >> The adaptor/virtual file interface allows different approaches to be used >> to take advantage of the characteristics of each persistence strategy, so a >> different approach can be used for Hadoop, etc. >> >> >> Why don't you move this thread to the github repo for VOS. You can open >> an issue and dialog can be built around that. >> >> >> Kingsley >> >> On Fri, Dec 1, 2017 at 11:55 AM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> fifo special files >>> <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> is >>> a named pipe, I must have missed that. >>> >>> On Fri, Dec 1, 2017 at 10:32 AM, Sherman Monroe <sdm...@gm...> >>> wrote: >>> >>>> After reviewing the dbe_storage_s struct and the code in disk.c, it is >>>> apparent that disk.c is the *.db file manager of Virtuoso, and this >>>> file manager is sufficiently sophisticated as to elicit an alternative to >>>> extracting a programming interface from disk.c. So my preliminary >>>> proposal is to leave the disk.c code alone and instead make virtuoso.db >>>> a virtual file such that reading from the file actually reads from the >>>> stdout of a command and writing to the file actually writes to the >>>> stdin of a command. Then the commands in question will interact with >>>> the persistent store of choice, including the eos.io smart contract >>>> running on the blockchain. Some additional requirements are: >>>> >>>> - Low level-of-effort, so implementing a custom file system (e.g. fuse >>>> <http://fuse.sourceforge.net/>) is out of the question >>>> - Must be cross platform, so e.g. linux fifo special files >>>> <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> >>>> are out of the question >>>> - Preserves performance, i.e. the solution cannot adversely impact >>>> Virtuoso's current performance metrics >>>> >>>> After some preliminary research, writing to a named pipe >>>> <https://en.wikipedia.org/wiki/Named_pipe> looks very promising. The >>>> downside is that you can't have concurrent processes reading/listening to >>>> the file, but AFAIK only one process is accessing virtuoso.db at any >>>> given time. Once the content is read its gone so disk.c will need to >>>> treat it like a buffered stream (e.g. the writer will continuously push the >>>> content to the buffer, and the reader will loop the buffered content >>>> indefinitely). >>>> >>>> The command writing to the buffer is what I'll call an adaptor. It's >>>> job is to persist the state of the "virtual file" based on what it receives >>>> from Virtuoso and to fulfill Virtuoso's reads. This way, the adaptor is >>>> agnostic to the contents of virtuoso.db and its logic only needs to >>>> handle the details of the persistence provider. >>>> >>>> What are your thoughts? I'm sure I've overlooked requirements you >>>> envision, if so please share those also. >>>> >>>> >>>> On Thu, Nov 30, 2017 at 2:34 PM, Sherman Monroe <sdm...@gm...> >>>> wrote: >>>> >>>>> Correction, dbe_storage_s.dbs_file >>>>> >>>>> On Thu, Nov 30, 2017 at 1:43 PM, Sherman Monroe <sdm...@gm...> >>>>> wrote: >>>>> >>>>>> I've found dbe_storage_s and dbe_storage_s->dbs_file. Design >>>>>> proposal en route... >>>>>> >>>>>> On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe <sdm...@gm...> >>>>>> wrote: >>>>>> >>>>>>> Ok, I will review the codebase and begin the design. I'll post for >>>>>>> peer review before I begin coding. >>>>>>> >>>>>>> On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen < >>>>>>> ki...@op...> wrote: >>>>>>> >>>>>>>> On 11/29/17 6:47 PM, Sherman Monroe wrote: >>>>>>>> >>>>>>>> So as a starting point, I abstract every single call to .db and >>>>>>>> also any dependent functions dealing with in-memory state. Next I implement >>>>>>>> the old "file backed" persistence using the new API, and run test cases to >>>>>>>> ensure nothing broke. Next I test it on some common alternative types like >>>>>>>> the ones you list above. Finally I implement the eos.io backed >>>>>>>> persistence. Does this sum up the requirements? >>>>>>>> >>>>>>>> A 2K feet, yes. >>>>>>>> >>>>>>>> >>>>>>>> Kingsley >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm... >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> Excellent, that's what I hoped. >>>>>>>>> >>>>>>>>> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen < >>>>>>>>> ki...@op...> wrote: >>>>>>>>> >>>>>>>>>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>>>>>>>>> >>>>>>>>>> Is Virtuoso loading the contents of "xyz.db" into an internal >>>>>>>>>> model, or is it performing transactions against "xyz.db"? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Persistence happens against "xyz.db". Other relations related >>>>>>>>>> activity happens in memory. >>>>>>>>>> >>>>>>>>>> Kingsley >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe < >>>>>>>>>> sdm...@gm...> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Kingsley, yes I would like to take this on and then >>>>>>>>>>> implement a smart contract-based persistence with it. Can you suggest >>>>>>>>>>> starting points or what work has been done so far? >>>>>>>>>>> >>>>>>>>>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>>>>>>>>> ki...@op...> wrote: >>>>>>>>>>> >>>>>>>>>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I would like to know what is the feasibility of implementing >>>>>>>>>>>> Virtuoso's "state" one a blockchain like eos.io? If for >>>>>>>>>>>> instance Virt is storing its quad-store state as a linked list, then if I >>>>>>>>>>>> wanted to swap that datatype out for a custom, EOSLinkedList, how difficult >>>>>>>>>>>> would that be? Does Virtuoso support a swappable persistence API of any >>>>>>>>>>>> sort, or interfaces for the data types it uses for persistence? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> We are looking at loose-coupling of persistent storage i.e., >>>>>>>>>>>> rather than being pegged to "xyz.db" you end up being able to use a variety >>>>>>>>>>>> of document types (CSV, ORC, Parquet etc..). >>>>>>>>>>>> >>>>>>>>>>>> Naturally, anyone can pick up this effort to accelerate matters >>>>>>>>>>>> :) >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Regards, >>>>>>>>>>>> >>>>>>>>>>>> Kingsley Idehen >>>>>>>>>>>> Founder & CEO >>>>>>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>>>>>> >>>>>>>>>>>> Weblogs (Blogs): >>>>>>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>>>>>> >>>>>>>>>>>> Profile Pages: >>>>>>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>>>>>> >>>>>>>>>>>> Web Identities (WebID): >>>>>>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ------------------------------------------------------------------------------ >>>>>>>>>>>> Check out the vibrant tech community on one of the world's most >>>>>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Virtuoso-devel mailing list >>>>>>>>>>>> Vir...@li... >>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> -sherman >>>>>>>>>>> >>>>>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>>>>> shadow of turning. >>>>>>>>>>> (James 1:17) >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> -sherman >>>>>>>>>> >>>>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>>>> shadow of turning. >>>>>>>>>> (James 1:17) >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Kingsley Idehen >>>>>>>>>> Founder & CEO >>>>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>>>> >>>>>>>>>> Weblogs (Blogs): >>>>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>>>> >>>>>>>>>> Profile Pages: >>>>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>>>> >>>>>>>>>> Web Identities (WebID): >>>>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> -sherman >>>>>>>>> >>>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>>> shadow of turning. >>>>>>>>> (James 1:17) >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -sherman >>>>>>>> >>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>> shadow of turning. >>>>>>>> (James 1:17) >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> >>>>>>>> Kingsley Idehen >>>>>>>> Founder & CEO >>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>> >>>>>>>> Weblogs (Blogs): >>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>> >>>>>>>> Profile Pages: >>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>> >>>>>>>> Web Identities (WebID): >>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks, >>>>>>> -sherman >>>>>>> >>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>> shadow of turning. >>>>>>> (James 1:17) >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> -sherman >>>>>> >>>>>> Every good gift and every perfect gift is from above, and cometh down >>>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>>> turning. >>>>>> (James 1:17) >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> -sherman >>>>> >>>>> Every good gift and every perfect gift is from above, and cometh down >>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>> turning. >>>>> (James 1:17) >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software (Home Page: http://www.openlinksw.com) >> >> Weblogs (Blogs): >> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >> Blogspot Blog: http://kidehen.blogspot.com >> Medium Blog: https://medium.com/@kidehen >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> LinkedIn: http://www.linkedin.com/in/kidehen >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> >> > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Hugh W. <hwi...@op...> - 2018-08-10 10:53:07
|
Hi Steve, Note I have been able to recreate the error you report running your query against an empty database and have reported to development to look into: SQL> SPARQL PREFIX ce: <http://ontology.causeex.com/ontology/odps/CauseEffect#> Type the rest of statement, end with a semicolon (;)> PREFIX doco: <http://purl.org/spar/doco/> Type the rest of statement, end with a semicolon (;)> PREFIX event: <http://ontology.causeex.com/ontology/odps/Event#> Type the rest of statement, end with a semicolon (;)> PREFIX general: <http://ontology.causeex.com/ontology/odps/GeneralConcepts#> Type the rest of statement, end with a semicolon (;)> PREFIX prov: <http://ontology.causeex.com/ontology/odps/DataProvenance#> Type the rest of statement, end with a semicolon (;)> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> Type the rest of statement, end with a semicolon (;)> Type the rest of statement, end with a semicolon (;)> SELECT DISTINCT Type the rest of statement, end with a semicolon (;)> ("LCC" AS ?performer) Type the rest of statement, end with a semicolon (;)> ?event ?eventClassName Type the rest of statement, end with a semicolon (;)> (IF(BOUND(?sourceOffset), ?sourceOffset, IF(BOUND(?sentenceOffset), ?sentenceOffset, UNDEF)) AS ?offset) Type the rest of statement, end with a semicolon (;)> (IF(BOUND(?eventTextValue) && STRLEN(STR(?eventTextValue)) > 0, ?eventTextValue, IF(BOUND(?eventTextLabel) && STRLEN(STR(?eventTextLabel)) > 0, ?eventTextLabel, UNDEF)) As ?eventText) Type the rest of statement, end with a semicolon (;)> ?startTime ?locLabel ?latitude ?longitude Type the rest of statement, end with a semicolon (;)> ?activeActorText ?affectedActorText ?recipientText ?providerText ?actorText Type the rest of statement, end with a semicolon (;)> ?eventTenseText ?eventTopicText ?eventInstrumentText Type the rest of statement, end with a semicolon (;)> ?eventSentenceText ?sentenceOffset Type the rest of statement, end with a semicolon (;)> FROM <http://graph.causeex.com> Type the rest of statement, end with a semicolon (;)> FROM <http://ontology.causeex.com> Type the rest of statement, end with a semicolon (;)> WHERE { Type the rest of statement, end with a semicolon (;)> ?doc a prov:Document ; Type the rest of statement, end with a semicolon (;)> prov:contains+ ?docPart ; Type the rest of statement, end with a semicolon (;)> prov:original_source ?docName FILTER CONTAINS(?docName, "040f42df962bc21aa222ecd900c32ccd") . Type the rest of statement, end with a semicolon (;)> ?event a ?eventClass ; prov:sourced_from ?docPart Type the rest of statement, end with a semicolon (;)> OPTIONAL { ?event prov:text_value ?eventTextValue } Type the rest of statement, end with a semicolon (;)> OPTIONAL { ?event rdfs:label ?eventTextLabel } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event general:located_at ?loc . Type the rest of statement, end with a semicolon (;)> OPTIONAL { ?loc general:latitude ?latitude ; general:longitude ?longitude } Type the rest of statement, end with a semicolon (;)> OPTIONAL { ?locUnifier general:unifies ?loc ; general:canonical_label ?locLabel } Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { ?event general:earliest_possible_start_time ?startTime } . Type the rest of statement, end with a semicolon (;)> { Type the rest of statement, end with a semicolon (;)> { ?event a event:Event } Type the rest of statement, end with a semicolon (;)> UNION Type the rest of statement, end with a semicolon (;)> { ?eventClass rdfs:subClassOf event:Event } Type the rest of statement, end with a semicolon (;)> UNION Type the rest of statement, end with a semicolon (;)> { ?eventClass rdfs:subClassOf/rdfs:subClassOf event:Event } Type the rest of statement, end with a semicolon (;)> UNION Type the rest of statement, end with a semicolon (;)> { ?eventClass rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf event:Event } Type the rest of statement, end with a semicolon (;)> UNION Type the rest of statement, end with a semicolon (;)> { ?eventClass rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf event:Event } Type the rest of statement, end with a semicolon (;)> UNION Type the rest of statement, end with a semicolon (;)> { ?eventClass rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf event:Event } Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { ?eventClass rdfs:label ?eventClassName } . Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?docPart prov:char_offset ?sourceOffset FILTER NOT EXISTS { ?docPart a doco:Sentence } . Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> _:sentence a doco:Sentence ; prov:contains ?docPart ; prov:text_value ?eventSentenceText ; prov:char_offset ?sentenceOffset Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?docPart a doco:Sentence ; prov:text_value ?eventSentenceText ; prov:char_offset ?sentenceOffset . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_actor ?actor . Type the rest of statement, end with a semicolon (;)> ?actor prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?actorText ] . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_active_actor ?activeActor . Type the rest of statement, end with a semicolon (;)> ?activeActor prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?activeActorText ] . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:hasaffected_actor ?affectedActor . Type the rest of statement, end with a semicolon (;)> ?affectedActor prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?affectedActorText ] . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_tense ?eventTense . Type the rest of statement, end with a semicolon (;)> ?eventTense rdfs:label ?eventTenseText . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_topic ?eventTopic . Type the rest of statement, end with a semicolon (;)> ?eventTopic rdfs:label ?eventTopicText . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_instrument ?eventInstrument . Type the rest of statement, end with a semicolon (;)> ?eventInstrument prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?eventInstrumentText ] . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_recipient ?recipient . Type the rest of statement, end with a semicolon (;)> ?recipient prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?recipientText ] . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> OPTIONAL { Type the rest of statement, end with a semicolon (;)> ?event event:has_provider ?provider . Type the rest of statement, end with a semicolon (;)> ?provider prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?providerText ] . Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> } Type the rest of statement, end with a semicolon (;)> ORDER BY ?event ?offset; *** Error 37000: VD [Virtuoso Server]SQ156: Internal Optimized compiler error : unknown table for a column dfe in sqldf.c:1529. Please report the statement compiled. in lines 2-89 of Top-Level: #line 2 "(console)" SPARQL PREFIX ce: <http://ontology.causeex.com/ontology/odps/CauseEffect#> PREFIX doco: <http://purl.org/spar/doco/> PREFIX event: <http://ontology.causeex.com/ontology/odps/Event#> PREFIX general: <http://ontology.causeex.com/ontology/odps/GeneralConcepts#> PREFIX prov: <http://ontology.causeex.com/ontology/odps/DataProvenance#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT DISTINCT ("LCC" AS ?performer) ?event ?eventClassName (IF(BOUND(?sourceOffset), ?sourceOffset, IF(BOUND(?sentenceOffset), ?sentenceOffset, UNDEF)) AS ?offset) (IF(BOUND(?eventTextValue) && STRLEN(STR(?eventTextValue)) > 0, ?eventTextValue, IF(BOUND(?eventTextLabel) && STRLEN(STR(?eventTextLabel)) > 0, ?eventTextLabel, UNDEF)) As ?eventText) ?startTime ?locLabel ?latitude ?longitude ?activeActorText ?affectedActorText ?recipientText ?providerText ?actorText ?eventTenseText ?eventTopicText ?eventInstrumentText ?eventSentenceText ?sentenceOffset FROM <http://graph.causeex.com> FROM <http://ontology.causeex.com> WHERE { ?doc a prov:Document ; prov:contains+ ?docPart ; prov:original_source ?docName FILTER CONTAINS(?docName, "040f42df962bc21aa222ecd900c32ccd") . ?event a ?eventClass ; prov:sourced_from ?docPart OPTIONAL { ?event prov:text_value ?eventTextValue } OPTIONAL { ?event rdfs:label ?eventTextLabel } OPTIONAL { ?event general:located_at ?loc . OPTIONAL { ?loc general:latitude ?latitude ; general:longitude ?longitude } OPTIONAL { ?locUnifier general:unifies ?loc ; general:canonical_label ?locLabel } } OPTIONAL { ?event general:earliest_possible_start_time ?startTime } . { { ?event a event:Event } UNION { ?eventClass rdfs:subClassOf event:Event } UNION { ?eventClass rdfs:subClassOf/rdfs:subClassOf event:Event } UNION { ?eventClass rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf event:Event } UNION { ?eventClass rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf event:Event } UNION { ?eventClass rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf/rdfs:subClassOf event:Event } } OPTIONAL { ?eventClass rdfs:label ?eventClassName } . OPTIONAL { ?docPart prov:char_offset ?sourceOffset FILTER NOT EXISTS { ?docPart a doco:Sentence } . OPTIONAL { _:sentence a doco:Sentence ; prov:contains ?docPart ; prov:text_value ?eventSentenceText ; prov:char_offset ?sentenceOffset } } OPTIONAL { ?docPart a doco:Sentence ; prov:text_value ?eventSentenceText ; prov:char_offset ?sentenceOffset . } OPTIONAL { ?event event:has_actor ?actor . ?actor prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?actorText ] . } OPTIONAL { ?event event:has_active_actor ?activeActor . ?activeActor prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?activeActorText ] . } OPTIONAL { ?event event:hasaffected_actor ?affectedActor . ?affectedActor prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?affectedActorText ] . } OPTIONAL { ?event event:has_tense ?eventTense . ?eventTense rdfs:label ?eventTenseText . } OPTIONAL { ?event event:has_topic ?eventTopic . ?eventTopic rdfs:label ?eventTopicText . } OPTIONAL { ?event event:has_instrument ?eventInstrument . ?eventInstrument prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?eventInstrumentText ] . } OPTIONAL { ?event event:has_recipient ?recipient . ?recipient prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?recipientText ] . } OPTIONAL { ?event event:has_provider ?provider . ?provider prov:sourced_from [ a prov:MentionSpan ; prov:text_value ?providerText ] . } } ORDER BY ?event ?offset SQL> Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers > On 9 Aug 2018, at 21:51, Hugh Williams <hwi...@op...> wrote: > > Hi Steve, > > What is: > > 1. The version for the Virtuoso server being used which can be obtained by running the following command from the bin directory of your Virtuoso installation: > > ./virtuoso-t -? > > 2. What is the version of the Virtuoso RDF4J Provider being used which can be obtained with the command: > > java -jar virt_rdf4j.jar > > 3. Does the query run when executed against the Virtuoso /sparql endpoint ie http:/hostname:portno/sparql or if run directly via the Virtuoso “isql” command line tool ? > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ <http://www.openlinksw.com/> > Weblog -- http://www.openlinksw.com/blogs/ <http://www.openlinksw.com/blogs/> > LinkedIn -- http://www.linkedin.com/company/openlink-software/ <http://www.linkedin.com/company/openlink-software/> > Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink> > Google+ -- http://plus.google.com/100570109519069333827/ <http://plus.google.com/100570109519069333827/> > Facebook -- http://www.facebook.com/OpenLinkSoftware <http://www.facebook.com/OpenLinkSoftware> > Universal Data Access, Integration, and Management Technology Providers > > > >> On 9 Aug 2018, at 19:20, Wartik, Steven P Steve <sw...@id... <mailto:sw...@id...>> wrote: >> >> Hi, >> >> I’ve got a rather long SPARQL query that, when submitted to the Virtuoso server, returns an error: >> >> Query evaluation error: : SPARQL execute failed:[...] Exception:virtuoso.jdbc4.VirtuosoException: SQ156: Internal Optimized compiler error : unknown table for a column dfe in sqldf.c:1528. Please report the statement compiled. >> >> The “…” is the query I’ve attached. >> >> I’ve no idea what’s happening here. Something in my query, or a Virtuoso bug? >> >> If it makes any difference, I’m running this through an RDF4J server that uses Virtuoso as the back-end repository. >> >> Thanks! >> >> Steve Wartik >> Institute for Defense Analyses >> (703) 845-6646 >> sw...@id... <mailto:sw...@id...> >> >> >> >> <bad-query.sparql>------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________> >> Virtuoso-devel mailing list >> Vir...@li... <mailto:Vir...@li...> >> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> |
|
From: Hugh W. <hwi...@op...> - 2018-08-09 20:51:58
|
Hi Steve, What is: 1. The version for the Virtuoso server being used which can be obtained by running the following command from the bin directory of your Virtuoso installation: ./virtuoso-t -? 2. What is the version of the Virtuoso RDF4J Provider being used which can be obtained with the command: java -jar virt_rdf4j.jar 3. Does the query run when executed against the Virtuoso /sparql endpoint ie http:/hostname:portno/sparql or if run directly via the Virtuoso “isql” command line tool ? Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers > On 9 Aug 2018, at 19:20, Wartik, Steven P Steve <sw...@id...> wrote: > > Hi, > > I’ve got a rather long SPARQL query that, when submitted to the Virtuoso server, returns an error: > > Query evaluation error: : SPARQL execute failed:[...] Exception:virtuoso.jdbc4.VirtuosoException: SQ156: Internal Optimized compiler error : unknown table for a column dfe in sqldf.c:1528. Please report the statement compiled. > > The “…” is the query I’ve attached. > > I’ve no idea what’s happening here. Something in my query, or a Virtuoso bug? > > If it makes any difference, I’m running this through an RDF4J server that uses Virtuoso as the back-end repository. > > Thanks! > > Steve Wartik > Institute for Defense Analyses > (703) 845-6646 > sw...@id... <mailto:sw...@id...> > > > > <bad-query.sparql>------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________> > Virtuoso-devel mailing list > Vir...@li... <mailto:Vir...@li...> > https://lists.sourceforge.net/lists/listinfo/virtuoso-devel <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> |
|
From: Wartik, S. P \Steve\ <sw...@id...> - 2018-08-09 18:56:14
|
Hi, I’ve got a rather long SPARQL query that, when submitted to the Virtuoso server, returns an error: Query evaluation error: : SPARQL execute failed:[...] Exception:virtuoso.jdbc4.VirtuosoException: SQ156: Internal Optimized compiler error : unknown table for a column dfe in sqldf.c:1528. Please report the statement compiled. The “…” is the query I’ve attached. I’ve no idea what’s happening here. Something in my query, or a Virtuoso bug? If it makes any difference, I’m running this through an RDF4J server that uses Virtuoso as the back-end repository. Thanks! Steve Wartik Institute for Defense Analyses (703) 845-6646 sw...@id... |
|
From: Hugh W. <hwi...@op...> - 2018-06-19 14:26:20
|
Hi Steve, Yes, this has been reported to development from the report in https://github.com/openlink/virtuoso-opensource/issues/424 - Wrong implementation of MINUS ... I am checking with development as to the status of a fix for this issue ... Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers > On 19 Jun 2018, at 15:07, Wartik, Steven P Steve <sw...@id...> wrote: > > Oops – I discovered through another channel that this has already been recorded <https://github.com/openlink/virtuoso-opensource/issues/424>. Standing by for the fix. > > Steve Wartik > Institute for Defense Analyses > (703) 845-6646 > sw...@id... <mailto:sw...@id...> > > From: Wartik, Steven P "Steve" [mailto:sw...@id... <mailto:sw...@id...>] > Sent: Tuesday, June 19, 2018 9:11 AM > To: vir...@li... <mailto:vir...@li...> > Subject: [Virtuoso-devel] Bug processing SPARQL MINUS? > > Hi, > > I’ve encountered a problem using Virtuoso 07.20.3317. I was trying one of the queries in the SPARQL documentation. Section 8.3.1 in SPARQL 1.1 Query Language <https://www.w3.org/TR/sparql11-query/#neg-example-1> says that, given a data set containing: > > @prefix : <http://example/ <http://example/>> . > :a :b :c . > > the result of executing the query: > > SELECT * > { > ?s ?p ?o > MINUS > { ?x ?y ?z } > } > > should be: > > s > p > o > <http://example/a <http://example/a>> > <http://example/b <http://example/b>> > <http://example/c <http://example/c>> > > > But my Virtuoso server returns no results. Can someone please check whether this is an error in Virtuoso? Thanks! > > Steve Wartik > Institute for Defense Analyses > (703) 845-6646 > sw...@id... <mailto:sw...@id...> > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org <http://slashdot.org/>! http://sdm.link/slashdot_______________________________________________ <http://sdm.link/slashdot_______________________________________________> > Virtuoso-devel mailing list > Vir...@li... <mailto:Vir...@li...> > https://lists.sourceforge.net/lists/listinfo/virtuoso-devel <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> |
|
From: Wartik, S. P \Steve\ <sw...@id...> - 2018-06-19 14:07:37
|
Oops – I discovered through another channel that this has already been recorded<https://github.com/openlink/virtuoso-opensource/issues/424>. Standing by for the fix. Steve Wartik Institute for Defense Analyses (703) 845-6646 sw...@id... From: Wartik, Steven P "Steve" [mailto:sw...@id...] Sent: Tuesday, June 19, 2018 9:11 AM To: vir...@li... Subject: [Virtuoso-devel] Bug processing SPARQL MINUS? Hi, I’ve encountered a problem using Virtuoso 07.20.3317. I was trying one of the queries in the SPARQL documentation. Section 8.3.1 in SPARQL 1.1 Query Language<https://www.w3.org/TR/sparql11-query/#neg-example-1> says that, given a data set containing: @prefix : <http://example/> . :a :b :c . the result of executing the query: SELECT * { ?s ?p ?o MINUS { ?x ?y ?z } } should be: s p o <http://example/a> <http://example/b> <http://example/c> But my Virtuoso server returns no results. Can someone please check whether this is an error in Virtuoso? Thanks! Steve Wartik Institute for Defense Analyses (703) 845-6646 sw...@id... |
|
From: Wartik, S. P \Steve\ <sw...@id...> - 2018-06-19 13:47:33
|
Hi, I’ve encountered a problem using Virtuoso 07.20.3317. I was trying one of the queries in the SPARQL documentation. Section 8.3.1 in SPARQL 1.1 Query Language<https://www.w3.org/TR/sparql11-query/#neg-example-1> says that, given a data set containing: @prefix : <http://example/> . :a :b :c . the result of executing the query: SELECT * { ?s ?p ?o MINUS { ?x ?y ?z } } should be: s p o <http://example/a> <http://example/b> <http://example/c> But my Virtuoso server returns no results. Can someone please check whether this is an error in Virtuoso? Thanks! Steve Wartik Institute for Defense Analyses (703) 845-6646 sw...@id... |
|
From: Sherman M. <sdm...@gm...> - 2017-12-02 00:04:47
|
Okay. On Fri, Dec 1, 2017 at 3:45 PM, Kingsley Idehen <ki...@op...> wrote: > On 12/1/17 1:22 PM, Sherman Monroe wrote: > > As an example of how a disk write will work for the eos.io adaptor, the > buffer used by disk.c will pass the write onset and the total_bytes to > the eos blockchain and this transaction (and nothing more) will be recorded > on the blockchain, respecting its index in the sequence of i/o > transactions. One a read request, the buffer will pass the read onset and > length to eos. To derive the current state of the file inside that char > window, it smart contract will do the equivalent of running the transaction > history from beginning to end (this will happens using the blockchain's > built-in parallel processing), but as it runs each transaction/state > transition, it will only consider (i.e. read and record) the bytes that > changed within the requested window. It will then return the state/content > in the window back to the adaptor which passes the content back to the > buffer. > > In eos.io, only the messages (the inputs, which in this case are the file > write/delete/insert requests) are stored on the chain, not the file itself > (the output). The state of the state machine at any given moment is > therefore only implicit (in the message/transaction history). With > knowledge of the message history and a description of the state transition > logic, one can derive the state on demand. I propose using the state > machine (the blockchain) to model a file of infinite length. > > The adaptor/virtual file interface allows different approaches to be used > to take advantage of the characteristics of each persistence strategy, so a > different approach can be used for Hadoop, etc. > > > Why don't you move this thread to the github repo for VOS. You can open an > issue and dialog can be built around that. > > > Kingsley > > On Fri, Dec 1, 2017 at 11:55 AM, Sherman Monroe <sdm...@gm...> > wrote: > >> fifo special files >> <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> is >> a named pipe, I must have missed that. >> >> On Fri, Dec 1, 2017 at 10:32 AM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> After reviewing the dbe_storage_s struct and the code in disk.c, it is >>> apparent that disk.c is the *.db file manager of Virtuoso, and this >>> file manager is sufficiently sophisticated as to elicit an alternative to >>> extracting a programming interface from disk.c. So my preliminary >>> proposal is to leave the disk.c code alone and instead make virtuoso.db >>> a virtual file such that reading from the file actually reads from the >>> stdout of a command and writing to the file actually writes to the stdin >>> of a command. Then the commands in question will interact with the >>> persistent store of choice, including the eos.io smart contract running >>> on the blockchain. Some additional requirements are: >>> >>> - Low level-of-effort, so implementing a custom file system (e.g. fuse >>> <http://fuse.sourceforge.net/>) is out of the question >>> - Must be cross platform, so e.g. linux fifo special files >>> <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> >>> are out of the question >>> - Preserves performance, i.e. the solution cannot adversely impact >>> Virtuoso's current performance metrics >>> >>> After some preliminary research, writing to a named pipe >>> <https://en.wikipedia.org/wiki/Named_pipe> looks very promising. The >>> downside is that you can't have concurrent processes reading/listening to >>> the file, but AFAIK only one process is accessing virtuoso.db at any >>> given time. Once the content is read its gone so disk.c will need to >>> treat it like a buffered stream (e.g. the writer will continuously push the >>> content to the buffer, and the reader will loop the buffered content >>> indefinitely). >>> >>> The command writing to the buffer is what I'll call an adaptor. It's job >>> is to persist the state of the "virtual file" based on what it receives >>> from Virtuoso and to fulfill Virtuoso's reads. This way, the adaptor is >>> agnostic to the contents of virtuoso.db and its logic only needs to >>> handle the details of the persistence provider. >>> >>> What are your thoughts? I'm sure I've overlooked requirements you >>> envision, if so please share those also. >>> >>> >>> On Thu, Nov 30, 2017 at 2:34 PM, Sherman Monroe <sdm...@gm...> >>> wrote: >>> >>>> Correction, dbe_storage_s.dbs_file >>>> >>>> On Thu, Nov 30, 2017 at 1:43 PM, Sherman Monroe <sdm...@gm...> >>>> wrote: >>>> >>>>> I've found dbe_storage_s and dbe_storage_s->dbs_file. Design proposal >>>>> en route... >>>>> >>>>> On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe <sdm...@gm...> >>>>> wrote: >>>>> >>>>>> Ok, I will review the codebase and begin the design. I'll post for >>>>>> peer review before I begin coding. >>>>>> >>>>>> On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen < >>>>>> ki...@op...> wrote: >>>>>> >>>>>>> On 11/29/17 6:47 PM, Sherman Monroe wrote: >>>>>>> >>>>>>> So as a starting point, I abstract every single call to .db and also >>>>>>> any dependent functions dealing with in-memory state. Next I implement the >>>>>>> old "file backed" persistence using the new API, and run test cases to >>>>>>> ensure nothing broke. Next I test it on some common alternative types like >>>>>>> the ones you list above. Finally I implement the eos.io backed >>>>>>> persistence. Does this sum up the requirements? >>>>>>> >>>>>>> A 2K feet, yes. >>>>>>> >>>>>>> >>>>>>> Kingsley >>>>>>> >>>>>>> >>>>>>> On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> >>>>>>> wrote: >>>>>>> >>>>>>>> Excellent, that's what I hoped. >>>>>>>> >>>>>>>> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen < >>>>>>>> ki...@op...> wrote: >>>>>>>> >>>>>>>>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>>>>>>>> >>>>>>>>> Is Virtuoso loading the contents of "xyz.db" into an internal >>>>>>>>> model, or is it performing transactions against "xyz.db"? >>>>>>>>> >>>>>>>>> >>>>>>>>> Persistence happens against "xyz.db". Other relations related >>>>>>>>> activity happens in memory. >>>>>>>>> >>>>>>>>> Kingsley >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe < >>>>>>>>> sdm...@gm...> wrote: >>>>>>>>> >>>>>>>>>> Hi Kingsley, yes I would like to take this on and then implement >>>>>>>>>> a smart contract-based persistence with it. Can you suggest starting >>>>>>>>>> points or what work has been done so far? >>>>>>>>>> >>>>>>>>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>>>>>>>> ki...@op...> wrote: >>>>>>>>>> >>>>>>>>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I would like to know what is the feasibility of implementing >>>>>>>>>>> Virtuoso's "state" one a blockchain like eos.io? If for >>>>>>>>>>> instance Virt is storing its quad-store state as a linked list, then if I >>>>>>>>>>> wanted to swap that datatype out for a custom, EOSLinkedList, how difficult >>>>>>>>>>> would that be? Does Virtuoso support a swappable persistence API of any >>>>>>>>>>> sort, or interfaces for the data types it uses for persistence? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We are looking at loose-coupling of persistent storage i.e., >>>>>>>>>>> rather than being pegged to "xyz.db" you end up being able to use a variety >>>>>>>>>>> of document types (CSV, ORC, Parquet etc..). >>>>>>>>>>> >>>>>>>>>>> Naturally, anyone can pick up this effort to accelerate matters >>>>>>>>>>> :) >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Kingsley Idehen >>>>>>>>>>> Founder & CEO >>>>>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>>>>> >>>>>>>>>>> Weblogs (Blogs): >>>>>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>>>>> >>>>>>>>>>> Profile Pages: >>>>>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>>>>> >>>>>>>>>>> Web Identities (WebID): >>>>>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ------------------------------------------------------------ >>>>>>>>>>> ------------------ >>>>>>>>>>> Check out the vibrant tech community on one of the world's most >>>>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Virtuoso-devel mailing list >>>>>>>>>>> Vir...@li... >>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> -sherman >>>>>>>>>> >>>>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>>>> shadow of turning. >>>>>>>>>> (James 1:17) >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> -sherman >>>>>>>>> >>>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>>> shadow of turning. >>>>>>>>> (James 1:17) >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Kingsley Idehen >>>>>>>>> Founder & CEO >>>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>>> >>>>>>>>> Weblogs (Blogs): >>>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>>> >>>>>>>>> Profile Pages: >>>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>>> >>>>>>>>> Web Identities (WebID): >>>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -sherman >>>>>>>> >>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>> shadow of turning. >>>>>>>> (James 1:17) >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks, >>>>>>> -sherman >>>>>>> >>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>> shadow of turning. >>>>>>> (James 1:17) >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Kingsley Idehen >>>>>>> Founder & CEO >>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>> >>>>>>> Weblogs (Blogs): >>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>> >>>>>>> Profile Pages: >>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>> Twitter: https://twitter.com/kidehen >>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>> >>>>>>> Web Identities (WebID): >>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> -sherman >>>>>> >>>>>> Every good gift and every perfect gift is from above, and cometh down >>>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>>> turning. >>>>>> (James 1:17) >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> -sherman >>>>> >>>>> Every good gift and every perfect gift is from above, and cometh down >>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>> turning. >>>>> (James 1:17) >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > LinkedIn: http://www.linkedin.com/in/kidehen > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Kingsley I. <ki...@op...> - 2017-12-01 21:45:23
|
On 12/1/17 1:22 PM, Sherman Monroe wrote: > As an example of how a disk write will work for the eos.io > <http://eos.io> adaptor, the buffer used by disk.cwill pass the write > onsetand the total_bytesto the eos blockchain and this transaction > (and nothing more) will be recorded on the blockchain, respecting its > index in the sequence of i/o transactions. One a read request, the > buffer will pass the read onsetand lengthto eos. To derive the current > state of the file inside that char window, it smart contract will do > the equivalent of running the transaction history from beginning to > end (this will happens using the blockchain's built-in parallel > processing), but as it runs each transaction/state transition, it will > only consider (i.e. read and record) the bytes that changed within the > requested window. It will then return the state/content in the window > back to the adaptor which passes the content back to the buffer. > > In eos.io <http://eos.io>, only the messages (the inputs, which in > this case are the file write/delete/insert requests) are stored on the > chain, not the file itself (the output). The state of the state > machine at any given moment is therefore only implicit (in the > message/transaction history). With knowledge of the message history > and a description of the state transition logic, one can derive the > state on demand. I propose using the state machine (the blockchain) to > model a file of infinite length. > > The adaptor/virtual file interface allows different approaches to be > used to take advantage of the characteristics of each persistence > strategy, so a different approach can be used for Hadoop, etc. > Why don't you move this thread to the github repo for VOS. You can open an issue and dialog can be built around that. Kingsley > On Fri, Dec 1, 2017 at 11:55 AM, Sherman Monroe <sdm...@gm... > <mailto:sdm...@gm...>> wrote: > > fifo special files > <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> is > a named pipe, I must have missed that. > > On Fri, Dec 1, 2017 at 10:32 AM, Sherman Monroe > <sdm...@gm... <mailto:sdm...@gm...>> wrote: > > After reviewing the dbe_storage_s struct and the code in > disk.c, it is apparent that disk.cis the *.dbfile manager of > Virtuoso, and this file manager is sufficiently sophisticated > as to elicit an alternative to extracting a programming > interface from disk.c. So my preliminary proposal is to leave > the disk.ccode alone and instead make virtuoso.dba virtual > file such that reading from the file actually reads from the > stdoutof a command and writing to the file actually writes to > the stdinof a command. Then the commands in question will > interact with the persistent store of choice, including the > eos.io <http://eos.io> smart contract running on the > blockchain. Some additional requirements are: > > - Low level-of-effort, so implementing a custom file system > (e.g. fuse <http://fuse.sourceforge.net/>) is out of the question > - Must be cross platform, so e.g. linux fifo special files > <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> > are out of the question > - Preserves performance, i.e. the solution cannot adversely > impact Virtuoso's current performance metrics > > After some preliminary research, writing to a named pipe > <https://en.wikipedia.org/wiki/Named_pipe> looks very > promising. The downside is that you can't have concurrent > processes reading/listening to the file, but AFAIK only one > process is accessing virtuoso.dbat any given time. Once the > content is read its gone so disk.cwill need to treat it like a > buffered stream (e.g. the writer will continuously push the > content to the buffer, and the reader will loop the buffered > content indefinitely). > > The command writing to the buffer is what I'll call an > adaptor. It's job is to persist the state of the "virtual > file" based on what it receives from Virtuoso and to > fulfill Virtuoso's reads. This way, the adaptor is agnostic to > the contents of virtuoso.dband its logic only needs to handle > the details of the persistence provider. > > What are your thoughts? I'm sure I've overlooked requirements > you envision, if so please share those also. > > > On Thu, Nov 30, 2017 at 2:34 PM, Sherman Monroe > <sdm...@gm... <mailto:sdm...@gm...>> wrote: > > Correction, dbe_storage_s.dbs_file > > On Thu, Nov 30, 2017 at 1:43 PM, Sherman Monroe > <sdm...@gm... <mailto:sdm...@gm...>> wrote: > > I've found dbe_storage_s and dbe_storage_s->dbs_file. > Design proposal en route... > > On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe > <sdm...@gm... <mailto:sdm...@gm...>> wrote: > > Ok, I will review the codebase and begin the > design. I'll post for peer review before I begin > coding. > > On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen > <ki...@op... > <mailto:ki...@op...>> wrote: > > On 11/29/17 6:47 PM, Sherman Monroe wrote: >> So as a starting point, I abstract every >> single call to .db and also any dependent >> functions dealing with in-memory state. Next >> I implement the old "file backed" persistence >> using the new API, and run test cases to >> ensure nothing broke. Next I test it on some >> common alternative types like the ones you >> list above. Finally I implement the eos.io >> <http://eos.io> backed persistence. Does this >> sum up the requirements? > A 2K feet, yes. > > > Kingsley >> >> On Wed, Nov 29, 2017 at 5:40 PM, Sherman >> Monroe <sdm...@gm... >> <mailto:sdm...@gm...>> wrote: >> >> Excellent, that's what I hoped. >> >> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley >> Idehen <ki...@op... >> <mailto:ki...@op...>> wrote: >> >> On 11/29/17 6:35 PM, Sherman Monroe >> wrote: >>> Is Virtuoso loading the contents of >>> "xyz.db" into an internal model, or >>> is it performing transactions >>> against "xyz.db"? >> >> Persistence happens against "xyz.db". >> Other relations related activity >> happens in memory. >> >> Kingsley >>> >>> On Wed, Nov 29, 2017 at 5:18 PM, >>> Sherman Monroe <sdm...@gm... >>> <mailto:sdm...@gm...>> wrote: >>> >>> Hi Kingsley, yes I would like >>> to take this on and then >>> implement a smart contract-based >>> persistence with it. Can you >>> suggest starting points or what >>> work has been done so far? >>> >>> On Wed, Nov 29, 2017 at 4:33 PM, >>> Kingsley Idehen >>> <ki...@op... >>> <mailto:ki...@op...>> >>> wrote: >>> >>> On 11/29/17 5:09 PM, Sherman >>> Monroe wrote: >>>> Hi, >>>> >>>> I would like to know what >>>> is the feasibility of >>>> implementing Virtuoso's >>>> "state" one a blockchain >>>> like eos.io >>>> <http://eos.io>? If for >>>> instance Virt is storing >>>> its quad-store state as a >>>> linked list, then if I >>>> wanted to swap that >>>> datatype out for a custom, >>>> EOSLinkedList, how >>>> difficult would that be? >>>> Does Virtuoso support a >>>> swappable persistence API >>>> of any sort, or interfaces >>>> for the data types it uses >>>> for persistence? >>> >>> >>> We are looking at >>> loose-coupling of persistent >>> storage i.e., rather than >>> being pegged to "xyz.db" you >>> end up being able to use a >>> variety of document types >>> (CSV, ORC, Parquet etc..). >>> >>> Naturally, anyone can pick >>> up this effort to accelerate >>> matters :) >>> >>> -- >>> Regards, >>> >>> Kingsley Idehen >>> Founder & CEO >>> OpenLink Software (Home Page: http://www.openlinksw.com) >>> >>> Weblogs (Blogs): >>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>> <http://www.openlinksw.com/blog/%7Ekidehen/> >>> Blogspot Blog: http://kidehen.blogspot.com >>> Medium Blog: https://medium.com/@kidehen >>> >>> Profile Pages: >>> Pinterest: https://www.pinterest.com/kidehen/ >>> <https://www.pinterest.com/kidehen/> >>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>> <https://www.quora.com/profile/Kingsley-Uyi-Idehen> >>> Twitter: https://twitter.com/kidehen >>> Google+: https://plus.google.com/+KingsleyIdehen/about >>> <https://plus.google.com/+KingsleyIdehen/about> >>> LinkedIn: http://www.linkedin.com/in/kidehen >>> <http://www.linkedin.com/in/kidehen> >>> >>> Web Identities (WebID): >>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>> <http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i> >>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>> <http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this> >>> >>> >>> ------------------------------------------------------------------------------ >>> Check out the vibrant tech >>> community on one of the >>> world's most >>> engaging tech sites, >>> Slashdot.org! >>> http://sdm.link/slashdot >>> _______________________________________________ >>> Virtuoso-devel mailing list >>> Vir...@li... >>> <mailto:Vir...@li...> >>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>> <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> >>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every >>> perfect gift is from above, and >>> cometh down from the Father of >>> lights, with whom is no >>> variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect >>> gift is from above, and cometh down >>> from the Father of lights, with whom >>> is no variableness, neither shadow >>> of turning. >>> (James 1:17) >> >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software (Home Page: http://www.openlinksw.com) >> >> Weblogs (Blogs): >> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >> <http://www.openlinksw.com/blog/%7Ekidehen/> >> Blogspot Blog: http://kidehen.blogspot.com >> Medium Blog: https://medium.com/@kidehen >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> <https://www.pinterest.com/kidehen/> >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> <https://www.quora.com/profile/Kingsley-Uyi-Idehen> >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> <https://plus.google.com/+KingsleyIdehen/about> >> LinkedIn: http://www.linkedin.com/in/kidehen >> <http://www.linkedin.com/in/kidehen> >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> <http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i> >> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> <http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this> >> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is >> from above, and cometh down from the >> Father of lights, with whom is no >> variableness, neither shadow of turning. >> (James 1:17) >> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is >> from above, and cometh down from the Father >> of lights, with whom is no variableness, >> neither shadow of turning. >> (James 1:17) > > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > <http://www.openlinksw.com/blog/%7Ekidehen/> > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > <https://www.pinterest.com/kidehen/> > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > <https://www.quora.com/profile/Kingsley-Uyi-Idehen> > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > <https://plus.google.com/+KingsleyIdehen/about> > LinkedIn: http://www.linkedin.com/in/kidehen > <http://www.linkedin.com/in/kidehen> > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > <http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i> > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > <http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this> > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from > above, and cometh down from the Father of lights, > with whom is no variableness, neither shadow of > turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, > and cometh down from the Father of lights, with whom > is no variableness, neither shadow of turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and > cometh down from the Father of lights, with whom is no > variableness, neither shadow of turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and > cometh down from the Father of lights, with whom is no > variableness, neither shadow of turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh > down from the Father of lights, with whom is no variableness, > neither shadow of turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down > from the Father of lights, with whom is no variableness, neither > shadow of turning. > (James 1:17) -- Regards, Kingsley Idehen Founder & CEO OpenLink Software (Home Page: http://www.openlinksw.com) Weblogs (Blogs): Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ Blogspot Blog: http://kidehen.blogspot.com Medium Blog: https://medium.com/@kidehen Profile Pages: Pinterest: https://www.pinterest.com/kidehen/ Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen Twitter: https://twitter.com/kidehen Google+: https://plus.google.com/+KingsleyIdehen/about LinkedIn: http://www.linkedin.com/in/kidehen Web Identities (WebID): Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this |
|
From: Sherman M. <sdm...@gm...> - 2017-12-01 18:22:14
|
As an example of how a disk write will work for the eos.io adaptor, the buffer used by disk.c will pass the write onset and the total_bytes to the eos blockchain and this transaction (and nothing more) will be recorded on the blockchain, respecting its index in the sequence of i/o transactions. One a read request, the buffer will pass the read onset and length to eos. To derive the current state of the file inside that char window, it smart contract will do the equivalent of running the transaction history from beginning to end (this will happens using the blockchain's built-in parallel processing), but as it runs each transaction/state transition, it will only consider (i.e. read and record) the bytes that changed within the requested window. It will then return the state/content in the window back to the adaptor which passes the content back to the buffer. In eos.io, only the messages (the inputs, which in this case are the file write/delete/insert requests) are stored on the chain, not the file itself (the output). The state of the state machine at any given moment is therefore only implicit (in the message/transaction history). With knowledge of the message history and a description of the state transition logic, one can derive the state on demand. I propose using the state machine (the blockchain) to model a file of infinite length. The adaptor/virtual file interface allows different approaches to be used to take advantage of the characteristics of each persistence strategy, so a different approach can be used for Hadoop, etc. On Fri, Dec 1, 2017 at 11:55 AM, Sherman Monroe <sdm...@gm...> wrote: > fifo special files > <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> is > a named pipe, I must have missed that. > > On Fri, Dec 1, 2017 at 10:32 AM, Sherman Monroe <sdm...@gm...> > wrote: > >> After reviewing the dbe_storage_s struct and the code in disk.c, it is >> apparent that disk.c is the *.db file manager of Virtuoso, and this file >> manager is sufficiently sophisticated as to elicit an alternative to >> extracting a programming interface from disk.c. So my preliminary >> proposal is to leave the disk.c code alone and instead make virtuoso.db >> a virtual file such that reading from the file actually reads from the >> stdout of a command and writing to the file actually writes to the stdin >> of a command. Then the commands in question will interact with the >> persistent store of choice, including the eos.io smart contract running >> on the blockchain. Some additional requirements are: >> >> - Low level-of-effort, so implementing a custom file system (e.g. fuse >> <http://fuse.sourceforge.net/>) is out of the question >> - Must be cross platform, so e.g. linux fifo special files >> <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> >> are out of the question >> - Preserves performance, i.e. the solution cannot adversely impact >> Virtuoso's current performance metrics >> >> After some preliminary research, writing to a named pipe >> <https://en.wikipedia.org/wiki/Named_pipe> looks very promising. The >> downside is that you can't have concurrent processes reading/listening to >> the file, but AFAIK only one process is accessing virtuoso.db at any >> given time. Once the content is read its gone so disk.c will need to >> treat it like a buffered stream (e.g. the writer will continuously push the >> content to the buffer, and the reader will loop the buffered content >> indefinitely). >> >> The command writing to the buffer is what I'll call an adaptor. It's job >> is to persist the state of the "virtual file" based on what it receives >> from Virtuoso and to fulfill Virtuoso's reads. This way, the adaptor is >> agnostic to the contents of virtuoso.db and its logic only needs to >> handle the details of the persistence provider. >> >> What are your thoughts? I'm sure I've overlooked requirements you >> envision, if so please share those also. >> >> >> On Thu, Nov 30, 2017 at 2:34 PM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> Correction, dbe_storage_s.dbs_file >>> >>> On Thu, Nov 30, 2017 at 1:43 PM, Sherman Monroe <sdm...@gm...> >>> wrote: >>> >>>> I've found dbe_storage_s and dbe_storage_s->dbs_file. Design proposal >>>> en route... >>>> >>>> On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe <sdm...@gm...> >>>> wrote: >>>> >>>>> Ok, I will review the codebase and begin the design. I'll post for >>>>> peer review before I begin coding. >>>>> >>>>> On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen < >>>>> ki...@op...> wrote: >>>>> >>>>>> On 11/29/17 6:47 PM, Sherman Monroe wrote: >>>>>> >>>>>> So as a starting point, I abstract every single call to .db and also >>>>>> any dependent functions dealing with in-memory state. Next I implement the >>>>>> old "file backed" persistence using the new API, and run test cases to >>>>>> ensure nothing broke. Next I test it on some common alternative types like >>>>>> the ones you list above. Finally I implement the eos.io backed >>>>>> persistence. Does this sum up the requirements? >>>>>> >>>>>> A 2K feet, yes. >>>>>> >>>>>> >>>>>> Kingsley >>>>>> >>>>>> >>>>>> On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> >>>>>> wrote: >>>>>> >>>>>>> Excellent, that's what I hoped. >>>>>>> >>>>>>> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen < >>>>>>> ki...@op...> wrote: >>>>>>> >>>>>>>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>>>>>>> >>>>>>>> Is Virtuoso loading the contents of "xyz.db" into an internal >>>>>>>> model, or is it performing transactions against "xyz.db"? >>>>>>>> >>>>>>>> >>>>>>>> Persistence happens against "xyz.db". Other relations related >>>>>>>> activity happens in memory. >>>>>>>> >>>>>>>> Kingsley >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm... >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hi Kingsley, yes I would like to take this on and then implement >>>>>>>>> a smart contract-based persistence with it. Can you suggest starting >>>>>>>>> points or what work has been done so far? >>>>>>>>> >>>>>>>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>>>>>>> ki...@op...> wrote: >>>>>>>>> >>>>>>>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I would like to know what is the feasibility of implementing >>>>>>>>>> Virtuoso's "state" one a blockchain like eos.io? If for instance >>>>>>>>>> Virt is storing its quad-store state as a linked list, then if I wanted to >>>>>>>>>> swap that datatype out for a custom, EOSLinkedList, how difficult would >>>>>>>>>> that be? Does Virtuoso support a swappable persistence API of any sort, or >>>>>>>>>> interfaces for the data types it uses for persistence? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We are looking at loose-coupling of persistent storage i.e., >>>>>>>>>> rather than being pegged to "xyz.db" you end up being able to use a variety >>>>>>>>>> of document types (CSV, ORC, Parquet etc..). >>>>>>>>>> >>>>>>>>>> Naturally, anyone can pick up this effort to accelerate matters >>>>>>>>>> :) >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Kingsley Idehen >>>>>>>>>> Founder & CEO >>>>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>>>> >>>>>>>>>> Weblogs (Blogs): >>>>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>>>> >>>>>>>>>> Profile Pages: >>>>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>>>> >>>>>>>>>> Web Identities (WebID): >>>>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ------------------------------------------------------------ >>>>>>>>>> ------------------ >>>>>>>>>> Check out the vibrant tech community on one of the world's most >>>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>>>>>>> _______________________________________________ >>>>>>>>>> Virtuoso-devel mailing list >>>>>>>>>> Vir...@li... >>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> -sherman >>>>>>>>> >>>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>>> shadow of turning. >>>>>>>>> (James 1:17) >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -sherman >>>>>>>> >>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>> shadow of turning. >>>>>>>> (James 1:17) >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> >>>>>>>> Kingsley Idehen >>>>>>>> Founder & CEO >>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>> >>>>>>>> Weblogs (Blogs): >>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>> >>>>>>>> Profile Pages: >>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>> >>>>>>>> Web Identities (WebID): >>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks, >>>>>>> -sherman >>>>>>> >>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>> shadow of turning. >>>>>>> (James 1:17) >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> -sherman >>>>>> >>>>>> Every good gift and every perfect gift is from above, and cometh down >>>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>>> turning. >>>>>> (James 1:17) >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Kingsley Idehen >>>>>> Founder & CEO >>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>> >>>>>> Weblogs (Blogs): >>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>> Medium Blog: https://medium.com/@kidehen >>>>>> >>>>>> Profile Pages: >>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>> Twitter: https://twitter.com/kidehen >>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>> >>>>>> Web Identities (WebID): >>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> -sherman >>>>> >>>>> Every good gift and every perfect gift is from above, and cometh down >>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>> turning. >>>>> (James 1:17) >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Sherman M. <sdm...@gm...> - 2017-12-01 17:55:19
|
fifo special files <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> is a named pipe, I must have missed that. On Fri, Dec 1, 2017 at 10:32 AM, Sherman Monroe <sdm...@gm...> wrote: > After reviewing the dbe_storage_s struct and the code in disk.c, it is > apparent that disk.c is the *.db file manager of Virtuoso, and this file > manager is sufficiently sophisticated as to elicit an alternative to > extracting a programming interface from disk.c. So my preliminary > proposal is to leave the disk.c code alone and instead make virtuoso.db a > virtual file such that reading from the file actually reads from the > stdout of a command and writing to the file actually writes to the stdin > of a command. Then the commands in question will interact with the > persistent store of choice, including the eos.io smart contract running > on the blockchain. Some additional requirements are: > > - Low level-of-effort, so implementing a custom file system (e.g. fuse > <http://fuse.sourceforge.net/>) is out of the question > - Must be cross platform, so e.g. linux fifo special files > <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> > are out of the question > - Preserves performance, i.e. the solution cannot adversely impact > Virtuoso's current performance metrics > > After some preliminary research, writing to a named pipe > <https://en.wikipedia.org/wiki/Named_pipe> looks very promising. The > downside is that you can't have concurrent processes reading/listening to > the file, but AFAIK only one process is accessing virtuoso.db at any > given time. Once the content is read its gone so disk.c will need to > treat it like a buffered stream (e.g. the writer will continuously push the > content to the buffer, and the reader will loop the buffered content > indefinitely). > > The command writing to the buffer is what I'll call an adaptor. It's job > is to persist the state of the "virtual file" based on what it receives > from Virtuoso and to fulfill Virtuoso's reads. This way, the adaptor is > agnostic to the contents of virtuoso.db and its logic only needs to > handle the details of the persistence provider. > > What are your thoughts? I'm sure I've overlooked requirements you > envision, if so please share those also. > > > On Thu, Nov 30, 2017 at 2:34 PM, Sherman Monroe <sdm...@gm...> > wrote: > >> Correction, dbe_storage_s.dbs_file >> >> On Thu, Nov 30, 2017 at 1:43 PM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> I've found dbe_storage_s and dbe_storage_s->dbs_file. Design proposal >>> en route... >>> >>> On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe <sdm...@gm...> >>> wrote: >>> >>>> Ok, I will review the codebase and begin the design. I'll post for >>>> peer review before I begin coding. >>>> >>>> On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen < >>>> ki...@op...> wrote: >>>> >>>>> On 11/29/17 6:47 PM, Sherman Monroe wrote: >>>>> >>>>> So as a starting point, I abstract every single call to .db and also >>>>> any dependent functions dealing with in-memory state. Next I implement the >>>>> old "file backed" persistence using the new API, and run test cases to >>>>> ensure nothing broke. Next I test it on some common alternative types like >>>>> the ones you list above. Finally I implement the eos.io backed >>>>> persistence. Does this sum up the requirements? >>>>> >>>>> A 2K feet, yes. >>>>> >>>>> >>>>> Kingsley >>>>> >>>>> >>>>> On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> >>>>> wrote: >>>>> >>>>>> Excellent, that's what I hoped. >>>>>> >>>>>> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen < >>>>>> ki...@op...> wrote: >>>>>> >>>>>>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>>>>>> >>>>>>> Is Virtuoso loading the contents of "xyz.db" into an internal model, >>>>>>> or is it performing transactions against "xyz.db"? >>>>>>> >>>>>>> >>>>>>> Persistence happens against "xyz.db". Other relations related >>>>>>> activity happens in memory. >>>>>>> >>>>>>> Kingsley >>>>>>> >>>>>>> >>>>>>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Kingsley, yes I would like to take this on and then implement a >>>>>>>> smart contract-based persistence with it. Can you suggest starting points >>>>>>>> or what work has been done so far? >>>>>>>> >>>>>>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>>>>>> ki...@op...> wrote: >>>>>>>> >>>>>>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I would like to know what is the feasibility of implementing >>>>>>>>> Virtuoso's "state" one a blockchain like eos.io? If for instance >>>>>>>>> Virt is storing its quad-store state as a linked list, then if I wanted to >>>>>>>>> swap that datatype out for a custom, EOSLinkedList, how difficult would >>>>>>>>> that be? Does Virtuoso support a swappable persistence API of any sort, or >>>>>>>>> interfaces for the data types it uses for persistence? >>>>>>>>> >>>>>>>>> >>>>>>>>> We are looking at loose-coupling of persistent storage i.e., >>>>>>>>> rather than being pegged to "xyz.db" you end up being able to use a variety >>>>>>>>> of document types (CSV, ORC, Parquet etc..). >>>>>>>>> >>>>>>>>> Naturally, anyone can pick up this effort to accelerate matters :) >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Kingsley Idehen >>>>>>>>> Founder & CEO >>>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>>> >>>>>>>>> Weblogs (Blogs): >>>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>>> >>>>>>>>> Profile Pages: >>>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>>> >>>>>>>>> Web Identities (WebID): >>>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>>> >>>>>>>>> >>>>>>>>> ------------------------------------------------------------ >>>>>>>>> ------------------ >>>>>>>>> Check out the vibrant tech community on one of the world's most >>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>>>>>> _______________________________________________ >>>>>>>>> Virtuoso-devel mailing list >>>>>>>>> Vir...@li... >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Thanks, >>>>>>>> -sherman >>>>>>>> >>>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>>> shadow of turning. >>>>>>>> (James 1:17) >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks, >>>>>>> -sherman >>>>>>> >>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>> shadow of turning. >>>>>>> (James 1:17) >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> >>>>>>> Kingsley Idehen >>>>>>> Founder & CEO >>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>> >>>>>>> Weblogs (Blogs): >>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>> >>>>>>> Profile Pages: >>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>> Twitter: https://twitter.com/kidehen >>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>> >>>>>>> Web Identities (WebID): >>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> -sherman >>>>>> >>>>>> Every good gift and every perfect gift is from above, and cometh down >>>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>>> turning. >>>>>> (James 1:17) >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> -sherman >>>>> >>>>> Every good gift and every perfect gift is from above, and cometh down >>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>> turning. >>>>> (James 1:17) >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Kingsley Idehen >>>>> Founder & CEO >>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>> >>>>> Weblogs (Blogs): >>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>> Medium Blog: https://medium.com/@kidehen >>>>> >>>>> Profile Pages: >>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>> Twitter: https://twitter.com/kidehen >>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>> >>>>> Web Identities (WebID): >>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Sherman M. <sdm...@gm...> - 2017-12-01 16:32:43
|
After reviewing the dbe_storage_s struct and the code in disk.c, it is apparent that disk.c is the *.db file manager of Virtuoso, and this file manager is sufficiently sophisticated as to elicit an alternative to extracting a programming interface from disk.c. So my preliminary proposal is to leave the disk.c code alone and instead make virtuoso.db a virtual file such that reading from the file actually reads from the stdout of a command and writing to the file actually writes to the stdin of a command. Then the commands in question will interact with the persistent store of choice, including the eos.io smart contract running on the blockchain. Some additional requirements are: - Low level-of-effort, so implementing a custom file system (e.g. fuse <http://fuse.sourceforge.net/>) is out of the question - Must be cross platform, so e.g. linux fifo special files <http://www.gnu.org/software/libc/manual/html_node/FIFO-Special-Files.html> are out of the question - Preserves performance, i.e. the solution cannot adversely impact Virtuoso's current performance metrics After some preliminary research, writing to a named pipe <https://en.wikipedia.org/wiki/Named_pipe> looks very promising. The downside is that you can't have concurrent processes reading/listening to the file, but AFAIK only one process is accessing virtuoso.db at any given time. Once the content is read its gone so disk.c will need to treat it like a buffered stream (e.g. the writer will continuously push the content to the buffer, and the reader will loop the buffered content indefinitely). The command writing to the buffer is what I'll call an adaptor. It's job is to persist the state of the "virtual file" based on what it receives from Virtuoso and to fulfill Virtuoso's reads. This way, the adaptor is agnostic to the contents of virtuoso.db and its logic only needs to handle the details of the persistence provider. What are your thoughts? I'm sure I've overlooked requirements you envision, if so please share those also. On Thu, Nov 30, 2017 at 2:34 PM, Sherman Monroe <sdm...@gm...> wrote: > Correction, dbe_storage_s.dbs_file > > On Thu, Nov 30, 2017 at 1:43 PM, Sherman Monroe <sdm...@gm...> > wrote: > >> I've found dbe_storage_s and dbe_storage_s->dbs_file. Design proposal en >> route... >> >> On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> Ok, I will review the codebase and begin the design. I'll post for >>> peer review before I begin coding. >>> >>> On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen <ki...@op... >>> > wrote: >>> >>>> On 11/29/17 6:47 PM, Sherman Monroe wrote: >>>> >>>> So as a starting point, I abstract every single call to .db and also >>>> any dependent functions dealing with in-memory state. Next I implement the >>>> old "file backed" persistence using the new API, and run test cases to >>>> ensure nothing broke. Next I test it on some common alternative types like >>>> the ones you list above. Finally I implement the eos.io backed >>>> persistence. Does this sum up the requirements? >>>> >>>> A 2K feet, yes. >>>> >>>> >>>> Kingsley >>>> >>>> >>>> On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> >>>> wrote: >>>> >>>>> Excellent, that's what I hoped. >>>>> >>>>> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen < >>>>> ki...@op...> wrote: >>>>> >>>>>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>>>>> >>>>>> Is Virtuoso loading the contents of "xyz.db" into an internal model, >>>>>> or is it performing transactions against "xyz.db"? >>>>>> >>>>>> >>>>>> Persistence happens against "xyz.db". Other relations related >>>>>> activity happens in memory. >>>>>> >>>>>> Kingsley >>>>>> >>>>>> >>>>>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> >>>>>> wrote: >>>>>> >>>>>>> Hi Kingsley, yes I would like to take this on and then implement a >>>>>>> smart contract-based persistence with it. Can you suggest starting points >>>>>>> or what work has been done so far? >>>>>>> >>>>>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>>>>> ki...@op...> wrote: >>>>>>> >>>>>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I would like to know what is the feasibility of implementing >>>>>>>> Virtuoso's "state" one a blockchain like eos.io? If for instance >>>>>>>> Virt is storing its quad-store state as a linked list, then if I wanted to >>>>>>>> swap that datatype out for a custom, EOSLinkedList, how difficult would >>>>>>>> that be? Does Virtuoso support a swappable persistence API of any sort, or >>>>>>>> interfaces for the data types it uses for persistence? >>>>>>>> >>>>>>>> >>>>>>>> We are looking at loose-coupling of persistent storage i.e., rather >>>>>>>> than being pegged to "xyz.db" you end up being able to use a variety of >>>>>>>> document types (CSV, ORC, Parquet etc..). >>>>>>>> >>>>>>>> Naturally, anyone can pick up this effort to accelerate matters :) >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> >>>>>>>> Kingsley Idehen >>>>>>>> Founder & CEO >>>>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>>>> >>>>>>>> Weblogs (Blogs): >>>>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>>>> Medium Blog: https://medium.com/@kidehen >>>>>>>> >>>>>>>> Profile Pages: >>>>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>>>> Twitter: https://twitter.com/kidehen >>>>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>>>> >>>>>>>> Web Identities (WebID): >>>>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>>>> >>>>>>>> >>>>>>>> ------------------------------------------------------------ >>>>>>>> ------------------ >>>>>>>> Check out the vibrant tech community on one of the world's most >>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>>>>> _______________________________________________ >>>>>>>> Virtuoso-devel mailing list >>>>>>>> Vir...@li... >>>>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Thanks, >>>>>>> -sherman >>>>>>> >>>>>>> Every good gift and every perfect gift is from above, and cometh >>>>>>> down from the Father of lights, with whom is no variableness, neither >>>>>>> shadow of turning. >>>>>>> (James 1:17) >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks, >>>>>> -sherman >>>>>> >>>>>> Every good gift and every perfect gift is from above, and cometh down >>>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>>> turning. >>>>>> (James 1:17) >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Kingsley Idehen >>>>>> Founder & CEO >>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>> >>>>>> Weblogs (Blogs): >>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>> Medium Blog: https://medium.com/@kidehen >>>>>> >>>>>> Profile Pages: >>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>> Twitter: https://twitter.com/kidehen >>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>> >>>>>> Web Identities (WebID): >>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> -sherman >>>>> >>>>> Every good gift and every perfect gift is from above, and cometh down >>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>> turning. >>>>> (James 1:17) >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Kingsley Idehen >>>> Founder & CEO >>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>> >>>> Weblogs (Blogs): >>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>> Blogspot Blog: http://kidehen.blogspot.com >>>> Medium Blog: https://medium.com/@kidehen >>>> >>>> Profile Pages: >>>> Pinterest: https://www.pinterest.com/kidehen/ >>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>> Twitter: https://twitter.com/kidehen >>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>> >>>> Web Identities (WebID): >>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>> >>>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Sherman M. <sdm...@gm...> - 2017-11-30 19:43:45
|
I've found dbe_storage_s and dbe_storage_s->dbs_file. Design proposal en route... On Thu, Nov 30, 2017 at 8:24 AM, Sherman Monroe <sdm...@gm...> wrote: > Ok, I will review the codebase and begin the design. I'll post for peer > review before I begin coding. > > On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen <ki...@op...> > wrote: > >> On 11/29/17 6:47 PM, Sherman Monroe wrote: >> >> So as a starting point, I abstract every single call to .db and also any >> dependent functions dealing with in-memory state. Next I implement the old >> "file backed" persistence using the new API, and run test cases to ensure >> nothing broke. Next I test it on some common alternative types like the >> ones you list above. Finally I implement the eos.io backed persistence. >> Does this sum up the requirements? >> >> A 2K feet, yes. >> >> >> Kingsley >> >> >> On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> Excellent, that's what I hoped. >>> >>> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen <ki...@op... >>> > wrote: >>> >>>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>>> >>>> Is Virtuoso loading the contents of "xyz.db" into an internal model, or >>>> is it performing transactions against "xyz.db"? >>>> >>>> >>>> Persistence happens against "xyz.db". Other relations related activity >>>> happens in memory. >>>> >>>> Kingsley >>>> >>>> >>>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> >>>> wrote: >>>> >>>>> Hi Kingsley, yes I would like to take this on and then implement a >>>>> smart contract-based persistence with it. Can you suggest starting points >>>>> or what work has been done so far? >>>>> >>>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>>> ki...@op...> wrote: >>>>> >>>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I would like to know what is the feasibility of implementing >>>>>> Virtuoso's "state" one a blockchain like eos.io? If for instance >>>>>> Virt is storing its quad-store state as a linked list, then if I wanted to >>>>>> swap that datatype out for a custom, EOSLinkedList, how difficult would >>>>>> that be? Does Virtuoso support a swappable persistence API of any sort, or >>>>>> interfaces for the data types it uses for persistence? >>>>>> >>>>>> >>>>>> We are looking at loose-coupling of persistent storage i.e., rather >>>>>> than being pegged to "xyz.db" you end up being able to use a variety of >>>>>> document types (CSV, ORC, Parquet etc..). >>>>>> >>>>>> Naturally, anyone can pick up this effort to accelerate matters :) >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Kingsley Idehen >>>>>> Founder & CEO >>>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>>> >>>>>> Weblogs (Blogs): >>>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>>> Medium Blog: https://medium.com/@kidehen >>>>>> >>>>>> Profile Pages: >>>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>>> Twitter: https://twitter.com/kidehen >>>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>>> >>>>>> Web Identities (WebID): >>>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>>> >>>>>> >>>>>> ------------------------------------------------------------ >>>>>> ------------------ >>>>>> Check out the vibrant tech community on one of the world's most >>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>>> _______________________________________________ >>>>>> Virtuoso-devel mailing list >>>>>> Vir...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks, >>>>> -sherman >>>>> >>>>> Every good gift and every perfect gift is from above, and cometh down >>>>> from the Father of lights, with whom is no variableness, neither shadow of >>>>> turning. >>>>> (James 1:17) >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Kingsley Idehen >>>> Founder & CEO >>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>> >>>> Weblogs (Blogs): >>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>> Blogspot Blog: http://kidehen.blogspot.com >>>> Medium Blog: https://medium.com/@kidehen >>>> >>>> Profile Pages: >>>> Pinterest: https://www.pinterest.com/kidehen/ >>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>> Twitter: https://twitter.com/kidehen >>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>> >>>> Web Identities (WebID): >>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>> >>>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software (Home Page: http://www.openlinksw.com) >> >> Weblogs (Blogs): >> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >> Blogspot Blog: http://kidehen.blogspot.com >> Medium Blog: https://medium.com/@kidehen >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> LinkedIn: http://www.linkedin.com/in/kidehen >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> >> > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Sherman M. <sdm...@gm...> - 2017-11-30 14:24:20
|
Ok, I will review the codebase and begin the design. I'll post for peer review before I begin coding. On Wed, Nov 29, 2017 at 8:22 PM, Kingsley Idehen <ki...@op...> wrote: > On 11/29/17 6:47 PM, Sherman Monroe wrote: > > So as a starting point, I abstract every single call to .db and also any > dependent functions dealing with in-memory state. Next I implement the old > "file backed" persistence using the new API, and run test cases to ensure > nothing broke. Next I test it on some common alternative types like the > ones you list above. Finally I implement the eos.io backed persistence. > Does this sum up the requirements? > > A 2K feet, yes. > > > Kingsley > > > On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> > wrote: > >> Excellent, that's what I hoped. >> >> On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen <ki...@op...> >> wrote: >> >>> On 11/29/17 6:35 PM, Sherman Monroe wrote: >>> >>> Is Virtuoso loading the contents of "xyz.db" into an internal model, or >>> is it performing transactions against "xyz.db"? >>> >>> >>> Persistence happens against "xyz.db". Other relations related activity >>> happens in memory. >>> >>> Kingsley >>> >>> >>> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> >>> wrote: >>> >>>> Hi Kingsley, yes I would like to take this on and then implement a >>>> smart contract-based persistence with it. Can you suggest starting points >>>> or what work has been done so far? >>>> >>>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen < >>>> ki...@op...> wrote: >>>> >>>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>>> >>>>> Hi, >>>>> >>>>> I would like to know what is the feasibility of implementing >>>>> Virtuoso's "state" one a blockchain like eos.io? If for instance Virt >>>>> is storing its quad-store state as a linked list, then if I wanted to swap >>>>> that datatype out for a custom, EOSLinkedList, how difficult would that be? >>>>> Does Virtuoso support a swappable persistence API of any sort, or >>>>> interfaces for the data types it uses for persistence? >>>>> >>>>> >>>>> We are looking at loose-coupling of persistent storage i.e., rather >>>>> than being pegged to "xyz.db" you end up being able to use a variety of >>>>> document types (CSV, ORC, Parquet etc..). >>>>> >>>>> Naturally, anyone can pick up this effort to accelerate matters :) >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Kingsley Idehen >>>>> Founder & CEO >>>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>>> >>>>> Weblogs (Blogs): >>>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>>> Blogspot Blog: http://kidehen.blogspot.com >>>>> Medium Blog: https://medium.com/@kidehen >>>>> >>>>> Profile Pages: >>>>> Pinterest: https://www.pinterest.com/kidehen/ >>>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>>> Twitter: https://twitter.com/kidehen >>>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>>> >>>>> Web Identities (WebID): >>>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>>> >>>>> >>>>> ------------------------------------------------------------ >>>>> ------------------ >>>>> Check out the vibrant tech community on one of the world's most >>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>>> _______________________________________________ >>>>> Virtuoso-devel mailing list >>>>> Vir...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks, >>>> -sherman >>>> >>>> Every good gift and every perfect gift is from above, and cometh down >>>> from the Father of lights, with whom is no variableness, neither shadow of >>>> turning. >>>> (James 1:17) >>>> >>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >>> >>> -- >>> Regards, >>> >>> Kingsley Idehen >>> Founder & CEO >>> OpenLink Software (Home Page: http://www.openlinksw.com) >>> >>> Weblogs (Blogs): >>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>> Blogspot Blog: http://kidehen.blogspot.com >>> Medium Blog: https://medium.com/@kidehen >>> >>> Profile Pages: >>> Pinterest: https://www.pinterest.com/kidehen/ >>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>> Twitter: https://twitter.com/kidehen >>> Google+: https://plus.google.com/+KingsleyIdehen/about >>> LinkedIn: http://www.linkedin.com/in/kidehen >>> >>> Web Identities (WebID): >>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>> >>> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > LinkedIn: http://www.linkedin.com/in/kidehen > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Kingsley I. <ki...@op...> - 2017-11-30 02:22:12
|
On 11/29/17 6:47 PM, Sherman Monroe wrote: > So as a starting point, I abstract every single call to .db and also > any dependent functions dealing with in-memory state. Next I implement > the old "file backed" persistence using the new API, and run test > cases to ensure nothing broke. Next I test it on some common > alternative types like the ones you list above. Finally I implement > the eos.io <http://eos.io> backed persistence. Does this sum up the > requirements? A 2K feet, yes. Kingsley > > On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm... > <mailto:sdm...@gm...>> wrote: > > Excellent, that's what I hoped. > > On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen > <ki...@op... <mailto:ki...@op...>> wrote: > > On 11/29/17 6:35 PM, Sherman Monroe wrote: >> Is Virtuoso loading the contents of "xyz.db" into an internal >> model, or is it performing transactions against "xyz.db"? > > Persistence happens against "xyz.db". Other relations related > activity happens in memory. > > Kingsley >> >> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe >> <sdm...@gm... <mailto:sdm...@gm...>> wrote: >> >> Hi Kingsley, yes I would like to take this on and then >> implement a smart contract-based persistence with it. >> Can you suggest starting points or what work has been >> done so far? >> >> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen >> <ki...@op... <mailto:ki...@op...>> >> wrote: >> >> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>> Hi, >>> >>> I would like to know what is the feasibility of >>> implementing Virtuoso's "state" one a blockchain >>> like eos.io <http://eos.io>? If for instance Virt is >>> storing its quad-store state as a linked list, then >>> if I wanted to swap that datatype out for a custom, >>> EOSLinkedList, how difficult would that be? Does >>> Virtuoso support a swappable persistence API of any >>> sort, or interfaces for the data types it uses for >>> persistence? >> >> >> We are looking at loose-coupling of persistent >> storage i.e., rather than being pegged to "xyz.db" >> you end up being able to use a variety of document >> types (CSV, ORC, Parquet etc..). >> >> Naturally, anyone can pick up this effort to >> accelerate matters :) >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software (Home Page: http://www.openlinksw.com) >> >> Weblogs (Blogs): >> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >> <http://www.openlinksw.com/blog/%7Ekidehen/> >> Blogspot Blog: http://kidehen.blogspot.com >> Medium Blog: https://medium.com/@kidehen >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> <https://www.pinterest.com/kidehen/> >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> <https://www.quora.com/profile/Kingsley-Uyi-Idehen> >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> <https://plus.google.com/+KingsleyIdehen/about> >> LinkedIn: http://www.linkedin.com/in/kidehen >> <http://www.linkedin.com/in/kidehen> >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> <http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i> >> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> <http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this> >> >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the >> world's most >> engaging tech sites, Slashdot.org! >> http://sdm.link/slashdot >> _______________________________________________ >> Virtuoso-devel mailing list >> Vir...@li... >> <mailto:Vir...@li...> >> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >> <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> >> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and >> cometh down from the Father of lights, with whom is no >> variableness, neither shadow of turning. >> (James 1:17) >> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and >> cometh down from the Father of lights, with whom is no >> variableness, neither shadow of turning. >> (James 1:17) > > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > <http://www.openlinksw.com/blog/%7Ekidehen/> > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > <https://www.pinterest.com/kidehen/> > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > <https://www.quora.com/profile/Kingsley-Uyi-Idehen> > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > <https://plus.google.com/+KingsleyIdehen/about> > LinkedIn: http://www.linkedin.com/in/kidehen > <http://www.linkedin.com/in/kidehen> > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > <http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i> > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > <http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this> > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh > down from the Father of lights, with whom is no variableness, > neither shadow of turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down > from the Father of lights, with whom is no variableness, neither > shadow of turning. > (James 1:17) -- Regards, Kingsley Idehen Founder & CEO OpenLink Software (Home Page: http://www.openlinksw.com) Weblogs (Blogs): Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ Blogspot Blog: http://kidehen.blogspot.com Medium Blog: https://medium.com/@kidehen Profile Pages: Pinterest: https://www.pinterest.com/kidehen/ Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen Twitter: https://twitter.com/kidehen Google+: https://plus.google.com/+KingsleyIdehen/about LinkedIn: http://www.linkedin.com/in/kidehen Web Identities (WebID): Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this |
|
From: Sherman M. <sdm...@gm...> - 2017-11-29 23:48:02
|
So as a starting point, I abstract every single call to .db and also any dependent functions dealing with in-memory state. Next I implement the old "file backed" persistence using the new API, and run test cases to ensure nothing broke. Next I test it on some common alternative types like the ones you list above. Finally I implement the eos.io backed persistence. Does this sum up the requirements? On Wed, Nov 29, 2017 at 5:40 PM, Sherman Monroe <sdm...@gm...> wrote: > Excellent, that's what I hoped. > > On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen <ki...@op...> > wrote: > >> On 11/29/17 6:35 PM, Sherman Monroe wrote: >> >> Is Virtuoso loading the contents of "xyz.db" into an internal model, or >> is it performing transactions against "xyz.db"? >> >> >> Persistence happens against "xyz.db". Other relations related activity >> happens in memory. >> >> Kingsley >> >> >> On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> >> wrote: >> >>> Hi Kingsley, yes I would like to take this on and then implement a >>> smart contract-based persistence with it. Can you suggest starting points >>> or what work has been done so far? >>> >>> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen <ki...@op... >>> > wrote: >>> >>>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>>> >>>> Hi, >>>> >>>> I would like to know what is the feasibility of implementing Virtuoso's >>>> "state" one a blockchain like eos.io? If for instance Virt is storing >>>> its quad-store state as a linked list, then if I wanted to swap that >>>> datatype out for a custom, EOSLinkedList, how difficult would that be? Does >>>> Virtuoso support a swappable persistence API of any sort, or interfaces for >>>> the data types it uses for persistence? >>>> >>>> >>>> We are looking at loose-coupling of persistent storage i.e., rather >>>> than being pegged to "xyz.db" you end up being able to use a variety of >>>> document types (CSV, ORC, Parquet etc..). >>>> >>>> Naturally, anyone can pick up this effort to accelerate matters :) >>>> >>>> -- >>>> Regards, >>>> >>>> Kingsley Idehen >>>> Founder & CEO >>>> OpenLink Software (Home Page: http://www.openlinksw.com) >>>> >>>> Weblogs (Blogs): >>>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>>> Blogspot Blog: http://kidehen.blogspot.com >>>> Medium Blog: https://medium.com/@kidehen >>>> >>>> Profile Pages: >>>> Pinterest: https://www.pinterest.com/kidehen/ >>>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>>> Twitter: https://twitter.com/kidehen >>>> Google+: https://plus.google.com/+KingsleyIdehen/about >>>> LinkedIn: http://www.linkedin.com/in/kidehen >>>> >>>> Web Identities (WebID): >>>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>>> >>>> >>>> ------------------------------------------------------------ >>>> ------------------ >>>> Check out the vibrant tech community on one of the world's most >>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>>> _______________________________________________ >>>> Virtuoso-devel mailing list >>>> Vir...@li... >>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>>> >>>> >>> >>> >>> -- >>> >>> Thanks, >>> -sherman >>> >>> Every good gift and every perfect gift is from above, and cometh down >>> from the Father of lights, with whom is no variableness, neither shadow of >>> turning. >>> (James 1:17) >>> >> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software (Home Page: http://www.openlinksw.com) >> >> Weblogs (Blogs): >> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >> Blogspot Blog: http://kidehen.blogspot.com >> Medium Blog: https://medium.com/@kidehen >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> LinkedIn: http://www.linkedin.com/in/kidehen >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> >> > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Sherman M. <sdm...@gm...> - 2017-11-29 23:40:32
|
Excellent, that's what I hoped. On Wed, Nov 29, 2017 at 5:38 PM, Kingsley Idehen <ki...@op...> wrote: > On 11/29/17 6:35 PM, Sherman Monroe wrote: > > Is Virtuoso loading the contents of "xyz.db" into an internal model, or is > it performing transactions against "xyz.db"? > > > Persistence happens against "xyz.db". Other relations related activity > happens in memory. > > Kingsley > > > On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> > wrote: > >> Hi Kingsley, yes I would like to take this on and then implement a smart >> contract-based persistence with it. Can you suggest starting points or >> what work has been done so far? >> >> On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen <ki...@op...> >> wrote: >> >>> On 11/29/17 5:09 PM, Sherman Monroe wrote: >>> >>> Hi, >>> >>> I would like to know what is the feasibility of implementing Virtuoso's >>> "state" one a blockchain like eos.io? If for instance Virt is storing >>> its quad-store state as a linked list, then if I wanted to swap that >>> datatype out for a custom, EOSLinkedList, how difficult would that be? Does >>> Virtuoso support a swappable persistence API of any sort, or interfaces for >>> the data types it uses for persistence? >>> >>> >>> We are looking at loose-coupling of persistent storage i.e., rather than >>> being pegged to "xyz.db" you end up being able to use a variety of document >>> types (CSV, ORC, Parquet etc..). >>> >>> Naturally, anyone can pick up this effort to accelerate matters :) >>> >>> -- >>> Regards, >>> >>> Kingsley Idehen >>> Founder & CEO >>> OpenLink Software (Home Page: http://www.openlinksw.com) >>> >>> Weblogs (Blogs): >>> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >>> Blogspot Blog: http://kidehen.blogspot.com >>> Medium Blog: https://medium.com/@kidehen >>> >>> Profile Pages: >>> Pinterest: https://www.pinterest.com/kidehen/ >>> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >>> Twitter: https://twitter.com/kidehen >>> Google+: https://plus.google.com/+KingsleyIdehen/about >>> LinkedIn: http://www.linkedin.com/in/kidehen >>> >>> Web Identities (WebID): >>> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >>> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >>> >>> >>> ------------------------------------------------------------ >>> ------------------ >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> _______________________________________________ >>> Virtuoso-devel mailing list >>> Vir...@li... >>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >>> >>> >> >> >> -- >> >> Thanks, >> -sherman >> >> Every good gift and every perfect gift is from above, and cometh down >> from the Father of lights, with whom is no variableness, neither shadow of >> turning. >> (James 1:17) >> > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > LinkedIn: http://www.linkedin.com/in/kidehen > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Kingsley I. <ki...@op...> - 2017-11-29 23:38:41
|
On 11/29/17 6:35 PM, Sherman Monroe wrote: > Is Virtuoso loading the contents of "xyz.db" into an internal model, > or is it performing transactions against "xyz.db"? Persistence happens against "xyz.db". Other relations related activity happens in memory. Kingsley > > On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm... > <mailto:sdm...@gm...>> wrote: > > Hi Kingsley, yes I would like to take this on and then implement > a smart contract-based persistence with it. Can you suggest > starting points or what work has been done so far? > > On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen > <ki...@op... <mailto:ki...@op...>> wrote: > > On 11/29/17 5:09 PM, Sherman Monroe wrote: >> Hi, >> >> I would like to know what is the feasibility of implementing >> Virtuoso's "state" one a blockchain like eos.io >> <http://eos.io>? If for instance Virt is storing its >> quad-store state as a linked list, then if I wanted to swap >> that datatype out for a custom, EOSLinkedList, how difficult >> would that be? Does Virtuoso support a swappable persistence >> API of any sort, or interfaces for the data types it uses for >> persistence? > > > We are looking at loose-coupling of persistent storage i.e., > rather than being pegged to "xyz.db" you end up being able to > use a variety of document types (CSV, ORC, Parquet etc..). > > Naturally, anyone can pick up this effort to accelerate > matters :) > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > <http://www.openlinksw.com/blog/%7Ekidehen/> > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > <https://www.pinterest.com/kidehen/> > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > <https://www.quora.com/profile/Kingsley-Uyi-Idehen> > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > <https://plus.google.com/+KingsleyIdehen/about> > LinkedIn: http://www.linkedin.com/in/kidehen > <http://www.linkedin.com/in/kidehen> > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > <http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i> > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > <http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this> > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Virtuoso-devel mailing list > Vir...@li... > <mailto:Vir...@li...> > https://lists.sourceforge.net/lists/listinfo/virtuoso-devel > <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh > down from the Father of lights, with whom is no variableness, > neither shadow of turning. > (James 1:17) > > > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down > from the Father of lights, with whom is no variableness, neither > shadow of turning. > (James 1:17) -- Regards, Kingsley Idehen Founder & CEO OpenLink Software (Home Page: http://www.openlinksw.com) Weblogs (Blogs): Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ Blogspot Blog: http://kidehen.blogspot.com Medium Blog: https://medium.com/@kidehen Profile Pages: Pinterest: https://www.pinterest.com/kidehen/ Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen Twitter: https://twitter.com/kidehen Google+: https://plus.google.com/+KingsleyIdehen/about LinkedIn: http://www.linkedin.com/in/kidehen Web Identities (WebID): Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this |
|
From: Sherman M. <sdm...@gm...> - 2017-11-29 23:35:40
|
Is Virtuoso loading the contents of "xyz.db" into an internal model, or is it performing transactions against "xyz.db"? On Wed, Nov 29, 2017 at 5:18 PM, Sherman Monroe <sdm...@gm...> wrote: > Hi Kingsley, yes I would like to take this on and then implement a smart > contract-based persistence with it. Can you suggest starting points or > what work has been done so far? > > On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen <ki...@op...> > wrote: > >> On 11/29/17 5:09 PM, Sherman Monroe wrote: >> >> Hi, >> >> I would like to know what is the feasibility of implementing Virtuoso's >> "state" one a blockchain like eos.io? If for instance Virt is storing >> its quad-store state as a linked list, then if I wanted to swap that >> datatype out for a custom, EOSLinkedList, how difficult would that be? Does >> Virtuoso support a swappable persistence API of any sort, or interfaces for >> the data types it uses for persistence? >> >> >> We are looking at loose-coupling of persistent storage i.e., rather than >> being pegged to "xyz.db" you end up being able to use a variety of document >> types (CSV, ORC, Parquet etc..). >> >> Naturally, anyone can pick up this effort to accelerate matters :) >> >> -- >> Regards, >> >> Kingsley Idehen >> Founder & CEO >> OpenLink Software (Home Page: http://www.openlinksw.com) >> >> Weblogs (Blogs): >> Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ >> Blogspot Blog: http://kidehen.blogspot.com >> Medium Blog: https://medium.com/@kidehen >> >> Profile Pages: >> Pinterest: https://www.pinterest.com/kidehen/ >> Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen >> Twitter: https://twitter.com/kidehen >> Google+: https://plus.google.com/+KingsleyIdehen/about >> LinkedIn: http://www.linkedin.com/in/kidehen >> >> Web Identities (WebID): >> Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i >> : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this >> >> >> ------------------------------------------------------------ >> ------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> _______________________________________________ >> Virtuoso-devel mailing list >> Vir...@li... >> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel >> >> > > > -- > > Thanks, > -sherman > > Every good gift and every perfect gift is from above, and cometh down from > the Father of lights, with whom is no variableness, neither shadow of > turning. > (James 1:17) > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Sherman M. <sdm...@gm...> - 2017-11-29 23:18:50
|
Hi Kingsley, yes I would like to take this on and then implement a smart contract-based persistence with it. Can you suggest starting points or what work has been done so far? On Wed, Nov 29, 2017 at 4:33 PM, Kingsley Idehen <ki...@op...> wrote: > On 11/29/17 5:09 PM, Sherman Monroe wrote: > > Hi, > > I would like to know what is the feasibility of implementing Virtuoso's > "state" one a blockchain like eos.io? If for instance Virt is storing its > quad-store state as a linked list, then if I wanted to swap that datatype > out for a custom, EOSLinkedList, how difficult would that be? Does Virtuoso > support a swappable persistence API of any sort, or interfaces for the data > types it uses for persistence? > > > We are looking at loose-coupling of persistent storage i.e., rather than > being pegged to "xyz.db" you end up being able to use a variety of document > types (CSV, ORC, Parquet etc..). > > Naturally, anyone can pick up this effort to accelerate matters :) > > -- > Regards, > > Kingsley Idehen > Founder & CEO > OpenLink Software (Home Page: http://www.openlinksw.com) > > Weblogs (Blogs): > Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ > Blogspot Blog: http://kidehen.blogspot.com > Medium Blog: https://medium.com/@kidehen > > Profile Pages: > Pinterest: https://www.pinterest.com/kidehen/ > Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen > Twitter: https://twitter.com/kidehen > Google+: https://plus.google.com/+KingsleyIdehen/about > LinkedIn: http://www.linkedin.com/in/kidehen > > Web Identities (WebID): > Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i > : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this > > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Virtuoso-devel mailing list > Vir...@li... > https://lists.sourceforge.net/lists/listinfo/virtuoso-devel > > -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Kingsley I. <ki...@op...> - 2017-11-29 22:33:16
|
On 11/29/17 5:09 PM, Sherman Monroe wrote: > Hi, > > I would like to know what is the feasibility of implementing > Virtuoso's "state" one a blockchain like eos.io <http://eos.io>? If > for instance Virt is storing its quad-store state as a linked list, > then if I wanted to swap that datatype out for a custom, > EOSLinkedList, how difficult would that be? Does Virtuoso support a > swappable persistence API of any sort, or interfaces for the data > types it uses for persistence? We are looking at loose-coupling of persistent storage i.e., rather than being pegged to "xyz.db" you end up being able to use a variety of document types (CSV, ORC, Parquet etc..). Naturally, anyone can pick up this effort to accelerate matters :) -- Regards, Kingsley Idehen Founder & CEO OpenLink Software (Home Page: http://www.openlinksw.com) Weblogs (Blogs): Legacy Blog: http://www.openlinksw.com/blog/~kidehen/ Blogspot Blog: http://kidehen.blogspot.com Medium Blog: https://medium.com/@kidehen Profile Pages: Pinterest: https://www.pinterest.com/kidehen/ Quora: https://www.quora.com/profile/Kingsley-Uyi-Idehen Twitter: https://twitter.com/kidehen Google+: https://plus.google.com/+KingsleyIdehen/about LinkedIn: http://www.linkedin.com/in/kidehen Web Identities (WebID): Personal: http://kingsley.idehen.net/public_home/kidehen/profile.ttl#i : http://id.myopenlink.net/DAV/home/KingsleyUyiIdehen/Public/kingsley.ttl#this |
|
From: Sherman M. <sdm...@gm...> - 2017-11-29 22:09:09
|
Hi, I would like to know what is the feasibility of implementing Virtuoso's "state" one a blockchain like eos.io? If for instance Virt is storing its quad-store state as a linked list, then if I wanted to swap that datatype out for a custom, EOSLinkedList, how difficult would that be? Does Virtuoso support a swappable persistence API of any sort, or interfaces for the data types it uses for persistence? -- Thanks, -sherman Every good gift and every perfect gift is from above, and cometh down from the Father of lights, with whom is no variableness, neither shadow of turning. (James 1:17) |
|
From: Hugh W. <hwi...@op...> - 2016-12-13 11:32:34
|
Hi Ivan, In speaking to development they indicate it is possible to build with the -DLARGE_QU_INST flag and that will increase the number of available state slots by 16x, but this has never been fully tested and thus may lead to problems … Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers > On 11 Dec 2016, at 19:32, Ivan Rygaev <ir...@gm...> wrote: > > Hi Hugh > > Thank you for your answer. > > You are right. Originally my query was longer than 64K and I was getting this error: > Error SQ200 Query too large, variables in state over the limit > > Then I shortened it to the shortest size that generates the error. Now it is slightly less than the limit from your code example (query example in my first email is 63 892 bytes) and I'm getting a different error: > Virtuoso 37000 Error SQ156: Internal Optimized compiler error : Query too large, variables in state over the limit in ..\libsrc\Wi\sqlvec.c:3026. Please report the statement compiled. > > Can you investigate a bit more for the other (lower) limit? Is it also on query text length? > > Would it be dangerous removing these checks from the code (or increasing the limit)? > > Thank you. > > Ivan. > > > ----- Original message ----- > From: Hugh Williams <hwi...@op... <mailto:hwi...@op...>> > To: Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>> > Cc: vir...@li... <mailto:vir...@li...> > Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables in state over the limit > Date: Sat, 10 Dec 2016 13:28:19 +0000 > > Hi Ivan > > Can you provide a sample query that can be run to generate the error ? > > Seems there is about a 64K limit that your inserts queries are exceeded looking at sqlvec.c > > #define MAX_STATE_SLOTS 0xfffe > #endif > #define STATE_SLOT_LIMIT (MAX_STATE_SLOTS-500) > > if (sc->sc_cc->cc_super_cc->cc_instance_fill >= STATE_SLOT_LIMIT) > SQL_GPF_T1 (sc->sc_cc, "Query too large, variables in state over the limit”); > > > > > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ <http://www.openlinksw.com/> > Weblog -- http://www.openlinksw.com/blogs/ <http://www.openlinksw.com/blogs/> > LinkedIn -- http://www.linkedin.com/company/openlink-software/ <http://www.linkedin.com/company/openlink-software/> > Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink> > Google+ -- http://plus.google.com/100570109519069333827/ <http://plus.google.com/100570109519069333827/> > Facebook -- http://www.facebook.com/OpenLinkSoftware <http://www.facebook.com/OpenLinkSoftware> > Universal Data Access, Integration, and Management Technology Providers > > > > >> On 7 Dec 2016, at 15:29, Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>> wrote: >> >> So, what is the limit I'm encountering? >> >> >> ----- Original message ----- >> From: Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>> >> To: vir...@li... <mailto:vir...@li...> >> Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables in state over the limit >> Date: Mon, 05 Dec 2016 13:46:38 +0300 >> >> Hi William, >> >> I'm using snorql or direct program interface to /sparql endpoint. >> >> Through virtuoso /sparql page I also get a syntax error in the line 88, but I believe that's because this page truncates the query. >> >> >> ----- Original message ----- >> From: Hugh Williams <hwi...@op... <mailto:hwi...@op...>> >> To: Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>> >> Cc: vir...@li... <mailto:vir...@li...> >> Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables in state over the limit >> Date: Mon, 5 Dec 2016 10:10:26 +0000 >> >> Hi Ivan, >> >> How exactly are you attempting to execute this query, as when I try via the /sparql endpoint I get a syntax error at line 88 ? >> >> >> >> >> Best Regards >> Hugh Williams >> Professional Services >> OpenLink Software, Inc. // http://www.openlinksw.com/ <http://www.openlinksw.com/> >> Weblog -- http://www.openlinksw.com/blogs/ <http://www.openlinksw.com/blogs/> >> LinkedIn -- http://www.linkedin.com/company/openlink-software/ <http://www.linkedin.com/company/openlink-software/> >> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink> >> Google+ -- http://plus.google.com/100570109519069333827/ <http://plus.google.com/100570109519069333827/> >> Facebook -- http://www.facebook.com/OpenLinkSoftware <http://www.facebook.com/OpenLinkSoftware> >> Universal Data Access, Integration, and Management Technology Providers >> >> >> >> >>> On 29 Nov 2016, at 12:56, Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>> wrote: >>> >>> Hello! >>> >>> When I try to execute a number of inserts in one query I get an error: >>> Error SQ200 Query too large, variables in state over the limit: >>> >>> Or occasionally another one: >>> Virtuoso 37000 Error SQ156: Internal Optimized compiler error : Query too large, variables in state over the limit in ..\libsrc\Wi\sqlvec.c:3026. Please report the statement compiled. >>> >>> The list of inserts is not very large - less then 700 lines. See an example attached. If I remove just the last line it works fine. >>> >>> What is the cause of this issue? How can it be avoided? What is exactly the limit and on which parameter? The length of the query? Number of variables/terms? >>> >>> I'm using Virtuoso Open-Source 07.20.3217 Build: Apr 25 2016 on Windows. >>> >>> Thank you. >>> >>> Ivan. >>> >>> >>> >>> >>> >>> >>> <test_query.txt>------------------------------------------------------------------------------ >>> _______________________________________________ >>> Virtuoso-devel mailing list >>> Vir...@li... <mailto:Vir...@li...> >>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel <https://lists.sourceforge.net/lists/listinfo/virtuoso-devel> >> >> Email had 1 attachment: >> >> smime.p7s >> 3k (application/pkcs7-signature) >> >> ------------------------------------------------------------------------------ >> Developer Access Program for Intel Xeon Phi Processors >> Access to Intel Xeon Phi processor-based developer platforms. >> With one year of Intel Parallel Studio XE. >> Training and support from Colfax. >> Order your platform today.http://sdm.link/xeonphi_______________________________________________ <http://sdm.link/xeonphi_______________________________________________> >> Virtuoso-devel mailing list >> Vir...@li... <mailto:Vir...@li...> >> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel > > Email had 1 attachment: > > smime.p7s > 3k (application/pkcs7-signature) > > ------------------------------------------------------------------------------ > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today.http://sdm.link/xeonphi_______________________________________________ > Virtuoso-devel mailing list > Vir...@li... > https://lists.sourceforge.net/lists/listinfo/virtuoso-devel |
|
From: Ivan R. <ir...@gm...> - 2016-12-11 19:32:09
|
Hi Hugh
Thank you for your answer.
You are right. Originally my query was longer than 64K and I was getting
this error:
Error SQ200 Query too large, variables in state over the limit
Then I shortened it to the shortest size that generates the error. Now
it is slightly less than the limit from your code example (query example
in my first email is 63 892 bytes) and I'm getting a different error:
Virtuoso 37000 Error SQ156: Internal Optimized compiler error : Query
too large, variables in state over the limit in
..\libsrc\Wi\sqlvec.c:3026. Please report the statement compiled.
Can you investigate a bit more for the other (lower) limit? Is it also
on query text length?
Would it be dangerous removing these checks from the code (or increasing
the limit)?
Thank you.
Ivan.
----- Original message -----
From: Hugh Williams <hwi...@op...>
To: Ivan Rygaev <ir...@gm...>
Cc: vir...@li...
Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables in state over the limit
Date: Sat, 10 Dec 2016 13:28:19 +0000
Hi Ivan
Can you provide a sample query that can be run to generate the error ?
Seems there is about a 64K limit that your inserts queries are exceeded
looking at sqlvec.c
#define MAX_STATE_SLOTS 0xfffe
#endif
#define STATE_SLOT_LIMIT (MAX_STATE_SLOTS-500)
if (sc->sc_cc->cc_super_cc->cc_instance_fill >= STATE_SLOT_LIMIT)
SQL_GPF_T1 (sc->sc_cc, "Query too large, variables in state over
the limit”);
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 7 Dec 2016, at 15:29, Ivan Rygaev <ir...@gm...> wrote:
>
> So, what is the limit I'm encountering?
>
>
> ----- Original message -----
> From: Ivan Rygaev <ir...@gm...>
> To: vir...@li...
> Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables
> in state over the limit
> Date: Mon, 05 Dec 2016 13:46:38 +0300
>
> Hi William,
>
> I'm using snorql or direct program interface to /sparql endpoint.
>
> Through virtuoso /sparql page I also get a syntax error in the line
> 88, but I believe that's because this page truncates the query.
>
>
> ----- Original message -----
> From: Hugh Williams <hwi...@op...>
> To: Ivan Rygaev <ir...@gm...>
> Cc: vir...@li...
> Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables
> in state over the limit
> Date: Mon, 5 Dec 2016 10:10:26 +0000
>
> Hi Ivan,
>
> How exactly are you attempting to execute this query, as when I try
> via the /sparql endpoint I get a syntax error at line 88 ?
>
>
>
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. //
> http://www.openlinksw.com/
> Weblog -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter -- http://twitter.com/OpenLink
> Google+ -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology
> Providers
>
>
>
>
>> On 29 Nov 2016, at 12:56, Ivan Rygaev <ir...@gm...> wrote:
>>
>> Hello!
>>
>> When I try to execute a number of inserts in one query I get
>> an error:
>> Error SQ200 Query too large, variables in state over the limit:
>>
>> Or occasionally another one:
>> Virtuoso 37000 Error SQ156: Internal Optimized compiler error : Query
>> too large, variables in state over the limit in
>> ..\libsrc\Wi\sqlvec.c:3026. Please report the statement compiled.
>>
>> The list of inserts is not very large - less then 700 lines. See an
>> example attached. If I remove just the last line it works fine.
>>
>> What is the cause of this issue? How can it be avoided? What is
>> exactly the limit and on which parameter? The length of the query?
>> Number of variables/terms?
>>
>> I'm using Virtuoso Open-Source 07.20.3217 Build: Apr 25 2016 on
>> Windows.
>>
>> Thank you.
>>
>> Ivan.
>>
>>
>>
>>
>>
>>
>> <test_query.txt>-----------------------------------------------------
>> -------------------------
>> _______________________________________________
>> Virtuoso-devel mailing list
>> Vir...@li...
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel
> Email had 1 attachment:
> * smime.p7s 3k (application/pkcs7-signature)
>
> ----------------------------------------------------------------------
> --------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.
> http://sdm.link/xeonphi_______________________________________________
> Virtuoso-devel mailing list
> Vir...@li...
> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel
Email had 1 attachment:
* smime.p7s 3k (application/pkcs7-signature)
|
|
From: Hugh W. <hwi...@op...> - 2016-12-10 13:28:29
|
Hi Ivan
Can you provide a sample query that can be run to generate the error ?
Seems there is about a 64K limit that your inserts queries are exceeded looking at sqlvec.c
#define MAX_STATE_SLOTS 0xfffe
#endif
#define STATE_SLOT_LIMIT (MAX_STATE_SLOTS-500)
if (sc->sc_cc->cc_super_cc->cc_instance_fill >= STATE_SLOT_LIMIT)
SQL_GPF_T1 (sc->sc_cc, "Query too large, variables in state over the limit”);
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 7 Dec 2016, at 15:29, Ivan Rygaev <ir...@gm...> wrote:
>
> So, what is the limit I'm encountering?
>
>
> ----- Original message -----
> From: Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>>
> To: vir...@li... <mailto:vir...@li...>
> Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables in state over the limit
> Date: Mon, 05 Dec 2016 13:46:38 +0300
>
> Hi William,
>
> I'm using snorql or direct program interface to /sparql endpoint.
>
> Through virtuoso /sparql page I also get a syntax error in the line 88, but I believe that's because this page truncates the query.
>
>
> ----- Original message -----
> From: Hugh Williams <hwi...@op... <mailto:hwi...@op...>>
> To: Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>>
> Cc: vir...@li... <mailto:vir...@li...>
> Subject: Re: [Virtuoso-devel] Error SQ200 Query too large, variables in state over the limit
> Date: Mon, 5 Dec 2016 10:10:26 +0000
>
> Hi Ivan,
>
> How exactly are you attempting to execute this query, as when I try via the /sparql endpoint I get a syntax error at line 88 ?
>
>
>
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/ <http://www.openlinksw.com/>
> Weblog -- http://www.openlinksw.com/blogs/ <http://www.openlinksw.com/blogs/>
> LinkedIn -- http://www.linkedin.com/company/openlink-software/ <http://www.linkedin.com/company/openlink-software/>
> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink>
> Google+ -- http://plus.google.com/100570109519069333827/ <http://plus.google.com/100570109519069333827/>
> Facebook -- http://www.facebook.com/OpenLinkSoftware <http://www.facebook.com/OpenLinkSoftware>
> Universal Data Access, Integration, and Management Technology Providers
>
>
>
>
>> On 29 Nov 2016, at 12:56, Ivan Rygaev <ir...@gm... <mailto:ir...@gm...>> wrote:
>>
>> Hello!
>>
>> When I try to execute a number of inserts in one query I get an error:
>> Error SQ200 Query too large, variables in state over the limit:
>>
>> Or occasionally another one:
>> Virtuoso 37000 Error SQ156: Internal Optimized compiler error : Query too large, variables in state over the limit in ..\libsrc\Wi\sqlvec.c:3026. Please report the statement compiled.
>>
>> The list of inserts is not very large - less then 700 lines. See an example attached. If I remove just the last line it works fine.
>>
>> What is the cause of this issue? How can it be avoided? What is exactly the limit and on which parameter? The length of the query? Number of variables/terms?
>>
>> I'm using Virtuoso Open-Source 07.20.3217 Build: Apr 25 2016 on Windows.
>>
>> Thank you.
>>
>> Ivan.
>>
>>
>>
>>
>>
>>
>> <test_query.txt>------------------------------------------------------------------------------
>> _______________________________________________
>> Virtuoso-devel mailing list
>> Vir...@li... <mailto:Vir...@li...>
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel
>
> Email had 1 attachment:
>
> smime.p7s
> 3k (application/pkcs7-signature)
>
> ------------------------------------------------------------------------------
> Developer Access Program for Intel Xeon Phi Processors
> Access to Intel Xeon Phi processor-based developer platforms.
> With one year of Intel Parallel Studio XE.
> Training and support from Colfax.
> Order your platform today.http://sdm.link/xeonphi_______________________________________________
> Virtuoso-devel mailing list
> Vir...@li...
> https://lists.sourceforge.net/lists/listinfo/virtuoso-devel
|
8 messages has been excluded from this view by a project administrator.