Menu
▾
▴
Re: [Exist-open] XQuery: comparing order of items in two sequences
|
From: Adam R. <ad...@ex...> - 2015-08-27 10:02:36
|
Josef, Yes you could write your business rules in XQuery if you like. For completeness, other options that come to mind include: 1) Using Schematron for your business rules which compiles down to XSLT, and the running it with transform:transform from XQuery. 2) Using Relaxng for both your validation and business rules. On 27 August 2015 at 10:57, Jozef Aerts <joz...@xm...> wrote: > That is very interesting! > > So we could do a schema validation in a first step and then check the > "business rules" which cannot be done by Schema by the remaining of the > XQuery ... > > Adam Retter <ad...@ex...> hat am 27. August 2015 um 11:42 geschrieben: > > > > >> >> Has anyone else experience with the use XQuery for validation of XML >> documents? Any examples where this is done? > > There is the `validation` XQuery extension module in eXist - > http://www.exist-db.org/exist/apps/doc/validation.xml#sect3 > >> It would be great if the FDA were using a native XML database for storing >> electronic submissions for new drugs, but I am afraid we are still far >> away >> from that. >> >> >> Thanks! Jozef >> >> Jens Østergaard Petersen <oe...@gm...> hat am 25. August 2015 um >> 10:58 >> geschrieben: >> >> >> Well, that was a whole lot shorter! >> >> And if one does not like the string-thing, one could run >> >> functx:sequence-deep-equal($vars, $new-list) >> >> instead. >> >> >> Jens >> >> On 25 Aug 2015 at 10:25:16, Alister Pillow (gsp...@gm...) wrote: >> >> >> From the dumb ideas department >> >> declare function local:compare($identifiers, $vars) { >> let $new-list := >> for $id in $identifiers >> where $id = $vars >> return $id >> return string-join($vars,'') eq string-join($new-list, '') >> }; >> >> let $identifiers := >> >> ('STUDYID','DOMAIN','USUBJID','POOLID','SPDEVID','AESEQ','AEGRPID','AEREFID','AESPID','AELNKID','AELNKGRP') >> let $vars := ('STUDYID','DOMAIN','SPDEVID','AESEQ','AESPID') >> let $vars-b := ('STUDYID','DOMAIN','AESEQ','AESPID','SPDEVID') >> >> return (local:compare($identifiers,$vars), >> local:compare($identifiers,$vars-b)) >> >> >> >> On 25 Aug 2015, at 5:30 pm, Jozef Aerts <joz...@xm...> >> wrote: >> >> Thanks Jens! >> >> >> I learn a lot today ... >> >> Regarding your question on eliminating duplicates: we don't need to do. >> The $identifiers list is hardcoded, it comes from a standards >> specification >> from CDISC for submissions of clinical studies to the FDA (copy-past from >> a >> PDF!). In fact, this list is much longer than what I showed... >> >> The $vars comes from an XML file (the so-called define.xml) containing the >> metadata of the submission, i.e. describing all the variables that are >> used >> in the submission. These must come out of the list $identifiiers and >> follow >> the same order. In the define.xml, we check for duplicates using >> Schematron >> - but one could of course use XQuery for that. Anyway, in case there are >> duplicates, this violates the standard and the define.xml file is in >> principle invalid and should never make it to any further tests. >> >> >> With best regards, >> >> >> Jozef >> >> >> P.S. Some background information about this project: there are a few >> hundred >> of "rules" (mostly plausibility rules) for the data that is submitted to >> the >> FDA. These have been published as Excel worksheets (not >> machine-executable) >> and we are currently trying to write all these rules as XQuery >> (human-readable AND machine-executable AND vendor-neutral). These are then >> made available over a RESTful web service. Some background information can >> be found at: >> >> http://cdiscguru.blogspot.com/2014/12/machine-executable-fda-rules-for-sdtm.html >> and http://xml4pharmaserver.com/WebServices/XQueryRules_webservices.html >> >> Jens Østergaard Petersen <oe...@gm...> hat am 25. August 2015 um >> 09:29 >> geschrieben: >> >> Hi Jozef >> >> It’s much simpler than what you are trying to do. >> >> I have put come comments in my variation of the approach used by Adam. >> >> let $identifiers := >> ('STUDYID', 'DOMAIN', 'USUBJID', 'POOLID', 'SPDEVID', 'AESEQ', >> 'AEGRPID', 'AEREFID', 'AESPID', 'AELNKID', 'AELNKGRP') >> let $vars := ('STUDYID', 'DOMAIN', 'SPDEVID', 'AESEQ', 'AESPID') >> (: let $vars := ('STUDYID', 'SPDEVID', 'DOMAIN', 'AESEQ', 'AESPID') :) >> let $identifiers := >> for $item in $identifiers >> return >> if ($item = $vars) >> then $item >> else () >> (:it is not here assumed that all vars will be in identifiers, but it is >> assumed that all identifiers and vars values will be unique:) >> let $vars := >> for $item in $vars >> return >> if ($item = $identifiers) >> then $item >> else () >> (: identifiers and vars are now equally long :) >> let $indexes := $vars ! index-of($identifiers, .) >> (: in case of identical ordering, this returns a sequence of integers from >> 1 >> to the count of indexes/count of vars/count of identifiers; >> if an item is out of order, the sequence of index values will not follow >> the >> index of the sequence of index values :) >> let $order-maintained := >> for $index at $i in $indexes >> return >> if ($index eq $i) >> then true() >> else false() >> let $order-maintained := >> if (not($order-maintained = false())) >> then true() >> else false() >> return >> $order-maintained >> >> When you eliminate duplicate vars, how can you maintain ordering, that is, >> how can you decide which instance of a number of duplicates to maintain? >> >> Jens >> >> >> On 25 Aug 2015 at 07:25:25, Jozef Aerts (joz...@xm...) >> wrote: >> >> >> >> First of all thanks for the tip for >> http://x-query.com/mailman/listinfo/talk >> . I have been searching for such a discussion site for some time but did >> not >> find that one - I immediately subscribed. >> >> >> Last night, I could get a bit further, using: >> >> >> (: iterate over all items in the sequence $vararray :) >> for $v1 in $vars >> let $positionlocal1 := index-of($vars,$v1)[1] >> (: get the position in the sequence representing the SDTM standard :) >> let $positionstandardarray1 := index-of($identifiers,$v1) >> (: iterate over all FOLLOWING in the sequence $vars :) >> for $v2 in subsequence($vars,$positionlocal1+1) >> (: get the position in the sequence representing the SDTM standard :) >> let $positionstandardarray2 := index-of($identifiers,$v2)[1] >> (: if the position in the sequence representing the SDTM standard >> for the FOLLOWING variable is smaller than for the previous variable >> then the order of our variables is incorrect :) >> where $positionstandardarray2 < $positionstandardarray1 >> return <error>Variable is in wrong order within domain: {data($vars)} >> was found</error> >> >> >> but this returns several <error> elements when there is more than 1 item. >> >> I will study your solution - I can learn a lot from it... >> >> >> Many thanks! >> >> >> Jozef >> >> Adam Retter <ad...@ex...> hat am 25. August 2015 um 00:18 >> geschrieben: >> >> >> Josef, >> >> If I have correctly understood what you want, then this was quite a >> fun XQuery teaser. Perhaps the following is something like what you >> are looking for: >> >> xquery version "3.0"; >> >> let $identifiers := >> >> ('STUDYID','DOMAIN','USUBJID','POOLID','SPDEVID','AESEQ','AEGRPID','AEREFID','AESPID','AELNKID','AELNKGRP') >> let $vars := ('STUDYID','DOMAIN','SPDEVID','AESEQ','AESPID') >> return >> >> let $idxs := $vars ! index-of($identifiers, .) >> let $contains-order := >> if(count($idxs) eq count($vars))then >> for $idx at $i in $idxs >> return >> if($i eq 1) then >> true() >> else >> $idxs[$i - 1] lt $idx >> else >> false() >> return >> not($contains-order = false()) >> >> On 24 August 2015 at 21:05, Jozef Aerts <joz...@xm...> >> wrote: >>> I have the following two sequences: >>> >>> >>> let $identifiers := >>> >>> >>> ('STUDYID','DOMAIN','USUBJID','POOLID','SPDEVID','AESEQ','AEGRPID','AEREFID','AESPID','AELNKID','AELNKGRP') >>> >>> >>> let $vars := ('STUDYID','DOMAIN','SPDEVID','AESEQ','AESPID') >>> >>> >>> comparing the two sequences should return "true" as all items from $vars >>> are >>> in $identifiers and are in the same order. >>> >>> >>> However, the following sequence should return "false" when comparing with >>> $identifiers: >>> >>> >>> let $vars := ('STUDYID','DOMAIN','AESEQ','AESPID','SPDEVID') >>> >>> >>> as the items in $vars are not in the same order as in $identifiers. >>> Remark: none of the two sequences contains duplicates, and we already >>> checked before whether all items from $vars are really in the sequence >>> $identifiers using functx:is-value-in-sequence >>> >>> >>> I tried a few things using "intersection" and comparing the result with >>> $vars, but did not find a solution sofar. >>> Has anyone an idea how this can be solved? I'm not that good yet in >>> XQuery >>> ... >>> >>> >>> Many thanks in advance, >>> >>> >>> Jozef >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Exist-open mailing list >>> Exi...@li... >>> https://lists.sourceforge.net/lists/listinfo/exist-open >>> >> >> >> >> -- >> Adam Retter >> >> eXist Developer >> { United Kingdom } >> ad...@ex... >> irc://irc.freenode.net/existdb >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Exist-open mailing list >> Exi...@li... >> https://lists.sourceforge.net/lists/listinfo/exist-open >> >> >> >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Exist-open mailing list >> Exi...@li... >> https://lists.sourceforge.net/lists/listinfo/exist-open >> >> >> >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Exist-open mailing list >> Exi...@li... >> https://lists.sourceforge.net/lists/listinfo/exist-open >> >> >> >> > > > > -- > Adam Retter > > eXist Developer > { United Kingdom } > ad...@ex... > irc://irc.freenode.net/existdb > > ------------------------------------------------------------------------------ > _______________________________________________ > Exist-open mailing list > Exi...@li... > https://lists.sourceforge.net/lists/listinfo/exist-open -- Adam Retter eXist Developer { United Kingdom } ad...@ex... irc://irc.freenode.net/existdb |