exist-open — Open discussion

Re: [Exist-open] static variables?

[Eduard D. <ed...@fr...>]

thanks, have tried it, will have a look again..
-----Original Message-----From: Jean-Paul Rehr <re...@gm...>To:
Eduard Drenth <ed...@fr...>Cc: 
exi...@li... <exi...@li...>Subj
ect: Re: [Exist-open] static variables?Date: Mon, 21 Dec 2020 09:15:09
+0100
If it helps, I've found that using fields and facets in 5.0+ in
creative ways has dramatically improved query speeds. But this will
really depend on what you are trying to get out of your data. For me I
have a lot of cross-linked files that I need to lookup and these
techniques allow me to store  the results of the lookup at indexing
time, in effect a sort-of "precompiling".
On Mon, Dec 21, 2020 at 7:50 AM Eduard Drenth <
ed...@fr...> wrote:
> ok, fair enough, the reason I ask these questions is because
> performance worries me, I have several optimized dictionary
> applications where querying 66000 articles takes ± 1s. I find this
> slow. db/jpa/ee/jax-rs applications I build run way faster on more
> data.
> 
> So, now I am exploring options I have to make exist-db apps run
> faster, apart from cache on server and/or client or faster machine. I
> followed all the exist-db docs and tips on performance, used monex,
> build indexes, etc. It just isn't fast enough.
> 
> That is why I am wondering about the underlying java processing of
> xquery and the influence I have there. But alas the java processing
> is carefully hidden.
> 
> Perhaps I should experiment with basex, see if that is faster. But
> I'd rather stick to exist-db, also because of teipublisher.
> 
> Regards, Eduard
> 
> -----Original Message-----
> From: Jean-Paul Rehr <re...@gm...>
> To: Eduard Drenth <ed...@fr...>
> Cc: exi...@li... <
> exi...@li...>
> Subject: Re: [Exist-open] static variables?
> Date: Sun, 20 Dec 2020 15:13:20 +0100
> 
> >xquery has no static
> 
> Indeed. But as I tried to describe in my use case, I use this method
> for assigning values which are in effect static across my application
> (not static in programming terms). For example:
> 
> let $globalvar:datastring := "/db/apps/myapp/data"
> 
> But I would not do this:
> 
> let $globalvar:somedoc := doc( "/db/apps/myapp/data/mydoc.xml") 
> 
> 
> unless I was sure that the underlying document was not going to be
> changed elsewhere at the moment the variable was being used. I use
> this method for config files that rarely change. (For me, this is an
> issue of concurrent changes to a file, not really a question of
> processing cost.)
> 
> The above may not be useful practice for your context, but in my
> applications it is as effective as the cache...
> 
> Cheers,
> JPR
> 
> 
> On Sun, Dec 20, 2020 at 2:23 PM Eduard Drenth <
> ed...@fr...> wrote:
> > Thanks, but, xquery has no static. And if I do
> > 
> > declare variable $x := mod:func();
> > 
> > either or not in an imported module,
> > 
> > mod:func() gets executed with every request I suspect, but correct
> > me if I am wrong.
> > 
> > Regards, Eduard
> > 
> > -----Original Message-----
> > From: Jean-Paul Rehr <re...@gm...>
> > To: Eduard Drenth <ed...@fr...>
> > Cc: exi...@li... <
> > exi...@li...>
> > Subject: Re: [Exist-open] static variables?
> > Date: Sun, 20 Dec 2020 13:03:35 +0100
> > 
> > I accomplish this with a module that just has my "global variables"
> > for use across the application. This "global variable" module is
> > literally only populated with static variables which I can then
> > declare and import as needed in my queries. I keep a lot of static
> > configuration data and reusable path strings in them.....
> > 
> > On Sun, Dec 20, 2020 at 12:45 PM Eduard Drenth <
> > ed...@fr...> wrote:
> > > Dear all,
> > > 
> > > For performance reasons I want to be able to work with static
> > > variables. My data in exist is readonly.
> > > 
> > > I get the impression that variables are evaluated on every
> > > request normally, so if I do:
> > > 
> > > declare variable $teidictjson:data := ....
> > > 
> > > this variable is request scoped. Can I somehow make that static
> > > or application scoped? Or is the cache module the only way?
> > > 
> > > Bye, Eduard
> > > -- 
> > > Eduard Drenth, Software Architekt
> > > 
> > > ed...@fr...
> > > 
> > > Doelestrjitte 8
> > > 8911 DX  Ljouwert
> > > +31 58 234 30 47
> > > +31 62 094 34 28 (privé)
> > > 
> > > skype: eduarddrenth
> > > https://github.com/eduarddrenth
> > > frisian.eu
> > > gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
> > > 
> > > 
> > > Op freed bin ik thús/wurkje ik minder
> > > 
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > 
> > > Exist-open mailing list
> > > 
> > > Exi...@li...
> > > 
> > > https://lists.sourceforge.net/lists/listinfo/exist-open
> > > 
> > 
> > -- 
> > Eduard Drenth, Software Architekt
> > 
> > ed...@fr...
> > 
> > Doelestrjitte 8
> > 8911 DX  Ljouwert
> > +31 58 234 30 47
> > +31 62 094 34 28 (privé)
> > 
> > skype: eduarddrenth
> > https://github.com/eduarddrenth
> > frisian.eu
> > gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
> > 
> > 
> > Op freed bin ik thús/wurkje ik minder
> > 
> > 
> > 
> > 
> 
> -- 
> Eduard Drenth, Software Architekt
> 
> ed...@fr...
> 
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
> 
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
> 
> 
> Op freed bin ik thús/wurkje ik minder
> 
> 
> 
> 


-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] static variables?

[Jean-Paul R. <re...@gm...>]

If it helps, I've found that using fields and facets in 5.0+ in creative
ways has dramatically improved query speeds. But this will really depend on
what you are trying to get out of your data. For me I have a lot of
cross-linked files that I need to lookup and these techniques allow me to
store the results of the lookup at indexing time, in effect a sort-of
"precompiling".

On Mon, Dec 21, 2020 at 7:50 AM Eduard Drenth <ed...@fr...>
wrote:

> ok, fair enough, the reason I ask these questions is because performance
> worries me, I have several optimized dictionary applications where querying
> 66000 articles takes ± 1s. I find this slow. db/jpa/ee/jax-rs applications
> I build run way faster on more data.
>
> So, now I am exploring options I have to make exist-db apps run faster,
> apart from cache on server and/or client or faster machine. I followed all
> the exist-db docs and tips on performance, used monex, build indexes, etc.
> It just isn't fast enough.
>
> That is why I am wondering about the underlying java processing of xquery
> and the influence I have there. But alas the java processing is carefully
> hidden.
>
> Perhaps I should experiment with basex, see if that is faster. But I'd
> rather stick to exist-db, also because of teipublisher.
>
> Regards, Eduard
>
> -----Original Message-----
> *From*: Jean-Paul Rehr <re...@gm...
> <Jean-Paul%20Rehr%20%3cr...@gm...%3e>>
> *To*: Eduard Drenth <ed...@fr...
> <Eduard%20Drenth%20%3ce...@fr...%3e>>
> *Cc*: exi...@li... <exi...@li...
> <%22e...@li...%22%20%3ce...@li...%3e>
> >
> *Subject*: Re: [Exist-open] static variables?
> *Date*: Sun, 20 Dec 2020 15:13:20 +0100
>
> >xquery has no static
>
> Indeed. But as I tried to describe in my use case, I use this method for
> assigning values which are *in effect* static across my application (*not
> static in programming terms*). For example:
>
> let $globalvar:datastring := "/db/apps/myapp/data"
>
> But I would not do this:
>
> let $globalvar:somedoc := doc( "/db/apps/myapp/data/mydoc.xml")
>
> *unless* I was sure that the underlying document was not going to be
> changed elsewhere at the moment the variable was being used. I use this
> method for config files that rarely change. (For me, this is an issue of
> concurrent changes to a file, not really a question of processing cost.)
>
> The above may not be useful practice for your context, but in my
> applications it is as effective as the cache...
>
> Cheers,
> JPR
>
>
> On Sun, Dec 20, 2020 at 2:23 PM Eduard Drenth <ed...@fr...>
> wrote:
>
> Thanks, but, xquery has no static. And if I do
>
> declare variable $x := mod:func();
>
> either or not in an imported module,
>
> mod:func() gets executed with every request I suspect, but correct me if I
> am wrong.
>
> Regards, Eduard
>
> -----Original Message-----
> *From*: Jean-Paul Rehr <re...@gm...
> <Jean-Paul%20Rehr%20%3cr...@gm...%3e>>
> *To*: Eduard Drenth <ed...@fr...
> <Eduard%20Drenth%20%3ce...@fr...%3e>>
> *Cc*: exi...@li... <exi...@li...
> <%22e...@li...%22%20%3ce...@li...%3e>
> >
> *Subject*: Re: [Exist-open] static variables?
> *Date*: Sun, 20 Dec 2020 13:03:35 +0100
>
> I accomplish this with a module that just has my "global variables" for
> use across the application. This "global variable" module is literally only
> populated with static variables which I can then declare and import as
> needed in my queries. I keep a lot of static configuration data and
> reusable path strings in them.....
>
> On Sun, Dec 20, 2020 at 12:45 PM Eduard Drenth <ed...@fr...>
> wrote:
>
> Dear all,
>
> For performance reasons I want to be able to work with static variables.
> My data in exist is readonly.
>
> I get the impression that variables are evaluated on every request
> normally, so if I do:
>
> declare variable
>
> $
>
> teidictjson
>
> :
>
> data := ....
>
>
> this variable is request scoped. Can I somehow make that static or
> application scoped? Or is the cache module the only way?
>
> Bye, Eduard
>
>
> --
>
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> --
>
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
> --
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
>

Re: [Exist-open] XML to JSON

[Florian S. <ml-...@fl...>]

Hi Nick,

yes, just try to set 'json-ignore-whitespace-text-nodes' to 'yes'. So,
just adding the line

declare option output:json-ignore-whitespace-text-nodes "yes";

should do the trick.

HTH
Florian

Am 21.12.20 um 08:20 schrieb Nick Sincaglia:

> (...) . Is there a serialization setting that I can
> set that will make these go away?
>
> Nick
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-630-303-7035
> nsi...@nu...
> http://www.nuemeta.com
> Skype: nsincaglia
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] XML to JSON

[Eduard D. <ed...@fr...>]

For our apps we use: https://github.com/bramstein/xsltjson, that gives
you some options.

Furthermore I guess what is in your "server" ends up in the output,
you'll have to normalize your "server" first.

Bye, Eduard

-----Original Message-----
From: Nick Sincaglia <nsi...@nu...>
To: Alberto Simões via Exist-open <Exi...@li...>
Subject: [Exist-open] XML to JSON
Date: Mon, 21 Dec 2020 01:20:32 -0600



  
  
    I have a script where I am trying to
      convert XML to JSON. It looks like this:

      

      xquery version "3.1";

      

      declare namespace output =
      "http://www.w3.org/2010/xslt-xquery-serialization";

      declare namespace json="http://www.json.org";

      

      (: Switch to JSON serialization :)

      declare option output:method "json";

      declare option output:media-type "text/javascript";

      

      let $servers-list :=
      collection('/db/apps/partners-packages/control-
center/data')//server

      

      return <servers>{$servers-list}</servers>

      

      The output looks like this:

      { "server" : [{ "#text" : ["\n        ", "\n        ", "\n       
      ", "\n        ", "\n        ", "\n        ", "\n        ", "\n   
      " ], "id" : "1", "name" : "Indian Server", "status" : "active",
      "ip-address" : "XXX.XXX.XXX.XXX", "port" : "8080", "type" :
      "single-delivery-system", "description" : "Server which supports
      many clients in India" }, { "#text" : ["\n        ", "\n       
",
      "\n        ", "\n        ", "\n        ", "\n        ",
"\n       
      ", "\n    " ], "id" : "2", "name" : "N.Va.A HUB", "status" :
      "active", "ip-address" : "XXX.XXX.XXX.XXX", "port" : "8080",
      "type" : "auto-scaling-hub", "description" : "Auto scaling hub in
      N.Va Region" }, { "#text" : ["\n        ", "\n        ",
      "\n        ", "\n        ", "\n        ", "\n        ",
"\n       
      ", "\n    " ], "id" : "3", "name" : "N.Va.B HUB", "status" :
      "active", "ip-address" : "XXX.XXX.XXX.XXX", "port" : "8080",
      "type" : "auto-scaling-hub", "description" : "Auto scaling hub in
      N.Va Region" }, { "#text" : ["\n        ", "\n        ",
      "\n        ", "\n        ", "\n        ", "\n        ", "\n    "
      ], "id" : "4", "name" : "Video", "status" : "active", "ip-
address"
      : "XXX.XXX.XXX.XXX",
      "port" : "8080", "type" : "single-delivery-system" }] }

      

      I was wondering if there was a way to get rid of all of these
      #text arrays that get generated:

    

    "#text" :
        ["\n        ", "\n        ", "\n        ", "\n        ",
        "\n        ", "\n        ", "\n        ", "\n    " ]

        

        This array of newline characters goes away if my XML is not
        formatted in a easy-to-read format. Is there a serialization
        setting that I can set that will make these go away?

        

        Nick

        

      
    -- Nick Sincaglia President/FounderNueMeta, LLCDigital Media &
TechnologyPhone: +1-...@nu... 
http://www.nuemeta.com
Skype: nsincaglia
  

_______________________________________________Exist-open mailing 
lis...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open

-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

[Exist-open] XML to JSON

[Nick S. <nsi...@nu...>]

I have a script where I am trying to convert XML to JSON. It looks like 
this:

xquery version "3.1";

declare namespace output = 
"http://www.w3.org/2010/xslt-xquery-serialization";
declare namespace json="http://www.json.org";

(: Switch to JSON serialization :)
declare option output:method "json";
declare option output:media-type "text/javascript";

let $servers-list := 
collection('/db/apps/partners-packages/control-center/data')//server

return <servers>{$servers-list}</servers>

The output looks like this:
{ "server" : [{ "#text" : ["\n        ", "\n        ", "\n ", "\n        
", "\n        ", "\n        ", "\n        ", "\n " ], "id" : "1", "name" 
: "Indian Server", "status" : "active", "ip-address" : 
"XXX.XXX.XXX.XXX", "port" : "8080", "type" : "single-delivery-system", 
"description" : "Server which supports many clients in India" }, { 
"#text" : ["\n        ", "\n        ", "\n        ", "\n        ", 
"\n        ", "\n        ", "\n ", "\n    " ], "id" : "2", "name" : 
"N.Va.A HUB", "status" : "active", "ip-address" : "XXX.XXX.XXX.XXX", 
"port" : "8080", "type" : "auto-scaling-hub", "description" : "Auto 
scaling hub in N.Va Region" }, { "#text" : ["\n        ", "\n        ", 
"\n        ", "\n        ", "\n        ", "\n        ", "\n ", "\n    " 
], "id" : "3", "name" : "N.Va.B HUB", "status" : "active", "ip-address" 
: "XXX.XXX.XXX.XXX", "port" : "8080", "type" : "auto-scaling-hub", 
"description" : "Auto scaling hub in N.Va Region" }, { "#text" : 
["\n        ", "\n        ", "\n        ", "\n        ", "\n        ", 
"\n        ", "\n    " ], "id" : "4", "name" : "Video", "status" : 
"active", "ip-address" : "XXX.XXX.XXX.XXX", "port" : "8080", "type" : 
"single-delivery-system" }] }

I was wondering if there was a way to get rid of all of these #text 
arrays that get generated:

"#text" : ["\n        ", "\n        ", "\n        ", "\n        ", 
"\n        ", "\n        ", "\n        ", "\n    " ]

This array of newline characters goes away if my XML is not formatted in 
a easy-to-read format. Is there a serialization setting that I can set 
that will make these go away?

Nick

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] static variables?

[Eduard D. <ed...@fr...>]

ok, fair enough, the reason I ask these questions is because
performance worries me, I have several optimized dictionary
applications where querying 66000 articles takes ± 1s. I find this
slow. db/jpa/ee/jax-rs applications I build run way faster on more
data.
So, now I am exploring options I have to make exist-db apps run faster,
apart from cache on server and/or client or faster machine. I followed
all the exist-db docs and tips on performance, used monex, build
indexes, etc. It just isn't fast enough.
That is why I am wondering about the underlying java processing of
xquery and the influence I have there. But alas the java processing is
carefully hidden.
Perhaps I should experiment with basex, see if that is faster. But I'd
rather stick to exist-db, also because of teipublisher.
Regards, Eduard
-----Original Message-----From: Jean-Paul Rehr <re...@gm...>To:
Eduard Drenth <ed...@fr...>Cc: 
exi...@li... <exi...@li...>Subj
ect: Re: [Exist-open] static variables?Date: Sun, 20 Dec 2020 15:13:20
+0100
>xquery has no static

Indeed. But as I tried to describe in my use case, I use this method
for assigning values which are in effect static across my application
(not static in programming terms). For example:

let $globalvar:datastring := "/db/apps/myapp/data"

But I would not do this:

let $globalvar:somedoc := doc( "/db/apps/myapp/data/mydoc.xml") 


unless I was sure that the underlying document was not going to be
changed elsewhere at the moment the variable was being used. I use this
method for config files that rarely change. (For me, this is an issue
of concurrent changes to a file, not really a question of processing
cost.)

The above may not be useful practice for your context, but in my
applications it is as effective as the cache...

Cheers,
JPR


On Sun, Dec 20, 2020 at 2:23 PM Eduard Drenth <
ed...@fr...> wrote:
> Thanks, but, xquery has no static. And if I do
> 
> declare variable $x := mod:func();
> 
> either or not in an imported module,
> 
> mod:func() gets executed with every request I suspect, but correct me
> if I am wrong.
> 
> Regards, Eduard
> 
> -----Original Message-----
> From: Jean-Paul Rehr <re...@gm...>
> To: Eduard Drenth <ed...@fr...>
> Cc: exi...@li... <
> exi...@li...>
> Subject: Re: [Exist-open] static variables?
> Date: Sun, 20 Dec 2020 13:03:35 +0100
> 
> I accomplish this with a module that just has my "global variables"
> for use across the application. This "global variable" module is
> literally only populated with static variables which I can then
> declare and import as needed in my queries. I keep a lot of static
> configuration data and reusable path strings in them.....
> 
> On Sun, Dec 20, 2020 at 12:45 PM Eduard Drenth <
> ed...@fr...> wrote:
> > Dear all,
> > 
> > For performance reasons I want to be able to work with static
> > variables. My data in exist is readonly.
> > 
> > I get the impression that variables are evaluated on every request
> > normally, so if I do:
> > 
> > declare variable $teidictjson:data := ....
> > 
> > this variable is request scoped. Can I somehow make that static or
> > application scoped? Or is the cache module the only way?
> > 
> > Bye, Eduard
> > -- 
> > Eduard Drenth, Software Architekt
> > 
> > ed...@fr...
> > 
> > Doelestrjitte 8
> > 8911 DX  Ljouwert
> > +31 58 234 30 47
> > +31 62 094 34 28 (privé)
> > 
> > skype: eduarddrenth
> > https://github.com/eduarddrenth
> > frisian.eu
> > gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
> > 
> > 
> > Op freed bin ik thús/wurkje ik minder
> > 
> > 
> > 
> > 
> > _______________________________________________
> > 
> > Exist-open mailing list
> > 
> > Exi...@li...
> > 
> > https://lists.sourceforge.net/lists/listinfo/exist-open
> > 
> 
> -- 
> Eduard Drenth, Software Architekt
> 
> ed...@fr...
> 
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
> 
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
> 
> 
> Op freed bin ik thús/wurkje ik minder
> 
> 
> 
> 


-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] static variables?

[Jean-Paul R. <re...@gm...>]

>xquery has no static

Indeed. But as I tried to describe in my use case, I use this method for
assigning values which are *in effect* static across my application (*not
static in programming terms*). For example:

let $globalvar:datastring := "/db/apps/myapp/data"

But I would not do this:

let $globalvar:somedoc := doc( "/db/apps/myapp/data/mydoc.xml")

*unless* I was sure that the underlying document was not going to be
changed elsewhere at the moment the variable was being used. I use this
method for config files that rarely change. (For me, this is an issue of
concurrent changes to a file, not really a question of processing cost.)

The above may not be useful practice for your context, but in my
applications it is as effective as the cache...

Cheers,
JPR


On Sun, Dec 20, 2020 at 2:23 PM Eduard Drenth <ed...@fr...>
wrote:

> Thanks, but, xquery has no static. And if I do
>
> declare variable $x := mod:func();
>
> either or not in an imported module,
>
> mod:func() gets executed with every request I suspect, but correct me if I
> am wrong.
>
> Regards, Eduard
>
> -----Original Message-----
> *From*: Jean-Paul Rehr <re...@gm...
> <Jean-Paul%20Rehr%20%3cr...@gm...%3e>>
> *To*: Eduard Drenth <ed...@fr...
> <Eduard%20Drenth%20%3ce...@fr...%3e>>
> *Cc*: exi...@li... <exi...@li...
> <%22e...@li...%22%20%3ce...@li...%3e>
> >
> *Subject*: Re: [Exist-open] static variables?
> *Date*: Sun, 20 Dec 2020 13:03:35 +0100
>
> I accomplish this with a module that just has my "global variables" for
> use across the application. This "global variable" module is literally only
> populated with static variables which I can then declare and import as
> needed in my queries. I keep a lot of static configuration data and
> reusable path strings in them.....
>
> On Sun, Dec 20, 2020 at 12:45 PM Eduard Drenth <ed...@fr...>
> wrote:
>
> Dear all,
>
> For performance reasons I want to be able to work with static variables.
> My data in exist is readonly.
>
> I get the impression that variables are evaluated on every request
> normally, so if I do:
>
> declare variable
>
> $
>
> teidictjson
>
> :
>
> data := ....
>
>
> this variable is request scoped. Can I somehow make that static or
> application scoped? Or is the cache module the only way?
>
> Bye, Eduard
>
>
> --
>
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> --
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
>

Re: [Exist-open] static variables?

[Eduard D. <ed...@fr...>]

Thanks, but, xquery has no static. And if I do
declare variable $x := mod:func();
either or not in an imported module,
mod:func() gets executed with every request I suspect, but correct me
if I am wrong.
Regards, Eduard
-----Original Message-----From: Jean-Paul Rehr <re...@gm...>To:
Eduard Drenth <ed...@fr...>Cc: 
exi...@li... <exi...@li...>Subj
ect: Re: [Exist-open] static variables?Date: Sun, 20 Dec 2020 13:03:35
+0100
I accomplish this with a module that just has my "global variables" for
use across the application. This "global variable" module is literally
only populated with static variables which I can then declare and
import as needed in my queries. I keep a lot of static configuration
data and reusable path strings in them.....
On Sun, Dec 20, 2020 at 12:45 PM Eduard Drenth <
ed...@fr...> wrote:
> Dear all,
> 
> For performance reasons I want to be able to work with static
> variables. My data in exist is readonly.
> 
> I get the impression that variables are evaluated on every request
> normally, so if I do:
> 
> declare variable $teidictjson:data := ....
> 
> this variable is request scoped. Can I somehow make that static or
> application scoped? Or is the cache module the only way?
> 
> Bye, Eduard
> -- 
> Eduard Drenth, Software Architekt
> 
> ed...@fr...
> 
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
> 
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
> 
> 
> Op freed bin ik thús/wurkje ik minder
> 
> 
> 
> 
> _______________________________________________
> 
> Exist-open mailing list
> 
> Exi...@li...
> 
> https://lists.sourceforge.net/lists/listinfo/exist-open
> 


-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] static variables?

[Jean-Paul R. <re...@gm...>]

I accomplish this with a module that just has my "global variables" for use
across the application. This "global variable" module is literally only
populated with static variables which I can then declare and import as
needed in my queries. I keep a lot of static configuration data and
reusable path strings in them.....

On Sun, Dec 20, 2020 at 12:45 PM Eduard Drenth <ed...@fr...>
wrote:

> Dear all,
>
> For performance reasons I want to be able to work with static variables.
> My data in exist is readonly.
>
> I get the impression that variables are evaluated on every request
> normally, so if I do:
>
> declare variable $teidictjson:data := ....
>
>
> this variable is request scoped. Can I somehow make that static or
> application scoped? Or is the cache module the only way?
>
>
> Bye, Eduard
>
>
> --
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

[Exist-open] static variables?

[Eduard D. <ed...@fr...>]

Dear all,

For performance reasons I want to be able to work with static
variables. My data in exist is readonly.

I get the impression that variables are evaluated on every request
normally, so if I do:

declare variable $teidictjson:data := ....

this variable is request scoped. Can I somehow make that static or
application scoped? Or is the cache module the only way?

Bye, Eduard
-- 
Eduard Drenth, Software Architekt

ed...@fr...

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

Re: [Exist-open] Backup and restore

[Jean-Paul R. <re...@gm...>]

Coming back to the original problem:

" If so, is there a way to kick off the reindex through the non-gui client?
I'm running 4.1.0 on a linux platform"

Reindexing can be done anytime through Xquery: xmldb:reindex()
<http://exist-db.org/exist/apps/fundocs/view.html?uri=http://exist-db.org/xquery/xmldb>


JPR




On Fri, Dec 18, 2020 at 9:00 PM Dai, Tino <td...@lo...> wrote:

> Hi Dannes,
>
>
>       My bosses refuse to upgrade and they shut off the dashboard. How
> would I go about checking the WEB-INF/data directory?
>
>
> Thanks,
>
> Tino
>
>
> ------------------------------
> *From:* Dannes Wessels <da...@ex...>
> *Sent:* Friday, December 18, 2020 2:44 PM
> *To:* Dai, Tino
> *Cc:* exi...@li...
> *Subject:* Re: [Exist-open] Backup and restore
>
> Hi,
>
> v4.1 is 2+ years ago, that is a long time :-P
>
> In the current v5 the indices are updated during the restore, so after a
> restore a database is ready to go; A lot has changed since 4.x but I think
> this should still be the same :-)
>
> If you are not sure, please check monex (easy), or the contents of the
> WEB-INF/data directory (more complex).
>
> cheers
>
> Dannes
>
> eXist-db Native XML Database
> http://www.exist-db.org
>
>
>
> On 15 Dec 2020, at 20:02 , Dai, Tino <td...@lo...> wrote:
>
> Quick question about backup and restore. I'm using the backup.sh and the
> non gui client to backup/restore from one machine to another. When I do the
> restore, I don't think that I'm doing a re-index of the /db subdirectories.
> Is that a correct assumption? If so, is there a way to kick off the reindex
> through the non-gui client? I'm running 4.1.0 on a linux platform
>
> Thanks,
> Tino
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] Backup and restore

[Dai, T. <td...@lo...>]

Hi Dannes,


      My bosses refuse to upgrade and they shut off the dashboard. How would I go about checking the WEB-INF/data directory?


Thanks,

Tino


________________________________
From: Dannes Wessels <da...@ex...>
Sent: Friday, December 18, 2020 2:44 PM
To: Dai, Tino
Cc: exi...@li...
Subject: Re: [Exist-open] Backup and restore

Hi,

v4.1 is 2+ years ago, that is a long time :-P

In the current v5 the indices are updated during the restore, so after a restore a database is ready to go; A lot has changed since 4.x but I think this should still be the same :-)

If you are not sure, please check monex (easy), or the contents of the WEB-INF/data directory (more complex).

cheers

Dannes

eXist-db Native XML Database
http://www.exist-db.org



On 15 Dec 2020, at 20:02 , Dai, Tino <td...@lo...<mailto:td...@lo...>> wrote:

Quick question about backup and restore. I'm using the backup.sh and the non gui client to backup/restore from one machine to another. When I do the restore, I don't think that I'm doing a re-index of the /db subdirectories. Is that a correct assumption? If so, is there a way to kick off the reindex through the non-gui client? I'm running 4.1.0 on a linux platform

Thanks,
Tino
_______________________________________________
Exist-open mailing list
Exi...@li...<mailto:Exi...@li...>
https://lists.sourceforge.net/lists/listinfo/exist-open

Re: [Exist-open] Backup and restore

[Dannes W. <da...@ex...>]

Hi,

v4.1 is 2+ years ago, that is a long time :-P

In the current v5 the indices are updated during the restore, so after a restore a database is ready to go; A lot has changed since 4.x but I think this should still be the same :-)

If you are not sure, please check monex (easy), or the contents of the WEB-INF/data directory (more complex).

cheers

Dannes

eXist-db Native XML Database
http://www.exist-db.org



> On 15 Dec 2020, at 20:02 , Dai, Tino <td...@lo...> wrote:
> 
> Quick question about backup and restore. I'm using the backup.sh and the non gui client to backup/restore from one machine to another. When I do the restore, I don't think that I'm doing a re-index of the /db subdirectories. Is that a correct assumption? If so, is there a way to kick off the reindex through the non-gui client? I'm running 4.1.0 on a linux platform
> 
> Thanks,
> Tino
> _______________________________________________
> Exist-open mailing list
> Exi...@li... <mailto:Exi...@li...>
> https://lists.sourceforge.net/lists/listinfo/exist-open <https://lists.sourceforge.net/lists/listinfo/exist-open>

[Exist-open] Backup and restore

[Dai, T. <td...@lo...>]

Quick question about backup and restore. I'm using the backup.sh and the non gui client to backup/restore from one machine to another. When I do the restore, I don't think that I'm doing a re-index of the /db subdirectories. Is that a correct assumption? If so, is there a way to kick off the reindex through the non-gui client? I'm running 4.1.0 on a linux platform


Thanks,

Tino

Re: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".

[AG <gru...@gm...>]

Thanks to all for all your help! Especially to Joe for his detailed
explanations and suggestions!
Ok, the problem is in the losing identity of original nodes once they are
wrapped in a new tag... I got it.
I'm not yet very familiar with mapping techniques, so I will read up about
it. Thanks for the suggestion!
"Defer output generation until really needed" - absolutely agree. It's my
biggest problem. In my complex query I try to find a balance between
"generate the output too early" and "still have access to original data to
extract that I need".
So the problem is solved (or rather there is no solution as is).
Thanks once again to all the eXist-db community!

P.S. There is my related question on stackoverflow:
https://stackoverflow.com/questions/65174232/fulltext-xquery-lucene-kwic-doesnt-work-on-tagged-result-exist-db-bug
I'm going to suggest to close it as well (with the link to the mailing list
answer : https://sourceforge.net/p/exist/mailman/message/37170946/ )


On Mon, Dec 7, 2020 at 3:39 PM Joe Wicentowski <jo...@gm...> wrote:

> Hi AG,
>
> When you wrap a node in an element, you are constructing a new node, in
> memory. As a result, the newly constructed element has no connection to the
> original one (i.e., the wrapped node loses its identity), and you are no
> longer able to query it using the full text index, since the full text
> index only queries nodes stored in the database and your newly constructed
> node is in memory, not stored in the database. Here are two alternatives:
>
> 1. Perform ft:query first, then wrap the results as needed. You still lose
> the node's original identity, but you may no longer need it once you've
> performed the full text index query.
>
> collection("path_to_my_collection")//node[ft:query(., "KEYWORD")] !
> <tag>{.}</tag>
>
> Do keep in mind the advice to "Defer output generation until really
> needed" (https://exist-db.org/exist/apps/doc/tuning#defer-output).
>
> 2. A bit more advanced technique is to use a map instead of an element.
> This preserves the node's original identity while letting you associate it
> with arbitrary entries, for labeling.
>
> collection("path_to_my_collection")//node[ft:query(., "KEYWORD")] ! map {
> "label": "my tag", "hit": . }
>
> You could use these entries for grouping, etc.
>
> Joe
>
> On Mon, Dec 7, 2020 at 8:29 AM AG <gru...@gm...> wrote:
>
>> Thank you, Eduard, for all your suggestions.
>>
>> - there is no index for <tag>, so ft:query on it yields no results, yes
>>> you tried adding index config for it, but your <tag> isn't there at the
>>> time the index is built
>>>
>>
>> You are absolutely right. So the question is how to put an index on the
>> intermediate internal result. Is it even possible? Maybe someone has some
>> leads? eXist-db documentation is not very helpful. The closest to this I
>> have found is:
>> https://exist-db.org/exist/apps/doc/lucene#constructed-fields
>>
>> P.S.
>>
>> - I don't understand your need to wrap in <tag>
>>
>>
>> I have two collections with quite similar data but the different XML
>> schemas, so I have to query them separately (but I need a common result).
>> So for now I have two fulltext queries on each collection and then I
>> combine obtained results. My goal is optimization: go from two fulltext
>> queries (slow) to only one (fast). For this I do 1) from each collection
>> select the files that meet my criteria; 2) from selected files (from two
>> collections) extract data I need; 3) from these data construct combined
>> intermediate internal result (here I put <tag> on the part of this result
>> where I want to make fulltext query); 4) make fulltext query (only one) on
>> this combined intermediate internal result.
>> Maybe I'm wrong and this approach is not most optimised…
>>
>> On Mon, Dec 7, 2020 at 12:34 PM Eduard Drenth <ed...@fr...>
>> wrote:
>>
>>> Hi,
>>>
>>> A few things:
>>>
>>> - there is no index for <tag>, so ft:query on it yields no results, yes
>>> you tried adding index config for it, but your <tag> isn't there at the
>>> time the index is built
>>> - use exist 5.2.0, but that isn't your problem
>>> - you query can be simpler, but perhaps you know:
>>>
>>>> let $results := collection("path_to_my_collection")//node[ft:query(., "KEYWORD")]
>>>>
>>>> - I don't understand your need to wrap in <tag>
>>>
>>>
>>> Good luck, Eduard
>>>
>>>
>>> -----Original Message-----
>>> *From*: AG <gru...@gm... <AG%20%3cg...@gm...%3e>
>>> >
>>> *To*: exi...@li...
>>> *Subject*: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if
>>> variable in path was "tagged" in preceding "return".
>>> *Date*: Mon, 07 Dec 2020 12:04:38 +0100
>>>
>>>      First of all, Hello everybody!
>>>      I'm new here and very excited to be on this list!
>>>      Here is my problem. After reading XQuery and eXist-db
>>> documentations, I can't figure it out, so I decided to ask the question
>>> here.
>>>
>>> *In a nutshell:* The fulltext search with KWIC doesn't work if the
>>> variable in the path for fulltext search was put in a tag in preceding
>>> "result". It returns an empty result.
>>>
>>> *Explanations: *
>>>
>>> *XML file*
>>>
>>> <
>>>
>>> root
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> node
>>>
>>> >
>>>
>>> blablabla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> node
>>>
>>> >
>>>
>>> blab KEYWORD labla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> node
>>>
>>> >
>>>
>>> blablabla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>> </
>>>
>>> root
>>>
>>> >
>>>
>>>
>>>
>>> *Index configuration (collection.xconf)*
>>>
>>> <
>>>
>>> collection
>>>
>>>  xmlns
>>>
>>> =
>>>
>>> "
>>>
>>> http://exist-db.org/collection-config/1.0
>>>
>>> "
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> index
>>>
>>>  xmlns:xs
>>>
>>> =
>>>
>>> "
>>>
>>> http://www.w3.org/2001/XMLSchema
>>>
>>> "
>>>
>>> >
>>>
>>>
>>>         <
>>>
>>> lucene
>>>
>>> >
>>>
>>>
>>>             <
>>>
>>> text
>>>
>>>  qname
>>>
>>> =
>>>
>>> "root"
>>>
>>> />
>>>
>>>
>>>             <
>>>
>>> text
>>>
>>>  qname
>>>
>>> =
>>>
>>> "node"
>>>
>>> />
>>>
>>>
>>>         </
>>>
>>> lucene
>>>
>>> >
>>>
>>>
>>>     </
>>>
>>> index
>>>
>>> >
>>>
>>>
>>> </
>>>
>>> collection
>>>
>>> >
>>>
>>>
>>>
>>> *XQuery without "tagged" result (it works)* *(look at "return $node")*
>>>
>>> let $my_texts :=
>>>
>>>     for $node in collection("path_to_my_collection")//node
>>>
>>>     return
>>>
>>>         $node
>>>
>>>
>>> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
>>>
>>> return
>>>
>>>     $my_hit
>>>
>>>
>>> The Xquery code above works and I get a result.
>>>
>>> 1
>>>
>>> <
>>>
>>> node
>>>
>>> >
>>>
>>> blab KEYWORD labla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>>
>>> But it doesn't work when the first result on which fulltext search is
>>> launched was put in a tag. (My whole query is more complex and I need to
>>> put this result in the tag to use it in another place of my code.)
>>>
>>> *XQuery with "tagged" result (it doesn't work)* *(look at "return
>>> <tag>{$node}</tag>")*
>>>
>>> let $my_texts :=
>>>
>>>     for $node in collection("path_to_my_collection")//node
>>>
>>>     return
>>>
>>>         <
>>>
>>> tag
>>>
>>> >
>>>
>>> {$node}
>>>
>>> </
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>
>>> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
>>>
>>> return
>>>
>>>     $my_hit
>>>
>>>
>>> This query return 0 result.
>>>
>>> When I debug like this:
>>>
>>> *XQuery for debugging*
>>>
>>> let $my_texts :=
>>>
>>>     for $node in collection("path_to_my_collection")//node
>>>
>>>     return
>>>
>>>         <
>>>
>>> tag
>>>
>>> >
>>>
>>> {$node}
>>>
>>> </
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>
>>> return
>>>
>>>     $my_texts
>>>
>>>
>>> I get this:
>>>
>>> 1
>>>
>>> <
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> node
>>>
>>> >
>>>
>>> blablabla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>> </
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>
>>> 2
>>>
>>> <
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> node
>>>
>>> >
>>>
>>> blab KEYWORD labla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>> </
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>
>>> 3
>>>
>>> <
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>     <
>>>
>>> node
>>>
>>> >
>>>
>>> blablabla
>>>
>>> </
>>>
>>> node
>>>
>>> >
>>>
>>>
>>> </
>>>
>>> tag
>>>
>>> >
>>>
>>>
>>>
>>> What I tried:
>>>
>>>    - different path combinations: $my_texts/tag[ft:query(., "KEYWORD")],
>>>    $my_texts/tag/node[ft:query(., "KEYWORD")], $my_texts/*[ft:query(.,
>>>    "KEYWORD")], $my_texts/tag//*[ft:query(., "KEYWORD")], $my_texts//*//*[ft:query(.,
>>>    "KEYWORD")] etc...
>>>    - add <tag> in the Index configuration (<text qname="tag"/>)
>>>
>>> What I missed? Or it is an eXist-db bug? (my eXist version: 4.7.0)
>>> Many thanks in advance for all your help!
>>> Sincerely, AG.
>>>
>>> _______________________________________________
>>>
>>> Exist-open mailing list
>>>
>>> Exi...@li...
>>>
>>>
>>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>>
>>>
>>> --
>>>
>>> Eduard Drenth, Software Architekt
>>>
>>> ed...@fr...
>>>
>>> Doelestrjitte 8
>>> 8911 DX  Ljouwert
>>> +31 58 234 30 47
>>> +31 62 094 34 28 (privé)
>>>
>>> skype: eduarddrenth
>>> https://github.com/eduarddrenth
>>> frisian.eu
>>> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>>>
>>>
>>> Op freed bin ik thús/wurkje ik minder
>>>
>>>
>>>
>>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>

Re: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".

[Joe W. <jo...@gm...>]

Hi AG,

When you wrap a node in an element, you are constructing a new node, in
memory. As a result, the newly constructed element has no connection to the
original one (i.e., the wrapped node loses its identity), and you are no
longer able to query it using the full text index, since the full text
index only queries nodes stored in the database and your newly constructed
node is in memory, not stored in the database. Here are two alternatives:

1. Perform ft:query first, then wrap the results as needed. You still lose
the node's original identity, but you may no longer need it once you've
performed the full text index query.

collection("path_to_my_collection")//node[ft:query(., "KEYWORD")] !
<tag>{.}</tag>

Do keep in mind the advice to "Defer output generation until really needed"
(https://exist-db.org/exist/apps/doc/tuning#defer-output).

2. A bit more advanced technique is to use a map instead of an element.
This preserves the node's original identity while letting you associate it
with arbitrary entries, for labeling.

collection("path_to_my_collection")//node[ft:query(., "KEYWORD")] ! map {
"label": "my tag", "hit": . }

You could use these entries for grouping, etc.

Joe

On Mon, Dec 7, 2020 at 8:29 AM AG <gru...@gm...> wrote:

> Thank you, Eduard, for all your suggestions.
>
> - there is no index for <tag>, so ft:query on it yields no results, yes
>> you tried adding index config for it, but your <tag> isn't there at the
>> time the index is built
>>
>
> You are absolutely right. So the question is how to put an index on the
> intermediate internal result. Is it even possible? Maybe someone has some
> leads? eXist-db documentation is not very helpful. The closest to this I
> have found is:
> https://exist-db.org/exist/apps/doc/lucene#constructed-fields
>
> P.S.
>
> - I don't understand your need to wrap in <tag>
>
>
> I have two collections with quite similar data but the different XML
> schemas, so I have to query them separately (but I need a common result).
> So for now I have two fulltext queries on each collection and then I
> combine obtained results. My goal is optimization: go from two fulltext
> queries (slow) to only one (fast). For this I do 1) from each collection
> select the files that meet my criteria; 2) from selected files (from two
> collections) extract data I need; 3) from these data construct combined
> intermediate internal result (here I put <tag> on the part of this result
> where I want to make fulltext query); 4) make fulltext query (only one) on
> this combined intermediate internal result.
> Maybe I'm wrong and this approach is not most optimised…
>
> On Mon, Dec 7, 2020 at 12:34 PM Eduard Drenth <ed...@fr...>
> wrote:
>
>> Hi,
>>
>> A few things:
>>
>> - there is no index for <tag>, so ft:query on it yields no results, yes
>> you tried adding index config for it, but your <tag> isn't there at the
>> time the index is built
>> - use exist 5.2.0, but that isn't your problem
>> - you query can be simpler, but perhaps you know:
>>
>>> let $results := collection("path_to_my_collection")//node[ft:query(., "KEYWORD")]
>>>
>>> - I don't understand your need to wrap in <tag>
>>
>>
>> Good luck, Eduard
>>
>>
>> -----Original Message-----
>> *From*: AG <gru...@gm... <AG%20%3cg...@gm...%3e>>
>> *To*: exi...@li...
>> *Subject*: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if
>> variable in path was "tagged" in preceding "return".
>> *Date*: Mon, 07 Dec 2020 12:04:38 +0100
>>
>>      First of all, Hello everybody!
>>      I'm new here and very excited to be on this list!
>>      Here is my problem. After reading XQuery and eXist-db
>> documentations, I can't figure it out, so I decided to ask the question
>> here.
>>
>> *In a nutshell:* The fulltext search with KWIC doesn't work if the
>> variable in the path for fulltext search was put in a tag in preceding
>> "result". It returns an empty result.
>>
>> *Explanations: *
>>
>> *XML file*
>>
>> <
>>
>> root
>>
>> >
>>
>>
>>     <
>>
>> node
>>
>> >
>>
>> blablabla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>>     <
>>
>> node
>>
>> >
>>
>> blab KEYWORD labla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>>     <
>>
>> node
>>
>> >
>>
>> blablabla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>> </
>>
>> root
>>
>> >
>>
>>
>>
>> *Index configuration (collection.xconf)*
>>
>> <
>>
>> collection
>>
>>  xmlns
>>
>> =
>>
>> "
>>
>> http://exist-db.org/collection-config/1.0
>>
>> "
>>
>> >
>>
>>
>>     <
>>
>> index
>>
>>  xmlns:xs
>>
>> =
>>
>> "
>>
>> http://www.w3.org/2001/XMLSchema
>>
>> "
>>
>> >
>>
>>
>>         <
>>
>> lucene
>>
>> >
>>
>>
>>             <
>>
>> text
>>
>>  qname
>>
>> =
>>
>> "root"
>>
>> />
>>
>>
>>             <
>>
>> text
>>
>>  qname
>>
>> =
>>
>> "node"
>>
>> />
>>
>>
>>         </
>>
>> lucene
>>
>> >
>>
>>
>>     </
>>
>> index
>>
>> >
>>
>>
>> </
>>
>> collection
>>
>> >
>>
>>
>>
>> *XQuery without "tagged" result (it works)* *(look at "return $node")*
>>
>> let $my_texts :=
>>
>>     for $node in collection("path_to_my_collection")//node
>>
>>     return
>>
>>         $node
>>
>>
>> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
>>
>> return
>>
>>     $my_hit
>>
>>
>> The Xquery code above works and I get a result.
>>
>> 1
>>
>> <
>>
>> node
>>
>> >
>>
>> blab KEYWORD labla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>>
>> But it doesn't work when the first result on which fulltext search is
>> launched was put in a tag. (My whole query is more complex and I need to
>> put this result in the tag to use it in another place of my code.)
>>
>> *XQuery with "tagged" result (it doesn't work)* *(look at "return
>> <tag>{$node}</tag>")*
>>
>> let $my_texts :=
>>
>>     for $node in collection("path_to_my_collection")//node
>>
>>     return
>>
>>         <
>>
>> tag
>>
>> >
>>
>> {$node}
>>
>> </
>>
>> tag
>>
>> >
>>
>>
>>
>> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
>>
>> return
>>
>>     $my_hit
>>
>>
>> This query return 0 result.
>>
>> When I debug like this:
>>
>> *XQuery for debugging*
>>
>> let $my_texts :=
>>
>>     for $node in collection("path_to_my_collection")//node
>>
>>     return
>>
>>         <
>>
>> tag
>>
>> >
>>
>> {$node}
>>
>> </
>>
>> tag
>>
>> >
>>
>>
>>
>> return
>>
>>     $my_texts
>>
>>
>> I get this:
>>
>> 1
>>
>> <
>>
>> tag
>>
>> >
>>
>>
>>     <
>>
>> node
>>
>> >
>>
>> blablabla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>> </
>>
>> tag
>>
>> >
>>
>>
>>
>> 2
>>
>> <
>>
>> tag
>>
>> >
>>
>>
>>     <
>>
>> node
>>
>> >
>>
>> blab KEYWORD labla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>> </
>>
>> tag
>>
>> >
>>
>>
>>
>> 3
>>
>> <
>>
>> tag
>>
>> >
>>
>>
>>     <
>>
>> node
>>
>> >
>>
>> blablabla
>>
>> </
>>
>> node
>>
>> >
>>
>>
>> </
>>
>> tag
>>
>> >
>>
>>
>>
>> What I tried:
>>
>>    - different path combinations: $my_texts/tag[ft:query(., "KEYWORD")], $my_texts/tag/node[ft:query(.,
>>    "KEYWORD")], $my_texts/*[ft:query(., "KEYWORD")], $my_texts/tag//*[ft:query(.,
>>    "KEYWORD")], $my_texts//*//*[ft:query(., "KEYWORD")] etc...
>>    - add <tag> in the Index configuration (<text qname="tag"/>)
>>
>> What I missed? Or it is an eXist-db bug? (my eXist version: 4.7.0)
>> Many thanks in advance for all your help!
>> Sincerely, AG.
>>
>> _______________________________________________
>>
>> Exist-open mailing list
>>
>> Exi...@li...
>>
>>
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>>
>> --
>>
>> Eduard Drenth, Software Architekt
>>
>> ed...@fr...
>>
>> Doelestrjitte 8
>> 8911 DX  Ljouwert
>> +31 58 234 30 47
>> +31 62 094 34 28 (privé)
>>
>> skype: eduarddrenth
>> https://github.com/eduarddrenth
>> frisian.eu
>> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>>
>>
>> Op freed bin ik thús/wurkje ik minder
>>
>>
>>
>> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".

[Roy W. <gar...@ya...>]

 Hi,
You can't create an index on a dynamic result.
ft:query is fast so no problem to run two queries and union the results.
Regards,Roy

    On Monday, 7 December 2020, 13:29:21 GMT, AG <gru...@gm...> wrote:  
 
 Thank you, Eduard, for all your suggestions.

- there is no index for <tag>, so ft:query on it yields no results, yes you tried adding index config for it, but your <tag> isn't there at the time the index is built


You are absolutely right. So the question is how to put an index on the intermediate internal result. Is it even possible? Maybe someone has some leads? eXist-db documentation is not very helpful. The closest to this I have found is: https://exist-db.org/exist/apps/doc/lucene#constructed-fields
P.S.

- I don't understand your need to wrap in <tag>


I have two collections with quite similar data but the different XML schemas, so I have to query them separately (but I need a common result). So for now I have two fulltext queries on each collection and then I combine obtained results. My goal is optimization: go from two fulltext queries (slow) to only one (fast). For this I do 1) from each collection select the files that meet my criteria; 2) from selected files (from two collections) extract data I need; 3) from these data construct combined intermediate internal result (here I put <tag> on the part of this result where I want to make fulltext query); 4) make fulltext query (only one) on this combined intermediate internal result.
Maybe I'm wrong and this approach is not most optimised…
On Mon, Dec 7, 2020 at 12:34 PM Eduard Drenth <ed...@fr...> wrote:

Hi,
A few things:
- there is no index for <tag>, so ft:query on it yields no results, yes you tried adding index config for it, but your <tag> isn't there at the time the index is built- use exist 5.2.0, but that isn't your problem- you query can be simpler, but perhaps you know:
let $results := collection("path_to_my_collection")//node[ft:query(., "KEYWORD")]
- I don't understand your need to wrap in <tag>

Good luck, Eduard

-----Original Message-----From: AG <gru...@gm...>To: exi...@li...Subject: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".Date: Mon, 07 Dec 2020 12:04:38 +0100
     First of all, Hello everybody!
     I'm new here and very excited to be on this list!    Here is my problem. After reading XQuery and eXist-db documentations, I can't figure it out, soI decided to ask the question here. 
In a nutshell: The fulltext search with KWIC doesn't work if the variable in the path for fulltext search was put in a tag in preceding "result". It returns an empty result.   
Explanations:

XML file

<root>
    <node>blablabla</node>
    <node>blab KEYWORD labla</node>
    <node>blablabla</node>
</root>



Index configuration (collection.xconf)

<collection xmlns="http://exist-db.org/collection-config/1.0">
    <index xmlns:xs="http://www.w3.org/2001/XMLSchema">
        <lucene>
            <text qname="root"/>
            <text qname="node"/>
        </lucene>
    </index>
</collection>



XQuery without "tagged" result (it works) (look at "return $node")

let $my_texts :=     for $node in collection("path_to_my_collection")//node    return        $node
for $my_hit in $my_texts[ft:query(., "KEYWORD")]return     $my_hit


The Xquery code above works and I get a result.

1<node>blab KEYWORD labla</node>



But it doesn't work when the first result on which fulltext search is launched was put in a tag. (My whole query is more complex and I need to put this result in the tag to use it in another place of my code.)

XQuery with "tagged" result (it doesn't work) (look at "return <tag>{$node}</tag>")

let $my_texts :=     for $node in collection("path_to_my_collection")//node    return        <tag>{$node}</tag>

for $my_hit in $my_texts[ft:query(., "KEYWORD")]return     $my_hit


This query return 0 result.

When I debug like this:

XQuery for debugging

let $my_texts :=     for $node in collection("path_to_my_collection")//node    return        <tag>{$node}</tag>

return     $my_texts


I get this:

1<tag>
    <node>blablabla</node>
</tag>

2<tag>
    <node>blab KEYWORD labla</node>
</tag>

3<tag>
    <node>blablabla</node>
</tag>



What I tried:
   
   - different path combinations: $my_texts/tag[ft:query(., "KEYWORD")],$my_texts/tag/node[ft:query(., "KEYWORD")], $my_texts/*[ft:query(., "KEYWORD")],$my_texts/tag//*[ft:query(., "KEYWORD")], $my_texts//*//*[ft:query(., "KEYWORD")] etc...
   - add <tag> in the Index configuration (<text qname="tag"/>)

What I missed? Or it is an eXist-db bug? (my eXist version: 4.7.0)
Many thanks in advance for all your help!Sincerely, AG.

_______________________________________________Exist-open mailing lis...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open
-- 
Eduard Drenth, Software Architekt
ed...@fr...
Doelestrjitte 88911 DX  Ljouwert+31 58 234 30 47+31 62 094 34 28 (privé)
skype: eduarddrenthhttps://github.com/eduarddrenthfrisian.eugpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder



_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open
  

Re: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".

[AG <gru...@gm...>]

Thank you, Eduard, for all your suggestions.

- there is no index for <tag>, so ft:query on it yields no results, yes you
> tried adding index config for it, but your <tag> isn't there at the time
> the index is built
>

You are absolutely right. So the question is how to put an index on the
intermediate internal result. Is it even possible? Maybe someone has some
leads? eXist-db documentation is not very helpful. The closest to this I
have found is: https://exist-db.org/exist/apps/doc/lucene#constructed-fields

P.S.

- I don't understand your need to wrap in <tag>


I have two collections with quite similar data but the different XML
schemas, so I have to query them separately (but I need a common result).
So for now I have two fulltext queries on each collection and then I
combine obtained results. My goal is optimization: go from two fulltext
queries (slow) to only one (fast). For this I do 1) from each collection
select the files that meet my criteria; 2) from selected files (from two
collections) extract data I need; 3) from these data construct combined
intermediate internal result (here I put <tag> on the part of this result
where I want to make fulltext query); 4) make fulltext query (only one) on
this combined intermediate internal result.
Maybe I'm wrong and this approach is not most optimised…

On Mon, Dec 7, 2020 at 12:34 PM Eduard Drenth <ed...@fr...>
wrote:

> Hi,
>
> A few things:
>
> - there is no index for <tag>, so ft:query on it yields no results, yes
> you tried adding index config for it, but your <tag> isn't there at the
> time the index is built
> - use exist 5.2.0, but that isn't your problem
> - you query can be simpler, but perhaps you know:
>
>> let $results := collection("path_to_my_collection")//node[ft:query(., "KEYWORD")]
>>
>> - I don't understand your need to wrap in <tag>
>
>
> Good luck, Eduard
>
>
> -----Original Message-----
> *From*: AG <gru...@gm... <AG%20%3cg...@gm...%3e>>
> *To*: exi...@li...
> *Subject*: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if
> variable in path was "tagged" in preceding "return".
> *Date*: Mon, 07 Dec 2020 12:04:38 +0100
>
>      First of all, Hello everybody!
>      I'm new here and very excited to be on this list!
>      Here is my problem. After reading XQuery and eXist-db
> documentations, I can't figure it out, so I decided to ask the question
> here.
>
> *In a nutshell:* The fulltext search with KWIC doesn't work if the
> variable in the path for fulltext search was put in a tag in preceding
> "result". It returns an empty result.
>
> *Explanations: *
>
> *XML file*
>
> <
>
> root
>
> >
>
>
>     <
>
> node
>
> >
>
> blablabla
>
> </
>
> node
>
> >
>
>
>     <
>
> node
>
> >
>
> blab KEYWORD labla
>
> </
>
> node
>
> >
>
>
>     <
>
> node
>
> >
>
> blablabla
>
> </
>
> node
>
> >
>
>
> </
>
> root
>
> >
>
>
>
> *Index configuration (collection.xconf)*
>
> <
>
> collection
>
>  xmlns
>
> =
>
> "
>
> http://exist-db.org/collection-config/1.0
>
> "
>
> >
>
>
>     <
>
> index
>
>  xmlns:xs
>
> =
>
> "
>
> http://www.w3.org/2001/XMLSchema
>
> "
>
> >
>
>
>         <
>
> lucene
>
> >
>
>
>             <
>
> text
>
>  qname
>
> =
>
> "root"
>
> />
>
>
>             <
>
> text
>
>  qname
>
> =
>
> "node"
>
> />
>
>
>         </
>
> lucene
>
> >
>
>
>     </
>
> index
>
> >
>
>
> </
>
> collection
>
> >
>
>
>
> *XQuery without "tagged" result (it works)* *(look at "return $node")*
>
> let $my_texts :=
>
>     for $node in collection("path_to_my_collection")//node
>
>     return
>
>         $node
>
>
> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
>
> return
>
>     $my_hit
>
>
> The Xquery code above works and I get a result.
>
> 1
>
> <
>
> node
>
> >
>
> blab KEYWORD labla
>
> </
>
> node
>
> >
>
>
>
> But it doesn't work when the first result on which fulltext search is
> launched was put in a tag. (My whole query is more complex and I need to
> put this result in the tag to use it in another place of my code.)
>
> *XQuery with "tagged" result (it doesn't work)* *(look at "return
> <tag>{$node}</tag>")*
>
> let $my_texts :=
>
>     for $node in collection("path_to_my_collection")//node
>
>     return
>
>         <
>
> tag
>
> >
>
> {$node}
>
> </
>
> tag
>
> >
>
>
>
> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
>
> return
>
>     $my_hit
>
>
> This query return 0 result.
>
> When I debug like this:
>
> *XQuery for debugging*
>
> let $my_texts :=
>
>     for $node in collection("path_to_my_collection")//node
>
>     return
>
>         <
>
> tag
>
> >
>
> {$node}
>
> </
>
> tag
>
> >
>
>
>
> return
>
>     $my_texts
>
>
> I get this:
>
> 1
>
> <
>
> tag
>
> >
>
>
>     <
>
> node
>
> >
>
> blablabla
>
> </
>
> node
>
> >
>
>
> </
>
> tag
>
> >
>
>
>
> 2
>
> <
>
> tag
>
> >
>
>
>     <
>
> node
>
> >
>
> blab KEYWORD labla
>
> </
>
> node
>
> >
>
>
> </
>
> tag
>
> >
>
>
>
> 3
>
> <
>
> tag
>
> >
>
>
>     <
>
> node
>
> >
>
> blablabla
>
> </
>
> node
>
> >
>
>
> </
>
> tag
>
> >
>
>
>
> What I tried:
>
>    - different path combinations: $my_texts/tag[ft:query(., "KEYWORD")], $my_texts/tag/node[ft:query(.,
>    "KEYWORD")], $my_texts/*[ft:query(., "KEYWORD")], $my_texts/tag//*[ft:query(.,
>    "KEYWORD")], $my_texts//*//*[ft:query(., "KEYWORD")] etc...
>    - add <tag> in the Index configuration (<text qname="tag"/>)
>
> What I missed? Or it is an eXist-db bug? (my eXist version: 4.7.0)
> Many thanks in advance for all your help!
> Sincerely, AG.
>
> _______________________________________________
>
> Exist-open mailing list
>
> Exi...@li...
>
>
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> --
>
> Eduard Drenth, Software Architekt
>
> ed...@fr...
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
> +31 62 094 34 28 (privé)
>
> skype: eduarddrenth
> https://github.com/eduarddrenth
> frisian.eu
> gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth
>
>
> Op freed bin ik thús/wurkje ik minder
>
>
>
>

Re: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".

[Eduard D. <ed...@fr...>]

Hi,

A few things:

- there is no index for <tag>, so ft:query on it yields no results, yes you tried adding index config for it, but your <tag> isn't there at the time the index is built
- use exist 5.2.0, but that isn't your problem
- you query can be simpler, but perhaps you know:

let $results := collection("path_to_my_collection")//node[ft:query(., "KEYWORD")]

- I don't understand your need to wrap in <tag>


Good luck, Eduard


-----Original Message-----
From: AG <gru...@gm...<mailto:AG%20%3cg...@gm...%3e>>
To: exi...@li...<mailto:exi...@li...>
Subject: [Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".
Date: Mon, 07 Dec 2020 12:04:38 +0100

     First of all, Hello everybody!
     I'm new here and very excited to be on this list!
     Here is my problem. After reading XQuery and eXist-db documentations, I can't figure it out, so I decided to ask the question here.

In a nutshell: The fulltext search with KWIC doesn't work if the variable in the path for fulltext search was put in a tag in preceding "result". It returns an empty result.

Explanations:

XML file

<

root

>




<

node

>

blablabla

</

node

>




<

node

>

blab KEYWORD labla

</

node

>




<

node

>

blablabla

</

node

>


</

root

>



Index configuration (collection.xconf)

<

collection



xmlns

=

"

<http://exist-db.org/collection-config/1.0>

http://exist-db.org/collection-config/1.0

"

>




<

index



xmlns:xs

=

"

<http://www.w3.org/2001/XMLSchema>

http://www.w3.org/2001/XMLSchema

"

>




<

lucene

>




<

text



qname

=

"root"

/>




<

text



qname

=

"node"

/>




</

lucene

>




</

index

>


</

collection

>



XQuery without "tagged" result (it works) (look at "return $node")

let $my_texts :=

    for $node in collection("path_to_my_collection")//node

    return

        $node


for $my_hit in $my_texts[ft:query(., "KEYWORD")]

return

    $my_hit


The Xquery code above works and I get a result.

1

<

node

>

blab KEYWORD labla

</

node

>



But it doesn't work when the first result on which fulltext search is launched was put in a tag. (My whole query is more complex and I need to put this result in the tag to use it in another place of my code.)

XQuery with "tagged" result (it doesn't work) (look at "return <tag>{$node}</tag>")

let $my_texts :=

    for $node in collection("path_to_my_collection")//node

    return



<

tag

>

{$node}

</

tag

>



for $my_hit in $my_texts[ft:query(., "KEYWORD")]

return

    $my_hit


This query return 0 result.

When I debug like this:

XQuery for debugging

let $my_texts :=

    for $node in collection("path_to_my_collection")//node

    return



<

tag

>

{$node}

</

tag

>



return

    $my_texts


I get this:

1

<

tag

>




<

node

>

blablabla

</

node

>


</

tag

>



2

<

tag

>




<

node

>

blab KEYWORD labla

</

node

>


</

tag

>



3

<

tag

>




<

node

>

blablabla

</

node

>


</

tag

>



What I tried:

  *   different path combinations: $my_texts/tag[ft:query(., "KEYWORD")], $my_texts/tag/node[ft:query(., "KEYWORD")], $my_texts/*[ft:query(., "KEYWORD")], $my_texts/tag//*[ft:query(., "KEYWORD")], $my_texts//*//*[ft:query(., "KEYWORD")] etc...
  *   add <tag> in the Index configuration (<text qname="tag"/>)

What I missed? Or it is an eXist-db bug? (my eXist version: 4.7.0)

Many thanks in advance for all your help!
Sincerely, AG.


_______________________________________________

Exist-open mailing list

<mailto:Exi...@li...>

Exi...@li...


<https://lists.sourceforge.net/lists/listinfo/exist-open>

https://lists.sourceforge.net/lists/listinfo/exist-open


--

Eduard Drenth, Software Architekt

ed...@fr...<mailto:ed...@fr...>

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47
+31 62 094 34 28 (privé)

skype: eduarddrenth
https://github.com/eduarddrenth
frisian.eu
gpg: https://pgp.surfnet.nl/pks/lookup?search=eduarddrenth


Op freed bin ik thús/wurkje ik minder

[Exist-open] Bug? KWIC fulltext (ft:query) doesn't work if variable in path was "tagged" in preceding "return".

[AG <gru...@gm...>]

     First of all, Hello everybody!
     I'm new here and very excited to be on this list!
     Here is my problem. After reading XQuery and eXist-db documentations,
I can't figure it out, so I decided to ask the question here.

*In a nutshell:* The fulltext search with KWIC doesn't work if the variable
in the path for fulltext search was put in a tag in preceding "result". It
returns an empty result.

*Explanations: *

*XML file*

> <root>
>     <node>blablabla</node>
>     <node>blab KEYWORD labla</node>
>     <node>blablabla</node></root>
>
> *Index configuration (collection.xconf)*

> <collection xmlns="http://exist-db.org/collection-config/1.0">
>     <index xmlns:xs="http://www.w3.org/2001/XMLSchema">
>         <lucene>
>             <text qname="root"/>
>             <text qname="node"/>
>         </lucene>
>     </index></collection>
>
> *XQuery without "tagged" result (it works)* *(look at "return $node")*

> let $my_texts :=
>     for $node in collection("path_to_my_collection")//node
>     return
>         $node
>
> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
> return
>     $my_hit
>
> The Xquery code above works and I get a result.

> 1<node>blab KEYWORD labla</node>
>
> But it doesn't work when the first result on which fulltext search is
launched was put in a tag. (My whole query is more complex and I need to
put this result in the tag to use it in another place of my code.)

*XQuery with "tagged" result (it doesn't work)* *(look at "return
<tag>{$node}</tag>")*

> let $my_texts :=
>     for $node in collection("path_to_my_collection")//node
>     return
>         <tag>{$node}</tag>
>
> for $my_hit in $my_texts[ft:query(., "KEYWORD")]
> return
>     $my_hit
>
> This query return 0 result.

When I debug like this:

*XQuery for debugging*

> let $my_texts :=
>     for $node in collection("path_to_my_collection")//node
>     return
>         <tag>{$node}</tag>
>
> return
>     $my_texts
>
> I get this:

> 1<tag>
>     <node>blablabla</node></tag>
>
> 2<tag>
>     <node>blab KEYWORD labla</node></tag>
>
> 3<tag>
>     <node>blablabla</node></tag>
>
> What I tried:

   - different path combinations: $my_texts/tag[ft:query(.,
"KEYWORD")], $my_texts/tag/node[ft:query(.,
   "KEYWORD")], $my_texts/*[ft:query(., "KEYWORD")],
$my_texts/tag//*[ft:query(.,
   "KEYWORD")], $my_texts//*//*[ft:query(., "KEYWORD")] etc...
   - add <tag> in the Index configuration (<text qname="tag"/>)

What I missed? Or it is an eXist-db bug? (my eXist version: 4.7.0)
Many thanks in advance for all your help!
Sincerely, AG.

Re: [Exist-open] eXist-db does not see /db/apps

[Joe W. <jo...@gm...>]

So the question is what to do if the documented emergency recovery
procedures do not yield the files you expected it to restore? I am afraid
you may be out of options, other than restoring from an earlier backup.

It may be advisable to upgrade the system to eXist 5 to benefit from its
improved transaction lock handling and stability, and to schedule more
frequent backups.

On Sun, Dec 6, 2020 at 7:48 PM Nick Sincaglia <nsi...@nu...>
wrote:

> Yes, according to the emergency backup log, it was missing most of the
> folders.
>
> I have dealt with eXist-db crashes many times before. This one seems
> extreme compared to the others I have recovered from.
>
> I have followed most of my notes on what to do after a crash. Now I have
> run out of notes and ideas, so I am asking the community if there is
> anything more to try.
>
> Nick
>
>
> On 12/6/20 5:53 PM, Joe Wicentowski wrote:
>
> Hi Nick,
>
> Did you try the emergency backup and restore procedure? Was the backup
> missing data too?
>
> Joe
>
> On Sun, Dec 6, 2020 at 3:51 PM Nick Sincaglia <nsi...@nu...>
> wrote:
>
>> No, I am able to login. That is not the problem. It is just that eXist-db
>> only acknowledges the existence a few of the total folders that it once
>> acknowledged existed on the system.
>>
>> Nick
>>
>> On 12/6/20 12:17 PM, Dannes Wessels wrote:
>>
>> Hi Nick
>>
>> On 4 Dec 2020, at 19:57 , Nick Sincaglia <nsi...@nu...> wrote:
>>
>> This returns very quickly with true().
>>
>>
>> strange
>>
>> However, if I try to reindex '/db/apps' it returns false()
>>
>>
>> This probably means your login credentials were not accepted.
>>
>> D.
>>
>>
>> --
>> Nick Sincaglia
>> President/Founder
>> NueMeta, LLC
>> Digital Media & Technology
>> Phone: +1-...@nu... http://www.nuemeta.com
>> Skype: nsincaglia
>>
>> _______________________________________________
>> Exist-open mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-...@nu... http://www.nuemeta.com
> Skype: nsincaglia
>
> --
Sent from my iPhone

Re: [Exist-open] eXist-db does not see /db/apps

[Nick S. <nsi...@nu...>]

Yes, according to the emergency backup log, it was missing most of the 
folders.

I have dealt with eXist-db crashes many times before. This one seems 
extreme compared to the others I have recovered from.

I have followed most of my notes on what to do after a crash. Now I have 
run out of notes and ideas, so I am asking the community if there is 
anything more to try.

Nick


On 12/6/20 5:53 PM, Joe Wicentowski wrote:
> Hi Nick,
>
> Did you try the emergency backup and restore procedure? Was the backup 
> missing data too?
>
> Joe
>
> On Sun, Dec 6, 2020 at 3:51 PM Nick Sincaglia <nsi...@nu... 
> <mailto:nsi...@nu...>> wrote:
>
>     No, I am able to login. That is not the problem. It is just that
>     eXist-db only acknowledges the existence a few of the total
>     folders that it once acknowledged existed on the system.
>
>     Nick
>
>     On 12/6/20 12:17 PM, Dannes Wessels wrote:
>>     Hi Nick
>>
>>>     On 4 Dec 2020, at 19:57 , Nick Sincaglia <nsi...@nu...
>>>     <mailto:nsi...@nu...>> wrote:
>>>
>>>     This returns very quickly with true().
>>
>>     strange
>>
>>>     However, if I try to reindex '/db/apps' it returns false()
>>
>>     This probably means your login credentials were not accepted.
>>
>>     D.
>
>     -- 
>     Nick Sincaglia
>     President/Founder
>     NueMeta, LLC
>     Digital Media & Technology
>     Phone: +1-630-303-7035
>     nsi...@nu...  <mailto:nsi...@nu...>  
>     http://www.nuemeta.com  <http://www.nuemeta.com>
>     Skype: nsincaglia
>
>     _______________________________________________
>     Exist-open mailing list
>     Exi...@li...
>     <mailto:Exi...@li...>
>     https://lists.sourceforge.net/lists/listinfo/exist-open
>     <https://lists.sourceforge.net/lists/listinfo/exist-open>
>

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] eXist-db does not see /db/apps

[Joe W. <jo...@gm...>]

Hi Nick,

Did you try the emergency backup and restore procedure? Was the backup
missing data too?

Joe

On Sun, Dec 6, 2020 at 3:51 PM Nick Sincaglia <nsi...@nu...>
wrote:

> No, I am able to login. That is not the problem. It is just that eXist-db
> only acknowledges the existence a few of the total folders that it once
> acknowledged existed on the system.
>
> Nick
>
> On 12/6/20 12:17 PM, Dannes Wessels wrote:
>
> Hi Nick
>
> On 4 Dec 2020, at 19:57 , Nick Sincaglia <nsi...@nu...> wrote:
>
> This returns very quickly with true().
>
>
> strange
>
> However, if I try to reindex '/db/apps' it returns false()
>
>
> This probably means your login credentials were not accepted.
>
> D.
>
>
> --
> Nick Sincaglia
> President/Founder
> NueMeta, LLC
> Digital Media & Technology
> Phone: +1-...@nu... http://www.nuemeta.com
> Skype: nsincaglia
>
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

Re: [Exist-open] eXist-db does not see /db/apps

[Nick S. <nsi...@nu...>]

No, I am able to login. That is not the problem. It is just that 
eXist-db only acknowledges the existence a few of the total folders that 
it once acknowledged existed on the system.

Nick

On 12/6/20 12:17 PM, Dannes Wessels wrote:
> Hi Nick
>
>> On 4 Dec 2020, at 19:57 , Nick Sincaglia <nsi...@nu... 
>> <mailto:nsi...@nu...>> wrote:
>>
>> This returns very quickly with true().
>
> strange
>
>> However, if I try to reindex '/db/apps' it returns false()
>
> This probably means your login credentials were not accepted.
>
> D.

-- 
Nick Sincaglia
President/Founder
NueMeta, LLC
Digital Media & Technology
Phone: +1-630-303-7035
nsi...@nu...
http://www.nuemeta.com
Skype: nsincaglia

Re: [Exist-open] eXist-db does not see /db/apps

[Dannes W. <da...@ex...>]

Hi Nick

> On 4 Dec 2020, at 19:57 , Nick Sincaglia <nsi...@nu...> wrote:
> 
> This returns very quickly with true(). 

strange

> However, if I try to reindex '/db/apps' it returns false()

This probably means your login credentials were not accepted.

D.