exist-development — eXist Developer's List

Re: [Exist-development] how to disable xprocxq?

[Joe W. <jo...@gm...>]

>> But, as far as I can tell, xprocxq isn't an entry in extensions
>> properties (EXIST_HOME/extensions/build.properties).  Looking at
>> extensions/xprocxq/build.xml, it appears that the build file
>> references the build.properties and local.build.properties files but
>> doesn't look for a true/false value.
>
> hmm, that looks like it was changed/disabled by someone
>
>> I can manually delete the .jar if I don't need it, but I thought I'd
>> mention it since this appears to be the only extension in the
>> extensions directory without an entry in build.properties.
>
> in any event, xproc is core and I will be doing more work on it to
> bring it up to compliance now that I have scratched other itches.

Cool, thanks, Jim.  (Expathrepo and XSGI were some pretty productive
itches!  Neat stuff!)

Joe

Re: [Exist-development] how to disable xprocxq?

[Joe W. <jo...@gm...>]

Thanks for your reply, Jim.

>> In trunk, xprocxq isn't listed in extensions/build.properties, so it
>> doesn't appear there's a way to disable it using a
>> local.build.properties.  Is it a required piece of eXist, such that it
>> must be compiled with the distribution?  It appears that every other
>> folder in extensions/ has a corresponding entry in build.properties,
>> except for this one.
>
> for now its part of the core build, but if you wanted to disable
> building it, just do so in extensions properties (it should be there)

But, as far as I can tell, xprocxq isn't an entry in extensions
properties (EXIST_HOME/extensions/build.properties).  Looking at
extensions/xprocxq/build.xml, it appears that the build file
references the build.properties and local.build.properties files but
doesn't look for a true/false value.

I can manually delete the .jar if I don't need it, but I thought I'd
mention it since this appears to be the only extension in the
extensions directory without an entry in build.properties.

Joe

Re: [Exist-development] how to disable xprocxq?

[James F. <jam...@ex...>]

On 13 September 2010 19:10, Joe Wicentowski <jo...@gm...> wrote:
> Hi all,
>
> In trunk, xprocxq isn't listed in extensions/build.properties, so it
> doesn't appear there's a way to disable it using a
> local.build.properties.  Is it a required piece of eXist, such that it
> must be compiled with the distribution?  It appears that every other
> folder in extensions/ has a corresponding entry in build.properties,
> except for this one.

for now its part of the core build, but if you wanted to disable
building it, just do so in extensions properties (it should be there)

J

[Exist-development] how to disable xprocxq?

[Joe W. <jo...@gm...>]

Hi all,

In trunk, xprocxq isn't listed in extensions/build.properties, so it
doesn't appear there's a way to disable it using a
local.build.properties.  Is it a required piece of eXist, such that it
must be compiled with the distribution?  It appears that every other
folder in extensions/ has a corresponding entry in build.properties,
except for this one.

Thanks,
Joe

Re: [Exist-development] "Browse Indexes" admin panel

[Joe W. <jo...@gm...>]

> it should be /db/system/config

Indeed.  Thanks for catching my typo, Dmitriy.  To be clear, though,
the typo was only in the e-mail, not the code.

Joe

Re: [Exist-development] "Browse Indexes" admin panel

[Dmitriy S. <sha...@gm...>]

Hi Joe,

On Fri, 2010-09-10 at 17:36 -0400, Joe Wicentowski wrote:
> /db/config

it should be /db/system/config

-- 
Cheers,

Dmitriy Shabanov

[Exist-development] "Browse Indexes" admin panel

[Joe W. <jo...@gm...>]

Hi all,

I wanted to let you know that I committed a draft version of new eXist
admin panel to trunk today (1.5dev rev.12681), called "Browse
Indexes."

"Browse Indexes" displays an overview of all *.conf files stored in
/db/config, and allow you to browse through the index keys for any
indexed item (be it a qname or a path/match definition, and whether
the index is Lucene, Range, NGram, or Legacy Fulltext).  Under the
hood, it uses util:index-keys(), util:index-keys-by-qname(), and
text:index-keys().

I wrote this "Browse Index" panel to help users understand the
relationship between their queries (written in XQuery) and the index
keys -- the contents of the indexes themselves.  I was inspired by
Luke (http://code.google.com/p/luke/), the Java tool for exploring
Lucene indexes.  I had used Luke to look at index keys, and I realized
that it would be much more convenient for me (and other users) to have
a tool for examining the index keys right from within eXist.

If you want to try this out, update to rev. 12681 and open
http://localhost:8080/exist/admin/admin.xql?panel=indexes.  If you
don't have any indexed content, I'd suggest going to the "Examples
Setup" and "Install Documentation > Generate" admin panels; installing
these will give you 5 sample indexed collections to browse.

I hope you find it useful, and that you think it merits a place in the
admin panels.  If not, I can  create a separate page with the same
functions elsewhere in webapp, or remove it altogether from trunk.

Some notes about the current version:

Browsing index keys currently works for Lucene, NGram, and Legacy
Fulltext indexes.  Range and NGram indexes have some problems: 1.
Range indexes are showing 0 index keys.  2. NGram and Lucene only work
when browsing by 'node', not by 'qname'.  I e-mailed the list about
these problems earlier today.  If there are fixes to those issues, I
will incorporate the fixes into the panel.

I need to work one reporting if kinds of extra Lucene analyzers are
being used.  I expect the reports might not properly indicate which
non-default analyzers are being used.

I haven't tested the spatial index or sort index.  Would it make sense
to include these?

The panel could also be improved with the addition of some XQuery
functions to help me derive the indexing defaults in conf.xml.  For
example, I have no way of knowing what default index settings are in
conf.xml, or what the ngram index is set to, or what the various
legacy fulltext index settings are (some might be relevant to
understanding the index keys).  If anyone is interested, I'd
appreciate your help creating such a function.

Lastly, my focus was on the exposing the index keys themselves, so I
haven't included every possible detail about the collection.xconf
files, such as @boost values for Lucene indexes, or indicating cases
where there are multiple <lucene> elements in a single .xconf file (I
actually don't understand the significance of this latter practice,
evident in the /db/xqdocs .xconf file).  But if anyone has ideas for
ways to make this panel more useful, please let me know.

Cheers,
Joe

Re: [Exist-development] possible issue with controller.xql in database with latest security changes

[Dannes W. <da...@ex...>]

I have seen the same messages in the webdav trunk that did not contain yet the security changes. So... I think it it not related.

On 10 Sep 2010, at 14:51 , James Fuller wrote:

> I think there is a problem here with the latest security related change

Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] possible issue with controller.xql in database with latest security changes

[James F. <jam...@ex...>]

apologies, all works once I forced rebuild which I thought I had done
int he first place .

ta, J

On 10 September 2010 15:05, James Fuller <jam...@ex...> wrote:
> latest trunk
>
> J
>
> On 10 September 2010 15:00, Dmitriy Shabanov <sha...@gm...> wrote:
>> On Fri, 2010-09-10 at 14:51 +0200, James Fuller wrote:
>>> Hello,
>>>
>>> I updated to latest trunk and a controller.xql that worked in the
>>> database all of a sudden stopped working and throws the following
>>> error
>>>
>>> 2010-09-10 14:49:07,655 [eXistThread-22] WARN
>>> (ReentrantReadWriteLock.java [release]:305) - Possible lock problem:
>>> thread Thread[eXistThread-22,5,main] released a lock on /db/testing it
>>> didn't hold. Either the thread was interrupted or it never acquired
>>> the lock. The lock was owned by: null
>>
>> I did see it before refactoring, it something else. what revision?
>>
>> --
>> Cheers,
>>
>> Dmitriy Shabanov
>>
>

Re: [Exist-development] possible issue with controller.xql in database with latest security changes

[James F. <jam...@ex...>]

latest trunk

J

On 10 September 2010 15:00, Dmitriy Shabanov <sha...@gm...> wrote:
> On Fri, 2010-09-10 at 14:51 +0200, James Fuller wrote:
>> Hello,
>>
>> I updated to latest trunk and a controller.xql that worked in the
>> database all of a sudden stopped working and throws the following
>> error
>>
>> 2010-09-10 14:49:07,655 [eXistThread-22] WARN
>> (ReentrantReadWriteLock.java [release]:305) - Possible lock problem:
>> thread Thread[eXistThread-22,5,main] released a lock on /db/testing it
>> didn't hold. Either the thread was interrupted or it never acquired
>> the lock. The lock was owned by: null
>
> I did see it before refactoring, it something else. what revision?
>
> --
> Cheers,
>
> Dmitriy Shabanov
>

Re: [Exist-development] possible issue with controller.xql in database with latest security changes

[Dmitriy S. <sha...@gm...>]

On Fri, 2010-09-10 at 14:51 +0200, James Fuller wrote:
> Hello,
> 
> I updated to latest trunk and a controller.xql that worked in the
> database all of a sudden stopped working and throws the following
> error
> 
> 2010-09-10 14:49:07,655 [eXistThread-22] WARN
> (ReentrantReadWriteLock.java [release]:305) - Possible lock problem:
> thread Thread[eXistThread-22,5,main] released a lock on /db/testing it
> didn't hold. Either the thread was interrupted or it never acquired
> the lock. The lock was owned by: null

I did see it before refactoring, it something else. what revision?

-- 
Cheers,

Dmitriy Shabanov

[Exist-development] possible issue with controller.xql in database with latest security changes

[James F. <jam...@ex...>]

Hello,

I updated to latest trunk and a controller.xql that worked in the
database all of a sudden stopped working and throws the following
error

2010-09-10 14:49:07,655 [eXistThread-22] WARN
(ReentrantReadWriteLock.java [release]:305) - Possible lock problem:
thread Thread[eXistThread-22,5,main] released a lock on /db/testing it
didn't hold. Either the thread was interrupted or it never acquired
the lock. The lock was owned by: null

I think there is a problem here with the latest security related changes.

thoughts ?

J

Re: [Exist-development] eXist-db with Calabash and Saxon again

[Norman W. <nd...@nw...>]

James Fuller <jam...@ex...> writes:
> from pragmatic viewpoint, I would rather wait for a saxon release and
> then code up a proper extension, but that doesn't help Henry ;)

I think Henry's all set, but getting a Saxon release with the
classloader bug fixed would be nice. I don't think it's practical
to get XML Calabash working with 9.1 again.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <nd...@nw...> | We measure the excellency of other men,
http://nwalsh.com/            | by some excellency we conceive to be in
                              | ourselves.--John Selden

Re: [Exist-development] Broker twitter.xql on demo server

[Leif-Jöran O. <lj...@ex...>]

Den 2010-09-10 11:08, James Fuller skrev:

> the demo server is getting quite dated  .... any thoughts of when it
> is to be updated ;)

Yes, it is a few months old. I will try to get a date for you after the
weekend.

Re: [Exist-development] eXist-db with Calabash and Saxon again

[<ht...@in...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Adam Retter writes:

> Also, I have just discovered a way in which Henry could get this
> working today for his needs (although it wold not be suitable for a
> general release). We did try this together last night, but at that
> point it appeared not to work - however this morning with a fresh
> head, I have managed to get it working fine.
>
> Simply after building eXist-db with the xmlcalabash extension enabled -
>
> 1) rm $EXIST_HOME/lib/endorsed/saxonb*
>
> 2) rm $EXIST_HOME/lib/user/saxon9-s9api.jar
>
> 3) Place Saxon9he.jar (I used version 9.2.1.2) into $EXIST_HOME/lib/user

Yes, works now!  Thanks very much for your help.

ht
- -- 
       Henry S. Thompson, School of Informatics, University of Edinburgh
      10 Crichton Street, Edinburgh EH8 9AB, SCOTLAND -- (44) 131 650-4440
                Fax: (44) 131 651-1426, e-mail: ht...@in...
                       URL: http://www.ltg.ed.ac.uk/~ht/
 [mail from me _always_ has a .sig like this -- mail without it is forged spam]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFMifBDkjnJixAXWBoRAsjYAJ4pJopmXeWNBKnR2Vxc+TeglUxN0gCeOKw4
E0ZP6kwpiLmGBD9bp6vDpy8=
=pCh7
-----END PGP SIGNATURE-----

Re: [Exist-development] Broker twitter.xql on demo server

[James F. <jam...@ex...>]

On 10 September 2010 11:07, Adam Retter <ad...@ex...> wrote:
> Take a look here -
>
> http://demo.exist-db.org/exist/xquery/twitter.xql?mode=json&user=existdb&max=3
>
> Its trying to store resources in the database collection /db/twitter,
> it does appear to work on trunk here locally. Is this just an out of
> date revision on the demo server?

the demo server is getting quite dated  .... any thoughts of when it
is to be updated ;)

ta, J

[Exist-development] Broker twitter.xql on demo server

[Adam R. <ad...@ex...>]

Take a look here -

http://demo.exist-db.org/exist/xquery/twitter.xql?mode=json&user=existdb&max=3

Its trying to store resources in the database collection /db/twitter,
it does appear to work on trunk here locally. Is this just an out of
date revision on the demo server?

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] eXist-db with Calabash and Saxon again

[James F. <jam...@ex...>]

On 10 September 2010 10:45, Henry S. Thompson <ht...@in...> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Adam Retter writes:
>
>> Also, I have just discovered a way in which Henry could get this
>> working today for his needs (although it wold not be suitable for a
>> general release). We did try this together last night, but at that
>> point it appeared not to work - however this morning with a fresh
>> head, I have managed to get it working fine.
>>
>> Simply after building eXist-db with the xmlcalabash extension enabled -
>>
>> 1) rm $EXIST_HOME/lib/endorsed/saxonb*
>>
>> 2) rm $EXIST_HOME/lib/user/saxon9-s9api.jar
>>
>> 3) Place Saxon9he.jar (I used version 9.2.1.2) into $EXIST_HOME/lib/user
>
> Yes, works now!  Thanks very much for your help.

good to hear ... but once again the extension is super primitive ... I
hope you can get some use out of it.

J

Re: [Exist-development] eXist-db with Calabash and Saxon again

[James F. <jam...@ex...>]

On 10 September 2010 10:19, Adam Retter <ad...@ex...> wrote:
>> in the short term you maybe able to use standalone xmlcalabash with
>> James Sulak eXist step library
>>
>> http://bitbucket.org/jasulak/exist-xproc-library/wiki/Home
>
> Ah, a reverse integration, thats interesting!
>
> Also, I have just discovered a way in which Henry could get this
> working today for his needs (although it wold not be suitable for a
> general release). We did try this together last night, but at that
> point it appeared not to work - however this morning with a fresh
> head, I have managed to get it working fine.
>
> Simply after building eXist-db with the xmlcalabash extension enabled -
>
> 1) rm $EXIST_HOME/lib/endorsed/saxonb*
>
> 2) rm $EXIST_HOME/lib/user/saxon9-s9api.jar
>
> 3) Place Saxon9he.jar (I used version 9.2.1.2) into $EXIST_HOME/lib/user
>
> It should all work for you then...


yes that works for me ... maybe hopefully some day I will get eXist
own xproc impl robust enough ... thats my next focus ... just about to
push out a little toy I have been working on (will announce via
twitter).

ta, J

Re: [Exist-development] eXist-db with Calabash and Saxon again

[Adam R. <ad...@ex...>]

> in the short term you maybe able to use standalone xmlcalabash with
> James Sulak eXist step library
>
> http://bitbucket.org/jasulak/exist-xproc-library/wiki/Home

Ah, a reverse integration, thats interesting!

Also, I have just discovered a way in which Henry could get this
working today for his needs (although it wold not be suitable for a
general release). We did try this together last night, but at that
point it appeared not to work - however this morning with a fresh
head, I have managed to get it working fine.

Simply after building eXist-db with the xmlcalabash extension enabled -

1) rm $EXIST_HOME/lib/endorsed/saxonb*

2) rm $EXIST_HOME/lib/user/saxon9-s9api.jar

3) Place Saxon9he.jar (I used version 9.2.1.2) into $EXIST_HOME/lib/user

It should all work for you then...

Cheers Adam.

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] eXist-db with Calabash and Saxon again

[James F. <jam...@ex...>]

On 10 September 2010 01:42, Adam Retter <ad...@ex...> wrote:
> Unfortunately I found this today whilst trying to show Henry Thompson
> how to use xmlcalabash XProc inside eXist-db -

thx Adam for the analysis,

Note that the current xmlcalabash extension is extremely primitive
because I noticed that Norman is refactoring calabash and I didn't
want to enshrine just yet.

I think the answer depends on Dr. Kay's response ... the eXist
community would be grateful if he could indicate if this is a priority
or something that he plans to work on; we did just switch eXist to use
SAXON by default now for XSLT processing perhaps we could announce
this a bit more widely (in the next release no doubt).

from pragmatic viewpoint, I would rather wait for a saxon release and
then code up a proper extension, but that doesn't help Henry ;)

J

[Exist-development] eXist-db with Calabash and Saxon again

[Adam R. <ad...@ex...>]

Unfortunately I found this today whilst trying to show Henry Thompson
how to use xmlcalabash XProc inside eXist-db -

{deep breath} Okay so whilst everything in the eXist-db codebase now
compiles nicely since I switched us over to XSLT2 (Saxon 9.1) by
default. The xmlcalabash xproc extension now throws an exception at
runtime...

Basically since xmlcalabash 0.9.16 (or thereabouts?), you have a
dependency on Saxon 9.2 as a minimum. Even though the xmlcalabash
documentation webpage states that Saxon 9.1.0.1 and up is supported, I
think that this must be incorrect as the changelog seems to indicate
otherwise as does the src code.

On the flip side, we are stuck using Saxon 9.1 and not 9.2, because
there is a classloader issue in Saxon 9.2 that causes a NPE when Saxon
9.2 is used from lib/endorsed (i.e. with the Java Application Servers
endorsed classloader), which it is in eXist-db.

Speaking to Mike Kay today (at the XML Summer School 2010) he has made
some fixes, but some more testing may need to be done. Also I am not
clear when the next release of Saxon 9.2 will be?

So we really have two options -

1) [Preferred] Revert to xmlcalabash 0.9.16 (current version is
0.9.23). This is the easiest approach, but I am not sure of the
impact. Looking at the changelog, there appears to of been quite a few
improvements since 0.9.16.
When the new release of Saxon 9.2 comes out, we can upgrade that also
upgtrade to the latest xmlcalabash etc.

2) Get or create a custom build of Saxon 9.2 where the classloader
issue is fixed.
We would need to also warn the betterForm guys if we move to Saxon
9.2, as they have a direct dependency on the Saxon version.

Or any other suggestions?

Cheers Adam.

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] [Exist-commits] SF.net SVN: exist:[12659] trunk/eXist/webapp/repo/packages

[Joe W. <jo...@gm...>]

>> Would anyone object to me adding *.xar to svn:ignore on
>> trunk/eXist/webapp/repo/packages/ ?
>
> no, but I have already removed it from source, should just be a
> generated file now.

Cool, thanks, I've made the commit.  This way, *.xar files don't
appear as changes to SVN clients.

Joe

Re: [Exist-development] [Exist-commits] SF.net SVN: exist:[12659] trunk/eXist/webapp/repo/packages

[James F. <jam...@ex...>]

On 9 September 2010 17:05, Joe Wicentowski <jo...@gm...> wrote:
> Would anyone object to me adding *.xar to svn:ignore on
> trunk/eXist/webapp/repo/packages/ ?

no, but I have already removed it from source, should just be a
generated file now.

J


>
> Joe
>
>
> On Wed, Sep 8, 2010 at 2:45 PM,  <cu...@us...> wrote:
>> Revision: 12659
>>          http://exist.svn.sourceforge.net/exist/?rev=12659&view=rev
>> Author:   cutlass
>> Date:     2010-09-08 18:45:17 +0000 (Wed, 08 Sep 2010)
>>
>> Log Message:
>> -----------
>> remove repo .xar from source
>>
>> Removed Paths:
>> -------------
>>    trunk/eXist/webapp/repo/packages/functx-1.0.xar
>>    trunk/eXist/webapp/repo/packages/math-1.0.xar
>>    trunk/eXist/webapp/repo/packages/myapp-0.1.xar
>>    trunk/eXist/webapp/repo/packages/xquery-json-0.1.xar
>>
>> Deleted: trunk/eXist/webapp/repo/packages/functx-1.0.xar
>> ===================================================================
>> (Binary files differ)
>>
>> Deleted: trunk/eXist/webapp/repo/packages/math-1.0.xar
>> ===================================================================
>> (Binary files differ)
>>
>> Deleted: trunk/eXist/webapp/repo/packages/myapp-0.1.xar
>> ===================================================================
>> (Binary files differ)
>>
>> Deleted: trunk/eXist/webapp/repo/packages/xquery-json-0.1.xar
>> ===================================================================
>> (Binary files differ)
>

Re: [Exist-development] [Exist-commits] SF.net SVN: exist:[12659] trunk/eXist/webapp/repo/packages

[Joe W. <jo...@gm...>]

Would anyone object to me adding *.xar to svn:ignore on
trunk/eXist/webapp/repo/packages/ ?

Joe


On Wed, Sep 8, 2010 at 2:45 PM,  <cu...@us...> wrote:
> Revision: 12659
>          http://exist.svn.sourceforge.net/exist/?rev=12659&view=rev
> Author:   cutlass
> Date:     2010-09-08 18:45:17 +0000 (Wed, 08 Sep 2010)
>
> Log Message:
> -----------
> remove repo .xar from source
>
> Removed Paths:
> -------------
>    trunk/eXist/webapp/repo/packages/functx-1.0.xar
>    trunk/eXist/webapp/repo/packages/math-1.0.xar
>    trunk/eXist/webapp/repo/packages/myapp-0.1.xar
>    trunk/eXist/webapp/repo/packages/xquery-json-0.1.xar
>
> Deleted: trunk/eXist/webapp/repo/packages/functx-1.0.xar
> ===================================================================
> (Binary files differ)
>
> Deleted: trunk/eXist/webapp/repo/packages/math-1.0.xar
> ===================================================================
> (Binary files differ)
>
> Deleted: trunk/eXist/webapp/repo/packages/myapp-0.1.xar
> ===================================================================
> (Binary files differ)
>
> Deleted: trunk/eXist/webapp/repo/packages/xquery-json-0.1.xar
> ===================================================================
> (Binary files differ)