exist-development — eXist Developer's List

Re: [Exist-development] Metadata in eXist

[Wolfgang M. <wol...@ex...>]

> It looks like there is a lot of thought around using key/value pairs. What
> value does key/value provide over using a stored XML (similar to MarkLogic)?
> Because on the inverse of that, I can see a lot of value in being able to
> store more complex structures, as they can be simple key/value pairs at
> their simplest and more complex for users who have that need.

I would prefer to use the appropriate standards for storing metadata,
for example, MODS for bibliographic information related to a document.
So basically I would not put any restrictions on the XML structure
which can be stored as metadata.

I don't like the idea of a property:: axis. If the metadata is an
ordinary document, you can just use standard XPath, e.g.
metadata("/db/my/doc.xml")//mods:titleInfo.

Wolfgang

[Exist-development] [FIXED] teamcity build should be working again

[James F. <jam...@ex...>]

found the problem, fixed and build should be working again

apologies for any interruptions, anyone give me a shout if you have
any problems.

James Fuller

[Exist-development] NOTE broken build on teamcity

[James F. <jam...@ex...>]

build is broken on teamcity at the moment, though I believe people
should be able to build locally.

working to resolve.

James Fuller

Re: [Exist-development] Metadata in eXist

[Andrzej J. T. <an...@ch...>]

Patrick offers:

> After reading over MarkLogic's implementation, I think it may be a very good
> idea to essentially mimic them. It looks like their system would accomplish
> most of the goals expressed here thus far. And I'm always a proponent of
> consistency.

+1 from me on that...seems like a very clean and usable approach.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Exist-development] IMPORTANT package management repository committed to trunk

[James F. <jam...@ex...>]

Hello All,

I have committed the package management repository which lets eXist
now uses expath repository to resolve external dependencies located in
repository to be used by xquery
and other XML technologies. You can add and remove .xar packages from eXist.

NOTE: I have tested pretty extensively but because this is affecting
the core you may have problems so please take precautions and email me
if you need any help

This is a heads up for everyone using trunk which is why I have posted
to both lists.

If you intend to update trunk you will *NEED* to do the following

* rebuild

* place the following in your conf.xml (its in conf.xml.tmpl)

    <module class="org.exist.xquery.modules.expathrepo.ExpathPackageModule"
            uri="http://exist-db.org/xquery/repo" />

I have written up some docs at
http://127.0.0.1:8080/exist/repo/repo.xml which should show you how to
install the functx-1.0.xar package.

Here is some more reference on how to create your own .xar packages.

* expath package:  http://expath.org/modules/pkg/
* expath package spec: http://expath.org/modules/pkg.html

Limitations

* you still need to restart eXist to pick up any changes to
repository, of course we will want to address this limitation of eXist
in the near future so we can have hot pluggable

* the repository is currently outside the database
(webapp/WEB-INF/expathrepo) but we may want to have an option to
create within the database

* we will want to review all the apps/examples/existing libs and pick
which ones are good candidates for converting to .xar archives


cheers, James Fuller

Re: [Exist-development] Metadata in eXist

[James F. <jam...@ex...>]

On 29 July 2010 16:27, Patrick Bosek <pat...@jo...> wrote:
> It looks like there is a lot of thought around using key/value pairs. What
> value does key/value provide over using a stored XML (similar to MarkLogic)?
> Because on the inverse of that, I can see a lot of value in being able to
> store more complex structures, as they can be simple key/value pairs at
> their simplest and more complex for users who have that need.

I think its safe to say that attributes do the same job as key/value
... e.g. I think introducing a new axis is a bit too much for this
kind of thing, we could just inject exist metadata attributes. This
means that when data is  backup/archive/extracted outside of eXist it
can retain its information.

> Also, which method is going to be easier to index with lucene?

good point and another reason why we should introduce metadata into
the natural xml structure instead of 'something else'.

James Fuller

Re: [Exist-development] Metadata in eXist

[Patrick B. <pat...@jo...>]

It looks like there is a lot of thought around using key/value pairs. What
value does key/value provide over using a stored XML (similar to MarkLogic)?
Because on the inverse of that, I can see a lot of value in being able to
store more complex structures, as they can be simple key/value pairs at
their simplest and more complex for users who have that need.

Also, which method is going to be easier to index with lucene?


Cheers,

Patrick

On Thu, Jul 29, 2010 at 4:32 AM, Evgeny Gazdovsky <gaz...@gm...>wrote:

> What about adding into metadata any kind of key/value pair
> whith fulltext indexing of values.
>
> We will able to add authors, descriptions, keywords,
> linked docs, and any more for any kind of XML or
> binary docs.
>
> Common value can be the XML fragment or all
> metadata can be one XML fragment like
> with fixed shem like
>
> <metada>
>    <property name="foo" value="bla-bla"/>
>    <property name="foo1">
>          <!--  any XML or base64 binary fragment here -->
>    </property>
> </metadata>
>
> or will have flex scheme
> <metadata>
>      <foo>bla-bla</foo>
>      <foo1>
>            <!--  any XML or base64 binary fragment here -->
>     </foo1>
> </metadata>
>
> the axis metadata:: will returns the "metadata" element for both cases
>
>
> --
> Evgeny
>
>
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Patrick Bosek
Jorsek Software
Cell (585) 820 9634
Office (585) 239 6060
Jorsek.com

Re: [Exist-development] betterform integration

[COUTHURES A. <ala...@ag...>]

Hello James and Joern,

James Fuller a écrit :
> On 29 July 2010 13:54, Joern Turner <joe...@go...> wrote:
>   
>> Ok, but did you notice that the links for XSLTForms do not work?
>>     
>
> though I did the initial integration of xsltforms, I do not maintain
> this area ... refactor as you see fit
>   
I can maintain myself XSLTForms within eXist if you're OK about that (I 
need an account for that, I presume ;-) ).

I saw recently that it didn't work, apparently because of missing config 
files.

I'm now busy with SVG support (I'm preparing my paper for SVG Open 2010 
in Paris at the end of August!) about graph2svg integration... but I'm 
also preparing the Beta 3 release with native XML documents.

Thanks!

 -Alain

Re: [Exist-development] util:system-property() should run under dba

[James F. <jam...@ex...>]

On 29 July 2010 14:03, Evgeny Gazdovsky <gaz...@gm...> wrote:
> My idea releted to security, like util:system-property() too.
>

ah, ok now I understand, nothing to do specifically with util:system-property()

> Also we can setup this kind of security
> (dba access for some func) in system config too
> instead harcoding in the java source code.

yes, this is a good idea but unsure if the definition should be inside
the database.

as you probably know, others (Dmitry, etc ) are discussing and working on this.

ta, J

Re: [Exist-development] util:system-property() should run under dba

[Evgeny G. <gaz...@gm...>]

My idea releted to security, like util:system-property() too.

Also we can setup this kind of security
(dba access for some func) in system config too
instead harcoding in the java source code.

--
Evgeny

Re: [Exist-development] betterform integration

[James F. <jam...@ex...>]

On 29 July 2010 13:54, Joern Turner <joe...@go...> wrote:
> On Thu, Jul 29, 2010 at 11:22 AM, James Fuller
> <jam...@ex...> wrote:
>> On 29 July 2010 11:12, Joern Turner <joe...@go...> wrote:
>>> Hi,
>>>
>>> after we have added betterform to the trunk there are some questions
>>> left to finish the first step:
>>>
>>> 1. we need to add ourselves to the start.config file. As we started as
>>> a intaller in an existing installation we have created our own copy of
>>> this file in the root of exist. I assume this is not the right way to
>>> do it now that betterform has become a part of the tree. I further
>>> assume that it's more appropriate to add our entries to
>>> src/org/exist/start/start.config? If yes, which condition would be
>>> appropriate - shall we use 'always'?
>>
>> u don't need to add here if any jars are being placed under user,
>> which is what I am assuming
> no, we have our libs below extensions/betteform ...
>
> and we have  a couple of class files there also.
>
> We tried to keep the libs separate as there is a clash with ehcache
> which we use in a newer version than cocoon (which also incorporates
> that)

if there are any java deps (jars, classes) they need to live under
$EXIST_HOME/lib
for extensions like betterforms that should be $EXIST_HOME/lib/user

your build will need to take care of deploying jars to these
directories if there is any runtime deps

>>
>>> 3. We have an issue with our demo application. This assumes that the
>>> admin password is defaulted to 'betterform' which is hardly
>>> appropriate any more. This there a way to determine this password from
>>> within an xquery or how can we otherwise manage to 'patch' our demo
>>> app files to have the right password. Of course we will remove this
>>> hardcoded value when the app moves on but for the time being it's
>>> necessary to make the demo run. Any guidance with this issue would be
>>> great.
>>
>> for trunk and testing purposes can you remove password for now ?
> sure, no problem.
>
> Another question regarding the demo: we have a target to install the
> demo but the database must be up and running to store the relevant
> files into the db. Is there a way to start the db from ant or at least
> a way to check that condition?
>
> Or - what would be the best practice to install that application?
> Probably an ant target is not the right thing here?

best practice is to install apps using xquery ... look under webapp
... you will see how admin area installs examples data/apps

though in the future, we will have repository mechanism (I am
committing this code today!).

>> I don't have time to review just at this moment to suggest other ways
>> to do this.
>>
>>> 4. What about docs? Once we have figured out how to (de)activate
>>> betterform i assume we need some docs about that. Is there someone
>>> having the hat on for documentation? Of course it's up to us to
>>> provide the docs but it should be linked from somewhere in the exist
>>> site (developer guide?). Do you have particular wishes about which
>>> docs you'd like to see?
>>
>> create a new directory under webapp/xforms e.g. webapp/xforms/betterforms
>>
>> and follow how other people have created docs
>>
>> I would like to see a top level xforms page with link to any and all
>> xforms extensions.
> Ok, but did you notice that the links for XSLTForms do not work?

though I did the initial integration of xsltforms, I do not maintain
this area ... refactor as you see fit

> I'd like to setup an introduction page for XForms listing the (now 2)
> option for using XForms in eXist and then link to separate pages for
> each engine - are you ok with that?

that seems like a reasonable approach, though other dev may have
different opinions.

ok lastly .... you need to make sure that you test everything before
you commit, I know you know this but its important, lots of people use
trunk and you will hear about it quickly if changes break or change
behavior ;)

ta, J

Re: [Exist-development] betterform integration

[Joern T. <joe...@go...>]

On Thu, Jul 29, 2010 at 11:22 AM, James Fuller
<jam...@ex...> wrote:
> On 29 July 2010 11:12, Joern Turner <joe...@go...> wrote:
>> Hi,
>>
>> after we have added betterform to the trunk there are some questions
>> left to finish the first step:
>>
>> 1. we need to add ourselves to the start.config file. As we started as
>> a intaller in an existing installation we have created our own copy of
>> this file in the root of exist. I assume this is not the right way to
>> do it now that betterform has become a part of the tree. I further
>> assume that it's more appropriate to add our entries to
>> src/org/exist/start/start.config? If yes, which condition would be
>> appropriate - shall we use 'always'?
>
> u don't need to add here if any jars are being placed under user,
> which is what I am assuming
no, we have our libs below extensions/betteform ...

and we have  a couple of class files there also.

We tried to keep the libs separate as there is a clash with ehcache
which we use in a newer version than cocoon (which also incorporates
that)

>
>> 2. we need some guidance of how to integrate with the exist build
>> system. Our extension files do not need compilation as we added the
>> binaries to the extension but we need to:
>> - modify the web.xml when betterform is supposed to be activated
>> - add ourselves to the start.config (as mentioned above)
>> - deploy our resources to the webapp/resources dir in exist
>> - copy a xquery error page to the webapp/xquery dir
>
> under build/scripts/build-impl.xml
>
> add a new target
>
>    <target name="extension-betterforms" depends="jar">
>        <ant antfile="your build file.xml"
> dir="extensions/betterforms" inheritall="false"/>
>    </target>
>
> and add in depends attribute in all target
ok, did that.

>
> you will also need to add in clean target the appropriate clean target
> of betterforms (using antfile)
>
>
>> 3. We have an issue with our demo application. This assumes that the
>> admin password is defaulted to 'betterform' which is hardly
>> appropriate any more. This there a way to determine this password from
>> within an xquery or how can we otherwise manage to 'patch' our demo
>> app files to have the right password. Of course we will remove this
>> hardcoded value when the app moves on but for the time being it's
>> necessary to make the demo run. Any guidance with this issue would be
>> great.
>
> for trunk and testing purposes can you remove password for now ?
sure, no problem.

Another question regarding the demo: we have a target to install the
demo but the database must be up and running to store the relevant
files into the db. Is there a way to start the db from ant or at least
a way to check that condition?

Or - what would be the best practice to install that application?
Probably an ant target is not the right thing here?


>
> I don't have time to review just at this moment to suggest other ways
> to do this.
>
>> 4. What about docs? Once we have figured out how to (de)activate
>> betterform i assume we need some docs about that. Is there someone
>> having the hat on for documentation? Of course it's up to us to
>> provide the docs but it should be linked from somewhere in the exist
>> site (developer guide?). Do you have particular wishes about which
>> docs you'd like to see?
>
> create a new directory under webapp/xforms e.g. webapp/xforms/betterforms
>
> and follow how other people have created docs
>
> I would like to see a top level xforms page with link to any and all
> xforms extensions.
Ok, but did you notice that the links for XSLTForms do not work?

I'd like to setup an introduction page for XForms listing the (now 2)
option for using XForms in eXist and then link to separate pages for
each engine - are you ok with that?

Thanks,

Joern Turner
>
> hth, James Fuller
>

Re: [Exist-development] util:system-property() should run under dba

[James F. <jam...@ex...>]

On 29 July 2010 11:04, Evgeny Gazdovsky <gaz...@gm...> wrote:
>> unsure what you mean by config doc ...  do you mean, for example, the
>> 'load function documentation'  which runs under admin now ?

> Just we must use password in xquery to run some from another user:
> system:as-user($user, $password, $sec)
>
> But but we can use system:as-user($user, $sec)
> and not use password in code.
> For this we can create config doc in system collection
> in which we set up wich script is able to delegate
> access rights to $user.

Hello Gev,

sure I always have a problem embedding secrets like passwords into
code, but what does this have to do with util:system-property()
function ... are you mentioning this because its related to security
or related to the function (and this post) ?

thx, J

Re: [Exist-development] util:system-property() should run under dba

[Dannes W. <da...@ex...>]

Hi,

On Thu, Jul 29, 2010 at 10:06 AM, James Fuller
<jam...@ex...> wrote:
> on second thought I would let it run under guest but only basic
> properties (vendor, vendor-url, etc)
>
> when run under dba java properties get exposed.

That is the cleanest way, this way it should be implemented.

Still unsure what the ideas of Gev are about....

cheers

Dannes



-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] betterform integration

[James F. <jam...@ex...>]

On 29 July 2010 11:12, Joern Turner <joe...@go...> wrote:
> Hi,
>
> after we have added betterform to the trunk there are some questions
> left to finish the first step:
>
> 1. we need to add ourselves to the start.config file. As we started as
> a intaller in an existing installation we have created our own copy of
> this file in the root of exist. I assume this is not the right way to
> do it now that betterform has become a part of the tree. I further
> assume that it's more appropriate to add our entries to
> src/org/exist/start/start.config? If yes, which condition would be
> appropriate - shall we use 'always'?

u don't need to add here if any jars are being placed under user,
which is what I am assuming

> 2. we need some guidance of how to integrate with the exist build
> system. Our extension files do not need compilation as we added the
> binaries to the extension but we need to:
> - modify the web.xml when betterform is supposed to be activated
> - add ourselves to the start.config (as mentioned above)
> - deploy our resources to the webapp/resources dir in exist
> - copy a xquery error page to the webapp/xquery dir

under build/scripts/build-impl.xml

add a new target

    <target name="extension-betterforms" depends="jar">
        <ant antfile="your build file.xml"
dir="extensions/betterforms" inheritall="false"/>
    </target>

and add in depends attribute in all target

you will also need to add in clean target the appropriate clean target
of betterforms (using antfile)


> 3. We have an issue with our demo application. This assumes that the
> admin password is defaulted to 'betterform' which is hardly
> appropriate any more. This there a way to determine this password from
> within an xquery or how can we otherwise manage to 'patch' our demo
> app files to have the right password. Of course we will remove this
> hardcoded value when the app moves on but for the time being it's
> necessary to make the demo run. Any guidance with this issue would be
> great.

for trunk and testing purposes can you remove password for now ?

I don't have time to review just at this moment to suggest other ways
to do this.

> 4. What about docs? Once we have figured out how to (de)activate
> betterform i assume we need some docs about that. Is there someone
> having the hat on for documentation? Of course it's up to us to
> provide the docs but it should be linked from somewhere in the exist
> site (developer guide?). Do you have particular wishes about which
> docs you'd like to see?

create a new directory under webapp/xforms e.g. webapp/xforms/betterforms

and follow how other people have created docs

I would like to see a top level xforms page with link to any and all
xforms extensions.

hth, James Fuller

[Exist-development] betterform integration

[Joern T. <joe...@go...>]

Hi,

after we have added betterform to the trunk there are some questions
left to finish the first step:

1. we need to add ourselves to the start.config file. As we started as
a intaller in an existing installation we have created our own copy of
this file in the root of exist. I assume this is not the right way to
do it now that betterform has become a part of the tree. I further
assume that it's more appropriate to add our entries to
src/org/exist/start/start.config? If yes, which condition would be
appropriate - shall we use 'always'?

2. we need some guidance of how to integrate with the exist build
system. Our extension files do not need compilation as we added the
binaries to the extension but we need to:
- modify the web.xml when betterform is supposed to be activated
- add ourselves to the start.config (as mentioned above)
- deploy our resources to the webapp/resources dir in exist
- copy a xquery error page to the webapp/xquery dir

All this is done by calling 'ant install' in extensions/betterform.
The changes on the tree can be reverted with 'ant uninstall'.

Now the questions is: where in the exist build files should the
betterform build file be called? Or do you prefer a different
mechanism?

3. We have an issue with our demo application. This assumes that the
admin password is defaulted to 'betterform' which is hardly
appropriate any more. This there a way to determine this password from
within an xquery or how can we otherwise manage to 'patch' our demo
app files to have the right password. Of course we will remove this
hardcoded value when the app moves on but for the time being it's
necessary to make the demo run. Any guidance with this issue would be
great.

Ah, not to forget - the demo application can be installed by calling
'ant install-xrx' on our build.xml.

4. What about docs? Once we have figured out how to (de)activate
betterform i assume we need some docs about that. Is there someone
having the hat on for documentation? Of course it's up to us to
provide the docs but it should be linked from somewhere in the exist
site (developer guide?). Do you have particular wishes about which
docs you'd like to see?

Thanks,

Joern

Re: [Exist-development] util:system-property() should run under dba

[Evgeny G. <gaz...@gm...>]

> unsure what you mean by config doc ...  do you mean, for example, the
> 'load function documentation'  which runs under admin now ?
>
> J
>

Just we must use password in xquery to run some from another user:
system:as-user($user, $password, $sec)

But but we can use system:as-user($user, $sec)
and not use password in code.
For this we can create config doc in system collection
in which we set up wich script is able to delegate
access rights to $user.

--
Evgeny

Re: [Exist-development] Granular access to system functions, WAS: util:system-property() should run under dba

[James F. <jam...@ex...>]

On 29 July 2010 10:40, Thomas White <tho...@gm...> wrote:
> James,
>
> I think we need to define the access to the variety of system commands on a
> more granular level.
>
> I would like to propose a design that will not only keep the current
> execution restrictions but also adds an option to grand access to a module
> or even to a specific function to group.
>
> For every function that needs to have a controlled manner of execution, we
> can have a list of group names that are allowed to call this function. The
> default group names are DBA and a group with name as the name of the
> module. Then every eXist instance may add application specific groups with
> rights to execute a function or all functions for a module.
>
> We can have the default list of groups in a config file that will be
> imported into the db initially and later only the db  version will be used.

good ideas, though whatever effort we do we need to be mindful that we
may eventually end up with

http://exist.sourceforge.net/xacml-intro.html

which is already integrated into eXist, though I know not the status
of this code (I think its best to view this code stale and untested).

I think we need to be pragmatic with our approach, but admittedly I
don't have well formed ideas of just how to do this.

J

[Exist-development] Granular access to system functions, WAS: util:system-property() should run under dba

[Thomas W. <tho...@gm...>]

James,

I think we need to define the access to the variety of system commands on a
more granular level.

 I would like to propose a design that will not only keep the current
execution restrictions but also adds an option to grand access to a module
or even to a specific function to group.

For every function that needs to have a controlled manner of execution, we
can have a list of group names that are allowed to call this function. The
default group names are DBA and a group with name as the name of the
module. Then every eXist instance may add application specific groups with
rights to execute a function or all functions for a module.

We can have the default list of groups in a config file that will be
imported into the db initially and later only the db  version will be used.

What do you think?

Thomas

------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007



On 29 July 2010 09:06, James Fuller <jam...@ex...> wrote:

> on second thought I would let it run under guest but only basic
> properties (vendor, vendor-url, etc)
>
> when run under dba java properties get exposed.
>
> On 29 July 2010 09:55, Evgeny Gazdovsky <gaz...@gm...> wrote:
> > Sure!
> >
> > Also, suggest add the config doc into system collection,
> > where we can setup the the xquery URI scripts, which
> > can use some functions from other user without password.
>
> unsure what you mean by config doc ...  do you mean, for example, the
> 'load function documentation'  which runs under admin now ?
>
> J
>
>
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>

Re: [Exist-development] Metadata in eXist

[Evgeny G. <gaz...@gm...>]

What about adding into metadata any kind of key/value pair
whith fulltext indexing of values.

We will able to add authors, descriptions, keywords,
linked docs, and any more for any kind of XML or
binary docs.

Common value can be the XML fragment or all
metadata can be one XML fragment like
with fixed shem like

<metada>
    <property name="foo" value="bla-bla"/>
    <property name="foo1">
          <!--  any XML or base64 binary fragment here -->
    </property>
</metadata>

or will have flex scheme
<metadata>
      <foo>bla-bla</foo>
      <foo1>
            <!--  any XML or base64 binary fragment here -->
     </foo1>
</metadata>

the axis metadata:: will returns the "metadata" element for both cases


--
Evgeny

Re: [Exist-development] Metadata in eXist

[Thomas W. <tho...@gm...>]

Adam,

I like the idea of metadata axis. Your proposal for name/value pairs is
simple and it can accommodate most of variations we may need.

2010/7/28 Adam Retter <ad...@ex...>

> >        MarkLogic implementation is interesting and a good starting point,
> but I guess it does not allow attaching binary contents as metadata, which
> is a limitation. Some scenarios can be, for instance, a JPEG image stored in
> eXist could have some metadata in XML describing its EXIF properties
>
> Accessing this metadata at the moment is possible via the image xquery
> extension module.
>

Actually the the existing image module gives vary limited metadata: hight,
width and image type.
It will really love to have all EXIF properties of an image as searchable
metadata fields.


>
> > , and binary metadata like a thumbnail, and SVG or XSL-FO documents could
> also have as binary metadata a thumbnail.
>
> Well my proposal was to use the XQuery datatypes, so then the user could
> set metadata to be base64binary data if desired.

 --
> Adam Retter
>

What will be the format of the metadata if we request all available pairs?
May be an element with all metadata pairs as attributes.

Thomas

Re: [Exist-development] util:system-property() should run under dba

[James F. <jam...@ex...>]

on second thought I would let it run under guest but only basic
properties (vendor, vendor-url, etc)

when run under dba java properties get exposed.

On 29 July 2010 09:55, Evgeny Gazdovsky <gaz...@gm...> wrote:
> Sure!
>
> Also, suggest add the config doc into system collection,
> where we can setup the the xquery URI scripts, which
> can use some functions from other user without password.

unsure what you mean by config doc ...  do you mean, for example, the
'load function documentation'  which runs under admin now ?

J

Re: [Exist-development] util:system-property() should run under dba

[Evgeny G. <gaz...@gm...>]

Sure!

Also, suggest add the config doc into system collection,
where we can setup the the xquery URI scripts, which
can use some functions from other user without password.

--
Evgeny

[Exist-development] util:system-property() should run under dba

[James F. <jam...@ex...>]

considering java properties are exposed from util:system-property, its
probably a security risk to let guest user run

e.g.

util:system-property("exist.home")

will reveal path information for example

thoughts ? Note I think we need to have a general activity or way of
defining rights for functions/modules and probably need to review at
some point.

James Fuller

Re: [Exist-development] [Exist-commits] Session Triggers - Re: SF.net SVN: exist:[12086] trunk/eXist

[Loren C. <lor...@gm...>]

Hello Adam,

I am all for that.  I will wait for your trigger refactoring to make the change.

Loren

On Jul 28, 2010, at 12:43 PM, Adam Retter wrote:

> This functionality should be integrated with Triggers.
> 
> It also make no sense to configure this separately via the command
> line! This should be done in conf.xml!
> 
> Thanks Adam.
> 
> 
> On 28 July 2010 18:06,  <ix...@us...> wrote:
>> Revision: 12086
>>          http://exist.svn.sourceforge.net/exist/?rev=12086&view=rev
>> Author:   ixitar
>> Date:     2010-07-28 17:06:45 +0000 (Wed, 28 Jul 2010)
>> 
>> Log Message:
>> -----------
>> [feature] Adding the ability to execute an XQuery script when an HTTP session is created and another XQuery script when an HTTP session is destroyed.
>> 
>> Two listeners are being added to the server:
>>  -  org.exist.http.AuditTrailSessionListener
>>  -  org.exist.http.SessionCountListener
>> 
>> NOTE: This feature is disabled by default, so it will have no impact on a system until enabled.
>> 
>> These listeners can be enabled by uncommenting the following in tools/jetty/etc/webdefaults.xml:
>> 
>>    <listener>
>>        <listener-class>org.exist.http.AuditTrailSessionListener</listener-class>
>>    </listener>
>>    <listener>
>>        <listener-class>org.exist.http.SessionCountListener</listener-class>
>>    </listener>
>> 
>> To change the session timeout from the default of 30 minutes, change the following value in webdefaults.xml:
>> 
>>  <session-config>
>>    <session-timeout>30</session-timeout>
>>  </session-config>
>> 
>> 
>> When the AuditTrailSessionListener is enabled, then there are two Java properties that need to be set:
>> 
>>  -  org.exist.http.session_create_listener
>>  -  org.exist.http.session_destroy_listener
>> 
>> If a property is not set, then the listener will do nothing for the create or destroy.
>> 
>> If the property is set, but the XQuery script does not exist, then a Log4J error message "Resource [resource path] does not exist." is processed and then no other action is taken.
>> 
>> If the resource exists and there is an execution failure, then a Log4J error message "Exception while executing [resource path] script for admin" is processed and then no other action is taken.
>> 
>> If the resource exists and the execution succeeds, then a Log4J info message "XQuery execution results: results" is processed and then the method is completed.
>> 
>> To set these properties, you need to edit bin/functions.d/eXist-settings.sh if starting from the command line in Unix/Linux:
>> 
>>        set_java_options() {
>>            if [ -z "${JAVA_OPTIONS}" ]; then
>>                JAVA_OPTIONS="-Xms128m -Xmx512m -Dfile.encoding=UTF-8 -Dorg.exist.http.session_destroy_listener=/db/session/session-destroy.xq";
>>            fi
>>            JAVA_OPTIONS="${JAVA_OPTIONS} -Djava.endorsed.dirs=${JAVA_ENDORSED_DIRS}";
>>        }
>> 
>> To set these properties for The command line execution in Windows, you need to edit bin/startup.bat:
>> 
>>        :gotExistHome
>>        set JAVA_ENDORSED_DIRS="%EXIST_HOME%"\lib\endorsed
>>        set JAVA_OPTS="-Xms128m -Xmx512m -Dfile.encoding=UTF-8 -Dorg.exist.http.session_destroy_listener=/db/session/session-destroy.xq -Djava.endorsed.dirs=%JAVA_ENDORSED_DIRS%"
>> 
>> To set these properties in wrapper, you need to edit tools/wrapper/conf/wrapper.conf:
>> 
>>        # Java Additional Parameters
>>        wrapper.java.additional.1=-Djava.endorsed.dirs=../../../lib/endorsed
>>        wrapper.java.additional.2=-Dfile.encoding=UTF-8
>>        wrapper.java.additional.3=-Dexist.home=../../..
>>        wrapper.java.additional.4=-Djetty.home=../../../tools/jetty
>>        wrapper.java.additional.5=-Dorg.exist.http.session_create_listener=/db/session/session-create.xq
>>        wrapper.java.additional.6=-Dorg.exist.http.session_destroy_listener=/db/session/session-destroy.xq
>> 
>> 
>> - - - - - - - - - - - - - - -
>> The SessionCountListener will return the number of open sessions are on the server, the way to access the value is through SessionCountListener.getActiveSessions() which returns a long. An XQuery function will be implemented to return this value.
>> 
>> Modified Paths:
>> --------------
>>    trunk/eXist/tools/jetty/etc/webdefault.xml
>> 
>> Added Paths:
>> -----------
>>    trunk/eXist/src/org/exist/http/AuditTrailSessionListener.java
>>    trunk/eXist/src/org/exist/http/SessionCountListener.java
>> 
>> Added: trunk/eXist/src/org/exist/http/AuditTrailSessionListener.java
>> ===================================================================
>> --- trunk/eXist/src/org/exist/http/AuditTrailSessionListener.java                               (rev 0)
>> +++ trunk/eXist/src/org/exist/http/AuditTrailSessionListener.java       2010-07-28 17:06:45 UTC (rev 12086)
>> @@ -0,0 +1,151 @@
>> +/*
>> + *  eXist Open Source Native XML Database
>> + *  Copyright (C) 2010 The eXist Project
>> + *  http://exist-db.org
>> + *
>> + *  This program is free software; you can redistribute it and/or
>> + *  modify it under the terms of the GNU Lesser General Public License
>> + *  as published by the Free Software Foundation; either version 2
>> + *  of the License, or (at your option) any later version.
>> + *
>> + *  This program is distributed in the hope that it will be useful,
>> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + *  GNU Lesser General Public License for more details.
>> + *
>> + *  You should have received a copy of the GNU Lesser General Public License
>> + *  along with this program; if not, write to the Free Software
>> + *  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
>> + *
>> + * $Id$
>> + */
>> +package org.exist.http;
>> +
>> +import org.apache.log4j.Logger;
>> +import org.exist.dom.BinaryDocument;
>> +import org.exist.dom.DocumentImpl;
>> +import org.exist.security.User;
>> +import org.exist.security.xacml.AccessContext;
>> +import org.exist.source.DBSource;
>> +import org.exist.source.Source;
>> +import org.exist.storage.BrokerPool;
>> +import org.exist.storage.DBBroker;
>> +import org.exist.storage.lock.Lock;
>> +import org.exist.xmldb.XmldbURI;
>> +import org.exist.xquery.CompiledXQuery;
>> +import org.exist.xquery.XQuery;
>> +import org.exist.xquery.XQueryContext;
>> +import org.exist.xquery.value.Sequence;
>> +
>> +import javax.servlet.http.HttpSessionListener;
>> +import javax.servlet.http.HttpSessionEvent;
>> +import javax.servlet.http.HttpSession;
>> +import java.util.Properties;
>> +
>> +/**
>> + * Executes an XQuery script whose filename is retrieved from the
>> + * java option 'org.exist.http.session_create_listener' when an
>> + * HTTP session is created and 'org.exist.http.session_destroy_listener'
>> + * when an HTTP session is destroyed.
>> + *
>> + * If the java option is not set, then do nothing.
>> + *
>> + * If the java option is set, then retrieve the script from the file
>> + * or resource designated by the value of the property.  Execute the
>> + * XQuery script to record the creation or destruction of a HTTP session.
>> + */
>> +public class AuditTrailSessionListener implements HttpSessionListener {
>> +
>> +    private final static Logger LOG = Logger.getLogger(AuditTrailSessionListener.class);
>> +    private static final String REGISTER_CREATE_XQUERY_SCRIPT_PROPERTY = "org.exist.http.session_create_listener";
>> +    private static final String REGISTER_DESTROY_XQUERY_SCRIPT_PROPERTY = "org.exist.http.session_destroy_listener";
>> +
>> +
>> +
>> +    /**
>> +     *
>> +     * @param sessionEvent
>> +     */
>> +    public void sessionCreated(HttpSessionEvent sessionEvent) {
>> +        HttpSession session = sessionEvent.getSession();
>> +
>> +        LOG.info("session created " + session.getId());
>> +        String xqueryResourcePath = System.getProperty(REGISTER_CREATE_XQUERY_SCRIPT_PROPERTY);
>> +        executeXQuery(xqueryResourcePath);
>> +    }
>> +
>> +    /**
>> +     *
>> +     * @param sessionEvent
>> +     */
>> +    public void sessionDestroyed(HttpSessionEvent sessionEvent) {
>> +        HttpSession session = (sessionEvent != null) ? sessionEvent.getSession() : null;
>> +        if (session != null)
>> +            LOG.info("destroy session " + session.getId());
>> +        else
>> +            LOG.info("destroy session");
>> +
>> +        String xqueryResourcePath = System.getProperty(REGISTER_DESTROY_XQUERY_SCRIPT_PROPERTY);
>> +        executeXQuery(xqueryResourcePath);
>> +    }
>> +
>> +    private void executeXQuery(String xqueryResourcePath) {
>> +        if (xqueryResourcePath != null && xqueryResourcePath.length() > 0) {
>> +            xqueryResourcePath = xqueryResourcePath.trim();
>> +            BrokerPool pool = null;
>> +            DBBroker broker = null;
>> +            User principal = null;
>> +
>> +            try {
>> +                DocumentImpl resource = null;
>> +                Source source = null;
>> +
>> +                pool = BrokerPool.getInstance();
>> +                principal = pool.getSecurityManager().getSystemAccount();
>> +
>> +                broker = pool.get(principal);
>> +                if (broker == null) {
>> +                    LOG.error("Unable to retrieve DBBroker for " + principal.getName());
>> +                    return;
>> +                }
>> +
>> +                XmldbURI pathUri = XmldbURI.create(xqueryResourcePath);
>> +
>> +
>> +                resource = broker.getXMLResource(pathUri, Lock.READ_LOCK);
>> +
>> +                if(resource != null) {
>> +                    LOG.info("Resource [" + xqueryResourcePath + "] exists.");
>> +                    source = new DBSource(broker, (BinaryDocument)resource, true);
>> +                } else {
>> +                    LOG.error("Resource [" + xqueryResourcePath + "] does not exist.");
>> +                    return;
>> +                }
>> +
>> +
>> +                XQuery xquery = broker.getXQueryService();
>> +
>> +                if (xquery == null) {
>> +                    LOG.error("broker unable to retrieve XQueryService");
>> +                    return;
>> +                }
>> +
>> +                XQueryContext context = xquery.newContext(AccessContext.REST);
>> +
>> +                CompiledXQuery compiled = xquery.compile(context, source);
>> +
>> +                Properties outputProperties = new Properties();
>> +
>> +                Sequence result = xquery.execute(compiled, null, outputProperties);
>> +                LOG.info("XQuery execution results: " + result.toString());
>> +
>> +            } catch (Exception e) {
>> +                LOG.error("Exception while executing [" + xqueryResourcePath + "] script for " + principal.getName(), e);
>> +            }
>> +            finally {
>> +                if (pool != null)
>> +                    pool.release(broker);
>> +            }
>> +        }
>> +    }
>> +}
>> \ No newline at end of file
>> 
>> 
>> Property changes on: trunk/eXist/src/org/exist/http/AuditTrailSessionListener.java
>> ___________________________________________________________________
>> Added: svn:keywords
>>   + Id
>> 
>> Added: trunk/eXist/src/org/exist/http/SessionCountListener.java
>> ===================================================================
>> --- trunk/eXist/src/org/exist/http/SessionCountListener.java                            (rev 0)
>> +++ trunk/eXist/src/org/exist/http/SessionCountListener.java    2010-07-28 17:06:45 UTC (rev 12086)
>> @@ -0,0 +1,50 @@
>> +/*
>> + *  eXist Open Source Native XML Database
>> + *  Copyright (C) 2010 The eXist Project
>> + *  http://exist-db.org
>> + *
>> + *  This program is free software; you can redistribute it and/or
>> + *  modify it under the terms of the GNU Lesser General Public License
>> + *  as published by the Free Software Foundation; either version 2
>> + *  of the License, or (at your option) any later version.
>> + *
>> + *  This program is distributed in the hope that it will be useful,
>> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + *  GNU Lesser General Public License for more details.
>> + *
>> + *  You should have received a copy of the GNU Lesser General Public License
>> + *  along with this program; if not, write to the Free Software
>> + *  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
>> + *
>> + * $Id$
>> + */
>> +package org.exist.http;
>> +
>> +import javax.servlet.http.HttpSessionEvent;
>> +import javax.servlet.http.HttpSessionListener;
>> +
>> +/**
>> + * Created by IntelliJ IDEA.
>> + * User: lcahlander
>> + * Date: Jul 27, 2010
>> + * Time: 10:31:47 PM
>> + * To change this template use File | Settings | File Templates.
>> + */
>> +public class SessionCountListener implements HttpSessionListener {
>> +
>> +    private static long activeSessions = 0;
>> +
>> +    public void sessionCreated(HttpSessionEvent httpSessionEvent) {
>> +        activeSessions++;
>> +    }
>> +
>> +    public void sessionDestroyed(HttpSessionEvent httpSessionEvent) {
>> +        if (activeSessions > 0)
>> +            activeSessions--;
>> +    }
>> +
>> +    public static long getActiveSessions() {
>> +        return activeSessions;
>> +    }
>> +}
>> 
>> 
>> Property changes on: trunk/eXist/src/org/exist/http/SessionCountListener.java
>> ___________________________________________________________________
>> Added: svn:keywords
>>   + Id
>> 
>> Modified: trunk/eXist/tools/jetty/etc/webdefault.xml
>> ===================================================================
>> --- trunk/eXist/tools/jetty/etc/webdefault.xml  2010-07-28 16:49:11 UTC (rev 12085)
>> +++ trunk/eXist/tools/jetty/etc/webdefault.xml  2010-07-28 17:06:45 UTC (rev 12086)
>> @@ -29,6 +29,22 @@
>>     This file is applied to a Web application before it's own WEB_INF/web.xml file
>>   </description>
>> 
>> +    <!-- ======================================================================== -->
>> +    <!-- The AuditTrailSessionListener allows for the execution of XQuery scripts -->
>> +    <!-- when an HTTP session is created or destroyed.  Set the following Java    -->
>> +    <!-- properties to specify the scripts to run.                                -->
>> +    <!--                                                                          -->
>> +    <!-- org.exist.http.session_create_listener                                   -->
>> +    <!-- org.exist.http.session_destroy_listener                                  -->
>> +    <!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  -->
>> +    <!-- UNCOMMENT TO ACTIVATE
>> +    <listener>
>> +        <listener-class>org.exist.http.AuditTrailSessionListener</listener-class>
>> +    </listener>
>> +    <listener>
>> +        <listener-class>org.exist.http.SessionCountListener</listener-class>
>> +    </listener>
>> +    -->
>> 
>>   <!-- ==================================================================== -->
>>   <!-- Context params to control Session Cookies                            -->
>> 
>> 
>> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
>> 
>> ------------------------------------------------------------------------------
>> The Palm PDK Hot Apps Program offers developers who use the
>> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> of $1 Million in cash or HP Products. Visit us here for more details:
>> http://p.sf.net/sfu/dev2dev-palm
>> _______________________________________________
>> Exist-commits mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-commits
>> 
> 
> 
> 
> -- 
> Adam Retter
> 
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
> 
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> Exist-commits mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-commits