exist-development — eXist Developer's List

[Exist-development] DAVFS: Could not authenticate to server: rejected Basic challenge

[Вячеслав С. <sch...@gm...>]

got from fresh trunk

something new?

Re: [Exist-development] [ANN] Community Teleconf Weekly -> Monthly

[Adam R. <ad...@ex...>]

So I was asked to arrange this poll, and the vote indicates that it
will be on the 3rd Wednesday of each month at 20:00 CEST = 14:00 EDT

The first is tomorrow i.e. 20th July 2011.

However, I also noticed that nothing had been added to Google
Moderator, so I am not sure how much interest there really is.

Either way I will make sure that I am available for this tomorrow,
everyone should meet in our eXist-db IRC
(irc://chat.freenode.org/#existdb) channel at 20:00 CEST, from there
we can move onto Skype or whatever if there is enough demand.

Thanks Adam.



On 8 July 2011 20:29, Adam Retter <ad...@ex...> wrote:
> I have had two community members both ask me today what happened to
> the Friday teleconf -
>
> The weekly Friday teleconf was not really working IMHO as it was two
> frequent, therefore the same topics were often re-discussed and it did
> not give people time to take action on decisions, and therefore it
> fizzled out :-/
>
> So... I propose that we have a Monthly teleconf instead, which
> involves all available developers and interested users.
>
> I think a Weds evening at 8pm CEST would work well for Europe and the
> US at 2pm EDT.
> I would also propose that the first one would be on July 20th.
> But in the spirit of the community I have created a Doodle Poll so you
> can vote - http://www.doodle.com/qzrbnvqxbrdg2p6h
>
> I have also setup this Google Moderator series -
> http://www.google.com/moderator/#15/e=9cebb&t=9cebb.40
> You can use it to add topics that you want to discuss/demo/present or
> suggestions/requests for topics for others to
> discuss/demo/present/explain.
>
> Cheers Adam.
>
> --
> Adam Retter
>
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Problem backing up resource permissions

[Kasper L. <kl...@vi...>]

On Tue, Jul 12, 2011 at 2:31 PM, Dmitriy Shabanov <sha...@gm...>wrote:

> eXist version?
>
>
I have reproduced it both in version 1.4 (stable) and 1.5 from trunk. The
code is basically identical.

/Kasper

Re: [Exist-development] Problem backing up resource permissions

[Dmitriy S. <sha...@gm...>]

eXist version?

On 7/12/11, Kasper Laudrup <kl...@vi...> wrote:
> Hi eXist developers and users,
>
> I've noticed a problem when using the backup functionality in eXist.
> Sometimes, but not always, the ownership and permissions stored in the
> __contents__.xml metadata file, are incorrect. A way to reproduce this is to
> create an empty xml dcoument in a collection, change the ownership and/or
> permissions and then backup that collection. Attempting to restore that
> collection or looking at the contents of the __contents__.xml file, it is
> clear that the permissions are not backed up correctly.
>
> I've had a look at the code in org/exist/backup/Backup.java and near the
> start of the method backup(Collection current, BackupWriter output,
> BackupDialog dialog) I think I've spotted the problem. First of all, if I
> understand the code correctly, it works with two different arrays - one
> which contains the resources to backup and another which contains the
> permissions to backup. IMHO that seems quite dangerous, since there isn't
> even, as far as I can tell, a way to make sure that these two arrays are
> synchronized. One must simply assume that the contents are in the same
> order. Now where it gets really interesting is the point in the method where
> there is a call to Arrays.sort(resources), without a similar call for the
> permissions. In other words it seems like effort has been made to make sure
> that the arrays are not synchronized. I have a very hard time understanding
> the reason for that.
>
> Anyway, if I remove the offending call to sort, I'm unable to reproduce the
> bug. I am of course more than willing the submit a patch that removes the
> offending line, but someone might have some insight into why this was
> written this way?
>
> Hope this is the correct mailing list, otherwise please point me to the
> right one.
>
> Thanks for your help.
>
> Kasper Laudrup
>
> ViewWorld Aps.
>


-- 
Dmitriy Shabanov

[Exist-development] Problem backing up resource permissions

[Kasper L. <kl...@vi...>]

Hi eXist developers and users,

I've noticed a problem when using the backup functionality in eXist.
Sometimes, but not always, the ownership and permissions stored in the
__contents__.xml metadata file, are incorrect. A way to reproduce this is to
create an empty xml dcoument in a collection, change the ownership and/or
permissions and then backup that collection. Attempting to restore that
collection or looking at the contents of the __contents__.xml file, it is
clear that the permissions are not backed up correctly.

I've had a look at the code in org/exist/backup/Backup.java and near the
start of the method backup(Collection current, BackupWriter output,
BackupDialog dialog) I think I've spotted the problem. First of all, if I
understand the code correctly, it works with two different arrays - one
which contains the resources to backup and another which contains the
permissions to backup. IMHO that seems quite dangerous, since there isn't
even, as far as I can tell, a way to make sure that these two arrays are
synchronized. One must simply assume that the contents are in the same
order. Now where it gets really interesting is the point in the method where
there is a call to Arrays.sort(resources), without a similar call for the
permissions. In other words it seems like effort has been made to make sure
that the arrays are not synchronized. I have a very hard time understanding
the reason for that.

Anyway, if I remove the offending call to sort, I'm unable to reproduce the
bug. I am of course more than willing the submit a patch that removes the
offending line, but someone might have some insight into why this was
written this way?

Hope this is the correct mailing list, otherwise please point me to the
right one.

Thanks for your help.

Kasper Laudrup

ViewWorld Aps.

Re: [Exist-development] Conflicting (double) jar files in trunk

[Dmitriy S. <sha...@gm...>]

On Tue, Jul 12, 2011 at 1:51 AM, Dannes Wessels <da...@ex...> wrote:

> All,
>
> recently  some 'double' jar files have been added to trunk:
>
> ./extensions/betterform/src/main/lib/httpmime-4.1.1.jar
> ./extensions/expath/lib/httpclient-4.1.jar
> ./extensions/expath/lib/httpcore-4.1.jar
> ./extensions/security/oauth/lib/httpclient-4.0.1.jar
> ./extensions/security/oauth/lib/httpcore-4.0.1.jar
> ./extensions/security/oauth/lib/httpmime-4.0.1.jar
>
> This is a situation we clearly do not want, it will cause problems in the
> end (we have no clean classloader in place).  I'd like to have this cleaned,
> and have the latest & greatest versions only.
>
> Probably we should move these jars to a location where these jars are
> shared, in lib/extentions, lib/optional or in extensions/lib or so....
>

Name the place, please :-)


>
-- 
Dmitriy Shabanov

[Exist-development] Conflicting (double) jar files in trunk

[Dannes W. <da...@ex...>]

All,

recently  some 'double' jar files have been added to trunk:

./extensions/betterform/src/main/lib/httpmime-4.1.1.jar
./extensions/expath/lib/httpclient-4.1.jar
./extensions/expath/lib/httpcore-4.1.jar
./extensions/security/oauth/lib/httpclient-4.0.1.jar
./extensions/security/oauth/lib/httpcore-4.0.1.jar
./extensions/security/oauth/lib/httpmime-4.0.1.jar

This is a situation we clearly do not want, it will cause problems in the end (we have no clean classloader in place).  I'd like to have this cleaned, and have the latest & greatest versions only. 

Probably we should move these jars to a location where these jars are shared, in lib/extentions, lib/optional or in extensions/lib or so....

regards

Dannes


--
Dannes Wessels
eXist-db Open Source Native XML Database
e: da...@ex...
w: http://www.exist-db.org 

[Exist-development] [ANN] Community Teleconf Weekly -> Monthly

[Adam R. <ad...@ex...>]

I have had two community members both ask me today what happened to
the Friday teleconf -

The weekly Friday teleconf was not really working IMHO as it was two
frequent, therefore the same topics were often re-discussed and it did
not give people time to take action on decisions, and therefore it
fizzled out :-/

So... I propose that we have a Monthly teleconf instead, which
involves all available developers and interested users.

I think a Weds evening at 8pm CEST would work well for Europe and the
US at 2pm EDT.
I would also propose that the first one would be on July 20th.
But in the spirit of the community I have created a Doodle Poll so you
can vote - http://www.doodle.com/qzrbnvqxbrdg2p6h

I have also setup this Google Moderator series -
http://www.google.com/moderator/#15/e=9cebb&t=9cebb.40
You can use it to add topics that you want to discuss/demo/present or
suggestions/requests for topics for others to
discuss/demo/present/explain.

Cheers Adam.

-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] How can i test tamboti on my computer?

[Dmitriy S. <sha...@gm...>]

2011/7/8 Вячеслав Седов <sch...@gm...>

> Look like this is not in trunk?
>

yes, it under apps

http://exist.svn.sourceforge.net/viewvc/exist/apps/


-- 
Dmitriy Shabanov

[Exist-development] How can i test tamboti on my computer?

[Вячеслав С. <sch...@gm...>]

Look like this is not in trunk?

Re: [Exist-development] scheduler revision

[Dmitriy S. <sha...@gm...>]

Nothing, all I want is runtime reconfiguration feature.

On Wed, Jul 6, 2011 at 2:17 PM, Adam Retter <ad...@ex...> wrote:

> So for just the Scheduler changes, I presume without moving it to
> Configurator yet, do you foresee any breaking changes with regards
> backwards compatibility?
>


-- 
Dmitriy Shabanov

Re: [Exist-development] scheduler revision

[Adam R. <ad...@ex...>]

On 5 July 2011 06:38, Dmitriy Shabanov <sha...@gm...> wrote:
>
>
> On Mon, Jul 4, 2011 at 12:49 PM, Wolfgang Meier <wol...@ex...>
> wrote:
>>
>> I know it's difficult, but the goal right now still is to get trunk
>> into a releasable state. After several months, we just managed to get
>> the major new features in trunk stable. Now it's time to wrap up. I
>> would rather not start yet another redesign effort which affects the
>> very core. Instead, we should test and work on those issues users will
>> have when upgrading from 1.4.x to trunk. For example, backups are not
>> compatible between 1.4.x and trunk.
>>
>> There are also a lot of things on the surface which need to be fixed
>> and cleaned up (demo apps, app repository). We should concentrate on
>> those things first before moving on to new core redesigns.
>
> How long it can take?
> All I need so far is runtime schedule reconfiguration and add "tries" before
> unscheduling.

So for just the Scheduler changes, I presume without moving it to
Configurator yet, do you foresee any breaking changes with regards
backwards compatibility?

> --
> Dmitriy Shabanov
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] scheduler revision

[Dmitriy S. <sha...@gm...>]

On Mon, Jul 4, 2011 at 12:49 PM, Wolfgang Meier <wol...@ex...>wrote:

> I know it's difficult, but the goal right now still is to get trunk
> into a releasable state. After several months, we just managed to get
> the major new features in trunk stable. Now it's time to wrap up. I
> would rather not start yet another redesign effort which affects the
> very core. Instead, we should test and work on those issues users will
> have when upgrading from 1.4.x to trunk. For example, backups are not
> compatible between 1.4.x and trunk.
>
> There are also a lot of things on the surface which need to be fixed
> and cleaned up (demo apps, app repository). We should concentrate on
> those things first before moving on to new core redesigns.
>

How long it can take?

All I need so far is runtime schedule reconfiguration and add "tries" before
unscheduling.

-- 
Dmitriy Shabanov

Re: [Exist-development] scheduler revision

[Dmitriy S. <sha...@gm...>]

On Mon, Jul 4, 2011 at 2:50 PM, Adam Retter <ad...@ex...> wrote:

> Sorry, I have just re-read my email and I realise how negative it
> sounded, apologies.
> I guess apart from yourself, I am the only person that has spent a lot
> of time working and adapting the current Configurator implementation
> and I do have some concerns about it.
>
> ** In General I am very happy with the concept of the new
> Configurator, I think it is a good thing that has been needed for some
> time. Kudos and thanks :-)
> ** I am happy to see that the Configurator works for the Security Manager
> stuff.
>
> However, I think the Configurator itself is rather fragile and that
> the annotations used can be simplified and improved. But rather that
> the general 'it needs improving', let me explain myself in more
> detail. To be fair you have asked me for comments in the past, but I
> have never quite found the time to write them down, so here they
> are...
>
>
> a) You should only have to Annotate a concrete class with
> @ConfigurationClass, you should not need to annotate all of its Super
> abstract classes with the same annotation, this is overkill and easily
>

this should be simple to fix, I can't remember why Configurator force to
have that annotation ... that is coded at line 114, remove this and you get
what want.... I get it back, it there because of optimization, it will scan
super class only till last @ConfigurationClass.


> missable. Personally I would like to see @ConfigurationClass become
> @AutoConfigure, but thats just naming and personal preference ;-)
> As a Developer who wants his class state managed by the Configurator,
> I want to just write (or re-use my existing) POJO's and thats that.
>

What do you mean by POJO?


>
> b) You should not have to Annotate methods or fields within the class.
> If the class is AutoConfigured then it is up to the Configurator to
> load and persist the state of the class, this means all fields. For
> the edge case you could have an @AutoConfigureIgnore for fields, if
> that state is transient, but I would argue that you should not have
> global transient state in your code, unless maybe you are writing an
> event processor such as SAX.
>

Annotation mapping configuration "element" (tag or attribute) by name. How
can you reach same without that core part?


> c) You should not have to implement any Interface e.g. Configurable.
> By virtue of the @AutoConfigure Annotations alone we can scan and
> locate the classes at db startup time to configure. These classes
> should be POJO's as far as the author is concerned and the injection
> of state should be transparent, just like IoC frameworks such as
> Guice, Spring etc.
>

vice versa ... just interface is simple (and faster) to use.


> d) You should not have to tell the Annotator when to persist your
> class, the Configurator can monitor this through AOP. Again, im
> talking clean POJO's. I should be able to take any class, and just add
> @AutoConfigure to the top and im done. Simple :-p
>

IMHO, it's very close to that,  is it?


> e) At the moment you have to tell the Configurator about every new
> class that you want to persist, and this has to be hardcoded into the
> code. Lines 248 through 295 of Configurator.java. IMHO this is not
> maintainable or dynamic. I think we both realise this is a workaround
> for some of the current issues. This code needs to be replaced,
> preferably by a 3rd party lib (see my notes at the end) that can
> extract and persist data from any class. If the field is completely
> unknown or binary, there is no reason why we cant base64 encode this
> before storing it in the XML serialized form.
>

There is ConfigurationFieldClassMask annotation, this used by realm instance
creation and I just implement NewClass annotation: it mapping not
Configurable instance by mapping instructions. OAuth realm implementation
use this feature.


> Ideas - perhaps we could adapt the processor from JAXB or XMLBeans to
> actually do almost all of the above for us, they already know and have
> the code to take a hierarchy of any java objects and convert them into
> XML, and go from XML back to a tree of java objects. We could probably
> just replace their serializer with our own SAX serializer. Also in
>

I did spend some time with this libs, but the way was not clear for me ...


> this manner we would not need to use the memtree document impl, which
> could get quite large, and we could just stream the persisted document
> in/out of the database.
> I think this coupled with a small amount of AOP to scan for the
> Annotations and trigger the loading/persisting of state and we would
> have a much cleaner implementation with much less code, where you
> could just add @AutoConfigure to your POJO.
>
> Anyways, something to discuss I think...
>

 I'm always ready for discuss, but never for 'not' :-)

>
-- 
Dmitriy Shabanov

Re: [Exist-development] scheduler revision

[Adam R. <ad...@ex...>]

Sorry, I have just re-read my email and I realise how negative it
sounded, apologies.
I guess apart from yourself, I am the only person that has spent a lot
of time working and adapting the current Configurator implementation
and I do have some concerns about it.

** In General I am very happy with the concept of the new
Configurator, I think it is a good thing that has been needed for some
time. Kudos and thanks :-)
** I am happy to see that the Configurator works for the Security Manager stuff.

However, I think the Configurator itself is rather fragile and that
the annotations used can be simplified and improved. But rather that
the general 'it needs improving', let me explain myself in more
detail. To be fair you have asked me for comments in the past, but I
have never quite found the time to write them down, so here they
are...


a) You should only have to Annotate a concrete class with
@ConfigurationClass, you should not need to annotate all of its Super
abstract classes with the same annotation, this is overkill and easily
missable. Personally I would like to see @ConfigurationClass become
@AutoConfigure, but thats just naming and personal preference ;-)
As a Developer who wants his class state managed by the Configurator,
I want to just write (or re-use my existing) POJO's and thats that.

b) You should not have to Annotate methods or fields within the class.
If the class is AutoConfigured then it is up to the Configurator to
load and persist the state of the class, this means all fields. For
the edge case you could have an @AutoConfigureIgnore for fields, if
that state is transient, but I would argue that you should not have
global transient state in your code, unless maybe you are writing an
event processor such as SAX.

c) You should not have to implement any Interface e.g. Configurable.
By virtue of the @AutoConfigure Annotations alone we can scan and
locate the classes at db startup time to configure. These classes
should be POJO's as far as the author is concerned and the injection
of state should be transparent, just like IoC frameworks such as
Guice, Spring etc.

d) You should not have to tell the Annotator when to persist your
class, the Configurator can monitor this through AOP. Again, im
talking clean POJO's. I should be able to take any class, and just add
@AutoConfigure to the top and im done. Simple :-p

e) At the moment you have to tell the Configurator about every new
class that you want to persist, and this has to be hardcoded into the
code. Lines 248 through 295 of Configurator.java. IMHO this is not
maintainable or dynamic. I think we both realise this is a workaround
for some of the current issues. This code needs to be replaced,
preferably by a 3rd party lib (see my notes at the end) that can
extract and persist data from any class. If the field is completely
unknown or binary, there is no reason why we cant base64 encode this
before storing it in the XML serialized form.

Ideas - perhaps we could adapt the processor from JAXB or XMLBeans to
actually do almost all of the above for us, they already know and have
the code to take a hierarchy of any java objects and convert them into
XML, and go from XML back to a tree of java objects. We could probably
just replace their serializer with our own SAX serializer. Also in
this manner we would not need to use the memtree document impl, which
could get quite large, and we could just stream the persisted document
in/out of the database.
I think this coupled with a small amount of AOP to scan for the
Annotations and trigger the loading/persisting of state and we would
have a much cleaner implementation with much less code, where you
could just add @AutoConfigure to your POJO.

Anyways, something to discuss I think...



On 4 July 2011 09:07, Dmitriy Shabanov <sha...@gm...> wrote:
> On Sun, Jul 3, 2011 at 9:28 PM, Adam Retter <ad...@ex...> wrote:
>>
>> I am not against changes to the Scheduler, but what are the problems?
>>
>> I AM against converting anything to the new Configuration framework.
>> IMHO it is a good first proof of concept but there is lots about it
>> that needs revising, it feels like a mess at the moment.
>>
>
> Do you have any problem with configurator now? eXist itself looks like a
> mess, so what? should we stop coding it?
>
> --
> Dmitriy Shabanov
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] scheduler revision

[Wolfgang M. <wol...@ex...>]

Hi Dmitryi,

I know it's difficult, but the goal right now still is to get trunk
into a releasable state. After several months, we just managed to get
the major new features in trunk stable. Now it's time to wrap up. I
would rather not start yet another redesign effort which affects the
very core. Instead, we should test and work on those issues users will
have when upgrading from 1.4.x to trunk. For example, backups are not
compatible between 1.4.x and trunk.

There are also a lot of things on the surface which need to be fixed
and cleaned up (demo apps, app repository). We should concentrate on
those things first before moving on to new core redesigns.

Wolfgang

Re: [Exist-development] scheduler revision

[Dmitriy S. <sha...@gm...>]

On Sun, Jul 3, 2011 at 9:28 PM, Adam Retter <ad...@ex...> wrote:

> I am not against changes to the Scheduler, but what are the problems?
>
> I AM against converting anything to the new Configuration framework.
> IMHO it is a good first proof of concept but there is lots about it
> that needs revising, it feels like a mess at the moment.
>
>
Do you have any problem with configurator now? eXist itself looks like a
mess, so what? should we stop coding it?

-- 
Dmitriy Shabanov

Re: [Exist-development] scheduler revision

[Adam R. <ad...@ex...>]

I am not against changes to the Scheduler, but what are the problems?

I AM against converting anything to the new Configuration framework.
IMHO it is a good first proof of concept but there is lots about it
that needs revising, it feels like a mess at the moment.

On 3 July 2011 17:58, Dmitriy Shabanov <sha...@gm...> wrote:
> Hello,
>
> I do want to revisit scheduler to solve problems it have, same time I do
> want to move configurations to /db/system/scheduler/... and convert to new
> configuration framework.
> Any objections?
> --
> Dmitriy Shabanov
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] scheduler revision

[Dmitriy S. <sha...@gm...>]

Hello,

I do want to revisit scheduler to solve problems it have, same time I do
want to move configurations to /db/system/scheduler/... and convert to new
configuration framework.

Any objections?

-- 
Dmitriy Shabanov

Re: [Exist-development] Scheduler and new permissions model....

[Dmitriy S. <sha...@gm...>]

On Wed, Jun 15, 2011 at 2:02 PM, Adam Retter <ad...@ex...> wrote:

> > On Wed, Jun 15, 2011 at 3:54 AM, Adam Retter <ad...@ex...> wrote:
> >>
> >> >  Do we need to extend the conf.xml scheduler elements/definitions to
> be
> >> > able to specify what user
> >> > should be used to fire up a scheduled xquery, rather than the default
> >> > guest?
> >>
> >> Nope, I have laid the foundations for SetUID and SetGID style
> >> permissions, these will be implemented in the near future and will
> >> also solve this issue I hope.
> >>
> >
> > Yes, that's looks reasonable, the conf file can have scheduler definition
> > 'account' (attribute) as cron do have.
>
> No! It doesnt need that as you can just setUID on the XQuery file itself.
>

Think a bit more, linux like systems have setuid, but cron do have user
attribute ... why?

-- 
Dmitriy Shabanov

Re: [Exist-development] Scheduler and new permissions model....

[Adam R. <ad...@ex...>]

> On Wed, Jun 15, 2011 at 3:54 AM, Adam Retter <ad...@ex...> wrote:
>>
>> >  Do we need to extend the conf.xml scheduler elements/definitions to be
>> > able to specify what user
>> > should be used to fire up a scheduled xquery, rather than the default
>> > guest?
>>
>> Nope, I have laid the foundations for SetUID and SetGID style
>> permissions, these will be implemented in the near future and will
>> also solve this issue I hope.
>>
>
> Yes, that's looks reasonable, the conf file can have scheduler definition
> 'account' (attribute) as cron do have.

No! It doesnt need that as you can just setUID on the XQuery file itself.

> + rule: account needs to be in the cron group

That is not necessary and will just overcomplicate the system.

> If no objections I can do this.

Thanks, but its okay. I will implement SetUID, SetGID and Sticky when
the permissions stuff has bedded down a bit - i.e. in a couple of
weeks...

> --
> Dmitriy Shabanov
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Scheduler and new permissions model....

[Dmitriy S. <sha...@gm...>]

On Wed, Jun 15, 2011 at 3:54 AM, Adam Retter <ad...@ex...> wrote:

> >  Do we need to extend the conf.xml scheduler elements/definitions to be
> able to specify what user
> > should be used to fire up a scheduled xquery, rather than the default
> guest?
>
> Nope, I have laid the foundations for SetUID and SetGID style
> permissions, these will be implemented in the near future and will
> also solve this issue I hope.
>
>
Yes, that's looks reasonable, the conf file can have scheduler definition
'account' (attribute) as cron do have.

+ rule: account needs to be in the cron group

If no objections I can do this.

-- 
Dmitriy Shabanov

Re: [Exist-development] Scheduler and new permissions model....

[Adam R. <ad...@ex...>]

> I've got a test environment running your latest permissions code (current SVN). I was hoping that
> Wolfgang's recent checkin might have fixed the duplicated attributes bug, but unfortunately it
> didn't.  So I plan to dive into the cause of that later this week, since I haven't been able to
> create a replicable, simple test case as I usually do.
>
> In the meantime, in our environment we fire off some scheduled User XQuery tasks from the conf.xml
> file at startup.
>
> Problem was, that our user jobs are now failing with the following exception:
>
> 2011-06-14 16:05:04,496 [DefaultQuartzScheduler_Worker-2] INFO  (JobRunShell.java [run]:221) - Job
> eXist.User.XQuery: /db/chaeron/raven/common/xquery/chaeronJobScheduler.xql threw a
> JobExecutionException:
> org.quartz.JobExecutionException: UserXQueryJob Failed: Permission denied for the scheduling user:
> guest! Unscheduling UserXQueryJob.
>        at org.exist.scheduler.UserXQueryJob.abort(UserXQueryJob.java:268)
>        at org.exist.scheduler.UserXQueryJob.abort(UserXQueryJob.java:263)
>        at org.exist.scheduler.UserXQueryJob.execute(UserXQueryJob.java:228)
>        at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
>        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
>
> found in the scheduler.log file.

So we need to understand why the guest user cannot execute the XQuery
Job that you want them to, I suspect that the guest user simply does
not have permissions necessary on the XQuery. Can you confirm this?
At present the guest would need to have 'update' permission, either as
the user, group or other - this is something that was implemented
previously before my changes by someone else. This was implemented as
the goal was to move from 'rwu' to 'rwx', this is still the goal,
however this work was previously never completed by the contributor,
however I will complete it myself in the near future.

> I'm not sure of how we fix this?  Do we need to set global execute permissions on all the xqueries
> we have stored in the database, or at least on the ones that are scheduled at startup from conf.xml?

(see above)

>  Do we need to extend the conf.xml scheduler elements/definitions to be able to specify what user
> should be used to fire up a scheduled xquery, rather than the default guest?

Nope, I have laid the foundations for SetUID and SetGID style
permissions, these will be implemented in the near future and will
also solve this issue I hope.

> Your advice on the best way to proceed would be much appreciated.
>
> Thanks!
>
> --
> Andrzej Taramina
> Chaeron Corporation: Enterprise System Solutions
> http://www.chaeron.com
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] Scheduler and new permissions model....

[Andrzej J. T. <an...@ch...>]

Adam:

I've got a test environment running your latest permissions code (current SVN). I was hoping that
Wolfgang's recent checkin might have fixed the duplicated attributes bug, but unfortunately it
didn't.  So I plan to dive into the cause of that later this week, since I haven't been able to
create a replicable, simple test case as I usually do.

In the meantime, in our environment we fire off some scheduled User XQuery tasks from the conf.xml
file at startup.

Problem was, that our user jobs are now failing with the following exception:

2011-06-14 16:05:04,496 [DefaultQuartzScheduler_Worker-2] INFO  (JobRunShell.java [run]:221) - Job
eXist.User.XQuery: /db/chaeron/raven/common/xquery/chaeronJobScheduler.xql threw a
JobExecutionException:
org.quartz.JobExecutionException: UserXQueryJob Failed: Permission denied for the scheduling user:
guest! Unscheduling UserXQueryJob.
	at org.exist.scheduler.UserXQueryJob.abort(UserXQueryJob.java:268)
	at org.exist.scheduler.UserXQueryJob.abort(UserXQueryJob.java:263)
	at org.exist.scheduler.UserXQueryJob.execute(UserXQueryJob.java:228)
	at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)

found in the scheduler.log file.

It seems that the scheduler cannot even initiate the user xquery any more due to the permissions
stuff, whereas in the past this was fine. In fact, in the documentation found in conf.xml, it
specifically states that the a user xquery will be started running as guest, but then can change the
login using xquery functions to perform controlled things, which is in fact what our xquery does.
But it doesn't even get to that stage, since the permissions stuff is not allowing the scheduled
xquery to even fire.

I'm not sure of how we fix this?  Do we need to set global execute permissions on all the xqueries
we have stored in the database, or at least on the ones that are scheduled at startup from conf.xml?
 Do we need to extend the conf.xml scheduler elements/definitions to be able to specify what user
should be used to fire up a scheduled xquery, rather than the default guest?

Your advice on the best way to proceed would be much appreciated.

Thanks!

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

[Exist-development] Nice Testing :-) Re: [Exist-commits] SF.net SVN: exist:[14679] trunk/eXist

[Adam R. <ad...@ex...>]

Great to see someone else making use of EasyMock in eXist-db to do
Unit testing as opposed to integration testing - very very pleased :-)

On 13 June 2011 23:47,  <rwa...@us...> wrote:
> Revision: 14679
>          http://exist.svn.sourceforge.net/exist/?rev=14679&view=rev
> Author:   rwalpole
> Date:     2011-06-13 21:47:52 +0000 (Mon, 13 Jun 2011)
>
> Log Message:
> -----------
> New EXI serializer [feature]
>
> Added Paths:
> -----------
>    trunk/eXist/src/org/exist/util/serializer/EXISerializer.java
>    trunk/eXist/test/src/org/exist/util/serializer/EXISerializerTest.java
>
> Added: trunk/eXist/src/org/exist/util/serializer/EXISerializer.java
> ===================================================================
> --- trunk/eXist/src/org/exist/util/serializer/EXISerializer.java                                (rev 0)
> +++ trunk/eXist/src/org/exist/util/serializer/EXISerializer.java        2011-06-13 21:47:52 UTC (rev 14679)
> @@ -0,0 +1,193 @@
> +/*
> + *  eXist Open Source Native XML Database
> + *  Copyright (C) 2011 The eXist Project
> + *  http://exist-db.org
> + *
> + *  This program is free software; you can redistribute it and/or
> + *  modify it under the terms of the GNU Lesser General Public License
> + *  as published by the Free Software Foundation; either version 2
> + *  of the License, or (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU Lesser General Public License for more details.
> + *
> + *  You should have received a copy of the GNU Lesser General Public
> + *  License along with this library; if not, write to the Free Software
> + *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> + */
> +package org.exist.util.serializer;
> +
> +import java.io.OutputStream;
> +
> +import org.exist.dom.QName;
> +import org.exist.dom.StoredNode;
> +import org.w3c.dom.Document;
> +import org.xml.sax.Attributes;
> +import org.xml.sax.ContentHandler;
> +import org.xml.sax.Locator;
> +import org.xml.sax.SAXException;
> +import org.xml.sax.helpers.AttributesImpl;
> +
> +import com.siemens.ct.exi.EXIFactory;
> +import com.siemens.ct.exi.api.sax.SAXEncoder;
> +import com.siemens.ct.exi.exceptions.EXIException;
> +import com.siemens.ct.exi.helpers.DefaultEXIFactory;
> +
> +public class EXISerializer implements ContentHandler, Receiver {
> +
> +       static final String UNKNOWN_TYPE = "";
> +
> +       private SAXEncoder encoder;
> +
> +       public EXISerializer(OutputStream outputStream) throws EXIException {
> +               EXIFactory exiFactory = DefaultEXIFactory.newInstance();
> +               encoder = new SAXEncoder(exiFactory, outputStream);
> +       }
> +
> +       public void startDocument() throws SAXException {
> +               encoder.startDocument();
> +       }
> +
> +       public void endDocument() throws SAXException {
> +               encoder.endDocument();
> +       }
> +
> +       @Override
> +       public void startPrefixMapping(String prefix, String namespaceURI)
> +                       throws SAXException {
> +               encoder.startPrefixMapping(prefix, namespaceURI);
> +       }
> +
> +       @Override
> +       public void endPrefixMapping(String prefix) throws SAXException {
> +               encoder.endPrefixMapping(prefix);
> +       }
> +
> +       @Override
> +       public void startElement(QName qname, AttrList attribs) throws SAXException {
> +               AttributesImpl attributes = null;
> +               if(attribs != null) {
> +                       attributes = new AttributesImpl();
> +                       for(int x=0; x < attribs.size; x++) {
> +                               QName attribQName = attribs.getQName(x);
> +                               attributes.addAttribute(attribQName.getNamespaceURI(),
> +                                               attribQName.getLocalName(),
> +                                               attribQName.getStringValue(),
> +                                               UNKNOWN_TYPE,
> +                                               attribs.getValue(x));
> +                       }
> +               }
> +               encoder.startElement(qname.getNamespaceURI(), qname.getLocalName(), null, attributes);
> +
> +       }
> +
> +       @Override
> +       public void endElement(QName qname) throws SAXException {
> +               encoder.endElement(qname.getNamespaceURI(), qname.getLocalName(), null);
> +
> +       }
> +
> +       @Override
> +       public void characters(CharSequence seq) throws SAXException {
> +               String sequence = seq.toString();
> +               encoder.characters(sequence.toCharArray(), 0, sequence.length());
> +
> +       }
> +
> +       @Override
> +       public void attribute(QName qname, String value) throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void comment(char[] ch, int start, int length) throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void cdataSection(char[] ch, int start, int len) throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void processingInstruction(String target, String data)
> +                       throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void documentType(String name, String publicId, String systemId)
> +                       throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void highlightText(CharSequence seq) throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void setCurrentNode(StoredNode node) {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public Document getDocument() {
> +               // TODO Auto-generated method stub
> +               return null;
> +       }
> +
> +       @Override
> +       public void setDocumentLocator(Locator locator) {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void startElement(String uri, String localName, String qName,
> +                       Attributes atts) throws SAXException {
> +               encoder.startElement(uri, localName, null, atts);
> +
> +       }
> +
> +       @Override
> +       public void endElement(String uri, String localName, String qName)
> +                       throws SAXException {
> +               encoder.endElement(uri, localName, null);
> +
> +       }
> +
> +       @Override
> +       public void characters(char[] ch, int start, int length)
> +                       throws SAXException {
> +               encoder.characters(ch, start, length);
> +
> +       }
> +
> +       @Override
> +       public void ignorableWhitespace(char[] ch, int start, int length)
> +                       throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       @Override
> +       public void skippedEntity(String name) throws SAXException {
> +               // TODO Auto-generated method stub
> +
> +       }
> +
> +       void setEncoder(SAXEncoder encoder) {
> +               this.encoder = encoder;
> +       }
> +
> +}
> \ No newline at end of file
>
> Added: trunk/eXist/test/src/org/exist/util/serializer/EXISerializerTest.java
> ===================================================================
> --- trunk/eXist/test/src/org/exist/util/serializer/EXISerializerTest.java                               (rev 0)
> +++ trunk/eXist/test/src/org/exist/util/serializer/EXISerializerTest.java       2011-06-13 21:47:52 UTC (rev 14679)
> @@ -0,0 +1,98 @@
> +package org.exist.util.serializer;
> +
> +import static org.easymock.EasyMock.capture;
> +import static org.easymock.EasyMock.isNull;
> +import static org.easymock.EasyMock.matches;
> +import static org.easymock.classextension.EasyMock.createMock;
> +import static org.easymock.classextension.EasyMock.replay;
> +import static org.easymock.classextension.EasyMock.verify;
> +
> +import java.io.OutputStream;
> +import java.util.List;
> +
> +import junit.framework.Assert;
> +
> +import org.easymock.Capture;
> +import org.exist.dom.QName;
> +import org.junit.Before;
> +import org.junit.Ignore;
> +import org.junit.Test;
> +import org.xml.sax.Attributes;
> +
> +import com.siemens.ct.exi.api.sax.SAXEncoder;
> +
> +public class EXISerializerTest {
> +
> +       private EXISerializer serializer;
> +       private OutputStream mockOutputStream;
> +       private SAXEncoder mockEncoder;
> +
> +       @Before
> +       public void setUp() throws Exception {
> +               mockOutputStream = createMock(OutputStream.class);
> +               serializer = new EXISerializer(mockOutputStream);
> +               mockEncoder = createMock(SAXEncoder.class);
> +               serializer.setEncoder(mockEncoder);
> +       }
> +
> +       @Test
> +       public void testStartDocument() throws Exception {
> +               mockEncoder.startDocument();
> +               replay(mockEncoder);
> +               serializer.startDocument();
> +               verify(mockEncoder);
> +       }
> +
> +       @Test
> +       public void testEndDocument() throws Exception {
> +               mockEncoder.endDocument();
> +               replay(mockEncoder);
> +               serializer.endDocument();
> +               verify(mockEncoder);
> +       }
> +
> +       @Test
> +       public void testStartPrefixMapping() throws Exception {
> +               mockEncoder.startPrefixMapping("prefix", "uri");
> +               replay(mockEncoder);
> +               serializer.startPrefixMapping("prefix", "uri");
> +               verify(mockEncoder);
> +       }
> +
> +       @Test
> +       public void testEndPrefixMapping() throws Exception {
> +               mockEncoder.endPrefixMapping("prefix");
> +               replay(mockEncoder);
> +               serializer.endPrefixMapping("prefix");
> +               verify(mockEncoder);
> +       }
> +
> +       @Test
> +       public void testStartElement() throws Exception {
> +               QName testQName = new QName("local", "uri", "prefix");
> +               AttrList testAttrList = new AttrList();
> +               testAttrList.addAttribute(new QName("local", "uri"), "value");
> +               Capture<Attributes> capturedAttributes = new Capture<Attributes>();
> +               mockEncoder.startElement(matches("uri"), matches("local"), (String)isNull(), capture(capturedAttributes));
> +               replay(mockEncoder);
> +               serializer.startElement(testQName, testAttrList);
> +               verify(mockEncoder);
> +               List<Attributes> capturedAttributeList = capturedAttributes.getValues();
> +               Assert.assertEquals("local", capturedAttributeList.get(0).getLocalName(0));
> +               Assert.assertEquals("uri", capturedAttributeList.get(0).getURI(0));
> +               Assert.assertEquals("value", capturedAttributeList.get(0).getValue(0));
> +       }
> +
> +       @Ignore("incomplete")
> +       @Test
> +       public void testEndElement() {
> +               // TODO
> +       }
> +
> +       @Ignore("incomplete")
> +       @Test
> +       public void testCharacters() {
> +               // TODO
> +       }
> +
> +}
> \ No newline at end of file
>
>
> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Exist-commits mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-commits
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb