exist-development — eXist Developer's List

Re: [Exist-development] optimizing positional predicates for fast paging

[Joe W. <jo...@gm...>]

Mike and Wolfgang,

A great topic!  I'd certainly welcome optimizations like this.  I
wonder:  If a FLWOR expression has an 'order by' clause, is it even
possible to optimize the expression's positional predicate (or
subsequence)?  Or does 'order by' make optimizations impossible?

>> collection("/")/ (//elem) [position() = (1 to 10)]
>
> Note that using subsequence instead of the predicate can be a lot
> faster sometimes.

I didn't realize that.  Can you generalize about when subsequence
would be faster?  And would you say it's always at least as fast as
the predicate?

Joe

Re: [Exist-development] optimizing positional predicates for fast paging

[Wolfgang M. <wol...@ex...>]

> I wonder whether eXist is currently optimizing xquery evaluations that use
> positional predicates, or similar.

I did implement shortcuts for the expensive axes: following and
preceding. For example, LocationStep.getFollowing checks the property
hasPositionalPredicate. If set to true, the method evaluates the
predicate in advance to get its numeric value. This is then forwarded
to the NodeSet method which calculates the join:

return currentSet.selectFollowing(contextSet, position, contextId);

We could apply similar optimizations to other axes, though their
execution paths are more complex as you have more possibilities to
choose from.

However, my positional predicate optimization would only work for
//elem[1], not //elem[position() = 1]. But I guess it would be
possible to add a rewrite rule here to automatically simplify the
expression.

> collection("/")/ (//elem) [position() = (1 to 10)]

Note that using subsequence instead of the predicate can be a lot
faster sometimes.

Wolfgang

Re: [Exist-development] Junit issue in testCollectionWithAccentsAndSpaces ?

[Pierrick B. <pie...@fr...>]

Hi,

Dannes Wessels a écrit :

> In AllXmlRpcTests.java the test testCollectionWithAccentsAndSpaces
> fails for me. Can anyone confirm? Maybe it is due to the recent
> MacOsX Java6 upgrade.....

I confirm on Windows :

> Forked Java VM exited abnormally. Please note the time in the report
> does not reflect the time until the VM exit.
> 
> junit.framework.AssertionFailedError: Forked Java VM exited
> abnormally. Please note the time in the report does not reflect the
> time until the VM exit.

In all, I currently have 4 failures and 2 errors (including a still 
uncommited "correction" in the spatial stuff).

Cheers,

p.b.

Re: [Exist-development] Junit issue in testCollectionWithAccentsAndSpaces ?

[Loren C. <lor...@gm...>]

Hello Dannes,

I have confirmed the error.  I get the same error with the most recent Java 6 update on my mac.

Loren

On Dec 13, 2009, at 01:20 PM, Dannes Wessels wrote:

> All,
> 
> In AllXmlRpcTests.java the test testCollectionWithAccentsAndSpaces fails for me. Can anyone confirm?
> Maybe it is due to the recent MacOsX Java6 upgrade.....
> 
> Kind regards
> 
> Dannes

[Exist-development] optimizing positional predicates for fast paging

[Michael S. <so...@if...>]

I wonder whether eXist is currently optimizing xquery evaluations that use
positional predicates, or similar.  It doesn't seem like it, based on some
poking around in the code, and I think this could be a very fruitful area.
What I mean is an expression like:

collection("/")/ (//elem) [position() = (1 to 10)]

- when there are a lot of //elem, evaluation can be short-cut by ignoring
all but the first 10: ie the first "page" of results.  Currently it
*appears* to me as if the entire (//elem) node sequence is being
constructed, and then filtered.

In web applications, a great many queries are limited in exactly this way:
[position() = ($n to $n + $page-size)], so it seems like an area that merits
attention.  And the query-writer doesn't have enough expressive power to
push the positional predicate into the innermost expression, especially if
they are passing paging parameters via a wrapping API layer like REST.  

There is also a problem when coding FLWOR expressions (which you have to do
in order to sort the results), where the only option for paging is to wrap
the entire FLWOR in a subsequence function call (or postfix a predicate like
above).  Some kind of internal expression rewriting that pushes the paging
predicate into the FLWOR evaluation could yield a huge performance
improvement in such cases.  Even when results are sorted, one might see a
performance gain by using a value index scan.

I started to look into implementing this myself, but I wanted to see whether
any work had been done already, or if anyone had any other thoughts about
whether this is worthwhile?

-Mike

[Exist-development] Junit issue in testCollectionWithAccentsAndSpaces ?

[Dannes W. <di...@ex...>]

All,

In AllXmlRpcTests.java the test testCollectionWithAccentsAndSpaces fails for me. Can anyone confirm?
Maybe it is due to the recent MacOsX Java6 upgrade.....

Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] [Exist-open] namespace issues

[Anthony F. <an...@re...>]

Wolfgang,

What do you suspect would be the primary issues to contend with to
implement indexed wildcarded namespaces? You might recall I briefly
mentioned this when we met earlier this summer. I'm interested in
tackling this problem since I find that wanting to use a wildcarded
namespace is a very frequent usage pattern for us.

We tend to store collections of documents that are "mostly similar"
as opposed to collections that are entirely uniform with respect to a
grammar. That is, differences between document grammars are due to
versioning that we employ between our exchange partners but which gets
reflected into the namespace. Consequently, most application tasks
can be satisfied by treating all documents uniformly and only looking
at the local name.

Is this a common usage pattern for other eXist users? Maybe we have
overlooked another pattern that makes this a non-issue? If not, I'm
happy to step up to implementing it if I can get a handle on the
appropriate code points in the indexer/optimizer code.


--Tony

> 
> ________________________________________
> From: Wolfgang Meier [wol...@gm...]
> Sent: Monday, November 30, 2009 5:04 PM
> To: swr...@gm...
> Cc: exi...@li...
> Subject: Re: [Exist-open] namespace issues
> 
> > i get xml from our erp system for invoice, po, pacingslips, etc. and load it
> > into exist. A recent change to the erp system has added a namespace
> > attribute at the root node to all xml documents, previously there was no
> > namespace specified. my queries do not pick up the new documents because of
> > the namespaces.
> 
> Yes, in XML, two elements with the same local name but different
> namespace URIs are completely different. You would indeed need two
> queries, using a different namespace prefix, e.g.:
> 
> declare default element namespace p='http://SalesInvoice.mycompany.com';
> 
> (for $forms in collection("/db/CreateForm/")/Root ...) union
> (for $forms in collection("/db/CreateForm/")/p:Root ...)
> 
> Alternatively you could continue with just one query but use a
> wildcard for the prefix everywhere, so you would write *:Root. Please
> note that this will be slow since those wildcard lookups cannot be
> optimized in the same way as explicit names.
> 
> Wolfgang
> 
> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing.
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> Exist-open mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-open
> 

Re: [Exist-development] My vision about admin RESTful URLs

[Joe W. <jo...@gm...>]

Hi Thomas,

> An ‘area’  is very similar to ‘role’, although an area could be created with
> roles or groups or whatever means the current implementation offer to allow
> XML resources to be read only by authorised users.

What you said sounds reasonable -- in that you're relying on other
mechanisms (read privileges, granted through the builtin unix-style
permissions or XACML) for authorization, rather than reinventing the
wheel through some predetermined, hardcoded URL-based scheme.

> In every area collection, for every action we have an XML file that can be
> read only by the group or role authorised for this area,  say ‘system-admin’
> group:
>
> /admin/system/server/action-restart.xml
> <action>
>     <id>restart</id>
> </action>

I like the idea of action XML files for augmenting authorization,
although these files needn't necessarily be stored in an area
subcollection.  These XML files could conceivably be stored in a
single, common collection.  I would say that 'area' in your proposal
is merely a convenient way to group actions together into a collection
so that permissions can be applied to the whole collection, rather
than on a file-by-file basis.  It's an internal permissions issue.  I
don't yet see a compelling reason why this issue should impact the URL
design, in terms of requiring that 'area' be a part of the URL.  In
other words, it seems to me that your notion of 'area' is still an
authorization concept, rather than a concept that should drive the URL
or application organization.  The weakness in organizing an app around
preconceived notions of user groups is that authorization needs are
different for just about every organization.

That said, I think the most compelling claim you've made for
organizing the admin section around 'area' is in that it could help
the admin section be modular -- letting developers select groups of
admin actions for their applications.  This is a very different goal
than an 'authorization' goal, and I agree that it's desirable.  But I
haven't seen how 'area' helps achieve that flexibility from what
you've said.  Is there anything you could add that would back this
claim up?  Could you flesh out where action-restart.xql/m would be
stored relative to the main admin xquery file, and how a developer
could disable this action from their app without trouble?

I think the questions for everyone are:

1. Do folks think XML files that describe atomic admin 'actions' are a
good way to handle authorization?  If so, should the 'action' XML file
be organized into pre-determined 'area' sub-collections, or should the
grouping of these files be left to the individual developer?

2. Does this notion of 'area' really help the admin app stay modular?
What's the best way to organize the admin app so that both groups of
actions and individual actions can be turned on/off?

Joe

[Exist-development] Module file: concatenation of contents to existing files?

[José M. F. G. <jm...@us...>]

Hi everybody!
	looking at the functionality of file:serialize and file:serialize-binary I have realized that there is no function which concatenates contents to an existing file. This could be useful, for instance, to write to a custom log file.

	If you don't mind I can write this feature for 'file' module.

	Best Regards,
		José María
-- 
"La violencia es el último recurso del incompetente"
	- Salvor Hardin en "La Fundación" de Isaac Asimov
"Premature optimization is the root of all evil." - Donald Knuth

José María Fernández González
e-mail: jos...@gm...

Re: [Exist-development] My vision about admin RESTful URLs

[Thomas W. <tho...@gm...>]

Joe,

An ‘area’  is very similar to ‘role’, although an area could be created with
roles or groups or whatever means the current implementation offer to allow
XML resources to be read only by authorised users.



The first implementation of the area access control could be the following:

In every area collection, for every action we have an XML file that can be
read only by the group or role authorised for this area,  say ‘system-admin’
group:


/admin/system/server/action-restart.xml

<action>

    <id>restart</id>

</action>

/admin/system/server/ action-reload.xml

<action>

    <id>reload</id>

</action>



/admin/ system /server/action-shutdown.xml

<action>

    <id> shutdown </id>
</action>




The controller in this area collection loads
xdb:xcollection($exist:controller)/action/id to enumerate the available
actions for the current user. If the user is not member of the
‘system-admin’ group, then the list of the actions will be empty.

When the requested action is in the list, then the controller will call the
function for the action with the extracted parameters from the URL.



The controller in the application collection will capture the calls for the
missing areas collections and return ‘unavailable-action’ result.

Regards,
Thomas


------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007



2009/11/28 Joe Wicentowski <jo...@gm...>

> Thomas,
>
> > What I
> > have in my mind is the everyday use of administrative tools that usually
> are
> > done by different people. Imagine a junior admin that usually adds users
> and
> > groups, who accidentally shutdowns the server or even worse - deletes the
> > main data collection.
> >
> > The proper word we can use to describe this perspective is 'area'. It may
> > help if we think about any area, almost like a job description that can
> be
> > given to somebody to perform a specific set of tasks without need to have
> > access to any other admin resources.
>
> Your description of 'area' sounds more like what I think of as 'role'
> - as in RBAC, which covers all actions/resources that users are
> authorized to access, including admin functions like those for this
> admin app.  Could you elaborate on the distinction, if any, between
> your 'area' and 'role'?
>
> How do you imagine preventing the user in your example from accessing
> other admin areas -- through some sort of URL-based access control, or
> through RBAC (which focuses on documents, collections, functions, and
> modules)?
>
> You also listed many ideas about isolating an area into a single
> directory with its own controller.xql.  Most of your points up to now
> have been about URLs, but this gets into coding structure.  Do you
> have ideas about ways to arrange the code so that it fits your
> criteria for modularity?
>
> Joe
>

Re: [Exist-development] How can I get the Created and Modified times?

[Daryn H. <dar...@gm...>]

Hi all,

I have found a solution. I have used 'describeResource' as I don't think
there is RCP support for the other method.

Thanks!

2009/11/30 Daryn Holmes <dar...@gm...>

>
> Hi,
>
> I am new to all of this and I cannot seem to solve a problem.
> I feel a bit stupid as you have basically given me the solution.
> Still, I have having some problems fetching the required data through PHP.
>
> I am trying to get the Created and Modified time stamps.
>
> This is the code I am using:
>
> $downloadmessage = new XML_RPC_Message('last-modified',
>  array( new XML_RPC_Value('/db/somecollection/anothercollection',
> 'string'),
>  new XML_RPC_Value('document.xml', 'string')));
>  $downloadresponse = $xmlrpcclient->send($downloadmessage);
> $downloadval = $downloadresponse->value();
>  $lastmodified = $downloadval->getval();
>
> $lastmodified seems to be an array with two empty elements.
> I do not think it is the connection, as other queries are working.
>
> My apologies for my ignorance and for bothering you all with this basic
> question.
>
> Can anyone help me?
>
> Thanks...
>
> 2009/11/27 Dan McCreary <dan...@gm...>
>
>> Hello Daryn,
>>
>>
>> I assume you are looking for the XQuery functions that access when a
>> resource was created and when a resource was last modified.
>>
>> Here are the two functions you seek:
>>
>> *xmldb:last-modified($collection, $resource)
>>
>> http://demo.exist-db.org/exist/functions/xmldb/last-modified
>>
>> xmldb:created($collection, $resource)*
>>
>> *http://demo.exist-db.org/exist/functions/xmldb/created*
>>
>>
>> Here is an example of how they are used to sync two collections:
>>
>> http://en.wikibooks.org/wiki/XQuery/Synchronizing_Remote_Collections
>>
>> - Dan
>>
>> On Thu, Nov 26, 2009 at 12:11 PM, Daryn Holmes <dar...@gm...>wrote:
>>
>>> Hello,
>>>
>>> Is there a way to access the created and modified data that is eXists
>>> maintains?
>>>
>>> Thanks...
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
>>> 30-Day
>>> trial. Simplify your report design, integration and deployment - and
>>> focus on
>>> what you do best, core application coding. Discover what's new with
>>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>>> _______________________________________________
>>> Exist-development mailing list
>>> Exi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>>
>>>
>>
>>
>> --
>> Dan McCreary
>> Semantic Solutions Architect
>> syntactica.com
>> 952-460-1674
>> VOIP: 111@69.199.167.229
>>
>
>

Re: [Exist-development] How can I get the Created and Modified times?

[Daryn H. <dar...@gm...>]

Hi,

I am new to all of this and I cannot seem to solve a problem.
I feel a bit stupid as you have basically given me the solution.
Still, I have having some problems fetching the required data through PHP.

I am trying to get the Created and Modified time stamps.

This is the code I am using:

$downloadmessage = new XML_RPC_Message('last-modified',
array( new XML_RPC_Value('/db/somecollection/anothercollection', 'string'),
new XML_RPC_Value('document.xml', 'string')));
 $downloadresponse = $xmlrpcclient->send($downloadmessage);
$downloadval = $downloadresponse->value();
$lastmodified = $downloadval->getval();

$lastmodified seems to be an array with two empty elements.
I do not think it is the connection, as other queries are working.

My apologies for my ignorance and for bothering you all with this basic
question.

Can anyone help me?

Thanks...

2009/11/27 Dan McCreary <dan...@gm...>

> Hello Daryn,
>
> I assume you are looking for the XQuery functions that access when a
> resource was created and when a resource was last modified.
>
> Here are the two functions you seek:
>
> *xmldb:last-modified($collection, $resource)
> http://demo.exist-db.org/exist/functions/xmldb/last-modified
>
> xmldb:created($collection, $resource)*
>
> *http://demo.exist-db.org/exist/functions/xmldb/created*
>
>
> Here is an example of how they are used to sync two collections:
>
> http://en.wikibooks.org/wiki/XQuery/Synchronizing_Remote_Collections
>
> - Dan
>
> On Thu, Nov 26, 2009 at 12:11 PM, Daryn Holmes <dar...@gm...>wrote:
>
>> Hello,
>>
>> Is there a way to access the created and modified data that is eXists
>> maintains?
>>
>> Thanks...
>>
>>
>> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
>> 30-Day
>> trial. Simplify your report design, integration and deployment - and focus
>> on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> Exist-development mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-development
>>
>>
>
>
> --
> Dan McCreary
> Semantic Solutions Architect
> syntactica.com
> 952-460-1674
> VOIP: 111@69.199.167.229
>

Re: [Exist-development] My vision about admin RESTful URLs

[Joe W. <jo...@gm...>]

Thomas,

> What I
> have in my mind is the everyday use of administrative tools that usually are
> done by different people. Imagine a junior admin that usually adds users and
> groups, who accidentally shutdowns the server or even worse - deletes the
> main data collection.
>
> The proper word we can use to describe this perspective is 'area'. It may
> help if we think about any area, almost like a job description that can be
> given to somebody to perform a specific set of tasks without need to have
> access to any other admin resources.

Your description of 'area' sounds more like what I think of as 'role'
- as in RBAC, which covers all actions/resources that users are
authorized to access, including admin functions like those for this
admin app.  Could you elaborate on the distinction, if any, between
your 'area' and 'role'?

How do you imagine preventing the user in your example from accessing
other admin areas -- through some sort of URL-based access control, or
through RBAC (which focuses on documents, collections, functions, and
modules)?

You also listed many ideas about isolating an area into a single
directory with its own controller.xql.  Most of your points up to now
have been about URLs, but this gets into coding structure.  Do you
have ideas about ways to arrange the code so that it fits your
criteria for modularity?

Joe

Re: [Exist-development] My vision about admin RESTful URLs

[Thomas W. <tho...@gm...>]

Dear all,

It seams I forgot to mention the reasons why I introduced the idea of
modules. But before I go to any details let me make a small adjustment in
the terminology.

The word 'module' brings associations related to implementation.  What I
have in my mind is the everyday use of administrative tools that usually are
done by different people. Imagine a junior admin that usually adds users and
groups, who accidentally shutdowns the server or even worse - deletes the
main data collection.

The proper word we can use to describe this perspective is *'area'*. It may
help if we think about any area, almost like a job description that can be
given to somebody to perform a specific set of tasks without need to have
access to any other admin resources.

 1) Areas cover different activities of the administration, access to which
is a likely to be granted in more granular way.
2) An area comprises of set of actions that are likely to be developed and
used together.
3) When we have areas, each of them will have its own controller.xql with
specific way of extracting parameters. The alternative is a mile long single
controller.xql for all functionalities.
4) In some cases (production db) not all admin functions are required. When
we have areas, simple by deleting the directory/collection with unwanted
code (or copy back when needed) the problem is solved quickly and easily.
Conversely if we have a single a mile long controller.xql with all code in a
single location, then we need to edit copy/paste|delete|comment/uncomment
fragments in the controller, apart from the copying/deleting the code, which
does not look very practical to me.
5) When we have areas of administration we deal with same objects, but from
different perspective. For example a collection in the area of data browser
may give us list of child resources and sub-collections, while in the area
of security the actions for a collection in this context are completely
different.

When we implement the modules we can choose to either have a monolithic
module for every object or multiple modules that cover only the
actions(functions) in an single area. Because different areas are likely to
be coded by different developers and we want to be able to remove some of
the areas in not needed, my preference will be to have multiple module
files, with names as '{object}_{area}.xqm'

I agree, the area name 'administration' is not good - 'system' is much
better.

We need to revisit the names of areas and choose the wording that fits
well.

   - config
   - state
   - jobs
   - system
   - I can't come up with a name for the backup & restore area.  The job
   description is backup admin. any ideas?
   - data - reindex, import, export.
   - security
   - browser


> /admin/log/period/eXist.log/{datetime}/{datetime}
> /admin/log/tail/{log-id}
> (funny, it can be /admin/eXist.log/period/{datetime}/{datetime} & it
> possible to parse & "understand"! "log" miss, because of "eXist.log" -
> it have single "meaning")


I think the idea when we build the URL is to go from more generic to a
specific, from what I am going to do, to parameters that describe how to do
it. That is why I would
prefer:  /admin/log/eXist.log/period/{datetime}/{datetime}
 BTW using negative page/chink numbers, we can address the file from the end
up:
 /admin/log/eXist.log/page/250/-1  will give us the last 250 lines of the
log file.


About the log files. The following sequence can allow us to navigate in a
log file easily and simply with a little bit of AJAX magic, using only
three methods:

1. start with the last xxx lines:
        /admin/log/eXist.log/page/{number-of-lines-to-get}/-1

2. Use the first timestamp in the chunk and create a link for the previous
page:

/admin/log/eXist.log/before/{time-stamp-of-the-first-line-in-the-chunk}/{number-of-lines-to-get}
      after click and receiving the new chunk of lines, insert them at the
top of the screen (using AJAX),
      Do this step again to produce the link for the new previous page.

3. Use the last timestamp of the chunk and create a link for the next page.

/admin/log/eXist.log/after/{time-stamp-of-the-last-line-in-the-chunk}/{number-of-lines-to-get}
       after click and receiving the new chunk of lines, insert them at the
bottom of the screen (using AJAX),
       Do this step again to produce the link for the new next page.

     The link for the next page can be executed, using AJAX, every XX
seconds to get the new lines appended to the log file.


I hope this long post has brought more clarity about my view point of the
admin RESTful URLs.

Regards,
Thomas

[Exist-development] Maintaining the 1.4 branch

[Wolfgang M. <wol...@ex...>]

Dear developers,

I'd like to remind you that all bug fixes or changes you want to see
in the stable 1.4.x series need to be ported back to the 1.4.x branch:

https://exist.svn.sourceforge.net/svnroot/exist/stable/eXist-1.4.x/

It is much easier to do this in time, while you still remember why and
what you changed. I spent several hours last week to scan my own
stuff. It would have been less work if I had done it earlier.

What should go into 1.4.x:

* well-tested code only; the goal should be to improve overall stability
* all bug fixes
* improvements to existing features, mainly to make them more complete
(debugger, xproc, indexes, ...) or improve user experience.

New features are usually not allowed. Ideally all changes should be
cross-checked. So please keep an eye on all commits going into the
1.4.x branch. I think we will need to have a 1.4.1 release early next
year (or even before the new year).

Wolfgang

Re: [Exist-development] My vision about admin RESTful URLs

[Dmitriy S. <sha...@gm...>]

On Fri, 2009-11-27 at 13:33 +0000, Thomas White wrote:
> Dan,
>  
> Thank you for the feedback.
>  
> I can see your preference to the REST URL is /admin/{object}/{action} 
>  
> I would still prefer /admin/{module}/{object}/{action} 

For me (as linguist :) no differences between "object" & "module". To
make it flexible: 

/admin/{object}/{object}/{action}/{object}/{object}

The parsing is quite simple:
1. analyze first "object"s till get the module. (possible to omit)
2. "action" = function
3. second "object" - the information for function

That is very simple, the "action" split "object"s on "which" & "where".
The parsing time should be low too.

> We need to think about admin work as set of activities where different
> people will have access to different subset of possible actions. It is
> an exception, usually for small projects where one person is admin and
> he is responsible for all admin tasks. More common case is when we
> have more then one administrator. If we group common activities to a
> module we can easily grant access to this module and these
> activities. 
>  
> Conversely if we put all actions for an object in one
> place /admin/{object}/{action}  then it will be much more difficult or
> complex do define in details who can do what. 
>  
> We are discussing URLs here and the implementation of all actions for
> an object, say collection, can still be in a single module.
>  
> I do not really like the idea of receiving the list of possible
> actions when URL is /admin/collection. To request the list of action
> is an action and to me it looks like
>  /admin/{module}/collection/action-list. It can return the subset of
> the available actions for the context of the module combined with user
> credentials including groups and roles. This way we can control the
> access to the every action depending on who is asking
> 
> I not sure about the meaning of  "enable/disable collection
> operation".  Do you mean to allow the user to perform the actions or
> not? This could be a case only if we think admin vs non-admin
> users. In real life wе may have Backup Admin, User Admin, System
> admin and all of them may have different subset of actions. User admin
> does not need to be able to shutdown the server, change
> configuration or access the data in any collection.
>  
> I like your note about replacing 'do' with 'server' . Then we can
> have:
>  
> /admin/administration/server/restart
> /admin/administration/server/reload
> /admin/administration/server/shutdown
> /admin/administration/server/backup/{root-collection-of-the
> backup}[?to=c:/backups]
> /admin/administration/server/restore/{root-collection-of-the
> backup}?from=c:/backups/backup-file.zip
> /admin/administration/server/reindex/{collection-URI}

/admin/administration is tautology. we did definition area by "admin",
why redefine it again by administration. It can be /siteN/admin/....

Your idea is based on different users usage, but "server" for system
admin, "server/backup" backup admin.

My view:

/admin/system/restart
/admin/system/reload
/admin/system/shutdown
/admin/backup/{root-collection-of-the-backup}[?to=c:/backups]
/admin/restore/{root-collection-of-the-backup}?from=c:/backups/backup-file.zip
/admin/reindex/{collection-URI}

"system" -> module; "restart" -> function

"backup" -> module & function

If be honest, I like single "module & function" concept more that
"module & functions". It's simple in developing & understanding.

> I think we need an extra parameter 'mode' that will define what will
> be returned - XML, JSON, HTML, XForm.
>  
> About the log files viewer, I will be happy to put our heads together
> and come up with a simple and quick solution.

{action}/{object}/{object} should be ok here. 

/admin/log/period/eXist.log/{datetime}/{datetime}
/admin/log/tail/{log-id}

(funny, it can be /admin/eXist.log/period/{datetime}/{datetime} & it
possible to parse & "understand"! "log" miss, because of "eXist.log" -
it have single "meaning")

-- 
Cheers,

Dmitriy Shabanov

PS I what to try coding, any svn? under eXist one? 

[Exist-development] Simple AJAX Collection Browser

[Dan M. <dan...@gm...>]

With some help from Chris Wallace I now have a very simple AJAX-style
as-you-type collection browser running here:

http://demo.syntactica.com/exist/rest/db/cust/exist-admin/apps/browse/browse.xq?collection=/db/cust/exist-admin/apps/

Note that as you append characters to the input path the
sub-collection set is narrowed.

I have put the sample code here in the XQuery Wikibook:

http://en.wikibooks.org/wiki/XQuery/Navigating_Collections

The ideal form would have a drop-down list directly under the input field.

- Dan

Re: [Exist-development] My vision about admin RESTful URLs

[Thomas W. <tho...@gm...>]

Dan,

Thank you for the feedback.

I can see your preference to the REST URL is /admin/{object}/{action}

I would still prefer /admin/{module}/{object}/{action}

We need to think about admin work as set of activities where different
people will have access to different subset of possible actions. It is an
exception, usually for small projects where one person is admin and he is
responsible for all admin tasks. More common case is when we have more then
one administrator. If we group common activities to a module we can easily
grant access to this module and these activities.

Conversely if we put all actions for an object in one place
/admin/{object}/{action}  then it will be much more difficult or complex do
define in details who can do what.

 We are discussing URLs here and the implementation of all actions for an
object, say collection, can still be in a single module.

I do not really like the idea of receiving the list of possible actions when
URL is /admin/collection. To request the list of action is an action and to
me it looks like  /admin/{module}/collection/action-list. It can return the
subset of the available actions for the context of the module combined with
user credentials including groups and roles. This way we can control the
access to the every action depending on who is asking
 I not sure about the meaning of  "enable/disable collection operation".  Do
you mean to allow the user to perform the actions or not? This could be a
case only if we think admin vs non-admin users. In real life wе may
have Backup Admin, User Admin, System admin and all of them may have
different subset of actions. User admin does not need to be able to shutdown
the server, change configuration or access the data in any collection.

I like your note about replacing 'do' with 'server' . Then we can have:

/admin/administration/server/restart
/admin/administration/server/reload
/admin/administration/server/shutdown
/admin/administration/server/backup/{root-collection-of-the
backup}[?to=c:/backups]
/admin/administration/server/restore/{root-collection-of-the
backup}?from=c:/backups/backup-file.zip
/admin/administration/server/reindex/{collection-URI}

I think we need an extra parameter 'mode' that will define what will be
returned - XML, JSON, HTML, XForm.

About the log files viewer, I will be happy to put our heads together and
come up with a simple and quick solution.

I just realised - we may be talking about two different sets of URLs. I
believe we are discussing an RESTful API to perform admin actions. These
URLs are not the URL that a web application will have - neither the main
application URL nor the application modules' URLs.

The way URLs are constructed within any web application depends on many
factors including the technology and even the personal preferences of the
author. Application URLs should not be an object of restrictions or any hard
rules.

Regards,
Thomas

------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007



2009/11/27 Dan McCreary <dan...@gm...>

> eXcellent feedback Tomas.
>
> You have a nice holistic view of other areas of administration that I
> had not considered for RESTful interfaces.
>
> A few small suggestions.
>
> To be consistent with the /admin/{object}/{action} format, perhaps we
> could officially use "server" as a named object.
>
> So the following might be some actions on the database server:
>
> /admin/server/shutdown
> /admin/server/restart
> /admin/state/status
> /admin/state/statistics
>
> etc.
>
> This might be a little more precise then the label "do".
>
> I would also like to keep all the operations on collections in a
> single areas so that we can enable/disable collection operation
> buttons.  So if you specify:
>
> /admin/collection
>
> The REST interface would return a list of all possible operations on a
> collection:
>
> Create New Index, Edit Index, Estimate Index Creation Time, Index
> (Reindex) Collection, Update Index, Change Owner, Change Permissions,
> Create New Trigger, Edit Trigger, Add Version, Move, Rename,
> Collection Size, Synchronize, Edit Access Policy, Upload Files,
> Collection Details, Delete Collection, Copy, Backup Collection,
> Restore Collection, Create a New Subcollection
>
> Perhaps I should refactor the collection admin database to include a
> single object type code list:
>
> The list of possible admin objects might include:
>
> configuration-file
> collection
> group
> index
> log-file
> policy
> resource
> server
> trigger
> user
>
> etc.
>
> I also would like to suggest we talk to Loren about how we might
> create a RESTful interface to an existing logfile analyzer like
> "Apache Chainsaw".
>
> See http://logging.apache.org/chainsaw
>
> Loren is an expert on logfile viewing and has already spent a great
> deal of time thinking about integrating Apache Chainsaw with eXist.
>
> - Dan
>

Re: [Exist-development] My vision about admin RESTful URLs

[Dan M. <dan...@gm...>]

eXcellent feedback Tomas.

You have a nice holistic view of other areas of administration that I
had not considered for RESTful interfaces.

A few small suggestions.

To be consistent with the /admin/{object}/{action} format, perhaps we
could officially use "server" as a named object.

So the following might be some actions on the database server:

/admin/server/shutdown
/admin/server/restart
/admin/state/status
/admin/state/statistics

etc.

This might be a little more precise then the label "do".

I would also like to keep all the operations on collections in a
single areas so that we can enable/disable collection operation
buttons.  So if you specify:

/admin/collection

The REST interface would return a list of all possible operations on a
collection:

Create New Index, Edit Index, Estimate Index Creation Time, Index
(Reindex) Collection, Update Index, Change Owner, Change Permissions,
Create New Trigger, Edit Trigger, Add Version, Move, Rename,
Collection Size, Synchronize, Edit Access Policy, Upload Files,
Collection Details, Delete Collection, Copy, Backup Collection,
Restore Collection, Create a New Subcollection

Perhaps I should refactor the collection admin database to include a
single object type code list:

The list of possible admin objects might include:

configuration-file
collection
group
index
log-file
policy
resource
server
trigger
user

etc.

I also would like to suggest we talk to Loren about how we might
create a RESTful interface to an existing logfile analyzer like
"Apache Chainsaw".

See http://logging.apache.org/chainsaw

Loren is an expert on logfile viewing and has already spent a great
deal of time thinking about integrating Apache Chainsaw with eXist.

- Dan

Re: [Exist-development] How can I get the Created and Modified times?

[Dan M. <dan...@gm...>]

Hello Daryn,

I assume you are looking for the XQuery functions that access when a
resource was created and when a resource was last modified.

Here are the two functions you seek:

*xmldb:last-modified($collection, $resource)
http://demo.exist-db.org/exist/functions/xmldb/last-modified

xmldb:created($collection, $resource)*

*http://demo.exist-db.org/exist/functions/xmldb/created*


Here is an example of how they are used to sync two collections:

http://en.wikibooks.org/wiki/XQuery/Synchronizing_Remote_Collections

- Dan

On Thu, Nov 26, 2009 at 12:11 PM, Daryn Holmes <dar...@gm...>wrote:

> Hello,
>
> Is there a way to access the created and modified data that is eXists
> maintains?
>
> Thanks...
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>


-- 
Dan McCreary
Semantic Solutions Architect
syntactica.com
952-460-1674
VOIP: 111@69.199.167.229

Re: [Exist-development] How can I get the Created and Modified times?

[Dannes W. <di...@ex...>]

Hi,

On Thu, Nov 26, 2009 at 7:11 PM, Daryn Holmes <dar...@gm...> wrote:
> Is there a way to access the created and modified data that is eXists
> maintains?

sorry? What is your question about?

probably this is the wrong mailinglist.......

regards

Dannes

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] How can I get the Created and Modified times?

[Daryn H. <dar...@gm...>]

Hello,

Is there a way to access the created and modified data that is eXists
maintains?

Thanks...

Re: [Exist-development] My vision about admin RESTful URLs

[Dmitriy S. <sha...@gm...>]

To make sure I did understand:

admin/logs/exist.log/line-range/fist-line/last-line

logs - script
exist.log - first parameter - file name
line-range - second parameter - command name
fist-line - first command parameter
fist-line - second command parameter

so, it going be 
{application}/{module}/{object-type}/{action}/{parameter}/{parameter}

-- 
Cheers,

Dmitriy Shabanov

On Thu, 2009-11-26 at 12:58 +0000, Thomas White wrote:
> admin/logs/exist.log/line-range/fist-line/last-line
> admin/logs/exist.log/page/page-size/page-number
> admin/logs/exist.log/time-range/2009-11-23 10:37:50:703/2009-11-23
> 10:37:55:903
> admin/logs/exist.log/all-after/2009-11-23 10:37:50:703
> admin/logs/exist.log/all
> 
>   a) first-line-number, last-line-number
>   b) page-size, page-number,
>   c) timestamp-beginning, timestamp-end
>   d) all-after-timestamp
>   e) all  ( bad idea! )
>  
> Thomas
> 
> ------
> 
> Thomas White
> 
> Mobile:+44 7711 922 966
> Skype: thomaswhite
> gTalk: thomas.0007
> Linked-In:http://www.linkedin.com/in/thomaswhite0007
> facebook: http://www.facebook.com/thomas.0007
> 
> 
> 
> 
> 2009/11/26 Dmitriy Shabanov <sha...@gm...>
>         On Thu, 2009-11-26 at 12:42 +0000, Thomas White wrote:
>         > c) timestamp-beginning, timestamp-end
>         
>         
>         How to apply it to url?
>         
>         --
>         Cheers,
>         
>         Dmitriy Shabanov
>         
> 

Re: [Exist-development] My vision about admin RESTful URLs

[Thomas W. <tho...@gm...>]

admin/logs/exist.log/line-range/fist-line/last-line
 admin/logs/exist.log/page/page-size/page-number
 admin/logs/exist.log/time-range/2009-11-23 10:37:50:703/2009-11-23
10:37:55:903
 admin/logs/exist.log/all-after/2009-11-23 10:37:50:703
 admin/logs/exist.log/all

  a) first-line-number, last-line-number
  b) page-size, page-number,
  c) timestamp-beginning, timestamp-end
  d) all-after-timestamp
  e) all  ( bad idea! )

Thomas

------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007



2009/11/26 Dmitriy Shabanov <sha...@gm...>

> On Thu, 2009-11-26 at 12:42 +0000, Thomas White wrote:
> > c) timestamp-beginning, timestamp-end
>
> How to apply it to url?
>
> --
> Cheers,
>
> Dmitriy Shabanov
>
>

Re: [Exist-development] My vision about admin RESTful URLs

[Dmitriy S. <sha...@gm...>]

On Thu, 2009-11-26 at 12:42 +0000, Thomas White wrote:
> c) timestamp-beginning, timestamp-end

How to apply it to url?

-- 
Cheers,

Dmitriy Shabanov