exist-development — eXist Developer's List

[Exist-development] new functionality: Execution Pipeline

[Thomas W. <tho...@gm...>]

I would like to propose a new functionality that I believe could be very
beneficial for eXist users:
Asynchronous Execution PipelineThis a mechanism for execution of number of
asynchronous jobs simultaneously.  It is very useful for executing long
running jobs or in cases where it is impossible to predict how long it will
take to perform the operation. Every job will run as a separated thread and
the jobID and the estimated delay will be returned immediately to the
caller.

Use cases: 1. Executing long running queries

   - Callback function will be used to store the result, at a location
   according to the function-parameters.
   - A client checking periodically the status of this job will take next
   action.

2. Fetching data from (large) number of remote URLs

   - An XQuery or a scheduled job creates XX execution pipeline entries for
   each remote server.
   - Callback functions are used to store the results, at a location
   according to the function-parameters.
   - The batch callback function will combine the result and trigger the
   next action.

3. Federated search, on a web client

   - A web client sends a search request to a local XQuery, that creates XX
   execution pipeline entries for each remote server and returns to the web
   client a batch-id.
   - The web client queries the status for the jobs with this batch-id
   periodically and when some of the jobs has status 'completed', web client
   gets the result for this job and displays it on the screen asynchronously.

4. Federated search, on a the server

   - A web client sends a search request to a local XQuery, that creates XX
   execution pipeline entries for each remote server and returns to the web
   client a batch-id.
   - Every job callback function will save the result at a location
   according to the function-parameters. The batch callback function will
   combine the result.
   - The web client queries the status for this batch periodically and when
   the batch is completed, web client gets the result and displays combined
   result set on the screen asynchronously.


*5. Data Replication *

   - An XQuery or a scheduled job creates XX execution pipeline entries for
   each remote server.
   - Execute-before function will identify what needs to be replicated.
   - The main function does the replication.
   - The batch callback function moves the replication marker.



A call to the Execution Pipe Line:
   *execution-pipeline:addJob*( function, function-parameters,
pipeline-parameters )*
 *returning :
    handlerID, estimated-delay,  function-parameters


To get the result we need to call another function:
    *execution-pipeline:getJobResults*( handlerID, autoClose )
returning either:
   * the result data set*. if autoClose is true then close the job and
release all used resources.
or
   same handlerID, new-estimated-delay,function -parameters
or
   unknown-handlerID error

*execution-pipeline:getJobStatus*( handlerID )
returns
        status of the job, function-parameters for this job

*execution-pipeline:getBatchStatus*(  batch-ID )
returns
        the status for all jobs from a particular batch ID.


*execution-pipeline:getStatus*(  )
returns
        the status for all jobs.


*execution-pipeline:closeJob*( handlerID )
*execution-pipeline:closeBatch*( batchID )
*execution-pipeline:closeAll*( )


*function-parameters:*

   - job-statistic-id: used to keep average time for execution of this
   function. average time= (previous-average-time + last-execution-time)/2.
   URL with specific parameters could be used as an ID.
   - execute-before function: when provided, it will be called before
   calling the main function for this job. If the result is 0 then proceed with
   the main function, otherwise use the result as number of milliseconds to
   put this job to sleep and try later.
   - callback function: when provided callback-function will be called as
   callback-function( handlerID, result, function-parameters ). if it
   returns true() the job will be closed.
   - any other parameters that may be used by the callback function.


*pipeline-parameters:*

   - batch-ID - to group
   - batch-callback-function: called when all jobs from the batch are
   completed.
   - any other parameters that may be used by the callback function.


Any comments?

Thomas


------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007

Re: [Exist-development] Using JavaScript Libraries

[Dmitriy S. <sha...@gm...>]

I agree with Thomas. It's not a question "how many", it's - "how".

We need "standard" pluggable mechanism for web applications.

-- 
Cheers,

Dmitriy Shabanov

PS I working under proposal, but I do need first order functions ...

On Mon, 2010-01-25 at 14:10 +0000, Thomas White wrote:
> Yes we can keep only one version per library but ir is still good idea
> to have the version included in the file name. Both jQuery library and
> jQuery plugins file's names already follow this practice.
> 
> About the the single implementation - it is like asking Nikon users to
> use Canon :-).  jQuery and Yahoo and many other JS libraries will have
> their followers for many good reasons. jQuery is 118kb and 56kb
> minfielded, Yahoo JS  is about 1Mb. These file sizes are much smaller
> then most of the other files in the installation. I do not think sizes
> of the JS files will be ever an issue.
> 
> Thomas
> 
> 
> 
> ------
> 
> Thomas White
> 
> Mobile:+44 7711 922 966
> Skype: thomaswhite
> gTalk: thomas.0007
> Linked-In:http://www.linkedin.com/in/thomaswhite0007
> facebook: http://www.facebook.com/thomas.0007
> 
> 
> 
> 
> 2010/1/25 Dannes Wessels <di...@ex...>:
> > Hi,
> >
> > On Mon, Jan 25, 2010 at 1:54 PM, Adam Retter <ad...@ex...> wrote:
> >>
> >> Can we standardise on one implementation and version?
> >>
> >
> > +1 for me.
> >
> >
> > In the context of exist, that is, what is deployed by the exist
> > installer/war file, just one version of a library should be deployed.
> > Applications can bring it's own versions, for sure, and we can discuss about
> > direcotiy structures, but let's keep exist-db simple... and small :-)
> >
> > D.
> >
> > --
> > eXist-db Native XML Database - http://exist-db.org
> > Join us on linked-in: http://www.linkedin.com/groups?gid=35624
> >
> 
> ------------------------------------------------------------------------------
> Throughout its 18-year history, RSA Conference consistently attracts the
> world's best and brightest in the field, creating opportunities for Conference
> attendees to learn about information security's most important issues through
> interactions with peers, luminaries and emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] servlet's refactoring

[Adam R. <ad...@ex...>]

I am sorry, but I dont understand how this will work or be managed in practice?

2010/1/25 Dmitriy Shabanov <sha...@gm...>:
> "traffic lights" - coding control proposal.
>
> * green - open for commits; after commit switch to yellow until another
> (different person, not commiter) will switch to green or red
>
> * red - no commits (issue description) (warnings to commiter by mail)
>
> "red trunk" does not mean untested code income. It information to be
> tested on different platforms by others (+warning that critical part was
> changed).
>
> Example: I have debian & TC for testing (*nix OS), so I did change
> critical code & I want to be sure that it did not introduce problems on
> another OS. Same time, I don't want more commits before check-up
> completed.
>
> --
> Cheers,
>
> Dmitriy Shabanov
>
> On Mon, 2010-01-25 at 10:05 +0000, Adam Retter wrote:
>> I am concerned by the lack of testing that is being carried out.
>>
>> trunk should always be stable, buildable and useable, this also means
>> that all tests in the test suite should pass. I am also not happy when
>> I see new code being added without new tests. This leads to all sorts
>> of worries and issues such as unmaintainable/ununderstandable code
>> should a commiter leave. The tests should reveal business logic
>> intention and bounds for execution.
>>
>> Branches are fine for writting code and testing yourself, but who else
>> has tested your branch? I would really like to see major changes,
>> tested by more than just one person. This could perhaps be as simple
>> as announcing the feature and request for testing on the mailing list
>> and producing a test release of some sort for those interested users
>> to test. Once their feedback is positive these changes could be merged
>> into trunk.
>>
>>
>> 2010/1/25 Dmitriy Shabanov <sha...@gm...>:
>> > On Mon, Jan 25, 2010 at 3:49 AM, Adam Retter <ad...@ex...> wrote:
>> >>
>> >> On 24 January 2010 15:27, Dmitriy Shabanov <sha...@gm...> wrote:
>> >> > Hello,
>> >> >
>> >> > I did rewrite servlets to integrate servlet's native authentication
>> >> > mechanism
>> >> > (http://jcp.org/aboutJava/communityprocess/mrel/jsr196/index.html -
>> >> > SRV.12.5 Authentication)
>> >> >
>> >> > I would like to switch trunk to "red". Waiting for "green" response ...
>> >>
>> >> What does this mean?
>> >>
>> > "Red" trunk mean no commits untill another person confirm that trunk have no
>> > issue (new as minimum)
>> >
>> > --
>> > Dmitriy Shabanov
>> >
>>
>>
>>
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] Using JavaScript Libraries

[Thomas W. <tho...@gm...>]

Yes we can keep only one version per library but ir is still good idea
to have the version included in the file name. Both jQuery library and
jQuery plugins file's names already follow this practice.

About the the single implementation - it is like asking Nikon users to
use Canon :-).  jQuery and Yahoo and many other JS libraries will have
their followers for many good reasons. jQuery is 118kb and 56kb
minfielded, Yahoo JS  is about 1Mb. These file sizes are much smaller
then most of the other files in the installation. I do not think sizes
of the JS files will be ever an issue.

Thomas



------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007




2010/1/25 Dannes Wessels <di...@ex...>:
> Hi,
>
> On Mon, Jan 25, 2010 at 1:54 PM, Adam Retter <ad...@ex...> wrote:
>>
>> Can we standardise on one implementation and version?
>>
>
> +1 for me.
>
>
> In the context of exist, that is, what is deployed by the exist
> installer/war file, just one version of a library should be deployed.
> Applications can bring it's own versions, for sure, and we can discuss about
> direcotiy structures, but let's keep exist-db simple... and small :-)
>
> D.
>
> --
> eXist-db Native XML Database - http://exist-db.org
> Join us on linked-in: http://www.linkedin.com/groups?gid=35624
>

Re: [Exist-development] Using JavaScript Libraries

[Dannes W. <di...@ex...>]

Hi,

On Mon, Jan 25, 2010 at 1:54 PM, Adam Retter <ad...@ex...> wrote:

> Can we standardise on one implementation and version?
>
>
+1 for me.


In the context of exist, that is, what is deployed by the exist
installer/war file, just one version of a library should be deployed.
Applications can bring it's own versions, for sure, and we can discuss about
direcotiy structures, but let's keep exist-db simple... and small :-)

D.

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] servlet's refactoring

[Dmitriy S. <sha...@gm...>]

"traffic lights" - coding control proposal.

* green - open for commits; after commit switch to yellow until another
(different person, not commiter) will switch to green or red

* red - no commits (issue description) (warnings to commiter by mail)

"red trunk" does not mean untested code income. It information to be
tested on different platforms by others (+warning that critical part was
changed).

Example: I have debian & TC for testing (*nix OS), so I did change
critical code & I want to be sure that it did not introduce problems on
another OS. Same time, I don't want more commits before check-up
completed.

-- 
Cheers,

Dmitriy Shabanov

On Mon, 2010-01-25 at 10:05 +0000, Adam Retter wrote:
> I am concerned by the lack of testing that is being carried out.
> 
> trunk should always be stable, buildable and useable, this also means
> that all tests in the test suite should pass. I am also not happy when
> I see new code being added without new tests. This leads to all sorts
> of worries and issues such as unmaintainable/ununderstandable code
> should a commiter leave. The tests should reveal business logic
> intention and bounds for execution.
> 
> Branches are fine for writting code and testing yourself, but who else
> has tested your branch? I would really like to see major changes,
> tested by more than just one person. This could perhaps be as simple
> as announcing the feature and request for testing on the mailing list
> and producing a test release of some sort for those interested users
> to test. Once their feedback is positive these changes could be merged
> into trunk.
> 
> 
> 2010/1/25 Dmitriy Shabanov <sha...@gm...>:
> > On Mon, Jan 25, 2010 at 3:49 AM, Adam Retter <ad...@ex...> wrote:
> >>
> >> On 24 January 2010 15:27, Dmitriy Shabanov <sha...@gm...> wrote:
> >> > Hello,
> >> >
> >> > I did rewrite servlets to integrate servlet's native authentication
> >> > mechanism
> >> > (http://jcp.org/aboutJava/communityprocess/mrel/jsr196/index.html -
> >> > SRV.12.5 Authentication)
> >> >
> >> > I would like to switch trunk to "red". Waiting for "green" response ...
> >>
> >> What does this mean?
> >>
> > "Red" trunk mean no commits untill another person confirm that trunk have no
> > issue (new as minimum)
> >
> > --
> > Dmitriy Shabanov
> >
> 
> 
> 

Re: [Exist-development] Using JavaScript Libraries

[Adam R. <ad...@ex...>]

Can we standardise on one implementation and version?

2010/1/25 Thomas White <tho...@gm...>:
> It seams we are going to use more and more JavaScript  libraries and I
> would like to propose a way to organise the code that has been used in
> many places and is proven as a good practise.
>
> There are two main challenges:
>
> 1. Libraries do change and evolve frequently and in general the code
> that worked before, with the next version may not work at all.
> 2. When a lot of people are contributing to a project, there is a
> strong believe that whatever is used in one application (or page) will
> be definitely used by many applications (or pages) later and as a
> result a lot of library code are places in a generic directory with
> the intention to be used later. In our case 'scripts' directory.
>
> This type of problems are very ofter when jQuery + jQuery plugins (my
> favorite JS library) are used. Some of the plugins require jQuery
> version 1.3, some other 1.2 and on the top of that every plugin have
> its own versions are not necessarily compatible with each other.
>
> My proposal is:
> 1. Keep Javascript libraries in a subdirectory for the application
> until other application uses it. This way every application can have
> matching versions.Another advantage is - the application is self
> contained and by deleting / coping the directory we can easily
> inslall/uninslatll an application. I do believe this is in the most
> cases the simplest and easiest way.
> 2. If we really have to share JavaScript libraries (more then one
> application uses it), then the libraries should be located in
> directories that have version number of the library is part of the
> directory name.
> 3. When  jQuery plugins (or other small libraries that depend on main
> libraries) are used, then the name of the file should contain the
> version number.
> 4. The minimised version of every JS file will have '.min.js' suffix.
>
>
> In a long run the best solution will be to have a Static Resources
> Manger module that receives the name and the version of a file and
> returns a relative or an absolute URL.
>
> I hope this will help.
> Thomas
>
>
>
>
>
> ------
>
> Thomas White
>
> Mobile:+44 7711 922 966
> Skype: thomaswhite
> gTalk: thomas.0007
> Linked-In:http://www.linkedin.com/in/thomaswhite0007
> facebook: http://www.facebook.com/thomas.0007
>
> ------------------------------------------------------------------------------
> Throughout its 18-year history, RSA Conference consistently attracts the
> world's best and brightest in the field, creating opportunities for Conference
> attendees to learn about information security's most important issues through
> interactions with peers, luminaries and emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] Using JavaScript Libraries

[Thomas W. <tho...@gm...>]

It seams we are going to use more and more JavaScript  libraries and I
would like to propose a way to organise the code that has been used in
many places and is proven as a good practise.

There are two main challenges:

1. Libraries do change and evolve frequently and in general the code
that worked before, with the next version may not work at all.
2. When a lot of people are contributing to a project, there is a
strong believe that whatever is used in one application (or page) will
be definitely used by many applications (or pages) later and as a
result a lot of library code are places in a generic directory with
the intention to be used later. In our case 'scripts' directory.

This type of problems are very ofter when jQuery + jQuery plugins (my
favorite JS library) are used. Some of the plugins require jQuery
version 1.3, some other 1.2 and on the top of that every plugin have
its own versions are not necessarily compatible with each other.

My proposal is:
1. Keep Javascript libraries in a subdirectory for the application
until other application uses it. This way every application can have
matching versions.Another advantage is - the application is self
contained and by deleting / coping the directory we can easily
inslall/uninslatll an application. I do believe this is in the most
cases the simplest and easiest way.
2. If we really have to share JavaScript libraries (more then one
application uses it), then the libraries should be located in
directories that have version number of the library is part of the
directory name.
3. When  jQuery plugins (or other small libraries that depend on main
libraries) are used, then the name of the file should contain the
version number.
4. The minimised version of every JS file will have '.min.js' suffix.


In a long run the best solution will be to have a Static Resources
Manger module that receives the name and the version of a file and
returns a relative or an absolute URL.

I hope this will help.
Thomas





------

Thomas White

Mobile:+44 7711 922 966
Skype: thomaswhite
gTalk: thomas.0007
Linked-In:http://www.linkedin.com/in/thomaswhite0007
facebook: http://www.facebook.com/thomas.0007

Re: [Exist-development] servlet's refactoring

[Adam R. <ad...@ex...>]

I am concerned by the lack of testing that is being carried out.

trunk should always be stable, buildable and useable, this also means
that all tests in the test suite should pass. I am also not happy when
I see new code being added without new tests. This leads to all sorts
of worries and issues such as unmaintainable/ununderstandable code
should a commiter leave. The tests should reveal business logic
intention and bounds for execution.

Branches are fine for writting code and testing yourself, but who else
has tested your branch? I would really like to see major changes,
tested by more than just one person. This could perhaps be as simple
as announcing the feature and request for testing on the mailing list
and producing a test release of some sort for those interested users
to test. Once their feedback is positive these changes could be merged
into trunk.


2010/1/25 Dmitriy Shabanov <sha...@gm...>:
> On Mon, Jan 25, 2010 at 3:49 AM, Adam Retter <ad...@ex...> wrote:
>>
>> On 24 January 2010 15:27, Dmitriy Shabanov <sha...@gm...> wrote:
>> > Hello,
>> >
>> > I did rewrite servlets to integrate servlet's native authentication
>> > mechanism
>> > (http://jcp.org/aboutJava/communityprocess/mrel/jsr196/index.html -
>> > SRV.12.5 Authentication)
>> >
>> > I would like to switch trunk to "red". Waiting for "green" response ...
>>
>> What does this mean?
>>
> "Red" trunk mean no commits untill another person confirm that trunk have no
> issue (new as minimum)
>
> --
> Dmitriy Shabanov
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] servlet's refactoring

[Dmitriy S. <sha...@gm...>]

People did ask to inform about critical changes, so I did it. And openid was
on branch for 2 months.

On Mon, Jan 25, 2010 at 12:34 PM, Dannes Wessels <di...@ex...>wrote:

> Hi,
>
>
> On Sun, Jan 24, 2010 at 11:49 PM, Adam Retter <ad...@ex...> wrote:
>
>> > I would like to switch trunk to "red". Waiting for "green" response ...
>>
>> What does this mean?
>>
>
> I think it is the TDD paradigm.....
>
> IMO we must try to have 'trunk' stable at all time, because people rely on
> it. If invasive / risky changes are made, never do it in trunk. We'll lose
> credibility if we make mistakes here.
>
> For risky changes, a short lived branch should be created. Probably for the
> openid changes, a branch would have been a good idea as well.
>
> regards
>
> Dannes
>
> --
> eXist-db Native XML Database - http://exist-db.org
> Join us on linked-in: http://www.linkedin.com/groups?gid=35624
>



-- 
Dmitriy Shabanov

Re: [Exist-development] servlet's refactoring

[Dannes W. <di...@ex...>]

Hi,

On Sun, Jan 24, 2010 at 11:49 PM, Adam Retter <ad...@ex...> wrote:

>  > I would like to switch trunk to "red". Waiting for "green" response ...
>
> What does this mean?
>

I think it is the TDD paradigm.....

IMO we must try to have 'trunk' stable at all time, because people rely on
it. If invasive / risky changes are made, never do it in trunk. We'll lose
credibility if we make mistakes here.

For risky changes, a short lived branch should be created. Probably for the
openid changes, a branch would have been a good idea as well.

regards

Dannes

-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] servlet's refactoring

[Adam R. <ad...@ex...>]

On 24 January 2010 15:27, Dmitriy Shabanov <sha...@gm...> wrote:
> Hello,
>
> I did rewrite servlets to integrate servlet's native authentication
> mechanism
> (http://jcp.org/aboutJava/communityprocess/mrel/jsr196/index.html -
> SRV.12.5 Authentication)
>
> I would like to switch trunk to "red". Waiting for "green" response ...

What does this mean?

> --
> Cheers,
>
> Dmitriy Shabanov
>
> ------------------------------------------------------------------------------
> Throughout its 18-year history, RSA Conference consistently attracts the
> world's best and brightest in the field, creating opportunities for Conference
> attendees to learn about information security's most important issues through
> interactions with peers, luminaries and emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

[Exist-development] Junit test suite......

[Dannes W. <di...@ex...>]

Hi,

since some time we have an issue in our test suite:

All Errors ClassNameStatusTypeTime(s)
AllXmlRpcTests<file:///Users/wessels/Development/existdb/1.5/trunk/test/junit/html/org/exist/xmlrpc/53_AllXmlRpcTests.html>
testCollectionWithAccentsAndSpaces<file:///Users/wessels/Development/existdb/1.5/trunk/test/junit/html/org/exist/xmlrpc/53_AllXmlRpcTests.html#testCollectionWithAccentsAndSpaces>
ErrorForked Java VM exited abnormally. Please note the time in the report
does not reflect the time until the VM exit.

junit.framework.AssertionFailedError: Forked Java VM exited abnormally.
Please note the time in the report does not reflect the time until the VM
exit.
0.000

this is quite a strange issue. Any idea where to search this issue? There is
no logging available, and in netbeans the test runs fine :-(

greets

Dannes


-- 
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

[Exist-development] servlet's refactoring

[Dmitriy S. <sha...@gm...>]

Hello,

I did rewrite servlets to integrate servlet's native authentication
mechanism
(http://jcp.org/aboutJava/communityprocess/mrel/jsr196/index.html -
SRV.12.5 Authentication)

I would like to switch trunk to "red". Waiting for "green" response ...

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] admin panel

[Dmitriy S. <sha...@gm...>]

On Mon, 2010-01-18 at 15:35 +0000, Thomas White wrote:
> Dan,
> 
> > 1) How to package the "web-admin" applications together in a set.
> Have a directory|collection as an entry point for the admin
> application and sub-directories|sub-collections  for all
> sub-applications. This way dealing with the single entry point
> directory we can easily copy, move or delete the whole application.

+1

> > 2) How to share common libraries like Blueprint CSS files and JQuery.
> Sharing libraries across applications is a really bad thing especially
> then these libraries have so many versions.
> Keeping the files as a subdirectory of the the main application entry
> point directory is a easy way to share the code between
> sub-applications.
> With such cheap disk space and small library files this is not an issue.

As opposite solution: application name & version as collection name for
lib's applications.

> > 3) How to make these applications "portable" and db collection location independant
> If you use the approach with a single application entry point
> directory (see my answer at point 1.) then using only relative paths
> will do the job perfectly.

+1 for relative paths

> > 4) How to build install and uninstall scripts
> > 5) How to document the basic functions like the RESTadmin actions so they can be reused in all the applications
> Some time ago I offered a URL schema that is task oriented, it is
> implementation agnostic and it allows to control the access to the
> admin functions to be controlled in details. I still believe is not a
> good idea to put all actions for an object in a single place, because
> there will be no formal way to distinguish how to give partial access
> to a sub sets of the actions.
> 
> I hope this will help.
> Thomas
-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] admin panel

[Thomas W. <tho...@gm...>]

Dan,

> 1) How to package the "web-admin" applications together in a set.
Have a directory|collection as an entry point for the admin
application and sub-directories|sub-collections  for all
sub-applications. This way dealing with the single entry point
directory we can easily copy, move or delete the whole application.

> 2) How to share common libraries like Blueprint CSS files and JQuery.
Sharing libraries across applications is a really bad thing especially
then these libraries have so many versions.
Keeping the files as a subdirectory of the the main application entry
point directory is a easy way to share the code between
sub-applications.
With such cheap disk space and small library files this is not an issue.

> 3) How to make these applications "portable" and db collection location independant
If you use the approach with a single application entry point
directory (see my answer at point 1.) then using only relative paths
will do the job perfectly.

> 4) How to build install and uninstall scripts
> 5) How to document the basic functions like the RESTadmin actions so they can be reused in all the applications
Some time ago I offered a URL schema that is task oriented, it is
implementation agnostic and it allows to control the access to the
admin functions to be controlled in details. I still believe is not a
good idea to put all actions for an object in a single place, because
there will be no formal way to distinguish how to give partial access
to a sub sets of the actions.

I hope this will help.
Thomas

Re: [Exist-development] admin panel

[Dmitriy S. <sha...@gm...>]

On Mon, 2010-01-18 at 06:29 -0600, Dan McCreary wrote:
> 5) How to document the basic functions like the RESTadmin actions so
> they can be reused in all the applications

This is xquery module? Pluggable xquery module solve this one.

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] admin panel

[Dmitriy S. <sha...@gm...>]

On Mon, 2010-01-18 at 06:29 -0600, Dan McCreary wrote:
> 2) How to share common libraries like Blueprint CSS files and JQuery.

This one should be simple. Library as application & other application
depend on library's application.

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] Questions on packaging for new administration applications

[Dmitriy S. <sha...@gm...>]

On Thu, 2010-01-14 at 15:55 +0100, Wolfgang Meier wrote:
> For a short time plan, it would probably be sufficient to distribute
> the administrative tools within a distinct section of the webapp
> directory (webapp/administration, which would then replace
> webapp/admin). This would allow us to properly test and improve the
> tools until they are ready to be repackaged in another way. It would
> also be easier to see how all the tools play together and what needs
> to be added.

+1 for webapp/administration

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] jetty 7

[Dmitriy S. <sha...@gm...>]

On Wed, 2010-01-13 at 19:01 +0100, Adam Retter wrote:
> 2010/1/12 Dannes Wessels <di...@ex...>:
> >
> > On 12 Jan 2010, at 19:48 , Dmitriy Shabanov wrote:
> >
> > * http://wiki.eclipse.org/Jetty/Feature/JAAS
> > * http://wiki.eclipse.org/Jetty/Feature/Continuations
> > (asynchronous servlets (updated continuations))
> > * faster (personal feeling, but it looks very fast)
> >
> >
> > Hmmm if we want to use JAAS... we can already use that? (just a jar). I am
> > afraid to lose compatibility with the servlet spec and thus... lose tomcat
> > etc....
> 
> Is it not backward compatible with 2.5 servlet spec? I kinda assumed
> it should be?!?

What is relation between servlet & JAAS do you see? As soon as you
answer that question, you will get answers on your questions. :-)

I going to update trunk to jetty 7 on this week-end.

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] Questions on packaging for new administration applications

[Wolfgang M. <wol...@ex...>]

Hi Dan,

> We have some questions for the team package the new administration tools we
> are prototyping and some other new tools like an index adviser we are
> working on.

Cool. It would indeed help if those tools could make it into SVN
somewhere to avoid a duplication of efforts. There seems to be some
overlap with work other people are doing (someone's creating a
collection browser with svn integration; I started working on an
"index adviser" - not sure if we had the same idea - as well).

> 1) Do you think that some or all of these tools should be optional in an
> installation?  Do we want to think of them as part of the core eXist system
> or are we now entering the more complex world of "applications"?

It depends on what tools we are talking about. Everything which is now
covered by the admin clients (plus backup/restore and index
configuration/adviser) should probably be considered "core" and should
not be optional, i.e. they should be part of the standard eXist
distribution. However, we could still manage them as self-contained
packages once a packaging mechanism is in place.

I guess we need to proceed step by step. You, me, Loren and others
were thinking about auto-deployment and packaging mechanisms for
XQuery apps for some time. Ideally it should be possible to deploy an
app by just dropping an archive into a directory or uploading it to an
URL. We don't have such a mechanism yet, but if we can work out the
few missing details, e.g. during our meeting in Prague, it should not
be a big issue to implement it.

For a short time plan, it would probably be sufficient to distribute
the administrative tools within a distinct section of the webapp
directory (webapp/administration, which would then replace
webapp/admin). This would allow us to properly test and improve the
tools until they are ready to be repackaged in another way. It would
also be easier to see how all the tools play together and what needs
to be added.

> 2) If these new tools will be considered "Applications" how should they be
> added or removed from eXist site?  How should we track shared resources that
> they depend on?

I think this has to be handled by the packaging mechanism. For
example, certain resources could be marked as shared in the descriptor
for the app, and the package management system would take care to
install them into a shared area.

> 2) Should we create some configuration file entry to have "admin
> applications" be conditionally loaded or not load into a system?  Should we
> have an XForms application to manage this configuration file?

It should at least be possible to disable the entire admin application
suite (for security reasons). Disabling single applications may make
sense if they pose a potential security issue. I don't think disk
space is an issue for the admin apps, so most people will probably
want to have as many enabled as possible.

> 3) Some of the tools will use XSLTForms, the Blueprint CSS frame and JQuery
> JavaScript libraries.  Is there a central place that these should be
> installed?

Right now I try to keep those tools in webapp/scripts (I just added
jquery) or webapp/styles. I use XQueryURLRewrite to redirect the
browser request to the scripts from within subdirectories. This way we
can keep the links simple.

Wolfgang

[Exist-development] Questions on packaging for new administration applications

[Dan M. <dan...@gm...>]

Hello Wolfgang and eXist developers.

I hope everyone is doing well in the new year.

We have some questions for the team package the new administration tools we
are prototyping and some other new tools like an index adviser we are
working on.

1) Do you think that some or all of these tools should be optional in an
installation?  Do we want to think of them as part of the core eXist system
or are we now entering the more complex world of "applications"?
2) If these new tools will be considered "Applications" how should they be
added or removed from eXist site?  How should we track shared resources that
they depend on?
2) Should we create some configuration file entry to have "admin
applications" be conditionally loaded or not load into a system?  Should we
have an XForms application to manage this configuration file?
3) Some of the tools will use XSLTForms, the Blueprint CSS frame and JQuery
JavaScript libraries.  Is there a central place that these should be
installed?

Let me know if you have any guidance for the team.  Perhaps we should write
up a short term plan and some longer term strategies as we move toward a
full OSGI environment.

Thanks - Dan & the New Admin Client Team

Re: [Exist-development] jetty 7

[Adam R. <ad...@ex...>]

2010/1/12 Dannes Wessels <di...@ex...>:
>
> On 12 Jan 2010, at 19:48 , Dmitriy Shabanov wrote:
>
> * http://wiki.eclipse.org/Jetty/Feature/JAAS
> * http://wiki.eclipse.org/Jetty/Feature/Continuations
> (asynchronous servlets (updated continuations))
> * faster (personal feeling, but it looks very fast)
>
>
> Hmmm if we want to use JAAS... we can already use that? (just a jar). I am
> afraid to lose compatibility with the servlet spec and thus... lose tomcat
> etc....

Is it not backward compatible with 2.5 servlet spec? I kinda assumed
it should be?!?


> These are no strong arguments, are they:
>
> * preparation for servlets 3.0
> * step forward =)
>
>
> Kind regards
> Dannes
> --
> eXist-db Native XML Database - http://exist-db.org
> Join us on linked-in: http://www.linkedin.com/groups?gid=35624
>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.Net email is sponsored by the Verizon Developer Community
> Take advantage of Verizon's best-in-class app development support
> A streamlined, 14 day to market process makes app distribution fast and easy
> Join now and get one step closer to millions of Verizon customers
> http://p.sf.net/sfu/verizon-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] jetty 7

[Dmitriy S. <sha...@gm...>]

On Tue, 2010-01-12 at 20:27 +0100, Dannes Wessels wrote:
> Hmmm if we want to use JAAS... we can already use that? (just a jar)

They did change security layer (totally).

> I am afraid to lose compatibility with the servlet spec and thus...
lose tomcat etc....

I watching to be in spec. I did understand that it's important. Current
jetty's security part is jetty only. 7 has JAAS way & it very close to
tomcat http://www.kopz.org/public/documents/tomcat/jaasintomcat.html

-- 
Cheers,

Dmitriy Shabanov

Re: [Exist-development] jetty 7

[Dannes W. <di...@ex...>]

On 12 Jan 2010, at 19:48 , Dmitriy Shabanov wrote:

> * http://wiki.eclipse.org/Jetty/Feature/JAAS 
> * http://wiki.eclipse.org/Jetty/Feature/Continuations 
> 	(asynchronous servlets (updated continuations))
> * faster (personal feeling, but it looks very fast)
> 

Hmmm if we want to use JAAS... we can already use that? (just a jar). I am afraid to lose compatibility with the servlet spec and thus... lose tomcat etc....

These are no strong arguments, are they:

> * preparation for servlets 3.0 
> * step forward =)



Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624