exist-development — eXist Developer's List

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Thu, Nov 4, 2010 at 5:45 AM, Chris Tomlinson <chr...@gm...
> wrote:

> We have been using VersioningTrigger and so will be interested in helping
> to test whatever changes are involved in the new design.
>

I'm very :-) TheVersioningTrigger should work with out any changers, well
just configured different way, the old configuration did have event
attribute

<exist:trigger event='store,remove'
class='org.exist.collections.triggers.TestTrigger'/>

new one do not require it, so configuration should be

<exist:trigger class='org.exist.collections.triggers.TestTrigger'/>


> I've looked through the various emails and wiki and so forth and I can not
> find a description or proposal for this significant revamp of triggers and
> their configuration.
>
> Can you please point me to such discussion so I can get up-to-speed on what
> the design goals are and what to expect?
>

The only public discussion  I can point is
http://demo.exist-db.org/exist/irclog/irclog.xql date 2010-11-01. I do plan
to document it with examples.

-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Chris T. <chr...@gm...>]

Dmitriy,

We have been using VersioningTrigger and so will be interested in helping to test whatever changes are involved in the new design. 

I've looked through the various emails and wiki and so forth and I can not find a description or proposal for this significant revamp of triggers and their configuration. 

Can you please point me to such discussion so I can get up-to-speed on what the design goals are and what to expect?

Thank you,
Chris Tomlinson


On Nov 3, 2010, at 12:41 PM, Dmitriy Shabanov wrote:

> Hello,
> 
> I did commit changers to trigger interface http://exist.svn.sourceforge.net/exist/?rev=13086&view=rev
> 
> "triggers interface: event's own method design
> 
> The events split base on nature: time, type and object type. Time have 'before' & 'after' methods. Type: 'create', 'update', 'copy', 'move' and 'delete'. Object type: 'document' & 'collection'.
> 
> Example: the document creation call two method: beforeCreateDocument & afterCreateDocument.
> 
> (WARNING: configuration are changing. It will be separated from index one, because if configuration defined it overwrite indexes & triggers setting. Plan to switch to new configuration framework)"
> 
> Are there any anyone who use triggers & ready to help with testing? (after I commit configuration changers) 
> 
> -- 
> Dmitriy Shabanov
> ------------------------------------------------------------------------------
> Achieve Improved Network Security with IP and DNS Reputation.
> Defend against bad network traffic, including botnets, malware, 
> phishing sites, and compromised hosts - saving your company time, 
> money, and embarrassment.   Learn More! 
> http://p.sf.net/sfu/hpdev2dev-nov_______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] ANN: upgrade of webdav interface in Trunk

[Dannes W. <da...@ex...>]

All,

so far I had a number of positive and constructive responses regarding the reimplementation of the webDAV interface, thnx for that!

I hope that the new implementation gives a more stable document exchange interface to the database. Please feel free to send me additional feedback regarding your specific clients. I'll update the documentation, e.g. with davfs specific settings later this week


Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] ANN: upgrade of webdav interface in Trunk

[Dannes W. <da...@ex...>]

Hi Peter,

I tired to solve the Lock issue last week. please could you confirm things work now?

regards

Dannes


On 27 Oct 2010, at 15:31 , Hungerburg wrote:

> Am 2010-10-26 21:46, schrieb Dannes Wessels:
>> - Need some help to check on recent version DAVFS , Konquerer and Nautilus
> 
> davfs2 1.4.5 tested fine, anecdotically only though, no test suite:
> - editing with komodo-edit: smooth (komodo edit does a lot of scanning 
> and is very slow with cifs over the wan, unlike eXist webdav.)
> - copying 300 xml files into validating eXist collection via webdav 
> takes 10% longer than via curl/REST, but maybe due to network.
> - options used: use_locks 0, delay_upload 0
> 
> davfs2 1.2.1 also tests ok,
> options used: use_locks 0, delay_upload 0, use_expect100 0
> 
> gvfs1.6 (same as nautilus?) still fails trying to access "OPTIONS 
> /exist/webdav/" without "db".
> 
> NEW this time: uploading with the windows app "BitKinex" works. /In 
> place" editing there though fails when trying to get a lock.
> 
> 2010-10-27 15:28:49,202 [eXistThread-22] DEBUG (MiltonDocument.java 
> [getCurrentLock]:285) - No database lock token.
> 2010-10-27 15:28:49,202 [eXistThread-22] DEBUG (LockHandler.java 
> [processNewLock]:205) - locking: some-binary.txt
> 2010-10-27 15:28:49,202 [eXistThread-22] ERROR (StandardFilter.java 
> [process]:44) - process
> java.lang.NullPointerException
> 	at org.exist.webdav.MiltonResource.convertToken(MiltonResource.java:218)
> 	at org.exist.webdav.MiltonDocument.lock(MiltonDocument.java:197)
> 	at 
> com.bradmcevoy.http.webdav.LockHandler.processNewLock(LockHandler.java:208)
> 	at 
> com.bradmcevoy.http.webdav.LockHandler.processExistingResource(LockHandler.java:93)
> 	at com.bradmcevoy.http.webdav.LockHandler.process(LockHandler.java:69)
> 	at com.bradmcevoy.http.StandardFilter.process(StandardFilter.java:32)
> 	at com.bradmcevoy.http.FilterChain.process(FilterChain.java:21)
> 	at com.bradmcevoy.http.HttpManager.process(HttpManager.java:152)
> 	at com.bradmcevoy.http.MiltonServlet.service(MiltonServlet.java:169)

Kind regards

Dannes

--
eXist-db Native XML Database - http://exist-db.org
Join us on linked-in: http://www.linkedin.com/groups?gid=35624

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 9:31 PM, Evgeny Gazdovsky <gaz...@gm...>wrote:

> >> Use case is in restful applications: eg. validating data coming in by
> http
> >> PUT - from an xform eg. or from webdav, in any another way than
> >> wellformedness and schema conformance. Right now incoming binary data
> cannot
> >> be validated at all, and rejected for that, at least not on upload time.
> >> Wasn't that useful?
> >
> > Yes, it's, but rise more questions :-) how do you plan to check binary
> data?
> > (I do want to be sure that your case will be covered)
>
> You can map the PUT request to some xquery via urlrewriter.
> This even could be one uri and one simple script for all
> requests.
>
> Why use a harcoded java trigers here?
>

I was think same, but we have to wait for details.

-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Evgeny G. <gaz...@gm...>]

>> Use case is in restful applications: eg. validating data coming in by http
>> PUT - from an xform eg. or from webdav, in any another way than
>> wellformedness and schema conformance. Right now incoming binary data cannot
>> be validated at all, and rejected for that, at least not on upload time.
>> Wasn't that useful?
>
> Yes, it's, but rise more questions :-) how do you plan to check binary data?
> (I do want to be sure that your case will be covered)

You can map the PUT request to some xquery via urlrewriter.
This even could be one uri and one simple script for all
requests.

Why use a harcoded java trigers here?

Less code + more data (include config) = more maintainability.

--
Evgeny

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 8:53 PM, Hungerburg <pc...@my...> wrote:

> Am 2010-11-03 15:45, schrieb Dmitriy Shabanov:
>
>
>>> The information I missed the most is, if a trigger can mess with control
>>> flow, i.e. somehow return a result of its operation, or throw an error
>>> condition that will be usefully caught, and if there is a messaging
>>> format specified, so that differing frontends (rpc,rest,webdav) can
>>> communicate that information to the caller/user. Will such be part of
>>> the new interface?
>>>
>>
>>
>> Trigger can throw exception on before processing, but can not on after.
>> Can
>> you descibe your use case?
>>
>>
> Use case is in restful applications: eg. validating data coming in by http
> PUT - from an xform eg. or from webdav, in any another way than
> wellformedness and schema conformance. Right now incoming binary data cannot
> be validated at all, and rejected for that, at least not on upload time.
> Wasn't that useful?


Yes, it's, but rise more questions :-) how do you plan to check binary data?
(I do want to be sure that your case will be covered)

-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 8:01 PM, Evgeny Gazdovsky <gaz...@gm...>wrote:

> > with function trigger:beforeCreateDocument & trigger:afterCreateDocument.
> > I do think to store this xquery modules under /db/system somethere,
> because
> > of Evgeny's use case: trubles on recovery from backup in result of
> missing
> > files.
>
> I prefer names like: trigger:before-create-document
>
> Yes, sure.


> > I do think to store this xquery modules under /db/system somethere,
> > because of Evgeny's use case: trubles on recovery from backup
> > in result of missing files.
>
> This will not help, becouse trigger may use the imported modules
> Dmitry's idea about internal system triggers could help for this case.
>

Well, we have to limit ourselfs somehow ...


-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Hungerburg <pc...@my...>]

Am 2010-11-03 15:45, schrieb Dmitriy Shabanov:
>>
>> The information I missed the most is, if a trigger can mess with control
>> flow, i.e. somehow return a result of its operation, or throw an error
>> condition that will be usefully caught, and if there is a messaging
>> format specified, so that differing frontends (rpc,rest,webdav) can
>> communicate that information to the caller/user. Will such be part of
>> the new interface?
>
>
> Trigger can throw exception on before processing, but can not on after. Can
> you descibe your use case?
>

Use case is in restful applications: eg. validating data coming in by 
http PUT - from an xform eg. or from webdav, in any another way than 
wellformedness and schema conformance. Right now incoming binary data 
cannot be validated at all, and rejected for that, at least not on 
upload time. Wasn't that useful?

-- 
peter

Re: [Exist-development] [WARNING] Triggers interface was changed

[Evgeny G. <gaz...@gm...>]

> with function trigger:beforeCreateDocument & trigger:afterCreateDocument.
> I do think to store this xquery modules under /db/system somethere, because
> of Evgeny's use case: trubles on recovery from backup in result of missing
> files.

I prefer names like: trigger:before-create-document

> I do think to store this xquery modules under /db/system somethere,
> because of Evgeny's use case: trubles on recovery from backup
> in result of missing files.

This will not help, becouse trigger may use the imported modules
Dmitry's idea about internal system triggers could help for this case.

--
Evgeny

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 5:52 PM, Hungerburg <pc...@my...> wrote:

> are these the same triggers that are available in a collection.xconf
> stanza? If so, where can I read instructions on using the new interface?
> I toyed around with triggers a little and documented my findings in the
> xquery-wiki and would like to extend on that.
>

This will be documents, if be short new design do link event with java class
method or xquery function (last do not complited), so on document creation
it will can two methods: beforeCreateDocument & afterCreateDocument. For
xquery script it should be a module at namespace ... (I don't know so far)
with function trigger:beforeCreateDocument & trigger:afterCreateDocument.

I do think to store this xquery modules under /db/system somethere, because
of Evgeny's use case: trubles on recovery from backup in result of missing
files.


>
> The information I missed the most is, if a trigger can mess with control
> flow, i.e. somehow return a result of its operation, or throw an error
> condition that will be usefully caught, and if there is a messaging
> format specified, so that differing frontends (rpc,rest,webdav) can
> communicate that information to the caller/user. Will such be part of
> the new interface?


Trigger can throw exception on before processing, but can not on after. Can
you descibe your use case?

-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Hungerburg <pc...@my...>]

hello Dmitriy,

are these the same triggers that are available in a collection.xconf 
stanza? If so, where can I read instructions on using the new interface? 
I toyed around with triggers a little and documented my findings in the 
xquery-wiki and would like to extend on that.

The information I missed the most is, if a trigger can mess with control 
flow, i.e. somehow return a result of its operation, or throw an error 
condition that will be usefully caught, and if there is a messaging 
format specified, so that differing frontends (rpc,rest,webdav) can 
communicate that information to the caller/user. Will such be part of 
the new interface?

-- 
peter

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 3:35 PM, Pierrick Brihaye
<pie...@fr...>wrote:

> Le 03/11/2010 11:32, Dmitriy Shabanov a écrit :
>
>
>     A good introduction to this stress testing would be the 30 currently
>>    failing tests in the test suite, no ? :-)
>>
>>
>> mostly TriggerTest, working on tests rewrite ...
>>
>
> Can't you rewrite tests *before* committing... and commit them with the
> code ?
>

Yes and no ... as far as I didn't run a way, rest should be fine :-)

-- 
Dmitriy Shabanov

PS I did wait for more weeks while others will fix failing tests & did fix
it by self after all, please be loyally to dogbody :-)

Re: [Exist-development] [WARNING] Triggers interface was changed

[Adam R. <ad...@ex...>]

On 3 November 2010 11:35, Pierrick Brihaye <pie...@fr...> wrote:
> Le 03/11/2010 11:32, Dmitriy Shabanov a écrit :
>
>>     A good introduction to this stress testing would be the 30 currently
>>     failing tests in the test suite, no ? :-)
>>
>>
>> mostly TriggerTest, working on tests rewrite ...
>
> Can't you rewrite tests *before* committing... and commit them with the
> code ?

I concur. Also considering the developers manifesto that we published
as part of the eXist-db documentation to try and increase the
maintainability and quality of new code contributions from everyone -
there are a lack of tests cases provided with this new code.

> Cheers,
>
> p.b.
>
> ------------------------------------------------------------------------------
> Achieve Improved Network Security with IP and DNS Reputation.
> Defend against bad network traffic, including botnets, malware,
> phishing sites, and compromised hosts - saving your company time,
> money, and embarrassment.   Learn More!
> http://p.sf.net/sfu/hpdev2dev-nov
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] [WARNING] Triggers interface was changed

[Pierrick B. <pie...@fr...>]

Le 03/11/2010 11:32, Dmitriy Shabanov a écrit :

>     A good introduction to this stress testing would be the 30 currently
>     failing tests in the test suite, no ? :-)
>
>
> mostly TriggerTest, working on tests rewrite ...

Can't you rewrite tests *before* committing... and commit them with the 
code ?

Cheers,

p.b.

Re: [Exist-development] [WARNING] Triggers interface was changed

[Adam R. <ad...@ex...>]

On 3 November 2010 11:27, Pierrick Brihaye <pie...@fr...> wrote:
> Hi,
>
> Le 03/11/2010 11:22, Dmitriy Shabanov a écrit :
>
>>     We really need some heavy stress testing on the security manager as
>>     well as the collection configuration with large data sets!
>>
>>
>> That what I did ask for :-)
>
> A good introduction to this stress testing would be the 30 currently
> failing tests in the test suite, no ? :-)

I cannot replicate those failures on my local machine, I did try as I
previously saw that they were failing on the build server as well.
Hmmm...

> Cheers,
>
> p.b.
>
> ------------------------------------------------------------------------------
> Achieve Improved Network Security with IP and DNS Reputation.
> Defend against bad network traffic, including botnets, malware,
> phishing sites, and compromised hosts - saving your company time,
> money, and embarrassment.   Learn More!
> http://p.sf.net/sfu/hpdev2dev-nov
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 3:27 PM, Pierrick Brihaye
<pie...@fr...>wrote:

> Hi,
>
> Le 03/11/2010 11:22, Dmitriy Shabanov a écrit :
>
>
>     We really need some heavy stress testing on the security manager as
>>    well as the collection configuration with large data sets!
>>
>>
>> That what I did ask for :-)
>>
>
> A good introduction to this stress testing would be the 30 currently
> failing tests in the test suite, no ? :-)
>

mostly TriggerTest, working on tests rewrite ...

-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Pierrick B. <pie...@fr...>]

Hi,

Le 03/11/2010 11:22, Dmitriy Shabanov a écrit :

>     We really need some heavy stress testing on the security manager as
>     well as the collection configuration with large data sets!
>
>
> That what I did ask for :-)

A good introduction to this stress testing would be the 30 currently 
failing tests in the test suite, no ? :-)

Cheers,

p.b.

Re: [Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

On Wed, Nov 3, 2010 at 2:00 PM, Wolfgang Meier <wol...@ex...>wrote:

> > Are there any anyone who use triggers & ready to help with testing?
> (after I
> > commit configuration changers)
>
> I'm not really happy to see more changes to eXist core classes in
> trunk. We had quite a lot of redesign happening recently and it's time
> to get trunk into a well-tested and stable state again! I still see
> issues with the new security design. Some things are just not working
> as they should, e.g. modifying users via the Java admin client. Can
> everybody please concentrate more on testing and bug fixing and less
> on implementing new stuff?
>

It's time to go back to taskfreak or anything else to share the problems.
BTW, it is not possible to manager realms different from default at java
client, but for internal one is do the job as it was.


> I would thus like to ask everyone to stop committing new stuff and
> instead concentrate on fixing the remaining issues. The goal should be
> to release 1.4.1 and a 1.5 beta until (latest) the end of the year!
>

Again, there is should a list of the problems or tasks.


>
> I'm not sure about the implications this new trigger design has, but
> I'll have a look at the potential risks. The collection configuration
> manager is very critical and we are now using triggers internally
> where we did not use them before.
>

Collection configuration behavior wasn't touched.

>
> We really need some heavy stress testing on the security manager as
> well as the collection configuration with large data sets!
>

That what I did ask for :-)

-- 
Dmitriy Shabanov

Re: [Exist-development] [WARNING] Triggers interface was changed

[Wolfgang M. <wol...@ex...>]

> Are there any anyone who use triggers & ready to help with testing? (after I
> commit configuration changers)

I'm not really happy to see more changes to eXist core classes in
trunk. We had quite a lot of redesign happening recently and it's time
to get trunk into a well-tested and stable state again! I still see
issues with the new security design. Some things are just not working
as they should, e.g. modifying users via the Java admin client. Can
everybody please concentrate more on testing and bug fixing and less
on implementing new stuff?

I would thus like to ask everyone to stop committing new stuff and
instead concentrate on fixing the remaining issues. The goal should be
to release 1.4.1 and a 1.5 beta until (latest) the end of the year!

I'm not sure about the implications this new trigger design has, but
I'll have a look at the potential risks. The collection configuration
manager is very critical and we are now using triggers internally
where we did not use them before.

We really need some heavy stress testing on the security manager as
well as the collection configuration with large data sets!

Wolfgang

Re: [Exist-development] [WARNING] Triggers interface was changed

[Adam R. <ad...@ex...>]

> WARNING: configuration are changing. It will be separated from index one,
> because if configuration defined it overwrite indexes & triggers setting.
> Plan to switch to new configuration framework)"

What is the performance impact of having multiple collection
configuration documents?

Also I DONT want to see anything else switched to the new
configuration framework until the new configuration framework is
finished. IMHO the configuration framework still has many issues and
is only a partially complete implementation of what is intended. We
have a test instantiation of it via the Security Manager which is
fine. BUT I do not want to see anything else switched to the new
configuration framework until the new configuration framework is more
complete...


> Are there any anyone who use triggers & ready to help with testing? (after I
> commit configuration changers)
> --
> Dmitriy Shabanov
>
> ------------------------------------------------------------------------------
> Achieve Improved Network Security with IP and DNS Reputation.
> Defend against bad network traffic, including botnets, malware,
> phishing sites, and compromised hosts - saving your company time,
> money, and embarrassment.   Learn More!
> http://p.sf.net/sfu/hpdev2dev-nov
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>
>



-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [Exist-development] [WARNING] Triggers interface was changed

[Evgeny G. <gaz...@gm...>]

> (WARNING: configuration are changing. It will be separated from index one,
> because if configuration defined it overwrite indexes & triggers setting.
> Plan to switch to new configuration framework)"
>
Where will the default sec configuration be?

> Are there any anyone who use triggers & ready to help with testing? (after I
> commit configuration changers)
>
 I'm ready!

--
Evgeny

[Exist-development] [WARNING] Triggers interface was changed

[Dmitriy S. <sha...@gm...>]

Hello,

I did commit changers to trigger interface
http://exist.svn.sourceforge.net/exist/?rev=13086&view=rev

"triggers interface: event's own method design

The events split base on nature: time, type and object type. Time have
'before' & 'after' methods. Type: 'create', 'update', 'copy', 'move' and
'delete'. Object type: 'document' & 'collection'.

Example: the document creation call two method: beforeCreateDocument &
afterCreateDocument.

(WARNING: configuration are changing. It will be separated from index one,
because if configuration defined it overwrite indexes & triggers setting.
Plan to switch to new configuration framework)"

Are there any anyone who use triggers & ready to help with testing? (after I
commit configuration changers)

-- 
Dmitriy Shabanov

Re: [Exist-development] Software Architecture class team project for enhancements to eXist

[Dmitriy S. <sha...@gm...>]

Hi Loren,

On Wed, Nov 3, 2010 at 2:08 AM, Loren Cahlander
<lor...@gm...>wrote:

> I am working on a team project for my *Introduction to Software
> Architecture* class (
> http://www-users.cselabs.umn.edu/classes/Fall-2010/seng5861/).  I was able
> to convince my teammates to use eXist as the focus of our architectural
> documentation by the fact that their work will become part of the
> documentation of eXist.  They like the idea that the results of the project
> will actually be used by others, so they agreed to work on eXist.  The
> website for the textbook for the class is at
> http://www.viewpoints-and-perspectives.info/.
>
>
> Part of the project is to identify three change cases to enhance the
> architecture that we are analyzing.  Our phase 2 document got the highest
> score of 99/100.  Once the class is completed, then I will be working on
> adding the results into the documentation of eXist giving credit to my
> teammates.
>
> What I would like from the community is to supply a list of requirements
> for these change cases.  We will only be working one of the change cases,
> but I would appreciate what the community has as requirements for each of
> these change cases.
>
> Thank you,
>
> Loren Cahlander
> Team *Chaos Management*
>
>
> 6.  Change Cases describing likely evolutions of the architecture
>
> In our opinion three likely change cases impacting the architecture of
> eXist-db are:
>
>    - transactional support
>    - cluster support
>    - “hot reboot”
>
> 6.1 Transactional support
>
> One of the enhancements that we could see eXist-db receiving in the near
> future, is its ability to offer transactional support. Under the current
> implementation this feature is not available. Although, at the moment,
> eXist-db guarantees that each commit is an atomic operation, current
> implementation does not support a notion of a session. For example, if the
> user was to make changes to the data, those changes could not be “rolled
> back” or grouped with other changes under the same transactional unit.
> Moreover, if more than one user
>

As I do see it, the eXist can group changers into one transaction, look at
batch-transaction pragma. The real problem is "roll back". It's not because
of missing resources, mostly because of missing ideas. The source of this
problems is locking architecture or better to say architecture that have to
locking. There are two levels: collections-documents-(may be)node and b+
tree page. It possible that two documents will share one page. If you do
lock documents, you get concurrent access to page and dead lock if you this
page in the "middle" of documents. There is was one idea to detect the
dead-locks and restart (rollback) one of transaction, but it quite
"expensive" solution, especially for systems under heavy-load. I can
recommend several book on this subject.


> were to work with the same data, changes to the data made by one of the
> user’s could be inadvertently overwritten due to the lack of isolation
> between changes.
>
> Absence of this feature shifts the responsibility onto the shoulders of the
> user to do the “housekeeping” if transactional support is required. Based on
> the interview given by the founder of the project, eXist-db, as is, could
> not be recommended for banking applications. However, he did stress that not
> all of the applications require transactional support. Moreover, he added
> that in his opinion there are situations where XML databases in general are
> not the best choice to begin with, and using traditional RDBMS makes more
> sense.
>
> Despite the fact that not all of the applications require transactional
> support, in our opinion, it is one of those features expected from a
> database. Therefore, we strongly believe that transactional support is one
> of the features that will most likely be integrated into eXist-db, and thus,
> affect its current architecture.
>

I do search for possible solution and will be happy to work on this in a
group, several brains better that one.

6.2 Cluster support
>
> Due to the nature of databases, they are expected to be scalable and
> robust. At present, eXist-db does not scale very well and is prone to
> failures of the underlying hardware.  Due to the existing architectural
> constraints it is capable of running only on a single machine. This makes
> eXist-db very centralized and limits its scalability and reliability
> capabilities to the capabilities of the machine that it is running on.
>
> We think that this is a serious limitation for a database. We believe that
> it can be solved by adding clustering support.  The ability to decentralize
> will offer failover capabilities and increase uptime and reliability. It
> also opens up possibilities to perform some of the maintenance operations
> without impacting users of the database. For example, nodes can be
> “upgraded” one at a time without causing disruption to the users of the
> database.
>
> Switching from the centralized to the distributed mode will have a
> noticeable impact on the current architecture, due to the differences in the
> paradigms.
>

I did start this one:
http://exist.svn.sourceforge.net/viewvc/exist/trunk/eXist/extensions/synchro/

The design do not make "big" changers to core, it force to finish few staff
that on the middle :-) It do plug to eXist through triggers & transaction
manager and use jGroups as transport. The cluster units are collection &
document (I don't think that it will be great to work on btree level, mostly
because difficulties on conflict resolutions for man).

> 6.3 Ability to “hot reboot”
>
> At the moment, eXist-db administrators and eXist-db contributors do not
> have the ability to “restart” the database. After the application is
> stopped, eXist-db has to be “manually” launched from the “command-line”.
> Sometimes, it causes inconvenience and requires unnecessary effort. Although
> the feature sounds to be very simple it might result in changes to the
> architecture of database. Especially when considering the change case we
> have described before, i.e. clustering support. Because of eXist-db’s
> ability to run on a multitude of platforms, both proprietary and
> open-source, it creates a unique challenge in architecting a solution for
> what may be considered a relatively easy problem.
>

I don't see why this can require changers at core. The design quite simple
watchdog daemon base on RESTfull interface (Wolfgang did implement "ping"
(don't loose 'n' :-) feature at trunk), well it possible to use jGroups
after cluster is ready.

-- 
Dmitriy Shabanov

[Exist-development] Software Architecture class team project for enhancements to eXist

[Loren C. <lor...@gm...>]

I am working on a team project for my Introduction to Software Architecture class (http://www-users.cselabs.umn.edu/classes/Fall-2010/seng5861/).  I was able to convince my teammates to use eXist as the focus of our architectural documentation by the fact that their work will become part of the documentation of eXist.  They like the idea that the results of the project will actually be used by others, so they agreed to work on eXist.  The website for the textbook for the class is at http://www.viewpoints-and-perspectives.info/.


Part of the project is to identify three change cases to enhance the architecture that we are analyzing.  Our phase 2 document got the highest score of 99/100.  Once the class is completed, then I will be working on adding the results into the documentation of eXist giving credit to my teammates.  

What I would like from the community is to supply a list of requirements for these change cases.  We will only be working one of the change cases, but I would appreciate what the community has as requirements for each of these change cases.

Thank you,

Loren Cahlander
Team Chaos Management


6.  Change Cases describing likely evolutions of the architecture
In our opinion three likely change cases impacting the architecture of eXist-db are:
transactional support
cluster support
“hot reboot”
6.1 Transactional support

One of the enhancements that we could see eXist-db receiving in the near future, is its ability to offer transactional support. Under the current implementation this feature is not available. Although, at the moment, eXist-db guarantees that each commit is an atomic operation, current implementation does not support a notion of a session. For example, if the user was to make changes to the data, those changes could not be “rolled back” or grouped with other changes under the same transactional unit. Moreover, if more than one user were to work with the same data, changes to the data made by one of the user’s could be inadvertently overwritten due to the lack of isolation between changes.
Absence of this feature shifts the responsibility onto the shoulders of the user to do the “housekeeping” if transactional support is required. Based on the interview given by the founder of the project, eXist-db, as is, could not be recommended for banking applications. However, he did stress that not all of the applications require transactional support. Moreover, he added that in his opinion there are situations where XML databases in general are not the best choice to begin with, and using traditional RDBMS makes more sense.
Despite the fact that not all of the applications require transactional support, in our opinion, it is one of those features expected from a database. Therefore, we strongly believe that transactional support is one of the features that will most likely be integrated into eXist-db, and thus, affect its current architecture.
6.2 Cluster support

Due to the nature of databases, they are expected to be scalable and robust. At present, eXist-db does not scale very well and is prone to failures of the underlying hardware.  Due to the existing architectural constraints it is capable of running only on a single machine. This makes eXist-db very centralized and limits its scalability and reliability capabilities to the capabilities of the machine that it is running on.
We think that this is a serious limitation for a database. We believe that it can be solved by adding clustering support.  The ability to decentralize will offer failover capabilities and increase uptime and reliability. It also opens up possibilities to perform some of the maintenance operations without impacting users of the database. For example, nodes can be “upgraded” one at a time without causing disruption to the users of the database.
Switching from the centralized to the distributed mode will have a noticeable impact on the current architecture, due to the differences in the paradigms.
6.3 Ability to “hot reboot”

At the moment, eXist-db administrators and eXist-db contributors do not have the ability to “restart” the database. After the application is stopped, eXist-db has to be “manually” launched from the “command-line”. Sometimes, it causes inconvenience and requires unnecessary effort. Although the feature sounds to be very simple it might result in changes to the architecture of database. Especially when considering the change case we have described before, i.e. clustering support. Because of eXist-db’s ability to run on a multitude of platforms, both proprietary and open-source, it creates a unique challenge in architecting a solution for what may be considered a relatively easy problem.