exist-development — eXist Developer's List

[Exist-development] Request for feedback about certain documentation articles

[Joe W. <jo...@gm...>]

Hi all,

I would like to raise a few documentation-related items for feedback.

1. I think I might have made an error by importing the following files
into the "doc" app:

- acknowledge.xml
- community.xml
- credits.xml
- facts.xml
- index.xml
- roadmap.xml

These were all listed in the sidebar but were never listed in the
documentation.  For each of these, is there a better place planned for
these than the documentation app, or is the doc app really where they
should stay?

2. Several articles in the doc app are not linked to from the main
documentation landing page, and they appear to be incomplete or
abandoned:

- beginners-guide-to-xrx-v4.xml
- exist-stack.xml
- installing-exist-on-amazon-ec2.xml
- readying-centos-for-exist.xml
- ubuntu-server.xml

I would be interested to know people's opinions on whether these
should remain in the main eXist-db documentation, and if not, what we
should do with them?  They are currently listed in the documentation
spreadsheet (https://docs.google.com/spreadsheet/ccc?key=0AoNLNdpw86LPdDBYNkFXSi1uZG93VThjbzNPTHBaWWc#gid=0),
where I have given my assessment and provisionally flagged them with a
status of "To Delete".

3. One article, xinclude.xml, has several associated files, including
an .xq file.  Such files can't be in the doc app anymore, but rather
need to have their own demo app.  Is anyone an avid enough XInclude
user that they could volunteer to clean up this article and create a
demo app?

- xinclude.xml
- testvars.xq
- disclaimer.xml
- disclaimer-short.xml

Thanks,
Joe

Re: [Exist-development] A proposed addition to the XQDocs

[Loren C. <lor...@gm...>]

I am thinking that in the XQuery code that the entries in the XQDoc documents could be changed to 




@param

The @param tag identifies the parameters associated with a function. For each parameter in a function, there should be a @param tag. The @param tag should be followed by the parameter name (as indicated in the function signature) and then the parameter description.

@param $name The username @schema URI to the schema



@return

The @return tag describes what is returned from a function. Zero or one @return tags can be specified.

@return Sequence of names matching the search criteria @schema URI to the schema




If there is no schema, then there would be no @schema entry.  One question would be that during the XQDoc scan, do we harvest the schema data and insert that into the XQDoc structure assuring that there is actually a schema there or do we just store the URI?



On Oct 6, 2012, at 10:30 PM, Casey Jordan <cas...@jo...> wrote:

> I think this is a great idea. It would even be very useful to be able to design this so inputs (and outputs) could automatically be validated (but I digress)
> 
> For the java part, I suppose a new function signature sequence type (FunctionParameterSequenceType) constructor could be created, which would include a reference to a schema, the nice thing about this is it would work for both parameters and return types.
> 
> I don't know enough about the native XQuery part to really comment.
> 
> On Sat, Oct 6, 2012 at 2:48 PM, Loren Cahlander <lor...@gm...> wrote:
> Hello folks,
> 
> I am thinking that the functions docs could use an enhancement.  I was looking at the function docs for scheduler:get-secheduled-jobs()
> 
> 
> get-scheduled-jobs
> 
> scheduler:get-scheduled-jobs() node()
> Gets the details of all scheduled jobs in the form: <scheduler:jobs xmlns:scheduler="http://exist-db.org/xquery/scheduler" count="iJobs"> <scheduler:group name="group"> <scheduler:job name=""> <scheduler:trigger name=""> <expression></expression> <state></state> <start></start> <end></end> <previous></previous> <next></next> <final></final> </scheduler:trigger> </scheduler:job> </scheduler:group></scheduler:jobs>
> 
> Parameters:
> $a	$a
> Returns:
> the XML containing the list of jobs
> 
> I am thinking that there should be an option to add an XSD or a Relax NG to a parameter or the return value and then display a graphic of the schema of the value.  I will work on this, but would like some feedback from the group as to a proper mechanism to add this to the XQDocs in any XQuery code.  The java implemented function modules should be pretty straight forward to implement after this decision.
> 
> 
> I am thinking that the description should be changed to 'Get the details of all scheduled jobs.' and the returns show something similar to the attached:
> 
> <2012-10-06_13-45-47.png>
> 
> What do you think?
> 
> Cheers,
> Loren
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
> 
> 
> 
> 
> -- 
> --
> Casey Jordan
> easyDITA a product of Jorsek LLC
> "CaseyDJordan" on LinkedIn, Twitter & Facebook
> (585) 348 7399
> easydita.com
> 
> 
> This message is intended only for the use of the Addressee(s) and may
> contain information that is privileged, confidential, and/or exempt from
> disclosure under applicable law.  If you are not the intended recipient,
> please be advised that any disclosure  copying, distribution, or use of
> the information contained herein is prohibited.  If you have received
> this communication in error, please destroy all copies of the message,
> whether in electronic or hard copy format, as well as attachments, and
> immediately contact the sender by replying to this e-mail or by phone.
> Thank you.
> 

Re: [Exist-development] Requesting the addition of exist-development to markmail

[Joe W. <jo...@gm...>]

> Thanks to Jason Hunter's quick action, both exist-development and
> exist-teixml are now on MarkMail already!

That's great news, Ron!  Thanks for getting in touch with Jason Hunter
(and thanks to him too)!

Does anyone have the archive of exist-development in mbox format?

Joe

Re: [Exist-development] Requesting the addition of exist-development to markmail

[ron.vandenbranden <ron...@ka...>]

Hi again,

Thanks to Jason Hunter's quick action, both exist-development and 
exist-teixml are now on MarkMail already! See e.g. 
<http://exist-development.markmail.org/>. I guess exist-teixml will only 
be activated when the first message is mirrored by MarkMail.

That is, from now on new messages will appear there. If we want the 
archives on MarkMail, too, all that's needed is a dump in mbox format 
(up to the first message that was picked up by MarkMail, I guess).

If you send them to me, I'll forward them straight to Jason.

Best,

Ron

On 4/10/2012 9:05, ron...@ka... wrote:
> Hi Joe,
>
> On 4/10/2012 8:56, Joe Wicentowski wrote:
>> I also certainly wouldn't object to your putting in the application,
>> since you were successful in the past - but I'd also be happy to put
>> the request in and give a shot.
>>
>
> Sure, if that can take some weight off your shoulders I wouldn't mind 
> giving it a try. I'll start with the regular web form and see what 
> happens.
>
> Best,
>
> Ron
>
>

[Exist-development] A proposed addition to the XQDocs

[Loren C. <lor...@gm...>]

Hello folks,

I am thinking that the functions docs could use an enhancement.  I was looking at the function docs for scheduler:get-secheduled-jobs()


get-scheduled-jobs

scheduler:get-scheduled-jobs() node()
Gets the details of all scheduled jobs in the form: <scheduler:jobs xmlns:scheduler="http://exist-db.org/xquery/scheduler" count="iJobs"> <scheduler:group name="group"> <scheduler:job name=""> <scheduler:trigger name=""> <expression></expression> <state></state> <start></start> <end></end> <previous></previous> <next></next> <final></final> </scheduler:trigger> </scheduler:job> </scheduler:group></scheduler:jobs>

Parameters:
$a	$a
Returns:
the XML containing the list of jobs

I am thinking that there should be an option to add an XSD or a Relax NG to a parameter or the return value and then display a graphic of the schema of the value.  I will work on this, but would like some feedback from the group as to a proper mechanism to add this to the XQDocs in any XQuery code.  The java implemented function modules should be pretty straight forward to implement after this decision.


I am thinking that the description should be changed to 'Get the details of all scheduled jobs.' and the returns show something similar to the attached:



What do you think?

Cheers,
Loren

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Claudius T. <cla...@gm...>]

@Dmitry
Great, waiting to see!

@Wolfgang
Yes, I know they are archives, actually I generated them with Ant, as test.
But I  wanted to avoid my human errors when generating those descriptors
manually, this is why I asked about a tool. :)

Thanks for hints,
Claudius

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Wolfgang M. <wol...@ex...>]

Claudius, 

.xar files are ordinary zip files and I usually create them from ant. The xar needs to have three descriptors: expath-pkg.xml, repo.xml and exist.xml. Copy the first two from another app (or generate one via eXide). exist.xml lists the jars, e.g. as follows:

<package xmlns="http://exist-db.org/ns/expath-pkg">
<jar>math-0.1.jar</jar>
<java>
<namespace>http://exist-db.org/xquery/math</namespace>
<class>org.exist.xquery.modules.math.MathModule</class>
</java>
</package>


Any jar files should be put into a directory having the same name as the abbreviated name of the project (see abbrev in expath-pkg.xml). eXist-db will pick up those jars without requiring a restart.

Wolfgang

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Claudius T. <cla...@gm...>]

Hi,


Gladly I can create xar. any idea of a Java library doing this?

Claudius

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Dmitriy S. <sha...@gm...>]

On Thu, Oct 4, 2012 at 10:47 PM, Claudius Teodorescu <
cla...@gm...> wrote:

>
> At this very moment, a new version of the EXPath Crypto module is
> available at [1], which is aligned with EXPath Crypto spec (see [2]).
>
> In order to use it, add this in conf.xml:
> <module uri="http://expath.org/ns/crypto"
> class="org.expath.exist.crypto.ExistExpathCryptoModule" />
>

Can you create xar? This days eXist allow to install & active xar package
without restart.


-- 
Dmitriy Shabanov

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Claudius T. <cla...@gm...>]

Hi,

Thanks, the wrapper will be there.


Claudius

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Joe W. <jo...@gm...>]

Hi Claudius,

> At this very moment, a new version of the EXPath Crypto module is available
> at [1], which is aligned with EXPath Crypto spec (see [2]).

Excellent!  Thank you!

> P. S. Any idea where I can host the eXist wrapper of this module, along with
> its XQuery tests?

Assuming you're comfortable with subversion, one way would be to
commit the code directly to eXist's SVN repository.   There is already
an area in the source code for EXPath extensions, the
extensions/expath/src/org/expath directory:

  http://sourceforge.net/p/exist/code/17236/tree/trunk/eXist/extensions/expath/

What's best for you?

Joe

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Claudius T. <cla...@gm...>]

Hi,

At this very moment, a new version of the EXPath Crypto module is available
at [1], which is aligned with EXPath Crypto spec (see [2]).

In order to use it, add this in conf.xml:
<module uri="http://expath.org/ns/crypto"
class="org.expath.exist.crypto.ExistExpathCryptoModule" />

Claudius

P. S. Any idea where I can host the eXist wrapper of this module, along
with its XQuery tests?


[1] http://code.google.com/p/existdb-contrib/downloads/list
[2] http://expath.org/spec/crypto

[Exist-development] prepare-extensions ANT target gone/replaced?

[Andrzej J. T. <an...@ch...>]

Just noticed the prepare-extensions ant target is gone.

Was this replaced with a different target perhaps?

Thanks!

Yeah...I'm still around..just been super busy with work things.

-- 
Andrzej Taramina
Chaeron Corporation: Enterprise System Solutions
http://www.chaeron.com

Re: [Exist-development] Requesting the addition of exist-development to markmail

[<ron...@ka...>]

Hi Joe,

On 4/10/2012 8:56, Joe Wicentowski wrote:
> I also certainly wouldn't object to your putting in the application,
> since you were successful in the past - but I'd also be happy to put
> the request in and give a shot.
>

Sure, if that can take some weight off your shoulders I wouldn't mind 
giving it a try. I'll start with the regular web form and see what happens.

Best,

Ron

Re: [Exist-development] Requesting the addition of exist-development to markmail

[Joe W. <jo...@gm...>]

Hi Ron,

Sure, could you please send this info to me?

I also certainly wouldn't object to your putting in the application,
since you were successful in the past - but I'd also be happy to put
the request in and give a shot.

Thanks either way,
Joe

On Thu, Oct 4, 2012 at 2:46 AM, ron...@ka...
<ron...@ka...> wrote:
> Hi Joe,
>
> Coincidentally, I realized both that I had lost track of exist-development,
> and that its absence on markmail was the most probable cause. And the first
> message I came across was your excellent suggestion to address this issue.
>
>
>> I contacted them for this some time ago, but had no response. Pity since
>> the other two lists were indeed accepted smoothly....
>
>
> If it may help to speed things up, I could provide you with the email
> addresses of some Markmail people who very responsively added TEI-L to
> markmail a while ago.
>
> Best,
>
> Ron

Re: [Exist-development] Requesting the addition of exist-development to markmail

[<ron...@ka...>]

Hi Joe,

Coincidentally, I realized both that I had lost track of 
exist-development, and that its absence on markmail was the most 
probable cause. And the first message I came across was your excellent 
suggestion to address this issue.

> I contacted them for this some time ago, but had no response. Pity since the other two lists were indeed accepted smoothly....

If it may help to speed things up, I could provide you with the email 
addresses of some Markmail people who very responsively added TEI-L to 
markmail a while ago.

Best,

Ron

Re: [Exist-development] Requesting the addition of exist-development to markmail

[Dannes W. <di...@ex...>]

Hi Joe,

I contacted them for this some time ago, but had no response. Pity since the other two lists were indeed accepted smoothly....

You can set me as the admin?

Cheers

Dannes

On 30 sep. 2012, at 20:34, Joe Wicentowski <jo...@gm...> wrote:

> Hi all,
> 
> I'd like to ask the Markmail folks to add exist-development.  They
> already have exist-open and exist-commits, but not exist-development.
> Since important discussions take place here, it would be great to be
> able to search these too.
> 
> The request form (http://exist.markmail.org/docs/feedback.xqy) asks
> for this information:
> 
> "Want us to host a new list? Let us know! It helps if you can give us
> the homepage for the list and/or the contact info to the
> administrator. Also, let us know if you know how we can find the
> historical archives."
> 
> Who would be the best name to send them as the administrator?
> 
> And as to the historical archives, I've been subscribed since the
> list's inception (I think?), but if anyone has done this before, could
> you advise us on how to package up the archives?
> 
> Thanks,
> Joe
> 
> ------------------------------------------------------------------------------
> Got visibility?
> Most devs has no idea what their production app looks like.
> Find out how fast your code is with AppDynamics Lite.
> http://ad.doubleclick.net/clk;262219671;13503038;y?
> http://info.appdynamics.com/FreeJavaPerformanceDownload.html
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] eXist replication & clustering

[Dannes W. <di...@ex...>]

Alexei,

great to read you have interest in the replication extension. This extension is just the start for all kind of ideas we have...

The branch you mention is quite out-of-date, the wiki page as well.
The most recent code is in trunk, please get a copy of that.

Included is a Readme file, and some XML fragments, these should be sufficient to get started.

Last time I checked the extension worked, but I did some refactoring after that, I might have overlooked some things making it fail. Unfortunately I can't have look at it right now for some time.

There are a few rough edges:
- the users must be existing on both instances
- on the consumer side the 'receiver' directory must exist and be writable
- there are memory limitations

There is sufficient logging when set on debug level.....

Thnx for providing your feedback. I consider the extension code stable, but it requires additional testing 'in real life'. Your input is appreciated and will result into better software!

Cheers

Dannes


--
Dannes Wessels

On 2 okt. 2012, at 06:47, Alexei Tarnakin <ale...@gm...> wrote:

> I made a small research on eXist clustering capabilities. According to Wikibooks article I checked out dizzzz's branch and tuned up ActiveMQ server and a couple of eXist servers.
> 
> First of all, I've discovered that article is totally out of date. collection.xconf should look like 
> 
> <collection xmlns="http://exist-db.org/collection-config/1.0">
> <triggers>
> <trigger class="org.exist.replication.jms.publish.ReplicationTrigger">
> <parameter name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
> <parameter name="java.naming.provider.url" value="tcp://localhost:61616"/>
> <parameter name="connectionfactory" value="ConnectionFactory"/>
> <parameter name="destination" value="dynamicTopics/eXistdb"/>
> <parameter name="client-id" value="id1"/>
> </trigger>
> </triggers>
> </collection>
> 
> If  you dig svn log, you will notice that 
> org.exist.clustering.triggers.send.ClusterTrigger
> was renamed to
> org.exist.replication.jms,publish.ReplicationTrigger
> And also notice that client-id looks like mandatory, but not optional field and separated with a dash, not a dot (i.e. client-id). 
> 
> Then lets go check slave eXist server. Article is wrong again. Proper conf.xml should look like
> 
> <job type="startup" name="clustering"  class="org.exist.replication.jms.subscribe.MessageReceiverJob">
>         <parameter name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>         <parameter name="java.naming.provider.url" value="tcp://localhost:61616"/>
>         <parameter name="connectionfactory" value="ConnectionFactory"/>
>         <parameter name="destination" value="dynamicTopics/eXistdb"/>
>         <parameter name="client-id" value="id2"/>
>         <parameter name="subscriber-name" value="sub_name"/>
> </job>
> 
> OK, now looks like we configured everything properly, bot does cluster work now?... Nope! 
> Let's try discover why it does not work by means of xdebug. So simply add a couple of breakpoints to ReplicationTrigger and JMSMessageListener to check what we actually send to ActiveMQ and get on other side (yep, I actually skipped several steps like reconfiguring log4j.xml, restarting server & digging logs, cause it's boring). 
> So the problem is that master eXist sends eXistMessage with document type literally "XML" or "BINARY", while slave expects "DOCUMENT" or "COLLECTION". The problem as always was in indian style of code. Lets analyze it a little bit 
> 
>     @Override
>     public void afterCreateDocument(DBBroker broker, Txn transaction,
>             DocumentImpl document) throws TriggerException {
> 
>         ......
>         // Create Message
>         eXistMessage msg = new eXistMessage();
>         msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>         msg.setResourceOperation(eXistMessage.ResourceOperation.CREATE_UPDATE);
>         msg.setResourcePath(document.getURI().toString());
> 
>         // Retrieve Metadata
>         Map<String, Object> md = msg.getMetadata();
>         MessageHelper.retrieveDocMetadata(md, document.getMetadata());
>         MessageHelper.retrieveFromDocument(md, document);
>         MessageHelper.retrievePermission(md, document.getPermissions());
>         .......
>         sendMessage(msg);
>     }
> 
> so resourcePath in eXistMessage was setted twice 
> msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
> setted to "DOCUMENT" and then one more time
> MessageHelper.retrieveFromDocument(md, document);
> setted to "XML" due to a bit of hardcode in MessageHelper. To fix it we need to patch it
> 
> Index: extensions/replication/src/org/exist/replication/shared/MessageHelper.java 
> =================================================================== 
> --- extensions/replication/src/org/exist/replication/shared/MessageHelper.java    (revision 17019) 
> +++ extensions/replication/src/org/exist/replication/shared/MessageHelper.java    (working copy) 
> @@ -159,7 +159,9 @@ 
>      public static void retrieveFromDocument(Map<String, Object> props, DocumentImpl document){ 
>           
>              props.put(EXIST_RESOURCE_TYPE,  
> -                    (document.getResourceType() == DocumentImpl.XML_FILE ? "XML" : "BINARY"));  
> +                      (document.getResourceType() == DocumentImpl.XML_FILE ?  eXistMessage.ResourceType.DOCUMENT : "BINARY")); 
>              props.put(EXIST_RESOURCE_DOCUMENTID, document.getDocId());  
>              props.put(EXIST_RESOURCE_CONTENTLENGTH, document.getContentLength()); 
> 
> now we got document updates enabled, but no collection operations yet. Let's try to fix it....
>                 ...well, I'm not going to test your patience, so just attached svn diff below and I'll keep on working. And I do hope that 2.1 release will be clustering/replication enabled, cause eXist really needs it due to lack of reliability under pressure.
> 
> 
> P.S. repo moved so proper link for checkout is http://svn.code.sf.net/p/exist/code/branches/dizzzz/clustering/
> 
> P.P.S. Sorry for my English, it's not my native
> 
> 
> <svn.diff>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Joe W. <jo...@gm...>]

p.s. Please do keep us posted about your work on the spec and
implementation.  I think this is a tremendously useful set of
functions for eXist-db developers to have at their disposal.

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Joe W. <jo...@gm...>]

Many thanks, Claudius.  I think this information will be sufficient
for the purposes of the present documentation push for eXist-db 2.0.

Joe

On Tue, Oct 2, 2012 at 1:59 PM, Claudius Teodorescu
<cla...@gm...> wrote:
> Hi,
>
> I was just about to answer this topic.
>
> So, the current module's first version in eXist was developed before the
> spec as of 10 August 2011, as you remember.
>
> Afterwards, was developed and somehow aligned with the spec as to
> functionality and function names, with the exception of encrypt-string(),
> decrypt-string() and module namespace.
>
> As I said some time ago, I am currently working on complete refactoring of
> the module and fully alignement with the spec. The module will be standalone
> and there will be an eXist wrapper for it.
>
> My goal is to make the module conform to next version of spec (see [1]).
>
> If you need more info, I am glad to provide it.
>
>
> Claudius
>
> [1]
> http://extxsltforms.sourceforge.net/specs/expath-crypto/expath-crypto.html

Re: [Exist-development] EXPath Crypto specification & eXist-db implementation

[Claudius T. <cla...@gm...>]

Hi,

I was just about to answer this topic.

So, the current module's first version in eXist was developed before the
spec as of 10 August 2011, as you remember.

Afterwards, was developed and somehow aligned with the spec as to
functionality and function names, with the exception of encrypt-string(),
decrypt-string() and module namespace.

As I said some time ago, I am currently working on complete refactoring of
the module and fully alignement with the spec. The module will be
standalone and there will be an eXist wrapper for it.

My goal is to make the module conform to next version of spec (see [1]).

If you need more info, I am glad to provide it.


Claudius

[1]
http://extxsltforms.sourceforge.net/specs/expath-crypto/expath-crypto.html

[Exist-development] EXPath Crypto specification & eXist-db implementation

[Joe W. <jo...@gm...>]

Hi Claudius,

>> - Crypto / x-crypt module (we can link to the spec at
>> http://expath.org/spec/crypto)
>
> I am not sure, but I dont think this is the implementation of that
> spec, maybe check with Claudius. I think this is actually an early
> alpha version, before the spec was written and the namespaces etc
> changed.

For the eXist-db documentation update I would like to be able to
describe the relationship between the x-crypt module in eXist-db trunk
and the EXPath Crypto spec (10 August 2011 edition, at
http://expath.org/spec/crypto).  I recall that the x-crypt module
predated the spec, and I know the x-crypt module works well (I use it
on history.state.gov).  But could you confirm that there is not an
eXist-db implementation that corresponds to the 10 August 2011 version
of the spec somewhere else?  Also, if you have any other notes about
either the x-crypt module or the EXPath Crypto spec's future
direction, this info would be helpful to include.

Thanks!
Joe

Re: [Exist-development] eXist replication & clustering

[Dannes W. <di...@ex...>]

Please note that all is in trunk right now, with updated docs. The branche must be removed ..

--
Dannes Wessels

On 2 okt. 2012, at 08:04, Dan McCreary <dan...@gm...> wrote:

> Hello Alexei,
> 
> Thanks for noting the differences.  I have updated the Wikibook
> article to include the new collection.xconf and the new conf.xml
> files.
> 
> - Dan
> 
> On Tue, Oct 2, 2012 at 6:47 AM, Alexei Tarnakin
> <ale...@gm...> wrote:
>> I made a small research on eXist clustering capabilities. According to
>> Wikibooks article I checked out dizzzz's branch and tuned up ActiveMQ server
>> and a couple of eXist servers.
>> 
>> First of all, I've discovered that article is totally out of date.
>> collection.xconf should look like
>> 
>>> <collection xmlns="http://exist-db.org/collection-config/1.0">
>>> <triggers>
>>> <trigger class="org.exist.replication.jms.publish.ReplicationTrigger">
>>> <parameter name="java.naming.factory.initial"
>>> value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>>> <parameter name="java.naming.provider.url" value="tcp://localhost:61616"/>
>>> <parameter name="connectionfactory" value="ConnectionFactory"/>
>>> <parameter name="destination" value="dynamicTopics/eXistdb"/>
>>> <parameter name="client-id" value="id1"/>
>>> </trigger>
>>> </triggers>
>>> </collection>
>> 
>> 
>> If  you dig svn log, you will notice that
>> 
>> org.exist.clustering.triggers.send.ClusterTrigger
>> 
>> was renamed to
>> 
>> org.exist.replication.jms,publish.ReplicationTrigger
>> 
>> And also notice that client-id looks like mandatory, but not optional field
>> and separated with a dash, not a dot (i.e. client-id).
>> 
>> Then lets go check slave eXist server. Article is wrong again. Proper
>> conf.xml should look like
>> 
>>> <job type="startup" name="clustering"
>>> class="org.exist.replication.jms.subscribe.MessageReceiverJob">
>>>        <parameter name="java.naming.factory.initial"
>>> value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>>>        <parameter name="java.naming.provider.url"
>>> value="tcp://localhost:61616"/>
>>>        <parameter name="connectionfactory" value="ConnectionFactory"/>
>>>        <parameter name="destination" value="dynamicTopics/eXistdb"/>
>>>        <parameter name="client-id" value="id2"/>
>>>        <parameter name="subscriber-name" value="sub_name"/>
>>> </job>
>> 
>> 
>> OK, now looks like we configured everything properly, bot does cluster work
>> now?... Nope!
>> Let's try discover why it does not work by means of xdebug. So simply add a
>> couple of breakpoints to ReplicationTrigger and JMSMessageListener to check
>> what we actually send to ActiveMQ and get on other side (yep, I actually
>> skipped several steps like reconfiguring log4j.xml, restarting server &
>> digging logs, cause it's boring).
>> So the problem is that master eXist sends eXistMessage with document type
>> literally "XML" or "BINARY", while slave expects "DOCUMENT" or "COLLECTION".
>> The problem as always was in indian style of code. Lets analyze it a little
>> bit
>> 
>>>    @Override
>>>    public void afterCreateDocument(DBBroker broker, Txn transaction,
>>>            DocumentImpl document) throws TriggerException {
>>> 
>>>        ......
>>>        // Create Message
>>>        eXistMessage msg = new eXistMessage();
>>>        msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>>> 
>>> msg.setResourceOperation(eXistMessage.ResourceOperation.CREATE_UPDATE);
>>>        msg.setResourcePath(document.getURI().toString());
>>> 
>>>        // Retrieve Metadata
>>>        Map<String, Object> md = msg.getMetadata();
>>>        MessageHelper.retrieveDocMetadata(md, document.getMetadata());
>>>        MessageHelper.retrieveFromDocument(md, document);
>>>        MessageHelper.retrievePermission(md, document.getPermissions());
>>>        .......
>>>        sendMessage(msg);
>>>    }
>> 
>> 
>> so resourcePath in eXistMessage was setted twice
>>> 
>>> msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>> 
>> setted to "DOCUMENT" and then one more time
>>> 
>>> MessageHelper.retrieveFromDocument(md, document);
>> 
>> setted to "XML" due to a bit of hardcode in MessageHelper. To fix it we need
>> to patch it
>> 
>>> Index:
>>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>>> ===================================================================
>>> ---
>>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>>> (revision 17019)
>>> +++
>>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>>> (working copy)
>>> @@ -159,7 +159,9 @@
>>>     public static void retrieveFromDocument(Map<String, Object> props,
>>> DocumentImpl document){
>>> 
>>>             props.put(EXIST_RESOURCE_TYPE,
>>> -                    (document.getResourceType() == DocumentImpl.XML_FILE
>>> ? "XML" : "BINARY"));
>>> +                      (document.getResourceType() ==
>>> DocumentImpl.XML_FILE ?  eXistMessage.ResourceType.DOCUMENT : "BINARY"));
>>>             props.put(EXIST_RESOURCE_DOCUMENTID, document.getDocId());
>>>             props.put(EXIST_RESOURCE_CONTENTLENGTH,
>>> document.getContentLength());
>> 
>> 
>> now we got document updates enabled, but no collection operations yet. Let's
>> try to fix it....
>>                ...well, I'm not going to test your patience, so just
>> attached svn diff below and I'll keep on working. And I do hope that 2.1
>> release will be clustering/replication enabled, cause eXist really needs it
>> due to lack of reliability under pressure.
>> 
>> 
>> P.S. repo moved so proper link for checkout is
>> http://svn.code.sf.net/p/exist/code/branches/dizzzz/clustering/
>> 
>> P.P.S. Sorry for my English, it's not my native
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Don't let slow site performance ruin your business. Deploy New Relic APM
>> Deploy New Relic app performance management and know exactly
>> what is happening inside your Ruby, Python, PHP, Java, and .NET app
>> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
>> http://p.sf.net/sfu/newrelic-dev2dev
>> _______________________________________________
>> Exist-development mailing list
>> Exi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-development
>> 
> 
> 
> 
> -- 
> Dan McCreary
> NoSQL Evangelist
> office: (952) 931-9198
> cell: (612) 986-1552
> 
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development

Re: [Exist-development] eXist replication & clustering

[Dan M. <dan...@gm...>]

Hello Alexei,

Thanks for noting the differences.  I have updated the Wikibook
article to include the new collection.xconf and the new conf.xml
files.

- Dan

On Tue, Oct 2, 2012 at 6:47 AM, Alexei Tarnakin
<ale...@gm...> wrote:
> I made a small research on eXist clustering capabilities. According to
> Wikibooks article I checked out dizzzz's branch and tuned up ActiveMQ server
> and a couple of eXist servers.
>
> First of all, I've discovered that article is totally out of date.
> collection.xconf should look like
>
>> <collection xmlns="http://exist-db.org/collection-config/1.0">
>> <triggers>
>> <trigger class="org.exist.replication.jms.publish.ReplicationTrigger">
>> <parameter name="java.naming.factory.initial"
>> value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>> <parameter name="java.naming.provider.url" value="tcp://localhost:61616"/>
>> <parameter name="connectionfactory" value="ConnectionFactory"/>
>> <parameter name="destination" value="dynamicTopics/eXistdb"/>
>> <parameter name="client-id" value="id1"/>
>> </trigger>
>> </triggers>
>> </collection>
>
>
> If  you dig svn log, you will notice that
>
> org.exist.clustering.triggers.send.ClusterTrigger
>
> was renamed to
>
> org.exist.replication.jms,publish.ReplicationTrigger
>
> And also notice that client-id looks like mandatory, but not optional field
> and separated with a dash, not a dot (i.e. client-id).
>
> Then lets go check slave eXist server. Article is wrong again. Proper
> conf.xml should look like
>
>> <job type="startup" name="clustering"
>> class="org.exist.replication.jms.subscribe.MessageReceiverJob">
>>         <parameter name="java.naming.factory.initial"
>> value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>>         <parameter name="java.naming.provider.url"
>> value="tcp://localhost:61616"/>
>>         <parameter name="connectionfactory" value="ConnectionFactory"/>
>>         <parameter name="destination" value="dynamicTopics/eXistdb"/>
>>         <parameter name="client-id" value="id2"/>
>>         <parameter name="subscriber-name" value="sub_name"/>
>> </job>
>
>
> OK, now looks like we configured everything properly, bot does cluster work
> now?... Nope!
> Let's try discover why it does not work by means of xdebug. So simply add a
> couple of breakpoints to ReplicationTrigger and JMSMessageListener to check
> what we actually send to ActiveMQ and get on other side (yep, I actually
> skipped several steps like reconfiguring log4j.xml, restarting server &
> digging logs, cause it's boring).
> So the problem is that master eXist sends eXistMessage with document type
> literally "XML" or "BINARY", while slave expects "DOCUMENT" or "COLLECTION".
> The problem as always was in indian style of code. Lets analyze it a little
> bit
>
>>     @Override
>>     public void afterCreateDocument(DBBroker broker, Txn transaction,
>>             DocumentImpl document) throws TriggerException {
>>
>>         ......
>>         // Create Message
>>         eXistMessage msg = new eXistMessage();
>>         msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>>
>> msg.setResourceOperation(eXistMessage.ResourceOperation.CREATE_UPDATE);
>>         msg.setResourcePath(document.getURI().toString());
>>
>>         // Retrieve Metadata
>>         Map<String, Object> md = msg.getMetadata();
>>         MessageHelper.retrieveDocMetadata(md, document.getMetadata());
>>         MessageHelper.retrieveFromDocument(md, document);
>>         MessageHelper.retrievePermission(md, document.getPermissions());
>>         .......
>>         sendMessage(msg);
>>     }
>
>
> so resourcePath in eXistMessage was setted twice
>>
>> msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>
> setted to "DOCUMENT" and then one more time
>>
>> MessageHelper.retrieveFromDocument(md, document);
>
> setted to "XML" due to a bit of hardcode in MessageHelper. To fix it we need
> to patch it
>
>> Index:
>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>> ===================================================================
>> ---
>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>> (revision 17019)
>> +++
>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>> (working copy)
>> @@ -159,7 +159,9 @@
>>      public static void retrieveFromDocument(Map<String, Object> props,
>> DocumentImpl document){
>>
>>              props.put(EXIST_RESOURCE_TYPE,
>> -                    (document.getResourceType() == DocumentImpl.XML_FILE
>> ? "XML" : "BINARY"));
>> +                      (document.getResourceType() ==
>> DocumentImpl.XML_FILE ?  eXistMessage.ResourceType.DOCUMENT : "BINARY"));
>>              props.put(EXIST_RESOURCE_DOCUMENTID, document.getDocId());
>>              props.put(EXIST_RESOURCE_CONTENTLENGTH,
>> document.getContentLength());
>
>
> now we got document updates enabled, but no collection operations yet. Let's
> try to fix it....
>                 ...well, I'm not going to test your patience, so just
> attached svn diff below and I'll keep on working. And I do hope that 2.1
> release will be clustering/replication enabled, cause eXist really needs it
> due to lack of reliability under pressure.
>
>
> P.S. repo moved so proper link for checkout is
> http://svn.code.sf.net/p/exist/code/branches/dizzzz/clustering/
>
> P.P.S. Sorry for my English, it's not my native
>
>
>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Dan McCreary
NoSQL Evangelist
office: (952) 931-9198
cell: (612) 986-1552

[Exist-development] Requesting the addition of exist-development to markmail

[Joe W. <jo...@gm...>]

Hi all,

I'd like to ask the Markmail folks to add exist-development.  They
already have exist-open and exist-commits, but not exist-development.
Since important discussions take place here, it would be great to be
able to search these too.

The request form (http://exist.markmail.org/docs/feedback.xqy) asks
for this information:

"Want us to host a new list? Let us know! It helps if you can give us
the homepage for the list and/or the contact info to the
administrator. Also, let us know if you know how we can find the
historical archives."

Who would be the best name to send them as the administrator?

And as to the historical archives, I've been subscribed since the
list's inception (I think?), but if anyone has done this before, could
you advise us on how to package up the archives?

Thanks,
Joe