Re: [Exist-development] eXist replication & clustering

[Dan M. <dan...@gm...>]

Hello Alexei,

Thanks for noting the differences.  I have updated the Wikibook
article to include the new collection.xconf and the new conf.xml
files.

- Dan

On Tue, Oct 2, 2012 at 6:47 AM, Alexei Tarnakin
<ale...@gm...> wrote:
> I made a small research on eXist clustering capabilities. According to
> Wikibooks article I checked out dizzzz's branch and tuned up ActiveMQ server
> and a couple of eXist servers.
>
> First of all, I've discovered that article is totally out of date.
> collection.xconf should look like
>
>> <collection xmlns="http://exist-db.org/collection-config/1.0">
>> <triggers>
>> <trigger class="org.exist.replication.jms.publish.ReplicationTrigger">
>> <parameter name="java.naming.factory.initial"
>> value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>> <parameter name="java.naming.provider.url" value="tcp://localhost:61616"/>
>> <parameter name="connectionfactory" value="ConnectionFactory"/>
>> <parameter name="destination" value="dynamicTopics/eXistdb"/>
>> <parameter name="client-id" value="id1"/>
>> </trigger>
>> </triggers>
>> </collection>
>
>
> If  you dig svn log, you will notice that
>
> org.exist.clustering.triggers.send.ClusterTrigger
>
> was renamed to
>
> org.exist.replication.jms,publish.ReplicationTrigger
>
> And also notice that client-id looks like mandatory, but not optional field
> and separated with a dash, not a dot (i.e. client-id).
>
> Then lets go check slave eXist server. Article is wrong again. Proper
> conf.xml should look like
>
>> <job type="startup" name="clustering"
>> class="org.exist.replication.jms.subscribe.MessageReceiverJob">
>>         <parameter name="java.naming.factory.initial"
>> value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
>>         <parameter name="java.naming.provider.url"
>> value="tcp://localhost:61616"/>
>>         <parameter name="connectionfactory" value="ConnectionFactory"/>
>>         <parameter name="destination" value="dynamicTopics/eXistdb"/>
>>         <parameter name="client-id" value="id2"/>
>>         <parameter name="subscriber-name" value="sub_name"/>
>> </job>
>
>
> OK, now looks like we configured everything properly, bot does cluster work
> now?... Nope!
> Let's try discover why it does not work by means of xdebug. So simply add a
> couple of breakpoints to ReplicationTrigger and JMSMessageListener to check
> what we actually send to ActiveMQ and get on other side (yep, I actually
> skipped several steps like reconfiguring log4j.xml, restarting server &
> digging logs, cause it's boring).
> So the problem is that master eXist sends eXistMessage with document type
> literally "XML" or "BINARY", while slave expects "DOCUMENT" or "COLLECTION".
> The problem as always was in indian style of code. Lets analyze it a little
> bit
>
>>     @Override
>>     public void afterCreateDocument(DBBroker broker, Txn transaction,
>>             DocumentImpl document) throws TriggerException {
>>
>>         ......
>>         // Create Message
>>         eXistMessage msg = new eXistMessage();
>>         msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>>
>> msg.setResourceOperation(eXistMessage.ResourceOperation.CREATE_UPDATE);
>>         msg.setResourcePath(document.getURI().toString());
>>
>>         // Retrieve Metadata
>>         Map<String, Object> md = msg.getMetadata();
>>         MessageHelper.retrieveDocMetadata(md, document.getMetadata());
>>         MessageHelper.retrieveFromDocument(md, document);
>>         MessageHelper.retrievePermission(md, document.getPermissions());
>>         .......
>>         sendMessage(msg);
>>     }
>
>
> so resourcePath in eXistMessage was setted twice
>>
>> msg.setResourceType(eXistMessage.ResourceType.DOCUMENT);
>
> setted to "DOCUMENT" and then one more time
>>
>> MessageHelper.retrieveFromDocument(md, document);
>
> setted to "XML" due to a bit of hardcode in MessageHelper. To fix it we need
> to patch it
>
>> Index:
>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>> ===================================================================
>> ---
>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>> (revision 17019)
>> +++
>> extensions/replication/src/org/exist/replication/shared/MessageHelper.java
>> (working copy)
>> @@ -159,7 +159,9 @@
>>      public static void retrieveFromDocument(Map<String, Object> props,
>> DocumentImpl document){
>>
>>              props.put(EXIST_RESOURCE_TYPE,
>> -                    (document.getResourceType() == DocumentImpl.XML_FILE
>> ? "XML" : "BINARY"));
>> +                      (document.getResourceType() ==
>> DocumentImpl.XML_FILE ?  eXistMessage.ResourceType.DOCUMENT : "BINARY"));
>>              props.put(EXIST_RESOURCE_DOCUMENTID, document.getDocId());
>>              props.put(EXIST_RESOURCE_CONTENTLENGTH,
>> document.getContentLength());
>
>
> now we got document updates enabled, but no collection operations yet. Let's
> try to fix it....
>                 ...well, I'm not going to test your patience, so just
> attached svn diff below and I'll keep on working. And I do hope that 2.1
> release will be clustering/replication enabled, cause eXist really needs it
> due to lack of reliability under pressure.
>
>
> P.S. repo moved so proper link for checkout is
> http://svn.code.sf.net/p/exist/code/branches/dizzzz/clustering/
>
> P.P.S. Sorry for my English, it's not my native
>
>
>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Exist-development mailing list
> Exi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-development
>



-- 
Dan McCreary
NoSQL Evangelist
office: (952) 931-9198
cell: (612) 986-1552