Menu

JGroups tcp config - use/not-use pbcast.FLUSH ? [tcp.xml vs flush-tcp.xml]

toddler
2014-11-25
2014-11-25
  • toddler

    toddler - 2014-11-25

    We're using JBoss EAP 6.3.1 which uses jgroups-3.2.13-final configuring
    the TCP protocol. Our application extends RpcDispatcher and uses lots
    of RPC calls/etc. is small clusters.

    We're using TCP, but I'm not sure if we should be using pbcast.FLUSH or not -
    porting the application from 2.6.20.Final that previously used:

    <pbcast.FLUSH
            start_flush_timeout="10000"
            timeout="0" />
    

    I likey I need an answer from Bela Ban, but he previous said he doesn't support 3.2.x
    [crossing my fingers :-)]. We have JBoss EAP support, but they've been refusing to
    answer jgroups questions unless its related to HA clustering/etc...

    In the jgroups-3.2.13-final there is both a tcp.xml and a flush-tcp.xml. It doesn't
    specify when one should be used over the other. tcp.xml does not contain pbcast.FLUSH.

    From here
    http://sourceforge.net/p/javagroups/discussion/18794/thread/21406446/#574e
    Bela Ban posted "Yes, use tcp.xml from the JAR as a starting point and modify it."

    http://sourceforge.net/p/javagroups/discussion/18794/thread/21406446/#1aa1
    Bela Ban responded "BTW: why do you need FLUSH ?"

    Don't have an answer for this, its a legacy application I wasn't previously involved with.

    From here
    http://www.jgroups.org/manual/index.html#_pbcast_flush
    It states Flush is used in "State Transfer" and "View Changes" and both are being
    used.

    Other things I've found supporting the use of flush include...
    http://www.jgroups.org/manual/index.html#Flushing
    https://github.com/belaban/JGroups/blob/master/conf/flush-tcp.xml

    Also, tcp.xml contains [timer_type="old"], yet TP.java has a comment this is only used up to 2.10:
    @Property(description="Type of timer to be used. Valid values are \"old\" (DefaultTimeScheduler, used up to 2.10), " +
    "\"new\" (TimeScheduler2) and \"wheel\". Note that this property might disappear " +
    "in future releases, if one of the 3 timers is chosen as default timer")
    protected String timer_type="old";

    I suspect it would be better to use [timer_type="new"]?

    I did check 3.5, TP.java has for the default:
    protected String timer_type="new3";

    Todd

     
    • Bela Ban

      Bela Ban - 2014-11-25

      Which subsystem of EAP do you use ? If it's Infinispan (distributed caches, EJBs), IIRC they required FLUSH to be present in older versions. Otherwise, drop FLUSH unless you really need virtual synchrony.

      Re timer_type: I'd leave the default (old) for now.

       
      • toddler

        toddler - 2014-11-25

        Thanks for the quick response Bela, much appreciated.

        I have another configuration question related to use/not-use <UNICAST/> or <UNICAST2/>.
        If we do use one in jgroups-3.2.13-final, which one should we use?

        <UNICAST/> is in tcp.xml.

        We did not use "<UNICAST/>" previous. I'm planning to leave it out, as I don't want to
        prohibit concurrent delivery, but I'm not exactly sure what "concurrent delivery" means
        from a JGroups perspective - what does "concurrent delivery" mean? The underlying socket
        interface will prevent concurrent delivery on the same socket descriptor...

        Our application mainly uses async RPCs (extends RpcDispatcher), but some synchronous ones
        as well. We use a single JChannel connection, these RPCs may or may-not be done in parallel
        between the same or different cluster members (both ways).

        From here:
        http://www.jgroups.org/manual/index.html#UNICAST

        "On top of a reliable transport, such as TCP, UNICAST is not really needed. However,
        concurrent delivery of messages from the same sender is prevented by UNICAST by acquiring
        a lock on the sender’s retransmission table, so unless concurrent delivery is desired,
        UNICAST should not be removed from the stack even if TCP is used."

        Cheers

         
        • Bela Ban

          Bela Ban - 2014-11-25

          Re UNICAST or UNICAST2: the latter is faster, but has the last-msg-missing issue, the former generates more traffic (1 ack per message) but doesn't have that issue.
          I suggest neither, but upgrade to 3.6 and use UNICAST3. :-)

          No, you cannot remove UNICAST{2} as it performs ordering of messages on the receiver. While TCP delivers messages in order, the thread pools at the receiver can easily screw up that order.

          If you mark your messages as OOB then even messages from the same sender will be delivered in parallel, otherwise msgs from the same sender are delivered in send-order (FIFO).

           

Log in to post a comment.