"Connection reset by peer: socket write error" exception, calling DataOutputStream.flush

[g muhl]

I am new to modbus (getting ready to interface with a Labjack UE9).

I am trying the tutorial exercises at the jamod site:
http://jamod.sourceforge.net/kb/tcp_master_howto.html
http://jamod.sourceforge.net/kb/tcp_slave_howto.html

When I increase the number of "repeats" above 16, I start to get "connection reset by peer" exceptions when calling ModbusTCPTransport.writeMessage. It appears that writeMessage will retry 3 times and then close the connection. On the next iteration of my code's "repeat" loop, the socket will reconnect, and I'll get another 16 or 17 writes before the exception occurs again.

It seems that the connection is being reset by the peer at regular intervals. Is that interval adjustable? Is there a way to prevent the connection being reset by keeping the connection "busy?" Or is it common practice for the programmer to close the connection and re-connect between writes?

Also it seems pointless for writeMessage to retry three times on a "connection reset" exception without closing and re-opening the connection between retries.

My code for the master-side follows.

Thanks

import java.net.*;

import com.ghgande.j2mod.modbus.Modbus;
import com.ghgande.j2mod.modbus.io.ModbusTCPTransaction;
import com.ghgande.j2mod.modbus.msg.ReadInputDiscretesRequest;
import com.ghgande.j2mod.modbus.msg.ReadInputDiscretesResponse;
import com.ghgande.j2mod.modbus.msg.ReadMultipleRegistersRequest;
import com.ghgande.j2mod.modbus.msg.ReadMultipleRegistersResponse;
import com.ghgande.j2mod.modbus.net.TCPMasterConnection;

/**
 * <p>
 * <code>DITest</code> is used to
 * </p>
 *
 * @keyword //TODO
 */
public class DITest {

  public static void main(String[] args) {
    /* The important instances of the classes mentioned before */
    TCPMasterConnection con = null; //the connection
    ModbusTCPTransaction trans = null; //the transaction
    ModbusTCPTransaction trans2 = null; //the transaction (for a register request)
    ReadInputDiscretesRequest req = null; //the request
    ReadInputDiscretesResponse res = null; //the response
    ReadMultipleRegistersRequest regReq = null;// a register request
    ReadMultipleRegistersResponse regRes = null;// a register response

    /* Variables for storing the parameters */
    InetAddress addr = null; //the slave's address
    int port = Modbus.DEFAULT_PORT;
    int ref = 0; //the reference; offset where to start reading from
    int count = 0; //the number of DI's to read
    int repeat = 1; //a loop for repeating the transaction

    try {
      //1. Setup the parameters
      if (args.length < 3) {
        System.exit(1);
      } else {
        try {
          String astr = args[0];
          int idx = astr.indexOf(':');
          if(idx > 0) {
            port = Integer.parseInt(astr.substring(idx+1));
            astr = astr.substring(0,idx);
          }
          addr = InetAddress.getByName(astr);
          ref = Integer.decode(args[1]).intValue();
          count = Integer.decode(args[2]).intValue();
          if (args.length == 4) {
            repeat = Integer.parseInt(args[3]);
          }
        } catch (Exception ex) {
          ex.printStackTrace();
          System.exit(1);
        }
      }
      //2. Open the connection
      con = new TCPMasterConnection(addr);
      con.setPort(port);
      con.connect();

      //3. Prepare the request
      req = new ReadInputDiscretesRequest(ref, count);
      regReq = new ReadMultipleRegistersRequest(ref, 1);

      //4. Prepare the transaction
      trans = new ModbusTCPTransaction(con);
      trans.setRequest(req);

      //5. Execute the transaction repeat times
      int k = 0;
      do {
        System.out.println("Discrete Count: " + k);
        trans.execute();
        res = (ReadInputDiscretesResponse) trans.getResponse();
        System.out.println("Digital Inputs Status=" + res.getDiscretes().toString());
        System.out.println("Done with pass #: " + k);
        System.out.println("");
        k++;
      } while (k < repeat);

      //4a. Prepare the register-request transaction
      trans2 = new ModbusTCPTransaction(con);
      trans2.setRequest(regReq);

      //5a. Execute the register-request transaction repeat times
      k = 0;
      do {
        System.out.println("Register Count: " + k);
        trans2.execute();
        regRes = (ReadMultipleRegistersResponse) trans2.getResponse();
        System.out.println("Register: " +regRes.getRegisterValue(0));
        System.out.println("Done with pass #: " + k);
        System.out.println("");
        k++;
      } while (k < repeat);

      //6. Close the connection
      con.close();

    } catch (Exception ex) {
      ex.printStackTrace();
    }
  }//main

}

[g muhl]

I called TCPMasterConnection.setTimeout with a longer value than the default, but that did not change the interval between dropped connections.

Also, the following code in ModbusTCPTransaction.execute has a potential shortcoming. We test the connection to see if it is connected, and then we wait for m_IO. During that short time the connection is getting dropped.

~~~~~~~~~~~~~~
public void execute() throws ModbusIOException, ModbusSlaveException,
ModbusException {

    if (m_Request == null || m_Connection == null)
        throw new ModbusException("Invalid request or connection");

    /*
     * Automatically re-connect if disconnected.
     */
    if (!m_Connection.isConnected()) {
        try {
          System.out.println("Give me back the rug!");
            m_Connection.connect();
        } catch (Exception ex) {
            throw new ModbusIOException("Connection failed.");
        }
    }

    /*
     * Try sending the message up to m_Retries time. Note that the message
     * is read immediately after being written, with no flushing of buffers.
     */
    int retryCounter = 0;
System.out.println("Testing retry counter, m_Retries:  " + m_Retries);
    while (retryCounter < (m_Retries > 0 ? m_Retries:1)) {
        try {
            synchronized (m_IO) {
            System.out.println("Synchronized");
                  if (!m_Connection.isConnected()){
                    System.out.println("Someone pulled the rug out.");
                  }
                m_IO.writeMessage(m_Request);

~~~~~~~~~~~~~

[g muhl]

I am seeing an exception on the slave side corresponding with the resets. Here is the trace:

java.net.SocketException: Software caused connection abort: recv failed
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
    at java.io.DataInputStream.read(DataInputStream.java:132)
    at com.ghgande.j2mod.modbus.io.ModbusTCPTransport.readRequest(ModbusTCPTransport.java:193)
    at com.ghgande.j2mod.modbus.net.TCPConnectionHandler.run(TCPConnectionHandler.java:85)
    at com.ghgande.j2mod.modbus.util.ThreadPool$PoolThread.run(ThreadPool.java:103)

I used RawCap to capture the traffic on the local loopback. Here is the RST frame, along with the frame immediately preceding it (I am using port 6675):

~~~~~~~~~~~~~~~~~~~
No. Time Source Destination Protocol Length Info
407 4.980496 127.0.0.1 127.0.0.1 TCP 41 57679 > 6675 [PSH, ACK, URG] Seq=125 Ack=91 Win=7936 Urg=1 Len=1

Frame 407: 41 bytes on wire (328 bits), 41 bytes captured (328 bits)
Raw packet data
No link information available
Internet Protocol Version 4, Src: 127.0.0.1 (127.0.0.1), Dst: 127.0.0.1 (127.0.0.1)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
Total Length: 41
Identification: 0x2cb3 (11443)
Flags: 0x02 (Don't Fragment)
Fragment offset: 0
Time to live: 128
Protocol: TCP (6)
Header checksum: 0x0000 [incorrect, should be 0xd019 (may be caused by "IP checksum offload"?)]
Source: 127.0.0.1 (127.0.0.1)
Destination: 127.0.0.1 (127.0.0.1)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 57679 (57679), Dst Port: 6675 (6675), Seq: 125, Ack: 91, Len: 1
Source port: 57679 (57679)
Destination port: 6675 (6675)
[Stream index: 4]
Sequence number: 125 (relative sequence number)
[Next sequence number: 126 (relative sequence number)]
Acknowledgment number: 91 (relative ack number)
Header length: 20 bytes
Flags: 0x038 (PSH, ACK, URG)
Window size value: 31
[Calculated window size: 7936]
[Window size scaling factor: 256]
Checksum: 0x8652 [validation disabled]
Urgent pointer: 1
[SEQ/ACK analysis]
[Bytes in flight: 1]
Data (1 byte)

0000 00 .

No. Time Source Destination Protocol Length Info
408 4.980496 127.0.0.1 127.0.0.1 TCP 40 6675 > 57679 [RST, ACK] Seq=91 Ack=126 Win=0 Len=0

Frame 408: 40 bytes on wire (320 bits), 40 bytes captured (320 bits)
Raw packet data
No link information available
Internet Protocol Version 4, Src: 127.0.0.1 (127.0.0.1), Dst: 127.0.0.1 (127.0.0.1)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
Total Length: 40
Identification: 0x2cb4 (11444)
Flags: 0x02 (Don't Fragment)
Fragment offset: 0
Time to live: 128
Protocol: TCP (6)
Header checksum: 0x0000 [incorrect, should be 0xd019 (may be caused by "IP checksum offload"?)]
Source: 127.0.0.1 (127.0.0.1)
Destination: 127.0.0.1 (127.0.0.1)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Transmission Control Protocol, Src Port: 6675 (6675), Dst Port: 57679 (57679), Seq: 91, Ack: 126, Len: 0
Source port: 6675 (6675)
Destination port: 57679 (57679)
[Stream index: 4]
Sequence number: 91 (relative sequence number)
Acknowledgment number: 126 (relative ack number)
Header length: 20 bytes
Flags: 0x014 (RST, ACK)
Window size value: 0
[Calculated window size: 0]
[Window size scaling factor: 256]
Checksum: 0x8696 [validation disabled]
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 407]
~~~~~~~~~~~~~~~~~

It seems curious that the RST frame always occurs on Ack count 126.

[g muhl]

It was a chicken/egg thing for awhile, but I think I've found the originator of the resets. It's happening on the master side when calling isConnected. An exception occurs when calling sendUrgentData. As I indicated above, this only happens on about the sixteenth transmission when the connection has been up for awhile. Maybe things are getting plugged up by then. Is there something we should check before calling sendUrgentData?

    /**
     * Tests if this <tt>TCPMasterConnection</tt> is connected.
     * 
     * @return <tt>true</tt> if connected, <tt>false</tt> otherwise.
     */
    public synchronized boolean isConnected() {
        if (m_Connected && m_Socket != null) {
            if (!m_Socket.isConnected() || m_Socket.isClosed()
                    || m_Socket.isInputShutdown()
                    || m_Socket.isOutputShutdown()) {
                try {
                System.out.println("close33");
                    m_Socket.close();
                } catch (IOException e) {
                    // Blah.
                }
                m_Connected = false;
            } else {
                try {
                    m_Socket.sendUrgentData(0);
                } catch (IOException e) {
                    m_Connected = false;
                    try {
                    System.out.println("close43");
                        m_Socket.close();
                    } catch (IOException e1) {
                        // Do nothing.
                    }
                }
            }
        }
        return m_Connected;
    }// isConnected

[g muhl]

According to the following two forum threads, calling sendUrgentData buys us nothing, and tends to confuse the slave side:

http://stackoverflow.com/questions/7831452/tcp-connection-is-reset-after-sending-urgent-data

https://forums.oracle.com/forums/thread.jspa?threadID=1144387

So I commented that part out, and my test works fine.

[g muhl]

[David Turner]

I too have found issue with using sendUrgentData - I was evaluating the j2Mod library and set up using an ADAM-6066 module - apparently sending a single byte (as this function does as out of band data) causes this particular device to reply with the last sent response - obviously causing issue (or hopefully noticed at least) when expecting a different response.

[Julie Haugh]

I'm not ignoring y'all -- I'm trying to find a way to solve the problem that isConnected() is supposed to be solving without using urgent data, which has worked very well with my products.

The issue is that the failed write() when the connection has been lost isn't detected particularly well (as noted above ...). This causes a message which could have been sent successfully, if the connection had just been reestablished, to fail.

The entire Modbus/TCP transport code is in need a major re-work because there are too many layers between the socket and the user.

[Dave Buchwald]

This is a new question on what seems to be an old topic, but has this issue been fixed? When I run simple j2mod examples in Windows, I still get socket write errors with the identical symptoms detailed above. And when I run them in Linux, I get connection refused.

• Windows 10, ARM linux on Toradex iMX7 system on module
• no firewalls are running
• V2.3.4 of j2mod (released June 2017)
• running with loopback address

Is there a difference between V2.3.4 for Maven and the jar on this site (1.06)? I saw that someone built a 'j4mod' somewhere that talks about addressing this issue. How can I run simple Modbus examples?

thanks in advance,
Dave