#24 Possible socket closing issue

[Chris Mair]

(Submitted by Andrew Norman)

I observed a problem with the machine I was running a mock ftp site on running out of sockets. It appears that if you shut the connection without sending a quit command the mock doesn't shutdown the socket leaving it in the CLOSE_WAIT state.

I tested the following code (using Apache FTP Client) against a real ftp site (vsftpd) and the mock ftp site (MockFtpServer) and I could run the machine out of connections if I removed the client.quit() line with the mock, the real ftp site shut the connection down as expected.

public static void main( String[] args ) throws SocketException, IOException {
    while ( true ) {
        final FTPClient client = new FTPClient();
        client.connect( "localhost", 21 );
        client.login( "USER", "PASSWORD" );
        client.changeWorkingDirectory( "/tmp/" );
        client.quit();
        client.disconnect();
    }
}

[Chris Mair]

(Submitted by Andrew Norman)

I have noticed and figured out the fix for two problems I have seen. The test code is below, I have added the enterLocalPassiveMode() to better reflect what our production code is running.

While testing a fix for the socket closing problem, my machine ran out of sockets and I noticed that there were a lot of sockets left in the LISTEN state. I fixed the problem by closing the passiveModeDataSocket in the DefaultSession class when the run() method finishes.

I left the test I talked about last bug running overnight by accident and came in the morning to find my FakeFtpServer had thrown an out of memory error. I spent some time profiling the system and found that the sessions map in the AbstractFtpServer was not removing the session when it finishes, so I added in the removal of the session at the end of the session run() method.

I made the following changes and have no successfully had the test code below thrashing the FakeFtpSite @30-50 connect/disconnect cycles a second.

public static void main( String[] args ) throws SocketException, IOException, InterruptedException {
  while ( true ) {
    final FTPClient client = new FTPClient();
    client.connect( "localhost", 10000 );
    client.login( "USER", "PASS" );
    client.enterLocalPassiveMode();
    client.changeWorkingDirectory( "/tmp/" );
    client.listFiles();
    client.quit();
    client.disconnect();
  }
}

Thanks for making such a useful library.

Regards
Andrew

[Chris Mair]

(An earlier response submitted by Andrew Norman)

Just tried to open an bug and it says that I need to be authorised by the project admin.

Just a heads up I think I have found the issue, in fact you found it first with a TODO on line 157 in DefaultSession, where you state that you need to set the socket timeout.

I think that it is one fix for the problem. I am still looking through the code to see if I can get a patch for you.

Regards
Andrew

[Chris Mair]

See https://sourceforge.net/p/mockftpserver/bugs/25/

That addresses the OutOfMemoryError error (cleans up closed sessions), and is fixed for MockFtpServer 2.6.

[Chris Mair]

Just a clarification: The bug #25 fixes the "java.lang.OutOfMemoryError: Java heap space", but not the separate "java.lang.OutOfMemoryError: unable to create new native thread" error that you get if the client never sends the QUIT command.

[Chris Mair]

I created #26: "Clean up sessions/sockets if client does not send QUIT"
https://sourceforge.net/p/mockftpserver/bugs/26/

to cover the case of a long-running server getting "java.lang.OutOfMemoryError: unable to create new native thread" when lots of clients never send the QUIT command.