#75 ls on big folder get exception

closed-fixed
J2SSH (50)
5
2014-11-22
2007-06-10
cwseetoh
No

When the target path on the remote server has more than 430 files (around there), the ls command will generate an exception. Below is what was logged down by my application:

10-06-2007@17:46:53 FATAL - [IPACClientFtpUtils.copyPDFFileToServerTemp] - java.lang.NullPointerException
10-06-2007@17:46:53 FATAL - [IPACClientFtpUtils.copyPDFFileToServerTemp] - com.sshtools.j2ssh.sftp.SftpSubsystemClient.listChildren(Unknown Source)
10-06-2007@17:46:53 FATAL - [IPACClientFtpUtils.copyPDFFileToServerTemp] - com.sshtools.j2ssh.SftpClient.ls(Unknown Source)

I'm using j2ssh-core-0.2.9.jar

The console logs down the following:

Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.subsystem.SubsystemChannel startSubsystem
INFO: Starting sftp subsystem
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol sendChannelRequest
INFO: Sending subsystem request for the session channel
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol sendChannelRequest
INFO: Waiting for channel request reply
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol sendChannelRequest
INFO: Channel request succeeded
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.sftp.SftpSubsystemClient initialize
INFO: Server responded with version 3
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.transport.AsyncService run
SEVERE: Service message loop failed!
java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at com.sshtools.j2ssh.io.DynamicBuffer.write(Unknown Source)
at com.sshtools.j2ssh.io.DynamicBuffer$DynamicBufferOutputStream.write(Unknown Source)
at java.io.OutputStream.write(OutputStream.java:58)
at com.sshtools.j2ssh.subsystem.SubsystemChannel.onChannelData(Unknown Source)
at com.sshtools.j2ssh.connection.Channel.processChannelData(Unknown Source)
at com.sshtools.j2ssh.connection.ConnectionProtocol.onMsgChannelData(Unknown Source)
at com.sshtools.j2ssh.connection.ConnectionProtocol.onMessageReceived(Unknown Source)
at com.sshtools.j2ssh.transport.AsyncService.run(Unknown Source)
at java.lang.Thread.run(Thread.java:595)
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol onStop
INFO: Closing all active channels
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol onStop
INFO: thread has 2 active channels to stop
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.Channel close
INFO: Finializing channel close
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol freeChannel
INFO: Freeing channel 1 [sftp]
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.Channel close
INFO: Finializing channel close
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ChannelInputStream close
INFO: Closing ChannelInputStream
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ChannelOutputStream close
INFO: Closing ChannelOutputStream
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ChannelInputStream close
INFO: Closing ChannelInputStream
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.connection.ConnectionProtocol freeChannel
INFO: Freeing channel 0 [session]
Jun 8, 2007 7:30:36 PM com.sshtools.j2ssh.transport.AsyncService run
INFO: ssh-connection thread is exiting

Discussion

  • Sascha Hunold

    Sascha Hunold - 2007-06-12
    • assigned_to: nobody --> hunoldinho
     
  • Sascha Hunold

    Sascha Hunold - 2007-06-12

    Logged In: YES
    user_id=752659
    Originator: NO

    I think the problem is only triggered when listing a large directory.
    Please provide some infos about
    - JDK version
    - client and server os
    - ssh server (version)
    - perhaps some code snippets

    And send the cli output of "ls" to my personal email address.

     
  • cwseetoh

    cwseetoh - 2007-06-19

    Logged In: YES
    user_id=1813362
    Originator: YES

    Hi hunoldinho, I wonder if you've received the requested information I just sent you?

    Anyway some more info for you. The exception probably has something to do with total file size. The 430 files that caused the exception are pdf files, average around 26kb - 30kb each. I tried to list a folder with 1000+ xml files without any problem. The xml files are all around 1kb. So I guess the file size does matters.

    Regards,
    Alfred

     
  • cwseetoh

    cwseetoh - 2007-06-20

    Logged In: YES
    user_id=1813362
    Originator: YES

    Hi, I made a surprising discovery today, I can't even download files that are only 78KB. Below is the stack trace captured.

    java.lang.NullPointerException
    com.sshtools.j2ssh.sftp.SftpSubsystemClient.readFile(Unknown Source)
    com.sshtools.j2ssh.sftp.SftpFileInputStream.read(Unknown Source)
    java.io.InputStream.read(InputStream.java:89)
    com.sshtools.j2ssh.SftpClient.transferFile(Unknown Source)
    com.sshtools.j2ssh.SftpClient.get(Unknown Source)
    com.sshtools.j2ssh.SftpClient.get(Unknown Source)
    com.sshtools.j2ssh.SftpClient.get(Unknown Source)
    seb.ipac.service.utils.IPACClientFtpUtils.receiveXMLFileUsingSftp(Unknown Source)
    seb.ipac.service.utils.IPACClientFtpUtils.ftpReceiveXMLFiles(Unknown Source)
    seb.ipac.service.IPACClientFtpSvcs.run(Unknown Source)

    I noticed that there was another bug reported previously (970676) which states that the largest file he could download was 53KB. However there was no follow-up from there. I'm probably facing the same problem as he did. He's using version 0.2.7, and I'm using version 0.2.9. Can anybody help with these 2 problems that I'm facing? Thanks a million.

     
  • cwseetoh

    cwseetoh - 2007-06-20

    Logged In: YES
    user_id=1813362
    Originator: YES

    I believe I had solved both problems already. The culprit is in DynamicBuffer.verifyBufferSize(). This method is supposed to ensure the buffer size is enough for the current write operation. However it will only increase the buffer size by a fixed quantum once, even if its not enough after the increase, it doesn't care. All I did to solve the problem was to put in a while loop to make sure it increase the buffer until sufficient. Patch is attached.
    File Added: DynamicBuffer.java

     
  • cwseetoh

    cwseetoh - 2007-06-20
     
  • cwseetoh

    cwseetoh - 2007-06-20
    • status: open --> closed-fixed
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks