#115 Hidden IOException at isClosed()

[Bardocz István]

When a line ends exactly at the end of BufferedReader internal buffer, the CSVReader#isClosed() method one byte reading will refill the buffer, the mark is lost, therefore the reset() throws "java.io.IOException: Mark invalid" exception. This bug aborts further reads from the buffer.

This bug probably introduced by this fix: https://sourceforge.net/p/opencsv/bugs/108/

[Bardocz István]

Here is a complete example use case:

import java.io.IOException;
import java.io.Reader;
import java.io.StringReader;
import java.util.List;

import com.opencsv.CSVReader;

public class CSVReaderTest
{

private static final String ONE_ROW = "col\tcol\tcol\tco\r\n"; // 10byte
private static final int ROWS = 1000;

public static void main(String[] args) throws IOException
{
    StringBuilder all = new StringBuilder();
    for (int i = 0; i < ROWS; i++)
        all.append(ONE_ROW);

    Reader reader = new StringReader(all.toString());
    CSVReader csvr = new CSVReader(reader, '\t');
    List<String[]> rows = csvr.readAll();
    if (rows.size() != ROWS)
        throw new RuntimeException("Expected row count: " + ROWS + " found: " + rows.size());

    csvr.close();
}

}

[Anand Rathi]

I am also facing this issue while parsing big file ~20MB. As complete file is not read, we are losing some data because of this.

Is there any update on this ticket?

[Scott Conway]

Rathiar I apologize but no there has been no updates on this issue. I am
entering in the crazy part of the year with work, kids activities, Boy
Scout troop activities that there will not be much work done for the next
couple of months. I have one feature I am trying to get out before
September and even that I have not touched in a month.

Bardocz - thanks for the example.

I have an question though. Bardocz - you said this issue was caused by
Issue #108. Have either of you tried version 3.4 because there was a fix
to the isClosed in there.

If both of you are using 3.4 I will take the example that Bardocz made and
try and make a unit test out of it. If successful I will try and slate
this for 3.5 but with everything happening this Summer/Fall I cannot give
you a ETA for 3.5.

:(

On Tue, Jul 21, 2015 at 12:07 AM, Anand Rathi rathiar@users.sf.net wrote:

I am also facing this issue while parsing big file ~20MB. As complete file
is not read, we are losing some data because of this.

Is there any update on this ticket?

• [bugs:#115] http://sourceforge.net/p/opencsv/bugs/115 Hidden
  IOException at isClosed()*

Status: open
Group: v1.0 (example)
Created: Wed Jun 24, 2015 07:00 AM UTC by Bardocz István
Last Updated: Wed Jun 24, 2015 08:23 PM UTC
Owner: nobody

When a line ends exactly at the end of BufferedReader internal buffer, the
CSVReader#isClosed() method one byte reading will refill the buffer, the
mark is lost, therefore the reset() throws "java.io.IOException: Mark
invalid" exception. This bug aborts further reads from the buffer.

This bug probably introduced by this fix:
https://sourceforge.net/p/opencsv/bugs/108/

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/opencsv/bugs/115/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

--
Scott Conway
scott.conway@gmail.com
http://www.conwayfamily.name

[Anand Rathi]

My scenario is working fine with v3.3. So, it seems that this issue is because of changes made in isClosed() method in v3.4

[Scott Conway]

I was afraid of that. I will see if I can get a chance to look at the
isClosed this weekend.

On Mon, Jul 27, 2015 at 11:23 PM, Anand Rathi rathiar@users.sf.net wrote:

My scenario is working fine with v3.3. So, it seems that this issue is
because of changes made in isClosed() method in v3.4

---

• [bugs:#115] http://sourceforge.net/p/opencsv/bugs/115 Hidden
  IOException at isClosed()*

Status: open
Group: v1.0 (example)
Created: Wed Jun 24, 2015 07:00 AM UTC by Bardocz István
Last Updated: Tue Jul 21, 2015 05:07 AM UTC
Owner: nobody

When a line ends exactly at the end of BufferedReader internal buffer, the
CSVReader#isClosed() method one byte reading will refill the buffer, the
mark is lost, therefore the reset() throws "java.io.IOException: Mark
invalid" exception. This bug aborts further reads from the buffer.

This bug probably introduced by this fix:
https://sourceforge.net/p/opencsv/bugs/108/

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/opencsv/bugs/115/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

--
Scott Conway
scott.conway@gmail.com
http://www.conwayfamily.name

[Scott Conway]

Found it! The problem with the isClosed fix is that it was setting a mark of one byte but in Java the size of a character is two bytes.

I do not know why it did not fail immediately but after changing the size passed into the mark I ran the test that Bardocz sent in a loop of 10000 without issue.

I have merged the fix in trunk and will try and get a new version out this weekend if I can get a support request in.

[Anand Rathi]

Thanks Scott for quick turnaround. Is it possible to make that fix available in v3.4. I believe without this fix, v3.4 should not be used.

[Scott Conway]

Agreed - I am going to try and push out a new version this weekend and that
will be 3.5. Agreed that 3.4 should not be used but I would rather push a
new version than overwrite a previous version. That way if someone comes
up and says I have a problem with version 3.4 I won't have to ask when they
downloaded it.

On Thu, Jul 30, 2015 at 12:08 AM, Anand Rathi rathiar@users.sf.net wrote:

Thanks Scott for quick turnaround. Is it possible to make that fix
available in v3.4. I believe without this fix, v3.4 should not be used.

---

• [bugs:#115] http://sourceforge.net/p/opencsv/bugs/115 Hidden
  IOException at isClosed()*

Status: closed-fixed
Group: v1.0 (example)
Created: Wed Jun 24, 2015 07:00 AM UTC by Bardocz István
Last Updated: Wed Jul 29, 2015 02:27 AM UTC
Owner: Scott Conway

When a line ends exactly at the end of BufferedReader internal buffer, the
CSVReader#isClosed() method one byte reading will refill the buffer, the
mark is lost, therefore the reset() throws "java.io.IOException: Mark
invalid" exception. This bug aborts further reads from the buffer.

This bug probably introduced by this fix:
https://sourceforge.net/p/opencsv/bugs/108/

---

Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/opencsv/bugs/115/

To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/

--
Scott Conway
scott.conway@gmail.com
http://www.conwayfamily.name