#56 Implement ResultsetMetaData.getColumnDisplaySize()

closed
None
5
2012-11-18
2012-08-28
jbocarsly
No

Need ResultsetMetaData.getColumnDisplaySize() to return a real value for both fixed width and delimited files.

Discussion

  • Simon Chenery

    Simon Chenery - 2012-09-03

    Source code changes, unit tests and test data are in attached ZIP file.

     
  • Simon Chenery

    Simon Chenery - 2012-09-06
    • assigned_to: nobody --> simoc
     
  • Simon Chenery

    Simon Chenery - 2012-09-06

    For fixed width files, the width of each column is easily calculated from the connection property:

    props.put("fixedWidths", "1-16,17-24,25-27,35-42,43-50,51-58");

    And the width is returned from the following function:

    ResultSetMetaData.getColumnDisplaySize(1)

    But for CSV files it seems that the complete CSV file is read into memory and the length of the longest column is used. Is that correct?

    Do you need the column widths for CSV files, or only for fixed width files?

     
  • jbocarsly

    jbocarsly - 2012-09-06

    We need the column width metadata for both fixed width and delimited. Alas, for delimited, there is no simple way of determining the data width without scanning the whole file, since the width is implicit in the data.

    It doesn't seem that it would be useful to pre-scan and cache the widths, since they could easily vary from file to file. I also considered handling width with a property, but that wouldn't be a terribly user-friendly design. If you've got any thoughts on a better way to engineer this, love to hear them...

     
  • Simon Chenery

    Simon Chenery - 2012-09-09

    Implement ResultsetMetaData.getColumnDisplaySize() for fixed width files (using "fixedWidths" property) and DBF files.

    For selected columns containing expressions (SELECT A-1, DOMAIN+'.com', ...) the current default column size of 20 is still returned.

    It is also not possible to calculate the column size for CSV files and the current default column size of 20 is still returned.

    One possible solution for CSV files is for applications to run SQL queries two times. The first query determines the maximum size of each column. Then you run the query again and format the rows using the maximum column sizes.

    Files changed:
    src/java/org/relique/io/DataReader.java
    src/java/org/relique/io/ListDataReader.java
    src/java/org/relique/jdbc/csv/CsvRawReader.java
    src/java/org/relique/jdbc/csv/CsvReader.java
    src/java/org/relique/jdbc/csv/CsvResultSet.java
    src/java/org/relique/jdbc/csv/CsvResultSetMetaData.java
    src/java/org/relique/jdbc/dbf/DbfReader.java
    src/java/test/org/relique/jdbc/csv/TestCsvDriver.java
    src/java/test/org/relique/jdbc/csv/TestDbfDriver.java
    src/java/test/org/relique/jdbc/csv/TestFixedWidthFiles.java

     
  • Simon Chenery

    Simon Chenery - 2012-11-18

    Possible to set this ticket to closed now?

    There is no way to calculate column sizes for CSV files without reading the whole file into memory (and this is something I want to avoid).

     
  • jbocarsly

    jbocarsly - 2012-11-18

    Feel free to close the ticket. I do feel that the driver should implement this metadata, but I can't think of a better engineering solution than the fairly ugly one that I suggested.
    Thanks.

     
  • Simon Chenery

    Simon Chenery - 2012-11-18

    Set to Closed. Included in CsvJdbc version 1.0-11

     
  • Simon Chenery

    Simon Chenery - 2012-11-18
    • status: open --> closed
     
  • Nobody/Anonymous

    I observe something truly interesting about your world wide web blog i really saved to favorites. hotfile, fileserve, filesonic wupload data
    <a href="http://carol1son.blogoak.com?blogname=carol1son&postarch=7" title="Option for">Option for</a>