Menu

#1286 New library for reading CDFs will fix a number of problems

nextrelease
closed
nobody
None
7
2014-12-19
2014-11-04
No

Since May I've had Nand's new library for reading CDF files, and I'm just starting to switch over to it. This fixes a number of problems, such as:

  • There will be just one reader for the CDF files. Before there were two versions with slightly different capacities. The new library has the capacities of both libraries.
  • NIO is used for reading, which has the neat feature that I can use memory outside of the JVM.
  • NIO performance is superior to legacy methods which would copy data into JVM memory.
  • NIO can also memory map some cdf files, where the memory used to read the data is actually the file itself. This hints at an interesting new method for handling huge (1000000x1000000) datasets.

Discussion

  • Jeremy Faden

    Jeremy Faden - 2014-11-10

    I noticed an issue where a kludge for handling [n,1] results from vap+cdfj:file:///home/jbf/ct/hudson/data.backup/cdf/c4_cp_fgm_spin_20030102_v01.cdf?range__C4_CP_FGM_SPIN wasn't in the new code. The new code's where handles this properly.

    This fix also coerces the rank so that the filter is equal to the parameter rank. I've added a nightly test of this: file:/home/jbf/ct/hudson/vap/demos/cdf/cdfWhere.vap

     

    Last edit: Jeremy Faden 2014-11-10
  • Jeremy Faden

    Jeremy Faden - 2014-11-17

    The file .classpath, used by Eclipse, needs to be updated to use this new jar file.

     
  • Jeremy Faden

    Jeremy Faden - 2014-11-17
    • Priority: 5 --> 7
     
  • Jeremy Faden

    Jeremy Faden - 2014-12-19

    I've verified that eclipse can use the new cdfj library, and committed the change to .classpath.

     
  • Jeremy Faden

    Jeremy Faden - 2014-12-19
    • status: open --> closed