Barron

I submitted an issue to netcdf-c. I am very perplexed. Either, the performance gains are system dependent or something has changed. I look forward to learning which. https://github.com/Unidata/netcdf-c/issues/1773

I wonder if maybe some of the speed gains have been acheived through other changes in the default cache and buffering on the library side. I'm having trouble reproducing the speed gains in other software that allow NC_SHARE. In older versions of libnetcdf, this wasn't hard to do. I'll let you know what I find, but I suspect you correctly implemented it. It may be more useful for people using older libnetcdf.

Thank you for the update. I have been attempting to run tests. I am having a hard time reproducing the expected performance improvments on my current system. I wonder if this is tied to the underlying netcdf library version and/or platform. I am grateful for the update, and will try other platforms/configurations to try to reproduce to confirm that all is working. Again, thank you!

Some relevant discussion is available at the two netcdf4-python issues linked below. https://github.com/Unidata/netcdf4-python/pull/902 https://github.com/Unidata/netcdf4-python/issues/18 This would enable unbuffered read, which would dramatically speed up a commands that access all records of a subset of variables from NetCDF3 files. The command below is an ideallized test case. ncks -vVAR1 gridded_with_100_vars.nc gridded_with_var1.nc This would have no effect on RAM files or hdf5 backend files....

Does NCO support the NC_SHARE open mode? This disables library caching on the record dimensions, and can dramatically speed up read times when reading all times of a single variable from a file with hundreds of variables. I'm wondering if there is a way to trigger NC_SHARE from the comand prompt with tools like ncks etc. https://www.unidata.ucar.edu/software/netcdf/docs/netcdf_8h.html#a62bd97ac2fa4de8d515e4aa13ed7edc3 Thank you,