Using NCO with https opendap URL

[Kevin White]

Previously I've used nco successfully on an http opendap site. Now the site has changed to using https. I've tested nco on my Windows desktop using the "nco-4.9.5.windows.mvs.exe ( 27M): Windows Self-Extracting Installer (last updated Thursday, 24-Sep-2020 15:29:26 UTC). Maintained by Pedro Vicente."

When I try the following command I get an access failure. Question: Does nco support https opendap, or only http opendap ?

Thanks for your help.
Kevin White

c:\Program Files\nco>ncks -M -m -p https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07 nos.gomofs.2ds.f001.20210507.t18z.nc

syntax error, unexpected $end, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: ^
ERROR: nco__open() unable to open file "https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07\nos.gomofs.2ds.f001.20210507.t18z.nc"
ERROR NC_EACCESS Access failure
HINT: NC_EACCESS errors signify a problem receiving data from a DAP server. This can occur, e.g., when NCO requests (with nco_var_get()) more data than the server is configured to dispense at one time. A workaround might be to request smaller chunks of data at one time. This can be accomplished by accessing hyperslabs or multi-slabs of data as described at http://nco.sf.net/nco.html#mlt
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -77. Translation into English with nc_strerror(-77) is "NetCDF: Access failure"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

[Charlie Zender]

NCO supports http and https OpenDAP. I do not have a Windows machine to test on, though. MacOS follows. I am unsure how to resolve this. @pvicente if you're reading this, does this work for you?

zender@sastrugi:~$ ncks -M -m -p https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07 nos.gomofs.2ds.f001.20210507.t18z.nc
netcdf nos.gomofs.2ds.f001.20210507.t18z {
  dimensions:
    boundary = 4 ;
    eta_psi = 776 ;
    eta_rho = 777 ;
    eta_u = 777 ;
    eta_v = 776 ;
    ocean_time = UNLIMITED ; // (1 currently)
    s_rho = 30 ;
    s_w = 31 ;
    tracer = 2 ;
    xi_psi = 1172 ;
    xi_rho = 1173 ;
    xi_u = 1172 ;
    xi_v = 1173 ;

  variables:
    double Akk_bak ;
      Akk_bak:long_name = "background vertical mixing coefficient for turbulent energy" ;
      Akk_bak:units = "meter2 second-1" ;
      ...

[Pedro Vicente]

I do have the same error;
will try to debug further

C:\nco>ncks -M -m -p https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07 nos.gomofs.2ds.f001.20210507.t18z.nc
syntax error, unexpected $end, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: ^
ERROR: nco__open() unable to open file "https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07\nos.gomofs.2ds.f001.20210507.t18z.nc"
ERROR NC_EACCESS Access failure
HINT: NC_EACCESS errors signify a problem receiving data from a DAP server. This can occur, e.g., when NCO requests (with nco_var_get()) more data than the server is configured to dispense at one time. A workaround might be to request smaller chunks of data at one time. This can be accomplished by accessing hyperslabs or multi-slabs of data as described at http://nco.sf.net/nco.html#mlt
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -77. Translation into English with nc_strerror(-77) is "NetCDF: Access failure"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

[Pedro Vicente]

I did notice that there is a space between the -p URL and the file name

-p https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07 nos.gomofs.2ds.f001.20210507.t18z.nc

this is how the variable for the file name is assigned

 /* Make sure file is on local system and is readable or die trying */
  fl_in=nco_fl_mk_lcl(fl_in,fl_pth_lcl,HPSS_TRY,&FL_RTR_RMT_LCN);

0x0000022afcd994c0 "https://opendap.co-ops.nos.noaa.gov/thredds/dodsC/NOAA/GOMOFS/MODELS/2021/05/07\nos.gomofs.2ds.f001.20210507.t18z.nc"

then this call fails with error -77

rcd=nc__open(fl_nm,mode,bfr_sz_hnt,nc_id);

[Pedro Vicente]

@zender

the problem is that https is not considred here

this works for https (replacing 8 with 9, 7 with 8 and 'http' with 'https')
this will need to handle both http and https

/* Remote access detection; this should be replaced with NC_testurl; DAP-URL = "http://" host [ ":" port ] [ abs-path ] */
    if(strlen(fl_pth) < 8UL) is_url=False; else{
      sng=(char *)nco_malloc(9);
      sng=strncpy(sng,fl_pth,9);
      sng[8]='\0';    
      if(!strcmp("https://",sng)) is_url=True; else is_url=False;
      sng=(char *)nco_free(sng);
    } /* end else */

[Charlie Zender]

Ahh, thanks Pedro. Methinks you have found the bug. Would you like to submit a PR that works with both http and https on Windows, or would you like me to take it from here?

[Pedro Vicente]

Charlie, it's best for you to take over; I am still 2 versions behind (will update soon);
but my dependencies are shaky, I did try to do a complete build , with bld.bat, that git clones all dependencies ;

but since this was setup, some dependencies changed, and now some fail to build;
so, I have to rely on the existing dependencies already built on my laptop, not ideal;

the ideal would be to have a dedicated git repo with all working dependencies ;
but this is a major time consuming effort

[Charlie Zender]

OK, I'm making the patch now and will add a test for https DAP access under Windows.

[Pedro Vicente]

netcdf nos.gomofs.2ds.f001.20210507.t18z {
dimensions:
boundary = 4 ;
eta_psi = 776 ;
eta_rho = 777 ;
eta_u = 777 ;
eta_v = 776 ;
ocean_time = UNLIMITED ; // (1 currently)
s_rho = 30 ;
s_w = 31 ;
tracer = 2 ;
xi_psi = 1172 ;
xi_rho = 1173 ;
xi_u = 1172 ;
xi_v = 1173 ;

variables:
double Akk_bak ;
Akk_bak:long_name = "background vertical mixing coefficient for turbulent energy" ;
Akk_bak:units = "meter2 second-1" ;

double Akp_bak ;
  Akp_bak:long_name = "background vertical mixing coefficient for length scale" ;
  Akp_bak:units = "meter2 second-1" ;

double Akt_bak(tracer) ;
  Akt_bak:long_name = "background vertical mixing coefficient for tracers" ;
  Akt_bak:units = "meter2 second-1" ;

double Akv_bak ;
  Akv_bak:long_name = "background vertical mixing coefficient for momentum" ;
  Akv_bak:units = "meter2 second-1" ;

double Charnok_alpha ;
  Charnok_alpha:long_name = "Charnok factor for surface roughness" ;

double CrgBan_cw ;
  CrgBan_cw:long_name = "surface flux due to Craig and Banner wave breaking" ;

double Cs_r(s_rho) ;
  Cs_r:long_name = "S-coordinate stretching curves at RHO-points" ;
  Cs_r:valid_min = -1. ;
  Cs_r:valid_max = 0. ;
  Cs_r:field = "Cs_r, scalar" ;

double Cs_w(s_w) ;
  Cs_w:long_name = "S-coordinate stretching curves at W-points" ;
  Cs_w:valid_min = -1. ;
  Cs_w:valid_max = 0. ;
  Cs_w:field = "Cs_w, scalar" ;

double FSobc_in(boundary) ;
  FSobc_in:long_name = "free-surface inflow, nudging inverse time scale" ;
  FSobc_in:units = "second-1" ;

double FSobc_out(boundary) ;
  FSobc_out:long_name = "free-surface outflow, nudging inverse time scale" ;
  FSobc_out:units = "second-1" ;

double Falpha ;
  Falpha:long_name = "Power-law shape barotropic filter parameter" ;

double Fbeta ;
  Fbeta:long_name = "Power-law shape barotropic filter parameter" ;

double Fgamma ;
  Fgamma:long_name = "Power-law shape barotropic filter parameter" ;

int Lm2CLM ;
  Lm2CLM:long_name = "2D momentum climatology processing switch" ;
  Lm2CLM:flag_values = 0, 1 ;
  Lm2CLM:flag_meanings = ".FALSE. .TRUE." ;

int Lm3CLM ;
  Lm3CLM:long_name = "3D momentum climatology processing switch" ;
  Lm3CLM:flag_values = 0, 1 ;
  Lm3CLM:flag_meanings = ".FALSE. .TRUE." ;

int LnudgeM2CLM ;
  LnudgeM2CLM:long_name = "2D momentum climatology nudging activation switch" ;
  LnudgeM2CLM:flag_values = 0, 1 ;
  LnudgeM2CLM:flag_meanings = ".FALSE. .TRUE." ;

int LnudgeM3CLM ;
  LnudgeM3CLM:long_name = "3D momentum climatology nudging activation switch" ;
  LnudgeM3CLM:flag_values = 0, 1 ;
  LnudgeM3CLM:flag_meanings = ".FALSE. .TRUE." ;

int LnudgeTCLM(tracer) ;
  LnudgeTCLM:long_name = "tracer climatology nudging activation switch" ;
  LnudgeTCLM:flag_values = 0, 1 ;
  LnudgeTCLM:flag_meanings = ".FALSE. .TRUE." ;

int LsshCLM ;
  LsshCLM:long_name = "sea surface height climatology processing switch" ;
  LsshCLM:flag_values = 0, 1 ;
  LsshCLM:flag_meanings = ".FALSE. .TRUE." ;

int LtracerCLM(tracer) ;
  LtracerCLM:long_name = "tracer climatology processing switch" ;
  LtracerCLM:flag_values = 0, 1 ;
  LtracerCLM:flag_meanings = ".FALSE. .TRUE." ;

int LtracerSponge(tracer) ;
  LtracerSponge:long_name = "horizontal diffusivity sponge activation switch" ;
  LtracerSponge:flag_values = 0, 1 ;
  LtracerSponge:flag_meanings = ".FALSE. .TRUE." ;

int LtracerSrc(tracer) ;
  LtracerSrc:long_name = "tracer point sources and sink activation switch" ;
  LtracerSrc:flag_values = 0, 1 ;
  LtracerSrc:flag_meanings = ".FALSE. .TRUE." ;

int LuvSponge ;
  LuvSponge:long_name = "horizontal viscosity sponge activation switch" ;
  LuvSponge:flag_values = 0, 1 ;
  LuvSponge:flag_meanings = ".FALSE. .TRUE." ;

int LuvSrc ;
  LuvSrc:long_name = "momentum point sources and sink activation switch" ;
  LuvSrc:flag_values = 0, 1 ;
  LuvSrc:flag_meanings = ".FALSE. .TRUE." ;

int LwSrc ;
  LwSrc:long_name = "mass point sources and sink activation switch" ;
  LwSrc:flag_values = 0, 1 ;
  LwSrc:flag_meanings = ".FALSE. .TRUE." ;

double M2nudg ;
  M2nudg:long_name = "2D momentum nudging/relaxation inverse time scale" ;
  M2nudg:units = "day-1" ;

double M2obc_in(boundary) ;
  M2obc_in:long_name = "2D momentum inflow, nudging inverse time scale" ;
  M2obc_in:units = "second-1" ;

double M2obc_out(boundary) ;
  M2obc_out:long_name = "2D momentum outflow, nudging inverse time scale" ;
  M2obc_out:units = "second-1" ;

double M3nudg ;
  M3nudg:long_name = "3D momentum nudging/relaxation inverse time scale" ;
  M3nudg:units = "day-1" ;

double M3obc_in(boundary) ;
  M3obc_in:long_name = "3D momentum inflow, nudging inverse time scale" ;
  M3obc_in:units = "second-1" ;

double M3obc_out(boundary) ;
  M3obc_out:long_name = "3D momentum outflow, nudging inverse time scale" ;
  M3obc_out:units = "second-1" ;

float Pair(ocean_time,eta_rho,xi_rho) ;
  Pair:long_name = "surface air pressure" ;
  Pair:units = "millibar" ;
  Pair:time = "ocean_time" ;
  Pair:grid = "grid" ;
  Pair:location = "face" ;
  Pair:coordinates = "lon_rho lat_rho ocean_time" ;
  Pair:field = "Pair, scalar, series" ;
  Pair:_FillValue = 1.e+37f ;
  Pair:_ChunkSizes = 1, 777, 1173 ;

double Tcline ;
  Tcline:long_name = "S-coordinate surface/bottom layer width" ;
  Tcline:units = "meter" ;

double Tnudg(tracer) ;
  Tnudg:long_name = "Tracers nudging/relaxation inverse time scale" ;
  Tnudg:units = "day-1" ;

double Tobc_in(boundary,tracer) ;
  Tobc_in:long_name = "tracers inflow, nudging inverse time scale" ;
  Tobc_in:units = "second-1" ;

double Tobc_out(boundary,tracer) ;
  Tobc_out:long_name = "tracers outflow, nudging inverse time scale" ;
  Tobc_out:units = "second-1" ;

float Uwind(ocean_time,eta_rho,xi_rho) ;
  Uwind:long_name = "surface u-wind component" ;
  Uwind:units = "meter second-1" ;
  Uwind:time = "ocean_time" ;
  Uwind:grid = "grid" ;
  Uwind:location = "face" ;
  Uwind:coordinates = "lon_rho lat_rho ocean_time" ;
  Uwind:field = "u-wind, scalar, series" ;
  Uwind:_FillValue = 1.e+37f ;
  Uwind:_ChunkSizes = 1, 777, 1173 ;

int Vstretching ;
  Vstretching:long_name = "vertical terrain-following stretching function" ;

int Vtransform ;
  Vtransform:long_name = "vertical terrain-following transformation equation" ;

float Vwind(ocean_time,eta_rho,xi_rho) ;
  Vwind:long_name = "surface v-wind component" ;
  Vwind:units = "meter second-1" ;
  Vwind:time = "ocean_time" ;
  Vwind:grid = "grid" ;
  Vwind:location = "face" ;
  Vwind:coordinates = "lon_rho lat_rho ocean_time" ;
  Vwind:field = "v-wind, scalar, series" ;
  Vwind:_FillValue = 1.e+37f ;
  Vwind:_ChunkSizes = 1, 777, 1173 ;

double Znudg ;
  Znudg:long_name = "free-surface nudging/relaxation inverse time scale" ;
  Znudg:units = "day-1" ;

double Zob ;
  Zob:long_name = "bottom roughness" ;
  Zob:units = "meter" ;

double Zos ;
  Zos:long_name = "surface roughness" ;
  Zos:units = "meter" ;

double Zos_hsig_alpha ;
  Zos_hsig_alpha:long_name = "wave amplitude factor for surface roughness" ;

double angle(eta_rho,xi_rho) ;
  angle:long_name = "angle between XI-axis and EAST" ;
  angle:units = "radians" ;
  angle:grid = "grid" ;
  angle:location = "face" ;
  angle:coordinates = "lon_rho lat_rho" ;
  angle:field = "angle, scalar" ;

double dstart ;
  dstart:long_name = "time stamp assigned to model initilization" ;
  dstart:units = "days since 2016-01-01 00:00:00" ;
  dstart:calendar = "proleptic_gregorian" ;

double dt ;
  dt:long_name = "size of long time-steps" ;
  dt:units = "second" ;

double dtfast ;
  dtfast:long_name = "size of short time-steps" ;
  dtfast:units = "second" ;

double el ;
  el:long_name = "domain length in the ETA-direction" ;
  el:units = "meter" ;

double f(eta_rho,xi_rho) ;
  f:long_name = "Coriolis parameter at RHO-points" ;
  f:units = "second-1" ;
  f:grid = "grid" ;
  f:location = "face" ;
  f:coordinates = "lon_rho lat_rho" ;
  f:field = "coriolis, scalar" ;

double gamma2 ;
  gamma2:long_name = "slipperiness parameter" ;

double gls_Kmin ;
  gls_Kmin:long_name = "minimum value of specific turbulent kinetic energy" ;

double gls_Pmin ;
  gls_Pmin:long_name = "minimum Value of dissipation" ;

double gls_c1 ;
  gls_c1:long_name = "shear production coefficient" ;

double gls_c2 ;
  gls_c2:long_name = "dissipation coefficient" ;

double gls_c3m ;
  gls_c3m:long_name = "buoyancy production coefficient (minus)" ;

double gls_c3p ;
  gls_c3p:long_name = "buoyancy production coefficient (plus)" ;

double gls_cmu0 ;
  gls_cmu0:long_name = "stability coefficient" ;

double gls_m ;
  gls_m:long_name = "turbulent kinetic energy exponent" ;

double gls_n ;
  gls_n:long_name = "turbulent length scale exponent" ;

double gls_p ;
  gls_p:long_name = "stability exponent" ;

double gls_sigk ;
  gls_sigk:long_name = "constant Schmidt number for TKE" ;

double gls_sigp ;
  gls_sigp:long_name = "constant Schmidt number for PSI" ;

int grid ;
  grid:cf_role = "grid_topology" ;
  grid:topology_dimension = 2 ;
  grid:node_dimensions = "xi_psi eta_psi" ;
  grid:face_dimensions = "xi_rho: xi_psi (padding: both) eta_rho: eta_psi (padding: both)" ;
  grid:edge1_dimensions = "xi_u: xi_psi eta_u: eta_psi (padding: both)" ;
  grid:edge2_dimensions = "xi_v: xi_psi (padding: both) eta_v: eta_psi" ;
  grid:node_coordinates = "lon_psi lat_psi" ;
  grid:face_coordinates = "lon_rho lat_rho" ;
  grid:edge1_coordinates = "lon_u lat_u" ;
  grid:edge2_coordinates = "lon_v lat_v" ;
  grid:vertical_dimensions = "s_rho: s_w (padding: none)" ;

double h(eta_rho,xi_rho) ;
  h:long_name = "bathymetry at RHO-points" ;
  h:units = "meter" ;
  h:grid = "grid" ;
  h:location = "face" ;
  h:coordinates = "lon_rho lat_rho" ;
  h:field = "bath, scalar" ;

double hc ;
  hc:long_name = "S-coordinate parameter, critical depth" ;
  hc:units = "meter" ;

double lat_psi(eta_psi,xi_psi) ;
  lat_psi:long_name = "latitude of PSI-points" ;
  lat_psi:units = "degree_north" ;
  lat_psi:standard_name = "latitude" ;
  lat_psi:field = "lat_psi, scalar" ;

double lat_rho(eta_rho,xi_rho) ;
  lat_rho:long_name = "latitude of RHO-points" ;
  lat_rho:units = "degree_north" ;
  lat_rho:standard_name = "latitude" ;
  lat_rho:field = "lat_rho, scalar" ;

double lat_u(eta_u,xi_u) ;
  lat_u:long_name = "latitude of U-points" ;
  lat_u:units = "degree_north" ;
  lat_u:standard_name = "latitude" ;
  lat_u:field = "lat_u, scalar" ;

double lat_v(eta_v,xi_v) ;
  lat_v:long_name = "latitude of V-points" ;
  lat_v:units = "degree_north" ;
  lat_v:standard_name = "latitude" ;
  lat_v:field = "lat_v, scalar" ;

double lon_psi(eta_psi,xi_psi) ;
  lon_psi:long_name = "longitude of PSI-points" ;
  lon_psi:units = "degree_east" ;
  lon_psi:standard_name = "longitude" ;
  lon_psi:field = "lon_psi, scalar" ;

double lon_rho(eta_rho,xi_rho) ;
  lon_rho:long_name = "longitude of RHO-points" ;
  lon_rho:units = "degree_east" ;
  lon_rho:standard_name = "longitude" ;
  lon_rho:field = "lon_rho, scalar" ;

double lon_u(eta_u,xi_u) ;
  lon_u:long_name = "longitude of U-points" ;
  lon_u:units = "degree_east" ;
  lon_u:standard_name = "longitude" ;
  lon_u:field = "lon_u, scalar" ;

double lon_v(eta_v,xi_v) ;
  lon_v:long_name = "longitude of V-points" ;
  lon_v:units = "degree_east" ;
  lon_v:standard_name = "longitude" ;
  lon_v:field = "lon_v, scalar" ;

double mask_psi(eta_psi,xi_psi) ;
  mask_psi:long_name = "mask on psi-points" ;
  mask_psi:flag_values = 0., 1. ;
  mask_psi:flag_meanings = "land water" ;
  mask_psi:grid = "grid" ;
  mask_psi:location = "node" ;
  mask_psi:coordinates = "lon_psi lat_psi" ;

double mask_rho(eta_rho,xi_rho) ;
  mask_rho:long_name = "mask on RHO-points" ;
  mask_rho:flag_values = 0., 1. ;
  mask_rho:flag_meanings = "land water" ;
  mask_rho:grid = "grid" ;
  mask_rho:location = "face" ;
  mask_rho:coordinates = "lon_rho lat_rho" ;

double mask_u(eta_u,xi_u) ;
  mask_u:long_name = "mask on U-points" ;
  mask_u:flag_values = 0., 1. ;
  mask_u:flag_meanings = "land water" ;
  mask_u:grid = "grid" ;
  mask_u:location = "edge1" ;
  mask_u:coordinates = "lon_u lat_u" ;

double mask_v(eta_v,xi_v) ;
  mask_v:long_name = "mask on V-points" ;
  mask_v:flag_values = 0., 1. ;
  mask_v:flag_meanings = "land water" ;
  mask_v:grid = "grid" ;
  mask_v:location = "edge2" ;
  mask_v:coordinates = "lon_v lat_v" ;

int nHIS ;
  nHIS:long_name = "number of time-steps between history records" ;

int nRST ;
  nRST:long_name = "number of time-steps between restart records" ;
  nRST:cycle = "only latest two records are maintained" ;

int nSTA ;
  nSTA:long_name = "number of time-steps between stations records" ;

int ndefHIS ;
  ndefHIS:long_name = "number of time-steps between the creation of history files" ;

int ndtfast ;
  ndtfast:long_name = "number of short time-steps" ;

double nl_tnu2(tracer) ;
  nl_tnu2:long_name = "nonlinear model Laplacian mixing coefficient for tracers" ;
  nl_tnu2:units = "meter2 second-1" ;

double nl_visc2 ;
  nl_visc2:long_name = "nonlinear model Laplacian mixing coefficient for momentum" ;
  nl_visc2:units = "meter2 second-1" ;

int ntimes ;
  ntimes:long_name = "number of long time-steps" ;

double ocean_time(ocean_time) ;
  ocean_time:long_name = "time since initialization" ;
  ocean_time:units = "seconds since 2016-01-01 00:00:00" ;
  ocean_time:calendar = "proleptic_gregorian" ;
  ocean_time:field = "time, scalar, series" ;
  ocean_time:_ChunkSizes = 512 ;

double pm(eta_rho,xi_rho) ;
  pm:long_name = "curvilinear coordinate metric in XI" ;
  pm:units = "meter-1" ;
  pm:grid = "grid" ;
  pm:location = "face" ;
  pm:coordinates = "lon_rho lat_rho" ;
  pm:field = "pm, scalar" ;

double pn(eta_rho,xi_rho) ;
  pn:long_name = "curvilinear coordinate metric in ETA" ;
  pn:units = "meter-1" ;
  pn:grid = "grid" ;
  pn:location = "face" ;
  pn:coordinates = "lon_rho lat_rho" ;
  pn:field = "pn, scalar" ;

double rdrg ;
  rdrg:long_name = "linear drag coefficient" ;
  rdrg:units = "meter second-1" ;

double rdrg2 ;
  rdrg2:long_name = "quadratic drag coefficient" ;

double rho0 ;
  rho0:long_name = "mean density used in Boussinesq approximation" ;
  rho0:units = "kilogram meter-3" ;

double s_rho(s_rho) ;
  s_rho:long_name = "S-coordinate at RHO-points" ;
  s_rho:valid_min = -1. ;
  s_rho:valid_max = 0. ;
  s_rho:positive = "up" ;
  s_rho:standard_name = "ocean_s_coordinate_g2" ;
  s_rho:formula_terms = "s: s_rho C: Cs_r eta: zeta depth: h depth_c: hc" ;
  s_rho:field = "s_rho, scalar" ;

double s_w(s_w) ;
  s_w:long_name = "S-coordinate at W-points" ;
  s_w:valid_min = -1. ;
  s_w:valid_max = 0. ;
  s_w:positive = "up" ;
  s_w:standard_name = "ocean_s_coordinate_g2" ;
  s_w:formula_terms = "s: s_w C: Cs_w eta: zeta depth: h depth_c: hc" ;
  s_w:field = "s_w, scalar" ;

float salt_sur(ocean_time,eta_rho,xi_rho) ;
  salt_sur:long_name = "surface salinity" ;
  salt_sur:time = "ocean_time" ;
  salt_sur:grid = "grid" ;
  salt_sur:location = "face" ;
  salt_sur:coordinates = "lon_rho lat_rho ocean_time" ;
  salt_sur:field = "salinity, scalar, series" ;
  salt_sur:_FillValue = 1.e+37f ;
  salt_sur:_ChunkSizes = 1, 777, 1173 ;

int spherical ;
  spherical:long_name = "grid type logical switch" ;
  spherical:flag_values = 0, 1 ;
  spherical:flag_meanings = "Cartesian spherical" ;

double sz_alpha ;
  sz_alpha:long_name = "surface flux from wave dissipation" ;

float temp_sur(ocean_time,eta_rho,xi_rho) ;
  temp_sur:long_name = "surface potential temperature" ;
  temp_sur:units = "Celsius" ;
  temp_sur:time = "ocean_time" ;
  temp_sur:grid = "grid" ;
  temp_sur:location = "face" ;
  temp_sur:coordinates = "lon_rho lat_rho ocean_time" ;
  temp_sur:field = "temp_sur, scalar, series" ;
  temp_sur:_FillValue = 1.e+37f ;
  temp_sur:_ChunkSizes = 1, 777, 1173 ;

double theta_b ;
  theta_b:long_name = "S-coordinate bottom control parameter" ;

double theta_s ;
  theta_s:long_name = "S-coordinate surface control parameter" ;

float u_sur(ocean_time,eta_u,xi_u) ;
  u_sur:long_name = "surface u-momentum component" ;
  u_sur:units = "meter second-1" ;
  u_sur:time = "ocean_time" ;
  u_sur:grid = "grid" ;
  u_sur:location = "edge1" ;
  u_sur:coordinates = "lon_u lat_u ocean_time" ;
  u_sur:field = "u_sur, scalar, series" ;
  u_sur:_FillValue = 1.e+37f ;
  u_sur:_ChunkSizes = 1, 777, 1172 ;

float v_sur(ocean_time,eta_v,xi_v) ;
  v_sur:long_name = "surface v-momentum component" ;
  v_sur:units = "meter second-1" ;
  v_sur:time = "ocean_time" ;
  v_sur:grid = "grid" ;
  v_sur:location = "edge2" ;
  v_sur:coordinates = "lon_v lat_v ocean_time" ;
  v_sur:field = "v_sur, scalar, series" ;
  v_sur:_FillValue = 1.e+37f ;
  v_sur:_ChunkSizes = 1, 776, 1173 ;

float wetdry_mask_psi(ocean_time,eta_psi,xi_psi) ;
  wetdry_mask_psi:long_name = "wet/dry mask on PSI-points" ;
  wetdry_mask_psi:flag_values = 0.f, 1.f ;
  wetdry_mask_psi:flag_meanings = "land water" ;
  wetdry_mask_psi:time = "ocean_time" ;
  wetdry_mask_psi:grid = "grid" ;
  wetdry_mask_psi:location = "node" ;
  wetdry_mask_psi:coordinates = "lon_psi lat_psi ocean_time" ;
  wetdry_mask_psi:field = "wetdry_mask_psi, scalar, series" ;
  wetdry_mask_psi:_ChunkSizes = 1, 776, 1172 ;

float wetdry_mask_rho(ocean_time,eta_rho,xi_rho) ;
  wetdry_mask_rho:long_name = "wet/dry mask on RHO-points" ;
  wetdry_mask_rho:flag_values = 0.f, 1.f ;
  wetdry_mask_rho:flag_meanings = "land water" ;
  wetdry_mask_rho:time = "ocean_time" ;
  wetdry_mask_rho:grid = "grid" ;
  wetdry_mask_rho:location = "face" ;
  wetdry_mask_rho:coordinates = "lon_rho lat_rho ocean_time" ;
  wetdry_mask_rho:field = "wetdry_mask_rho, scalar, series" ;
  wetdry_mask_rho:_ChunkSizes = 1, 777, 1173 ;

float wetdry_mask_u(ocean_time,eta_u,xi_u) ;
  wetdry_mask_u:long_name = "wet/dry mask on U-points" ;
  wetdry_mask_u:flag_values = 0.f, 1.f ;
  wetdry_mask_u:flag_meanings = "land water" ;
  wetdry_mask_u:time = "ocean_time" ;
  wetdry_mask_u:grid = "grid" ;
  wetdry_mask_u:location = "edge1" ;
  wetdry_mask_u:coordinates = "lon_u lat_u ocean_time" ;
  wetdry_mask_u:field = "wetdry_mask_u, scalar, series" ;
  wetdry_mask_u:_ChunkSizes = 1, 777, 1172 ;

float wetdry_mask_v(ocean_time,eta_v,xi_v) ;
  wetdry_mask_v:long_name = "wet/dry mask on V-points" ;
  wetdry_mask_v:flag_values = 0.f, 1.f ;
  wetdry_mask_v:flag_meanings = "land water" ;
  wetdry_mask_v:time = "ocean_time" ;
  wetdry_mask_v:grid = "grid" ;
  wetdry_mask_v:location = "edge2" ;
  wetdry_mask_v:coordinates = "lon_v lat_v ocean_time" ;
  wetdry_mask_v:field = "wetdry_mask_v, scalar, series" ;
  wetdry_mask_v:_ChunkSizes = 1, 776, 1173 ;

double xl ;
  xl:long_name = "domain length in the XI-direction" ;
  xl:units = "meter" ;

float zeta(ocean_time,eta_rho,xi_rho) ;
  zeta:long_name = "free-surface" ;
  zeta:units = "meter" ;
  zeta:time = "ocean_time" ;
  zeta:grid = "grid" ;
  zeta:location = "face" ;
  zeta:coordinates = "lon_rho lat_rho ocean_time" ;
  zeta:field = "free-surface, scalar, series" ;
  zeta:_ChunkSizes = 1, 777, 1173 ;

// global attributes:
:file = "nos.gomofs.surface.forecast.20210507.t18z_0002.nc" ;
:format = "netCDF-4/HDF5 file" ;
:Conventions = "CF-1.4, SGRID-0.3" ;
:type = "ROMS/TOMS quicksave file" ;
:title = "gomofs forecast RUN in operational mode" ;
:var_info = "varinfo.dat" ;
:rst_file = "nos.gomofs.rst.forecast.20210507.t18z.nc" ;
:his_base = "nos.gomofs.fields.forecast.20210507.t18z" ;
:sta_file = "nos.gomofs.stations.forecast.20210507.t18z.nc" ;
:grd_file = "nos.gomofs.romsgrid.nc" ;
:ini_file = "nos.gomofs.rst.nowcast.20210507.t18z.nc" ;
:tide_file = "nos.gomofs.roms.tides.nc" ;
:frc_file_01 = "nos.gomofs.met.forecast.20210507.t18z.nc" ;
:bry_file_01 = "nos.gomofs.obc.20210507.t18z.nc" ;
:clm_file_01 = "nos.gomofs.clim.20210507.t18z.nc" ;
:nud_file = "nos.gomofs.nudgcoef.nc" ;
:script_file = "./gomofs_ROMS_forecast.in" ;
:spos_file = "nos.gomofs.stations.in" ;
:NLM_TADV = "\n",
"ADVECTION: HORIZONTAL VERTICAL \n",
"temp: Akima4 Akima4 \n",
"salt: Akima4 Akima4" ;
:NLM_LBC = "\n",
"EDGE: WEST SOUTH EAST NORTH \n",
"zeta: Cha Cha Cha Clo \n",
"ubar: Fla Fla Fla Clo \n",
"vbar: Fla Fla Fla Clo \n",
"u: Rad Rad Rad Clo \n",
"v: Rad Rad Rad Clo \n",
"temp: RadNud RadNud RadNud Clo \n",
"salt: RadNud RadNud RadNud Clo \n",
"tke: Gra Gra Gra Clo" ;
:svn_url = "https://svnemc.ncep.noaa.gov/projects/nosofs_shared/tags/release-3.2.4/sorc/ROMS.fd" ;
:svn_rev = "101201" ;
:code_dir = "/gpfs/dell1/nco/ops/nwtest/nosofs.v3.3.5/sorc/ROMS.fd" ;
:header_dir = "/gpfs/dell1/nco/ops/nwtest/nosofs.v3.3.5/sorc/ROMS.fd/ROMS/Include" ;
:header_file = "gomofs.h" ;
:os = "Linux" ;
:cpu = "x86_64" ;
:compiler_system = "ifort" ;
:compiler_command = "/usrx/local/prod/intel/2018UP01/compilers_and_libraries/linux/mpi/bin64/mpif90" ;
:compiler_flags = "-fp-model precise -ip -xHost" ;
:tiling = "040x028" ;
:history = "ROMS/TOMS, Version 3.9, Friday - May 7, 2021 - 7:09:30 PM" ;
:ana_file = "ROMS/Functionals/ana_btflux.h, ROMS/Functionals/ana_stflux.h" ;
:CPP_options = "mode, ADD_FSOBC, ADD_M2OBC, ANA_BSFLUX, ANA_BTFLUX, ANA_SSFLUX, ASSUMED_SHAPE, ATM_PRESS, !BOUNDARY_ALLGATHER, BULK_FLUXES, !COLLECT_ALL..., CURVGRID, DIFF_GRID, DJ_GRADPS, DOUBLE_PRECISION, EMINUSP, GLS_MIXING, HDF5, LIMIT_BSTRESS, LIMIT_STFLX_COOLING, KANTHA_CLAYSON, LONGWAVE_OUT, MASKING, MIX_GEO_TS, MIX_S_UV, MPI, NONLINEAR, NONLIN_EOS, NO_LBC_ATT, N2S2_HORAVG, PERFECT_RESTART, POWER_LAW, PROFILE, K_GSCHEME, RADIATION_2D, REDUCE_ALLREDUCE, !RST_SINGLE, SALINITY, SOLAR_SOURCE, SOLVE3D, SSH_TIDES, STATIONS, TS_DIF2, UV_ADV, UV_COR, UV_U3HADVECTION, UV_C4VADVECTION, UV_QDRAG, UV_TIDES, UV_VIS2, VAR_RHO_2D, VISC_GRID, WET_DRY" ;
:DODS_EXTRA.Unlimited_Dimension = "ocean_time" ;
:EXTRA_DIMENSION.N = 30 ;

} // group /

[Kevin White]

Thanks for looking into this. Will look forward to a future release that fixes the bug.

[Kevin White]

Any chance that Pedro will be able to release a Windows executable in the foreseeable future that fixes this problem?

[Charlie Zender]

I cannot speak on behalf of Pedro, who could post the executable on his own if he had them. In any case, the bug was fixed a year ago so if you use, say, the Anaconda NCO package for Windows then the problem should be fixed there, too.

[Charlie Zender]

I cannot speak on behalf of Pedro, who could post the executable on his own if he had them. In any case, the bug was fixed a year ago so if you use, say, the Anaconda NCO package for Windows then the problem should be fixed there, too.

[Kevin White]

I have taken your advice and installed miniconda3 on my Windows PC. 
ncks responds normally when addressing locally  stored files such as:

(base) C:\Users\kwhit>ncks -m -p C:\Users\kwhit\OneDrive\Documents\SailFastLLC\TidalCurrents iho.nc
netcdf iho {
  dimensions:
    component_string_length = 6 ;
    components = 125 ;
    idcode_string_length = 19 ;
    name_string_length = 20 ;
    stations = 4509 ;
    etc......

    When I try to access the same file from an http server it fails:

(base) C:\Users\kwhit>ncks -m -p http://opendap.deltares.nl/static/deltares/delftdashboard/toolboxes/TideStations iho.nc
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <html^><head><title>404 Not Found</title></head><body><center><h1>404 Not Found</h1></center><hr><center>nginx</center></body></html>
ERROR: nco__open() unable to open file "http://opendap.deltares.nl/static/deltares/delftdashboard/toolboxes/TideStations\iho.nc"
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -90. Translation into English with nc_strerror(-90) is "NetCDF: file not found"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

Likewise, it fails when addressing an https server,

(base) C:\Users\kwhit>ncks -m -p https://opendap.co-ops.nos.noaa.gov/thredds/fileServer/NOAA/GOMOFS/MODELS/2023/01/12 nos.gomofs.2ds.f001.20230112.t18z.nc
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <html^ xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>TDS - Error report</title> <link rel="stylesheet" href="/thredds/tds.css" type="text/css"/> </head> <body> <h1>HTTP Status 404 - Not Found</h1> <HR size="1" noshade="noshade"> <p><b>Status</b> 404 - Not Found</p> <HR size="1" noshade="noshade"> <h3>THREDDS Data Server Version 4.6 -- <a href='https://www.unidata.ucar.edu/software/thredds/v4.6/tds/TDS.html'>Documentation</a></h3> </body></html>
ERROR: nco__open() unable to open file "https://opendap.co-ops.nos.noaa.gov/thredds/fileServer/NOAA/GOMOFS/MODELS/2023/01/12/nos.gomofs.2ds.f001.20230112.t18z.nc"
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -90. Translation into English with nc_strerror(-90) is "NetCDF: file not found"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

So my original problem which only occurred with https now seems to include http.   Any thoughts/suggestions?
Thanks
Kevin

[Kevin White]

Can't open remote files with ncks running miniconda. Refer to above discussion. Can you confirm if this is a bug? Any help appreciated.
Thanks Kevin

[Charlie Zender]

I get the similar behavior as you on the file above. It could be an NCO error. However, my DAP access to another file works fine. So it could also be a remote DAP configuration issue. Please try this and report what happens:
ncks -m http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis.dailyavgs/surface/pres.sfc.1969.nc

[Kevin White]

I tested with your command and the response was fine. I tried again with my 2 failed examples and got the same errors as reported before. Hmmm?

[Charlie Zender]

This suggests a potential DAP server configuration issue. I am open to other ideas, I just do not have any bright ones myself.

[Kevin White]

I believe I found my problem. I was using ncks with a "thredds/fileserver" URL. I should have used a "thredds/dodsC" URL. When I make that change it works!
Kevin