Menu

#2132 windows bugs with aggregation

nextrelease
open-fixed
nobody
None
5
2019-03-16
2019-03-15
No

Seth pointed out windows bugs with aggregation. I'm struggling to get it to use network filesystems like \VBOXSVR\tmp\agg\rel04\$Y\rbspb_rel04_ect-hope-sci-l2sa_$Y$m$d_v$(v,sep).cdf

and seeing other problems while I'm experimenting on Windows.

For example, I get:
java.io.IOException: Incorrect function.
at java.base/sun.nio.fs.WindowsWatchService$Poller.implRegister(Unknown Source)
at java.base/sun.nio.fs.AbstractPoller.processRequests(Unknown Source)
at java.base/sun.nio.fs.WindowsWatchService$Poller.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
java.io.IOException: Incorrect function.

Discussion

  • Jeremy Faden

    Jeremy Faden - 2019-03-15

    The agg file:////VBOXSVR/tmp/agg/rel04/2017/rbspb_rel04_ect-hope-sci-l2sa_$Y$m$d_v$(v,sep).cdf?L_Ele&timerange=2017-01-10 works, but

    file:////VBOXSVR/tmp/agg/rel04/$Y/rbspb_rel04_ect-hope-sci-l2sa_$Y$m$d_v$(v,sep).cdf?L_Ele&timerange=2017-01-10 (with a second $Y) does not.

     
  • Jeremy Faden

    Jeremy Faden - 2019-03-15

    The bug was where it tries to create the child filesystem's URI. I had to experiment with this, but the only one which appears to work on Windows is to use the File reference to create a child File, and then get the URI from that.

     
  • Jeremy Faden

    Jeremy Faden - 2019-03-15

    Seth pointed out that it does not work to do completions with aggregations:

    The first is completions on the file directly, the second is after a representative file is identified, and is given an incorrect URI:

    getCompletions file://///VBOXSVR/tmp/agg/rel04/2017/01/rbspb_rel04_ect-hope-sci-l2sa_20170110_v6.2.0.cdf
    getting CDF variables
    got 43 variables
    getting CDF attributes
    done, get plottable 
    getCompletions file:/VBOXSVR/tmp/agg/rel04/2017/01/rbspb_rel04_ect-hope-sci-l2sa_20170108_v6.2.0.cdf
    local root does not exist: \VBOXSVR\tmp\agg\rel04\2017\01
    
     
  • Jeremy Faden

    Jeremy Faden - 2019-03-15

    I put in a getChildFilesystem to contain the logic with a different branch for local filesystems. I was reminded that the FileSystem type cannot change, and it would be neat to support:

    file:/tmp/data/$Y/$Y$m.zip/$Y$m$d.dat

    and I don't think this would be too hard to do.

     
  • Jeremy Faden

    Jeremy Faden - 2019-03-16

    This code shows that URI.resolve must not be used for local URIs:

    from java.io import File
    u= File("//VBOXSVR/tmp/agg/").toURI()
    print u
    print File( u.resolve('rel04') ).list()
    print  File( File("//VBOXSVR/tmp/agg/"), 'rel04' ).list()
    

    gives the result:

    file:////VBOXSVR/tmp/agg/
    None
    array(java.lang.String,['.listing', '2017', '2018', '2016']) 
    
     

    Last edit: Jeremy Faden 2019-03-16
  • Jeremy Faden

    Jeremy Faden - 2019-03-16
    • status: open --> open-fixed
     
  • Jeremy Faden

    Jeremy Faden - 2019-03-16

    I believe I've found all use of this, and I'll close the ticket.