#323 PDL::IO::FITS takes a long time to load

closed-fixed
core (120)
2
2013-02-27
2013-02-26
Craig DeForest
No

PDL::IO::FITS seems to require nearly as much time to load as the rest of the 'use PDL;' automatic modules combined. This load process should be streamlined in a future release, or PDL::IO::FITS should not be autoloaded as part of "use PDL".

I don't propose to fix this in the next couple of days, but I'm filing a bug as a placeholder to be pondered after the dust settles.

Discussion

  • Derek Lamb
    Derek Lamb
    2013-02-26

    Why do you think it is PDL::IO::FITS? That hasn't changed since 2011. But between 2.4.11_001 and 2.4.11_011 perldl does take a lot longer to load. Doing
    $ perl -Mblib -MPDL -d:NYTProf -we '1;'
    shows that most of the time is spent in Text::Balanced::extract_multiple. Text::Balanced is a requirement of Filter::Simple, which would instead seem to point to the new PDL::NiceSlice engine.

     
  • Craig DeForest
    Craig DeForest
    2013-02-26

    Aha. It probably *is* the NiceSlice filter being slow, then -- IO::FITS is huge and is NiceSliced. Loading could probably be sped up considerably by making it NiceSlice-clean.

     
  • Craig DeForest
    Craig DeForest
    2013-02-27

    Well, that was short-lived. I pulled the NiceSlice stuff out of FITS and out of Transform, and each one loads very quickly now.

     
  • Craig DeForest
    Craig DeForest
    2013-02-27

    • status: open --> closed-fixed
     
  • Chris Marshall
    Chris Marshall
    2013-02-27

    The PDL::NiceSlice work is aiming for correctness first. That said, the current Filter::Simple approach is expensive because it is currently processing the entire file *and* there is a problem where lines are being crunched rather that directly ignored which makes the process a slow no-op.