Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#185 CLEANest algorithm

Enhancement
open
David Benn
Algorithms (37)
8
2013-03-16
2010-04-08
David Benn
No

Should we implement CLEANest and Slick spectra as part of period analysis?

See also https://sourceforge.net/apps/mediawiki/vstar/index.php?title=Fourier_Analysis

Discussion

  • Aaron Price
    Aaron Price
    2010-04-30

    I have used them both in the past. CLEANest was used when working with gaps in the data. SLICK is a "prewhitening" routine that I used to use a lot, but haven't in many years. I think both are more useful when working with long period variables. These could also be possible options for a plug-in framework, if one is implemented. My vote is to put them on the list, but perhaps make it late phase 2 or even a phase 3 thing.

     
  • David Benn
    David Benn
    2010-09-05

    Doug Welch's comments at Citizen Sky 2 suggest that the priority of this should be higher.

     
  • David Benn
    David Benn
    2010-09-05

    • priority: 5 --> 8
     
  • Doug Welch
    Doug Welch
    2010-09-07

    Thanks for increasing the priority of this. As I mentioned to David, there is a Peranso implementation but it does not produce correct or usable residuals (which are important!) and Tonny V has confirmed the bug and also the fact that it won't be fixed anytime soon!

     
  • David Benn
    David Benn
    2010-09-07

    No problem Doug. Now that I'm back on the other side of the pond (just got to Sydney airport from San Francisco) I took a quick look at the ts1201.f Fortran code I'm planning to use. I don't know what the status of that implementation is, but I'll check with Matt T. I assume it's fine though, and should have no problem in translating this for VStar. It may take me a week or so before I start doing much development again, but I'll keep you posted. If you have any particular ideas about presentation of the output of the CLEANest spectrum algorithm, feel free to make suggestions. Without a closer look, I imagine there'll at least be a dialog with a line plot.

     
  • David Benn
    David Benn
    2011-01-21

    A couple of conversations with Doug since this have continued to keep this uppermost. Raising the priority.

     
  • David Benn
    David Benn
    2011-01-21

    • priority: 8 --> 9
     
  • David Benn
    David Benn
    2011-04-21

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=745:

    First implementation of CLEANest that passes a unit test based upon the tcas.dat example scenario provided in the docs accompanying ts12.f. Locked and variable periods have not yet been incorporated.

     
  • David Benn
    David Benn
    2011-04-21

    At this stage, only support for CLEANest from ts12.f is planned, not Slick. That can change however, depending upon demand from end users.

     
  • David Benn
    David Benn
    2011-06-02

    One curiosity I've noticed is that for each CLEANest run for a given DC DFT, the same amplitude value is returned! Is that the case for TS. Why?

     
  • David Benn
    David Benn
    2011-07-19

    • summary: CLEANest and Slick spectra --> CLEANest algorithm
     
  • David Benn
    David Benn
    2011-07-19

    Arising from email discussion with Doug and Grant, Grant recommended that ultimately VStar's CLEANest implementation should permit the inclusion of harmonics per frequency specified for refinement. Here is the last email exchange on the topic, a reply from Grant on July 17 2011 to comments made by Doug and I:

    > > The TS (and VStar) implementation of CLEANest permits these frequencies/periods categories to be specified:
    > >
    > > 1. User selected
    > > 2. Locked
    > > 3. Variable
    > >
    > > So are you saying that we essentially need another category:
    > >
    > > 4. Harmonics
    > >
    > > and that unlike 3. (variable), although these can vary, they must do so such that they maintain their harmonic relationship with their "parent" frequency?

    Yes, I think that describes it quite well

    > Ideally, the category would be "Variable fundamental frequency with
    > integer-value-locked harmonics up to some specified degree".

    There is merit in this description also. The harmonics must be tied to the fundamental, and must be integer multiples of same.

    In my R version, I pass it a list of frequencies (to be refined by CLEANEST) together with a list of harmonic orders. Then it finds the combination of fundamental frequencies which, *together with their integer multiples up to the specified order*, best fit the data.

    For example, if I had identified a frequency of f=0.177 for which I wanted 6 harmonics (harmonics up to 6f), and another frequency f=0.202 for which I only wanted 2 harmonics (f and 2f), applied to data "x" at times "t", I would say

    freq.raw = c(0.177,0.202)
    harmons = c(6,2)
    freq = clean(t,x,freq.raw,nharm=harmons)

    Maybe instead of including a new "type" of frequency, you can simply set a "number of harmonics" parameter for each frequency in the list (with all of them defaulting to 1)

    By the way -- some folks (most in fact) call 6 times fundamental the 5th harmonic, so if you want multiples up to 6 times then you'd have fundamental plus 5 harmonics. I call 6 times fundamental the 6th harmonic, so if you want multiples up to 6 times then you have 6 harmonics. It's nonstandard terminology -- I'm such a maverick! -- but maybe I'll be able to change the world (over the course of several decades) and introduce a new lexicon. But just so you know, when I say up to 6th harmonic I mean f, 2f, 3f, 4f, 5f, and 6f -- whereas most would say that's only up to the 5th harmonic.

    For VSTAR it might be best to go with the more common convention. You could then have check-boxes or a drop-down menu for the number of harmonics *starting with zero* (no harmonics), which is what I would refer to as "starting with 1" (just the fundamental).

    Sincerely,
    Grant

     
  • David Benn
    David Benn
    2011-07-24

    Fixed a bug in which the wrong period was being sent to the phase plot dialog after CLEANest. This lead to a rewrite of period analysis selection message generation which also fixed other bugs relating to selection of rows and plot coordinates. In:

    http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=785

     
  • David Benn
    David Benn
    2011-07-29

    Implemented unified CLEANest parameter dialog including harmonics per user frequency. These are passed to refinement method on period analyis algorithm but not yet implemented in CLEANest implementation.

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=793

     
  • David Benn
    David Benn
    2011-07-29

    Moved CLEANest source files into a "refinement" package and renamed CLEANest dialog to RefinementParameterDialog. Added temporary code to identify harmonics specified for each user-requested frequency.

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=794

     
  • David Benn
    David Benn
    2011-08-07

    Apart from the current work towards including harmonics in the analysis, here are some other possible features to implement:

    1. Be able to reset CLEANest, which would entail removing additions to the top-hits table, and remove plot annotations.
    2. Permit addition of frequencies to the set of user-selected frequencies other than locked or variable, assuming these categories won't suffice.

     
  • David Benn
    David Benn
    2012-06-18

    • priority: 9 --> 8