Menu

#185 CLEANest algorithm

Enhancement
open
Algorithms (37)
8
2013-03-16
2010-04-08
David Benn
No

Should we implement CLEANest and Slick spectra as part of period analysis?

See also https://sourceforge.net/apps/mediawiki/vstar/index.php?title=Fourier_Analysis

Discussion

  • Aaron Price

    Aaron Price - 2010-04-30

    I have used them both in the past. CLEANest was used when working with gaps in the data. SLICK is a "prewhitening" routine that I used to use a lot, but haven't in many years. I think both are more useful when working with long period variables. These could also be possible options for a plug-in framework, if one is implemented. My vote is to put them on the list, but perhaps make it late phase 2 or even a phase 3 thing.

     
  • David Benn

    David Benn - 2010-09-05

    Doug Welch's comments at Citizen Sky 2 suggest that the priority of this should be higher.

     
  • David Benn

    David Benn - 2010-09-05
    • priority: 5 --> 8
     
  • Doug Welch

    Doug Welch - 2010-09-07

    Thanks for increasing the priority of this. As I mentioned to David, there is a Peranso implementation but it does not produce correct or usable residuals (which are important!) and Tonny V has confirmed the bug and also the fact that it won't be fixed anytime soon!

     
  • David Benn

    David Benn - 2010-09-07

    No problem Doug. Now that I'm back on the other side of the pond (just got to Sydney airport from San Francisco) I took a quick look at the ts1201.f Fortran code I'm planning to use. I don't know what the status of that implementation is, but I'll check with Matt T. I assume it's fine though, and should have no problem in translating this for VStar. It may take me a week or so before I start doing much development again, but I'll keep you posted. If you have any particular ideas about presentation of the output of the CLEANest spectrum algorithm, feel free to make suggestions. Without a closer look, I imagine there'll at least be a dialog with a line plot.

     
  • David Benn

    David Benn - 2011-01-21

    A couple of conversations with Doug since this have continued to keep this uppermost. Raising the priority.

     
  • David Benn

    David Benn - 2011-01-21
    • priority: 8 --> 9
     
  • David Benn

    David Benn - 2011-04-21

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=745:

    First implementation of CLEANest that passes a unit test based upon the tcas.dat example scenario provided in the docs accompanying ts12.f. Locked and variable periods have not yet been incorporated.

     
  • David Benn

    David Benn - 2011-04-21

    At this stage, only support for CLEANest from ts12.f is planned, not Slick. That can change however, depending upon demand from end users.

     
  • David Benn

    David Benn - 2011-06-02

    One curiosity I've noticed is that for each CLEANest run for a given DC DFT, the same amplitude value is returned! Is that the case for TS. Why?

     
  • David Benn

    David Benn - 2011-07-13

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=778:

    Finished initial complete CLEANest implementation with variable and locked periods now requested (via dialog); added more unit tests to match new functionality.

     
  • David Benn

    David Benn - 2011-07-19
    • summary: CLEANest and Slick spectra --> CLEANest algorithm
     
  • David Benn

    David Benn - 2011-07-19

    Arising from email discussion with Doug and Grant, Grant recommended that ultimately VStar's CLEANest implementation should permit the inclusion of harmonics per frequency specified for refinement. Here is the last email exchange on the topic, a reply from Grant on July 17 2011 to comments made by Doug and I:

    > > The TS (and VStar) implementation of CLEANest permits these frequencies/periods categories to be specified:
    > >
    > > 1. User selected
    > > 2. Locked
    > > 3. Variable
    > >
    > > So are you saying that we essentially need another category:
    > >
    > > 4. Harmonics
    > >
    > > and that unlike 3. (variable), although these can vary, they must do so such that they maintain their harmonic relationship with their "parent" frequency?

    Yes, I think that describes it quite well

    > Ideally, the category would be "Variable fundamental frequency with
    > integer-value-locked harmonics up to some specified degree".

    There is merit in this description also. The harmonics must be tied to the fundamental, and must be integer multiples of same.

    In my R version, I pass it a list of frequencies (to be refined by CLEANEST) together with a list of harmonic orders. Then it finds the combination of fundamental frequencies which, *together with their integer multiples up to the specified order*, best fit the data.

    For example, if I had identified a frequency of f=0.177 for which I wanted 6 harmonics (harmonics up to 6f), and another frequency f=0.202 for which I only wanted 2 harmonics (f and 2f), applied to data "x" at times "t", I would say

    freq.raw = c(0.177,0.202)
    harmons = c(6,2)
    freq = clean(t,x,freq.raw,nharm=harmons)

    Maybe instead of including a new "type" of frequency, you can simply set a "number of harmonics" parameter for each frequency in the list (with all of them defaulting to 1)

    By the way -- some folks (most in fact) call 6 times fundamental the 5th harmonic, so if you want multiples up to 6 times then you'd have fundamental plus 5 harmonics. I call 6 times fundamental the 6th harmonic, so if you want multiples up to 6 times then you have 6 harmonics. It's nonstandard terminology -- I'm such a maverick! -- but maybe I'll be able to change the world (over the course of several decades) and introduce a new lexicon. But just so you know, when I say up to 6th harmonic I mean f, 2f, 3f, 4f, 5f, and 6f -- whereas most would say that's only up to the 5th harmonic.

    For VSTAR it might be best to go with the more common convention. You could then have check-boxes or a drop-down menu for the number of harmonics *starting with zero* (no harmonics), which is what I would refer to as "starting with 1" (just the fundamental).

    Sincerely,
    Grant

     
  • David Benn

    David Benn - 2011-07-24

    Fixed a bug in which the wrong period was being sent to the phase plot dialog after CLEANest. This lead to a rewrite of period analysis selection message generation which also fixed other bugs relating to selection of rows and plot coordinates. In:

    http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=785

     
  • David Benn

    David Benn - 2011-07-29

    Implemented unified CLEANest parameter dialog including harmonics per user frequency. These are passed to refinement method on period analyis algorithm but not yet implemented in CLEANest implementation.

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=793

     
  • David Benn

    David Benn - 2011-07-29

    Moved CLEANest source files into a "refinement" package and renamed CLEANest dialog to RefinementParameterDialog. Added temporary code to identify harmonics specified for each user-requested frequency.

    In http://vstar.svn.sourceforge.net/viewvc/vstar?view=revision&revision=794

     
  • David Benn

    David Benn - 2011-08-07

    Apart from the current work towards including harmonics in the analysis, here are some other possible features to implement:

    1. Be able to reset CLEANest, which would entail removing additions to the top-hits table, and remove plot annotations.
    2. Permit addition of frequencies to the set of user-selected frequencies other than locked or variable, assuming these categories won't suffice.

     
  • David Benn

    David Benn - 2012-06-18
    • priority: 9 --> 8
     

Log in to post a comment.