Question about "Statistics within rectangle"
Stacking program, astrometric (plate) solver, fits file viewer
Brought to you by:
han59
Menu
▾
▴
I am trying to understand a descrepancy in the "Statistics within rectangle" information. I've beeen comparing the max (or peak) value in a star as reported by different apps.
Star 1:
ASTAP: 63,096 (1 x)
VPhot: 69,799
SiriL: 69,799
AstroImageJ: 69,799
Star 2:
ASTAP: 36,020 (1x)
VPhot: 39,847
Siril: 39,846.5
AstroImageJ: 39,846.5
First, I'm curious what the 1x means after ASTAP's max value (M)? In some cases I've seen 4x and I think even 12 x.
Second, why does ASTAP not agree with the others? This is not always the case and I have see cases where ASTAP does agree. Is this due to different definitions of "Max"?
Third, on Star 1, how are they getting a value of 69,799? I thought the value had to fit in 16 bits giving a max of 65,535. However, in the FITS header it has:
BITPIX = -32 / number of bits per data pixel
This is what ASTAP put there after stacking some images. The original unstacked images had:
BITPIX = 16 / number of bits per data pixel
Can you please explain what is going on?
Hello Billy,
That difference would be weird. ASTAP is clearly the outlier. I checked an image with AstroimageJ and the "peak"value reported is the same as the M value in ASTAP. ASTAP doesn't subtract the background. So it doesn't provide something like "value"as in AstroImageJ. Could that be the explanation? Would a new reported "value" above background help like M-x̄ where x̄ is the median value? But is it useful information? The flux is much more useful.
The number behind indicates the how many times it is found. This becomes more useful near saturation of the image.
If you stack 16 bit images you have to store the pixel values in floating point values. So the -32 FITS format. Otherwise you will loose information. If you stack two pixels e.g with values (1000+1001)/2 you end up with 1000.5. You don't want to lose that accuracy/resolution and store it as 1000.5 and not as 1001.
Han
Han,
The reason this is important to me is that I'm doing photometry. I actually got into the question of peak (or max) value of a star is because I wanted to make sure my exposures that didn't yield ADU values that exceeded the linearity range of the sensor and didn't saturate the sensor.
I have been using CloudMakers software to capture images. They had a bug in their implementation of peak value of a star that I helped them track down. Their's did subtract the background and also did some averaging. Apparently Pixinsight reports it in a similar way.
For some stars ASTAP does give the same value as VPhot, SiriL and AstroImageJ. Not sure why it doesn't in this case.
I understand why you shift to floating point for a stacked image. What I don't understand is how the averaged values are greater than 65,535. If I look at the unstacked subs for that star, they are saturated at 65,535. Averaging shouldn't result in 69,799.
So anyway, wha I think I'm interested in if the raw peak value of a star. No averaging over the FWHM or subtrating the background. I just need to know if the exposure is valid for photometry.
Send me a sample FITS image where it goes wrong and identify the wrongly measured star by the x,y position or a marked screenshot. And tell me watother program measure. I will compare it with AstroImagej which has a intuitive interface. If there is a bug in ASTAP, it has to be fixed :)
Yes saturatation should not occur. That's why the raw value M or >64E3 (greater then 64000) are there.
Values greater then 65535 should normally not occur. In principle they doesnt' matter for floating point stacks. It can go much higher without any loss of information. Note some programs rearrange the values to 0..1.
In ASTAP values above 65535 can occur when colour correction are applied but that will never be done for photometry. Secondly applying a flat can pull up the corner values of saturated pixels above 65535. So flats raise the corners with 30%
Furthermore a " pedestal" value of 500 (or 1000?) is added if you apply dark to prevent that you go below 0.
Han
[Edited to correct the URL]
You can find the images at:
http://www.otherwise.com/ASTAP/S_Boo.zip
The OldStackedImage.fits is one I had previously stacked and processed in VPhot. The NewStackedImage.fits is one I stacked today and it uses slightly different calibration file (I recently updated them).
I have included the subs and calibration images so you can repoduce the stacking if needed. ASTAP rejected the 005.fits file and it wasn't included in the stack.
I have looked at the old and new stacks using a variety of programs. I looked at S Boo at (1536, 1064) and the brighter star at (2027, 830). For ASTAP I selected arectangle around the stars and used the "Statistics within rectangle" feature to measure them (the "M"
Newly stacked Image
...............................................
S Boo at location (1536, 1064)
ASTAP: 36,505
SiriL: 39,923.8
AstroImageJ: 39,923.8
Bright Star (2027, 830)
ASTAP: 64,565
SiriL: 70,612.4
AstroImageJ: 70,612.4
Old Stacked Image
......................................
S Boo at location (1536, 1064)
ASTAP: 39,924
SiriL: 39,846.5
AstroImageJ: 39,846.5
VPhot: 39,847
Bright Star at location (2027, 830)
ASTAP: 63,096
SiriL: 69,799.3
AStroImageJ: 69,799.3
VPhot: 69,799
In all cases ASTAP was an outlier. I also found that I somehow got different results at different times although the rectangle never included other pixels that should have had a greater Max.
As I mentioned before, I'm not sure how ASTAP's "average" stacking is yielding values greater than 65,535. I don't think I like your adding the "pedestal" value. That seems to be messing with the data. How can you do careful photometry if you add something just because you are applying darks? Maybe since this is differential photometry it really doesn't matter.
Last edit: Bill Tschumy 2022-09-09
This was very puzzeling but I found the cause. The maximum value in the image is 71673. When ASTAP reads a floating point FITS image it rearrange all values to max 65535. So 39,846x65535/71673 gives 36505. The reason is that some program like PI produce files with range 0..1 and others much more then 65535. This is not abnormal even ASTAP produces files a little above 65535 in the corners due to the flat correction for the corners. This is difficult to avoid but should be no problem for photometry. In principle it is impossible to measure saturation in a stacked image due to the flat correction. Saturation test should be done in the raw images.
The pedestal for stacking is impossible to avoid. All programs do it. This is due to the noise in the image. Lets assume your dark has a mean value of 500 and noise level of 50 pixels (standard deviation. So the dark background in the dark varies up to 3xSD with the noise from maybe 350 to 650. That same applies for the light. Noise is random. So if the sky is superblack your light will have values between 350 up to 65535. If you subtract the dark from the light you could end up with a value 350 (light) - 650 (dark) resulting in pixel value of -300 so negative pixel values. The sky glow helps to lift the light background values but not always enough. In other words for a perfect dark sky the mean value for a light minus dark is zero. So the resulting (dark corrected) light will have a mean value of zero and the noise will fluctuate around this zero. This you don't want so have to apply a pedestal.
Both topics don't influence the photometry. I have thought of relaxing the the above correction from 0..65535 to maybe 0..75000 but then the histogram has to adapt and more problems occurs. This is all caused by the flat correction.
p.s. the pedestal is automatic and is not applied if the images have a background value above 500. Else the mean light background is raised up to 500. It is an automatic protection against negative values.
Han
I had an other look. I think it is save to relax the rearranging. I did in the past the same for 0..1 range files. That only becomes active if the maximum value is above 2. PI and APP program also produce files with pixel values outside the range 0..1.
So ASTAP will only rearrange if the pixel values are exceeding 65535 * 1.5 so about 98000. Then ASTAP will produce the same measured pixel values as the other programs.
Tomorrow I will release a new Mac version with this mod.
Cheers, Han
S boo will produce then the attached statistics:
Han
Very interesting. Thanks for looking into this.
In all the AAVSO docs that I have seen, they are always careful to tell you to avoid saturation or going outside the linearity range of the sensor. They also tell you how important it is to calibrate your images with darks and flats.
What they don't mention is that you should check for saturation and linearity before calibrating. I will ask the AAVSO folks about that and see if they agree this should be spelled out in the docs. Now that you mention it, it seems obvious that you have to do the before calibration. I have just never thought about it much.
As far as the pedestal, aren't you supposed to set the bias in the camera so negative values don't happen? Maybe folks don't do that and so you have to?
Thanks for the informative lesson. I learn more each day about this.
Yes it should be made very clear in the docs.
ASTAP will refuse/ignore lights which are saturated. After dark subtraction the saturation level should be reduced with the dark bias/pedestal. For uncalibrated lights I set it at 64000 and for calibrated at 60000 so pretty conservative.
The bias/pedestal values in the light and dark are the same. If the light receives almost no light then it will be like a dark. Then applying the dark on the light (so subtracting) the results is the value zero. The original bias/pedestal value is nullified. Noise in the light and dark will be unrelated. So the resulting image will show noise around zero. You get the same effect as when you subtract a dark from a dark or a dark minus master dark.
ASTAP version 2022.09.10 is now available for the Mac. Tell me if it works as expected,
Cheers, Han
Han,
Yes, the new version matches the other programs now. Thanks so much!
Could you explain what the 1x, 4x etc means after the M (Max) value in the statistics panel?
This indicates the number of times the value occurs. You probably will only see a value above 1 when there is saturation.