We've not spoken in a while! I hope you had a relaxing Christmas and New Year?
And of course, Happy New Year to you! :-)
I was wondering if you are interested in testing your brain cells? :-)
As you may know (or not), Ivo has introduced a new focusing algorithm in to APT called InvPower. The way it works is to analyse the whole sky, rather than stars, using Fourier transforms. So it doesn't need to see any stars to work. Indeed bright stars tend to get in the way of the algorithm, and their affect can only be countered by using very, very short exposures. For example for HFD measurements I use 3s 4000gain exposures. For InvPower I use 0.25s 400gain exposures (about 1/120 of the exposure for HFD).
Unfortunately InvPower doesn't work with Sky Simulator. I assume this is because Sky Sim only changes the focus of the stars (both in the Artificial and Downloaded images) and the background "sky" doesn't change.
So, is there a way to give the background "sky" something that would equate to a "focus"? The only thing I can think of is to deliberately introduce some level of "noise" to the background. Such that the better the focus the less the noise.
As APT seems to prefer working with the Artificial images (it won't platesolve the downloaded images), then (as far as APT is concerned anyway) a good place to start is with the artificial option.
Of course, I have no idea if this would work. But if it could it would be extraordinarily useful.
Any thoughts?
Thanks, Jim
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The background of the images generated by the simulator is steady simple noise. The image output depth is currently limited to 8 bit since I use something like Timage to draw lines and text. The stars can be drawing in 16 bit. But currently the bit depth is a little limited to 8 bit or 255 grey levels.
I have no problem to look to measurement of Fourier transformations. But to simulate the sky accordingly, I have to understand the math behind it. Is it somewhere described what the algortihm is measuring? A assume it measures something like sharpness.
Cheers, Han
Last edit: han.k 2022-01-13
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As the image becomes out of focus, high frequency features of the image will tend to become blurred and the cooresponding high frequency power spectrum characteristics will decrease in magnitude
However, I can't access the "The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms" .pdf document.
I don't know if this helps?
Cheers, Jim
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The fourier transformation is clear to me. In CCDCiel using a high frequency detection for focusing on planets and Moon/Sun
For artificial sky mode:
In principle the APT algorithm should work with the simulator. One possible problem is that in the simulator due to the 255 grey levels, I keeps the star levels high when defocusing. This was done to prevent the stars fade too soon in the noise. The frequency shift will be there but the total flux signal increases when defocusing. So APT should look to the signal ratio high/low frequencies and not to the signal of high frequencies only.
An other possibility is to convert the simulator output to 16 bit depth and let the stars behave more normal when defocusing. Then the question comes up if the ascom camera simulator handle 16 bit.
DSS image based sky:
The DSS download is 8 bit. 16 bit is also possible but then I have to download them as fits files. Not as gif or jpeg.
Do you see any difference in using invPower with the artificial mode and DSS download modes?
I can have a look to APT first to see whats happening. (In a few days).
Cheers, Han
Last edit: han.k 2022-01-13
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In the v3.90 (full release) version of APT, InvPower won't work with the Artificial sky, But it does work (to a degree) with the DSSR2 downloads. Interestingly it doesn't work with the DSS downloads.
As mentioned before, the InvPower algorithm has a problem with stars, especially bright (saturated) stars. They cause a high peak value when at best focus. This means the focus graph becomes a "w" instead os a "v". The "w" means the hyperbolic focus curve tends to use one side or the other of the "w" leading to an incorrect focus. The only way to overcome this is to use a very, very short exposure, that reduces the height of the "best focus" peak, allowing the hyperbolic curve to ignore the peak and produce a curve using all the points.
With the new 3.91 beta version, InvPower has been modified to try and reduce the affect of bright (saturated) stars, but now it doesn't work with any of the Sky Sim options. All you get is a flat line response.
In a few days version 2.0 of the simulator will be released. SNR will be depending on the exposure time and produced images are of 16 bit resolution. Images with 16 bit resolution process much better.
Han
Last edit: han.k 2022-01-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
SNR will be influenced both for artificial simulation and DSS images. But DSS images are 8 bit so I would not recommend to use them for focussing. The program will use the noise in the ASCOM camera driver. There is the only place where also the exposure is know.
But the same is possible in the Alpaca mode.
Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1) Artificial images have now 16 bit resolution for both ASCOM and ALPACA camera mode.! This is a large improvement. Star detection will work much better.
2) Flux and SNR are now depending on exposure duration. The stars will fade away when exposure time reaches a fraction of a second exposure.
Please report any problem or bug or improvement proposal.
Han
Last edit: han.k 2022-01-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There is a bug in the current version. After installing on an other computer, I noticed the noise is not switched on as expected. I will fix that in version 2.0.0ßb2
Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We are getting somewhere (I think). When focusing with InvPower we do get focusser readings. However, it is producing an inverted curve. So the InvPower value at Best Focus is at it's highest, when it should be at it's lowest. See screen shot.
This happens because the stars are affect the Fourier transform measurement of the background noise. In a real world environment you can reliably fix this by using very short low gain exposures. But with Sky Sim, reducing the exposure (down to 0.001s) doesn't make any difference to the curve.
Thanks for testing. Since the program structure is now ready, I will now focus on the noise and signal math. At this moment the noise is steady and the signal is increased with exposure duration. But in real world the shot noise should increase with the square root of the exposure duration.
More later.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The jpgtmp file is the downloaded DSS image. It is then read and saved to png. When the focusser moves, the jpg is reused for blurring. Only when the mount moves to a new location a new jpg is downloaded (or gif for ESO)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I think I know why the curve was reverse. The peak value of the centre should reduce with a factor 1/(hfd x hfd) To keep out of focus stars visible in 8 bit resultion I changed it to 1/hfd. In the beta version I removed it completely but it should be 1/(hfd x hfd). That should turn the curve :)
I will fix that tomorrow or maybe in the next 15 minutes
For skyview DSS it creates image.tmpjpg
For eso DSS it creates image.tmppng
The result is written image.png
You can delete any of them They will be recreated if required.
Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That was easy to implement. I noted that above 40 hot pixel pixels Invpower start to struggle. I assume a filter could help. Probably in the frequency domain filtering will be difficult but maybe counting the number of star pixels involved could help.
The hot pixels have a random value between 65535.
Note the star flux increases with exposure duration. Gain setting is not yet implemented.
What I'm missing is APT is a backslash setting.
Cheers, Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Han,
We've not spoken in a while! I hope you had a relaxing Christmas and New Year?
And of course, Happy New Year to you! :-)
I was wondering if you are interested in testing your brain cells? :-)
As you may know (or not), Ivo has introduced a new focusing algorithm in to APT called InvPower. The way it works is to analyse the whole sky, rather than stars, using Fourier transforms. So it doesn't need to see any stars to work. Indeed bright stars tend to get in the way of the algorithm, and their affect can only be countered by using very, very short exposures. For example for HFD measurements I use 3s 4000gain exposures. For InvPower I use 0.25s 400gain exposures (about 1/120 of the exposure for HFD).
Unfortunately InvPower doesn't work with Sky Simulator. I assume this is because Sky Sim only changes the focus of the stars (both in the Artificial and Downloaded images) and the background "sky" doesn't change.
So, is there a way to give the background "sky" something that would equate to a "focus"? The only thing I can think of is to deliberately introduce some level of "noise" to the background. Such that the better the focus the less the noise.
As APT seems to prefer working with the Artificial images (it won't platesolve the downloaded images), then (as far as APT is concerned anyway) a good place to start is with the artificial option.
Of course, I have no idea if this would work. But if it could it would be extraordinarily useful.
Any thoughts?
Thanks, Jim
Hello Jim,
Also the best wishes for coming year.
The background of the images generated by the simulator is steady simple noise. The image output depth is currently limited to 8 bit since I use something like Timage to draw lines and text. The stars can be drawing in 16 bit. But currently the bit depth is a little limited to 8 bit or 255 grey levels.
I have no problem to look to measurement of Fourier transformations. But to simulate the sky accordingly, I have to understand the math behind it. Is it somewhere described what the algortihm is measuring? A assume it measures something like sharpness.
Cheers, Han
Last edit: han.k 2022-01-13
Hi Han,
Thanks. :-)
The only documentation I have found is this: https://www.ltrr.arizona.edu/pub/trees/TREESv1/documentation/focustest/index.html
I think the key statement is this:
This webpage in turn refers to this:
https://ieeexplore.ieee.org/document/1161901/references#references
However, I can't access the "The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms" .pdf document.
I don't know if this helps?
Cheers, Jim
The fourier transformation is clear to me. In CCDCiel using a high frequency detection for focusing on planets and Moon/Sun
For artificial sky mode:
In principle the APT algorithm should work with the simulator. One possible problem is that in the simulator due to the 255 grey levels, I keeps the star levels high when defocusing. This was done to prevent the stars fade too soon in the noise. The frequency shift will be there but the total flux signal increases when defocusing. So APT should look to the signal ratio high/low frequencies and not to the signal of high frequencies only.
An other possibility is to convert the simulator output to 16 bit depth and let the stars behave more normal when defocusing. Then the question comes up if the ascom camera simulator handle 16 bit.
DSS image based sky:
The DSS download is 8 bit. 16 bit is also possible but then I have to download them as fits files. Not as gif or jpeg.
Do you see any difference in using invPower with the artificial mode and DSS download modes?
I can have a look to APT first to see whats happening. (In a few days).
Cheers, Han
Last edit: han.k 2022-01-13
Hi Han,
In the v3.90 (full release) version of APT, InvPower won't work with the Artificial sky, But it does work (to a degree) with the DSSR2 downloads. Interestingly it doesn't work with the DSS downloads.
As mentioned before, the InvPower algorithm has a problem with stars, especially bright (saturated) stars. They cause a high peak value when at best focus. This means the focus graph becomes a "w" instead os a "v". The "w" means the hyperbolic focus curve tends to use one side or the other of the "w" leading to an incorrect focus. The only way to overcome this is to use a very, very short exposure, that reduces the height of the "best focus" peak, allowing the hyperbolic curve to ignore the peak and produce a curve using all the points.
With the new 3.91 beta version, InvPower has been modified to try and reduce the affect of bright (saturated) stars, but now it doesn't work with any of the Sky Sim options. All you get is a flat line response.
Cheers, Jim
Last edit: Jim Hunt 2022-01-13
I will play with APT in a few days. Making the simulator image output 16 bit resolution is one of the wishes anyhow and maybe that helps.
But if InvPower only works with very short exposures in the real world is worrying. They better work on the robustness.
Han
Hi Han,
Thanks.
That's is being done. However, there is no problem using very short exposures. It works really well and is, of course, fast! :-)
Cheers, Jim
In a few days version 2.0 of the simulator will be released. SNR will be depending on the exposure time and produced images are of 16 bit resolution. Images with 16 bit resolution process much better.
Han
Last edit: han.k 2022-01-27
Thanks Han,
I'll give it a go as soon as it's released. Will the SNR effect be for both virtual and downloaded images?
Have you tried it against APT or will I be tge first?
Cheers, Jim
Last edit: Jim Hunt 2022-01-27
I haven't looked to APT yet.
SNR will be influenced both for artificial simulation and DSS images. But DSS images are 8 bit so I would not recommend to use them for focussing. The program will use the noise in the ASCOM camera driver. There is the only place where also the exposure is know.
But the same is possible in the Alpaca mode.
Han
First beta version v2.0.0ß1 released.
1) Artificial images have now 16 bit resolution for both ASCOM and ALPACA camera mode.! This is a large improvement. Star detection will work much better.
2) Flux and SNR are now depending on exposure duration. The stars will fade away when exposure time reaches a fraction of a second exposure.
Please report any problem or bug or improvement proposal.
Han
Last edit: han.k 2022-01-27
There is a bug in the current version. After installing on an other computer, I noticed the noise is not switched on as expected. I will fix that in version 2.0.0ßb2
Han
beta2 is released
Hi Han,
I've tested the latest version against APT.
We are getting somewhere (I think). When focusing with InvPower we do get focusser readings. However, it is producing an inverted curve. So the InvPower value at Best Focus is at it's highest, when it should be at it's lowest. See screen shot.
This happens because the stars are affect the Fourier transform measurement of the background noise. In a real world environment you can reliably fix this by using very short low gain exposures. But with Sky Sim, reducing the exposure (down to 0.001s) doesn't make any difference to the curve.
Cheers, Jim
Thanks for testing. Since the program structure is now ready, I will now focus on the noise and signal math. At this moment the noise is steady and the signal is increased with exposure duration. But in real world the shot noise should increase with the square root of the exposure duration.
More later.
Hi Han,
My pleasure. It's the least I can do.
I did notice something else, though not related to focusing.
if I try using g the DSS images, an image.jpgtmp file is created in the folder. but it never changes into a .png file.
cheers, Jim
The jpgtmp file is the downloaded DSS image. It is then read and saved to png. When the focusser moves, the jpg is reused for blurring. Only when the mount moves to a new location a new jpg is downloaded (or gif for ESO)
OK. But it's not then being converted into a. png file for the driver to see it.
So I've seen multiple DSS images downloaded, which are displayed in the GUI. But the image.png file remains the same.
Cheers, Jim
I think I know why the curve was reverse. The peak value of the centre should reduce with a factor 1/(hfd x hfd) To keep out of focus stars visible in 8 bit resultion I changed it to 1/hfd. In the beta version I removed it completely but it should be 1/(hfd x hfd). That should turn the curve :)
I will fix that tomorrow or maybe in the next 15 minutes
For skyview DSS it creates image.tmpjpg
For eso DSS it creates image.tmppng
The result is written image.png
You can delete any of them They will be recreated if required.
Han
Try v2.0.0beta3
I did a quick test. Could not find a backlash setting but this without backlash:
This for the calc backlash. The backlash was set at 100.
Added hot pixels in version 2.0.0ß4
Hot pixels? That's going above and beyond the call of duty! lol
Jim 🙂
That was easy to implement. I noted that above 40 hot pixel pixels Invpower start to struggle. I assume a filter could help. Probably in the frequency domain filtering will be difficult but maybe counting the number of star pixels involved could help.
The hot pixels have a random value between 65535.
Note the star flux increases with exposure duration. Gain setting is not yet implemented.
What I'm missing is APT is a backslash setting.
Cheers, Han