ASTAP version ß0.9.450, 64 bit, dated 2020-11-18 / Linux 64bit
Hello all,
I must be doing something wrong again, can't get an ASTAP stack with my data from last night. Again Siril processed everything without hesitation, warnings or errors. I was out and up in the mountains and the conditions were quite good. I captured the SH2-230 region in Auriga (Aur).
Though advertised for 5kg payload, my camera and lens seems to be a bit too heavy for the SkyWatcher Star Adventurer: Canon EOS 5D4 + Canon EF 1:2.8/300mm L IS USM = almost 4 kg which very often leads to frame shifts over time due to lever action. I have to re-frame again and I have to adjust everything by hand with no technical support, just my eyes, my hands and red torch.
Nevertheless, I took 3 imaging series of 60x60s (180 min). Though knowing that there are a couple of really bad frames and more or less "bumps" / shifts between the series, Siril made something really useful out of it with the provided, very basic OSC-preprocessing script. I know from some first success I had ASTAP could do better and more precise stacking ... but again with this data set it runs for long just to finally tell me:
...
...
17:54:38Addinglightfile: 142-144"/foto/2020/11/18.11.20-Astro-IC405-NGC1893/lights/298A6252.fit dark compensated to light average. Using 30 dark(s), 52 flat(s), 65 flat-dark(s)17:54:42Creatingmonochromaticx2binningimageforsolving/staralignment.
17:54:44Notenoughquadmatches<3orinconsistentsolution, skippingthisimage.
17:54:53Addinglightfile: 143-144"/foto/2020/11/18.11.20-Astro-IC405-NGC1893/lights/298A6134.fit dark compensated to light average. Using 30 dark(s), 52 flat(s), 65 flat-dark(s)17:54:56Creatingmonochromaticx2binningimageforsolving/staralignment.
17:54:59Notenoughquadmatches<3orinconsistentsolution, skippingthisimage.
17:55:07Addinglightfile: 144-144"/foto/2020/11/18.11.20-Astro-IC405-NGC1893/lights/298A6225.fit dark compensated to light average. Using 30 dark(s), 52 flat(s), 65 flat-dark(s)17:55:11Creatingmonochromaticx2binningimageforsolving/staralignment.
17:55:13Notenoughquadmatches<3orinconsistentsolution, skippingthisimage.
17:55:25Calculatingpixels σ oflightfile1-144/foto/2020/11/18.11.20-Astro-IC405-NGC1893/lights/298A6056.fitUsing30dark(s), 52flat(s), 65flat-dark(s)17:55:50Combining1-144"/foto/2020/11/18.11.20-Astro-IC405-NGC1893/lights/298A6056.fit", ignoringoutliers. Using30dark(s), 52flat(s), 65flat-dark(s)17:56:12Adjustingcolourlevelsassetintab"stack method"17:56:16Applyingcolour-smoothingfilterimageassetintab"stack method". Factorsaresetintabpixelmath117:56:26 █ █ █ Savingresult144as/foto/2020/11/18.11.20-Astro-IC405-NGC1893/lights/', 2020-11-18, 1x0L _stacked.fitsFinishedin2287sec. TheFITSheadercontainsadetailedhistory.
Attached are some screen shots with my settings. I also had to enter FOV to receive a single solution ....
From the screenshot I can see that ASTAP detects more then 4000 stars in each image. So stacking should work flawlessly. I suspect that the reference image, so the image with the lowest HFD is bad and spoiling the stack.
Can you inspect the first image used for stack by the log. You could find it also by double click on the HFD column and inspect the image with the lowest HFD value by double click on it. If it is a poor image, remove it by rename by the popup menu to " * .bak"
The FOV is not required for stacking using star alignment.
If it doesn't work, you could share some images for further investion?
Clear skies,
Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I first let ASTAP analyze and organise the images. Next is sort the results by HFD and check a couple of images (in this case) below 4.0 to find the "threshold" where the star in the images start to show significant shift, blur or combined errors. Then I select and uncheck everything below this list position.
As I leave the list ordered by HFD value, best first, I assume this would be the order ASTAP takes for stacking - that is: starting with the best ones and stopping before the really bad ones.
How can I get ASTAP to write a log file to dis for easier investigation?
CS and stay healthy and well,
Tom
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I assume it now works for you. Yes it takes the image with the best HFD as reference. Maybe I should add an other condition if enough stars are detected. For my setup this phenomena occurs typically when the image list contains an image with tracking problems. The streaks are reported with an low HFD but typically with a low star count detection. So manually I also check the number of stars detected in each image and inspect/view them.
That process can probably automated. Something like take lowest HFD but under the condition that at least the mean amount of stars is detected. I will work on this probably this afternoon. Keep you bad images for some further testing
About the log. You can check mark "write log file" but that is the same log you already see. You can select option "Show extended solver log" but that is more for debugging by me.
Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Unfortunately not. I only documented in my message what I did before starting the stacking process. This was to show you that I am not doing something completely wrong and/or useless. Obviously, a stacking process should start with the best images as a reference. It would not make sense to stack coincidently bad frames in the beginning - how should the algorithm know whether much better frames are yet to come!
In addition, for every single try I remove all intermediate files (FITS) and stacks. I clean the whole project apart from my RAWs and let ASTAP start from scratch just to ensure a clean and well defined environment and avoid having some mess of working and not working files mixed in the project directories. Narrowing down user (or software) errors works only by reproducible steps and clean environments.
About the log.
I read that somewhere on the docs page. But where does ASTAP stores it? I could not find by find . -iname "*log*" in the project directory or in /tmp. I would like to look into it while the processing is still running which may take very long ... and occasionally delivers a pure white L file as the final result. Looking into the log file (tail -f log or grep ... log or less log plus searching within) would be an efficient option. I can't scroll back in the "live log" because it immediately jumps back to the end when a new step starts or ends.
Tom
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In this version I have changed the selection of the reference image from lowest hfd to best quality equals stars_detected/hfd. This should work better. The reference image is also indicated by a thumb-up icon Try this version.
If this doesn't work for you. could you upload your images somewhere or share them with Google drive so I can have a look.
The log is the same as displayed in the bottom. It is only saved after the stack completion. So it is intended to look at if you leave the program unattended with the option shutdown after finishing.
The log is written at the location of the input images. Same with the stack result.
Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This morning, before you wrote this and I was thinking about the problems last night, I made new flats because I had none for that ISO setting. I used available ISO 100 flats instead.
We (you and me) had talked about a week or so ago that ISO shouldn't matter for flats as the noise pattern is already available in the darks. To exclude any unsureness, I made new ones under today's very evenly lit gray sky. Not necessary I believe but nice to have more good ones, anyway.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In this version I have changed the selection of the reference image from lowest hfd to best quality equals stars_detected/hfd.
... this is indeed helpful. Thank you for the fast update!
After rethinking what happens last night when I couldn't get a useful result - but a complete white 1L output instead - after 1:40 hours processing time - I recall I had unknowingly two instances of ASTAP running simultaneously. One ASTAP instance was still opened on a another of my 4 (virtual) desktops. I am not sure at which point in time I killed that other ASTAP process but probably while the current (2nd instance) was in the middle of its processing.
Suggestion: As I read elsewhere in this forum that others also had trouble using more than one instance of ASTAP at the same time, I suggest that on startup you may check and inform the user if "ASTAP is already running in another instance. Please terminate the other instance first".
The current processing was successful, by the way, very accurate details. This is what I knew ASTAP would be capable of and thus expected it. Thank you for your great support!
Stay healthy and well,
Tom
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Good it works. Normally the lights and darks should be a pair in exposure, temperature and ISO setting. Same for flats and flat-darks. The star detection is working on images where dark is applied. Wrong dark could break the star detection.
There is an detection for an other ASTAP still in memory. But it allows a ASTAP solving and an ASTAP stacking/viewer at the same time. Maybe I missed something in that routine for Linux.
My idea for image quality (=nr stars detected/hfd) is now fully implemented in version 0.9.452 just released 5 minutes ago. I'm happy with it. Sorting on the new quality column works effective. Also the automatic un-selection of images with poor tracking works better now.
Han
Last edit: han.k 2020-11-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Many thanks for your help and reply. I have had big problems using Astrotortilla. So in this experiment I only wanted to test ASTAP for plate solving, not to take pictures, although at the end of the series of plate solving I did take 20 lights and darks of M 110.
So if I understand correctly, because M 31 is so big and occupies the entire FOV of the camera, it is very hard to plate solve since there is no real "dark" area to draw polygons using stars. Is that correct?
Good it works. Normally the lights and darks should be a pair in exposure, temperature and ISO setting. Same for flats and flat-darks. The star detection is working on images where dark is applied. Wrong dark could break the star detection.
There is an detection for an other ASTAP still in memory. But it allows a ASTAP solving and an ASTAP stacking/viewer at the same time. Maybe I missed something in that routine for Linux.
My idea for image quality (=nr stars detected/hfd) is now fully implemented in version 0.9.452 just released 5 minutes ago. I'm happy with it. Sorting on the new quality column works effective. Also the automatic un-selection of images with poor tracking work better now.
Most problems with previous ASTAP version was with global clusters. The
Gaia contain a big portion of the global cluster stars but they are so
close that an earth based telescope can not separate them. This should
be mostly fixed in the last ASTAP versions. Deep space galaxies
including M31 should be no problem because the don't contain many
visible MilkyWay stars.
I downloaded ASTAP-0.9.452 and tried my SH 2-230 data set. The little reference crown is really nice and usefull as it not only marks the best single frame but also takes care to eventually use it as the first (reference) image for processing.
Good idea - thank you!
Concerning lights and darks: Of course darks have to be taken with the same settings as the lights but with the lens covered/closed. I always take darks immediately after the lights. As DSLR lenses are closed optical systems, I don't need to take flats out there in the mountains. I comfortably take them next morning together with the biases.
Also the automatic un-selection of images with poor tracking works better now.
While I am currently trying ASTAP-0.9.452 I noticed that only one exceptionally bad image (HFD=6.2) became unselected though I had entered the threshold value 4.2 in the "untick worst images" field manually. Does this mean that ASTAP finds my HFD=4.2 and worse images still good enough?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The threshold value is in standard deviation, not in HFD. The threshold should be typical 3 or lower to filter anything. For you it is probably better to use % which will be correctly interpreted. Select 95% for example.
I will also look if I can give a warning if an other solving ASTAP is in memory.
Clear skies, Han
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ASTAP version ß0.9.450, 64 bit, dated 2020-11-18 / Linux 64bit
Hello all,
I must be doing something wrong again, can't get an ASTAP stack with my data from last night. Again Siril processed everything without hesitation, warnings or errors. I was out and up in the mountains and the conditions were quite good. I captured the SH2-230 region in Auriga (Aur).
Though advertised for 5kg payload, my camera and lens seems to be a bit too heavy for the SkyWatcher Star Adventurer: Canon EOS 5D4 + Canon EF 1:2.8/300mm L IS USM = almost 4 kg which very often leads to frame shifts over time due to lever action. I have to re-frame again and I have to adjust everything by hand with no technical support, just my eyes, my hands and red torch.
Nevertheless, I took 3 imaging series of 60x60s (180 min). Though knowing that there are a couple of really bad frames and more or less "bumps" / shifts between the series, Siril made something really useful out of it with the provided, very basic OSC-preprocessing script. I know from some first success I had ASTAP could do better and more precise stacking ... but again with this data set it runs for long just to finally tell me:
Attached are some screen shots with my settings. I also had to enter FOV to receive a single solution ....
Regards,
Tom
Last edit: Tom Oskar Ortleb 2020-11-19
Hello Tom,
From the screenshot I can see that ASTAP detects more then 4000 stars in each image. So stacking should work flawlessly. I suspect that the reference image, so the image with the lowest HFD is bad and spoiling the stack.
Can you inspect the first image used for stack by the log. You could find it also by double click on the HFD column and inspect the image with the lowest HFD value by double click on it. If it is a poor image, remove it by rename by the popup menu to " * .bak"
The FOV is not required for stacking using star alignment.
If it doesn't work, you could share some images for further investion?
Clear skies,
Han
Hello Han,
I first let ASTAP analyze and organise the images. Next is sort the results by HFD and check a couple of images (in this case) below 4.0 to find the "threshold" where the star in the images start to show significant shift, blur or combined errors. Then I select and uncheck everything below this list position.
As I leave the list ordered by HFD value, best first, I assume this would be the order ASTAP takes for stacking - that is: starting with the best ones and stopping before the really bad ones.
How can I get ASTAP to write a log file to dis for easier investigation?
CS and stay healthy and well,
Tom
I assume it now works for you. Yes it takes the image with the best HFD as reference. Maybe I should add an other condition if enough stars are detected. For my setup this phenomena occurs typically when the image list contains an image with tracking problems. The streaks are reported with an low HFD but typically with a low star count detection. So manually I also check the number of stars detected in each image and inspect/view them.
That process can probably automated. Something like take lowest HFD but under the condition that at least the mean amount of stars is detected. I will work on this probably this afternoon. Keep you bad images for some further testing
About the log. You can check mark "write log file" but that is the same log you already see. You can select option "Show extended solver log" but that is more for debugging by me.
Han
Unfortunately not. I only documented in my message what I did before starting the stacking process. This was to show you that I am not doing something completely wrong and/or useless. Obviously, a stacking process should start with the best images as a reference. It would not make sense to stack coincidently bad frames in the beginning - how should the algorithm know whether much better frames are yet to come!
In addition, for every single try I remove all intermediate files (FITS) and stacks. I clean the whole project apart from my RAWs and let ASTAP start from scratch just to ensure a clean and well defined environment and avoid having some mess of working and not working files mixed in the project directories. Narrowing down user (or software) errors works only by reproducible steps and clean environments.
I read that somewhere on the docs page. But where does ASTAP stores it? I could not find by
find . -iname "*log*"
in the project directory or in/tmp
. I would like to look into it while the processing is still running which may take very long ... and occasionally delivers a pure whiteL
file as the final result. Looking into the log file (tail -f log
orgrep ... log
orless log
plus searching within) would be an efficient option. I can't scroll back in the "live log" because it immediately jumps back to the end when a new step starts or ends.Tom
Hello Tom,
I have uploaded ASTAP v0.9.451 at:
http://www.hnsky.org/astap_amd64.deb
In this version I have changed the selection of the reference image from lowest hfd to best quality equals stars_detected/hfd. This should work better. The reference image is also indicated by a thumb-up icon Try this version.
If this doesn't work for you. could you upload your images somewhere or share them with Google drive so I can have a look.
The log is the same as displayed in the bottom. It is only saved after the stack completion. So it is intended to look at if you leave the program unattended with the option shutdown after finishing.
The log is written at the location of the input images. Same with the stack result.
Han
It is also possible there is something wrong with the darks or flats. Try to stack without them.
Han
This morning, before you wrote this and I was thinking about the problems last night, I made new flats because I had none for that ISO setting. I used available ISO 100 flats instead.
We (you and me) had talked about a week or so ago that ISO shouldn't matter for flats as the noise pattern is already available in the darks. To exclude any unsureness, I made new ones under today's very evenly lit gray sky. Not necessary I believe but nice to have more good ones, anyway.
Hello Han,
... this is indeed helpful. Thank you for the fast update!
After rethinking what happens last night when I couldn't get a useful result - but a complete white
1L
output instead - after 1:40 hours processing time - I recall I had unknowingly two instances of ASTAP running simultaneously. One ASTAP instance was still opened on a another of my 4 (virtual) desktops. I am not sure at which point in time I killed that other ASTAP process but probably while the current (2nd instance) was in the middle of its processing.Suggestion: As I read elsewhere in this forum that others also had trouble using more than one instance of ASTAP at the same time, I suggest that on startup you may check and inform the user if "ASTAP is already running in another instance. Please terminate the other instance first".
The current processing was successful, by the way, very accurate details. This is what I knew ASTAP would be capable of and thus expected it. Thank you for your great support!
Stay healthy and well,
Tom
Good it works. Normally the lights and darks should be a pair in exposure, temperature and ISO setting. Same for flats and flat-darks. The star detection is working on images where dark is applied. Wrong dark could break the star detection.
There is an detection for an other ASTAP still in memory. But it allows a ASTAP solving and an ASTAP stacking/viewer at the same time. Maybe I missed something in that routine for Linux.
My idea for image quality (=nr stars detected/hfd) is now fully implemented in version 0.9.452 just released 5 minutes ago. I'm happy with it. Sorting on the new quality column works effective. Also the automatic un-selection of images with poor tracking works better now.
Han
Last edit: han.k 2020-11-21
Many thanks for your help and reply. I have had big problems using Astrotortilla. So in this experiment I only wanted to test ASTAP for plate solving, not to take pictures, although at the end of the series of plate solving I did take 20 lights and darks of M 110.
So if I understand correctly, because M 31 is so big and occupies the entire FOV of the camera, it is very hard to plate solve since there is no real "dark" area to draw polygons using stars. Is that correct?
Best Regards,
Anthony Catalano
acatalano@techagroup.com
Landline: 303-444-1744
Mobile: 303-475-7912
Websites: www.boulderwx.comhttp://www.boulderwx.com/, www.boulderweather.orghttp://www.boulderweather.org/, www.apcat.orghttp://www.apcat.org/
From: han.k han59@users.sourceforge.net
Sent: Saturday, November 21, 2020 4:30 AM
To: [astap-program:discussion] general@discussion.astap-program.p.re.sourceforge.net
Subject: [astap-program:discussion] Whatever I do: Not enough quad matches <3 or inconsistent solution, skipping this image.
Good it works. Normally the lights and darks should be a pair in exposure, temperature and ISO setting. Same for flats and flat-darks. The star detection is working on images where dark is applied. Wrong dark could break the star detection.
There is an detection for an other ASTAP still in memory. But it allows a ASTAP solving and an ASTAP stacking/viewer at the same time. Maybe I missed something in that routine for Linux.
My idea for image quality (=nr stars detected/hfd) is now fully implemented in version 0.9.452 just released 5 minutes ago. I'm happy with it. Sorting on the new quality column works effective. Also the automatic un-selection of images with poor tracking work better now.
Han
Whatever I do: Not enough quad matches <3 or inconsistent solution, skipping this image.https://sourceforge.net/p/astap-program/discussion/general/thread/ab203e7411/?limit=25#7f46
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/astap-program/discussion/general/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
Hello Anthony
Most problems with previous ASTAP version was with global clusters. The
Gaia contain a big portion of the global cluster stars but they are so
close that an earth based telescope can not separate them. This should
be mostly fixed in the last ASTAP versions. Deep space galaxies
including M31 should be no problem because the don't contain many
visible MilkyWay stars.
Clear skies, Han
Last edit: han.k 2020-11-21
I downloaded ASTAP-0.9.452 and tried my SH 2-230 data set. The little reference crown is really nice and usefull as it not only marks the best single frame but also takes care to eventually use it as the first (reference) image for processing.
Good idea - thank you!
Concerning lights and darks: Of course darks have to be taken with the same settings as the lights but with the lens covered/closed. I always take darks immediately after the lights. As DSLR lenses are closed optical systems, I don't need to take flats out there in the mountains. I comfortably take them next morning together with the biases.
While I am currently trying ASTAP-0.9.452 I noticed that only one exceptionally bad image (HFD=6.2) became unselected though I had entered the threshold value 4.2 in the "untick worst images" field manually. Does this mean that ASTAP finds my HFD=4.2 and worse images still good enough?
Hello Tom,
The threshold value is in standard deviation, not in HFD. The threshold should be typical 3 or lower to filter anything. For you it is probably better to use % which will be correctly interpreted. Select 95% for example.
I will also look if I can give a warning if an other solving ASTAP is in memory.
Clear skies, Han