For the math you need 3 quads minimum. But if you have 500 stars and find only three matching quads, it is likely a false detection. So if you extract less then 200 quads, 3 matching quads or more are acceptable. If you extract from the image between 200 and 500 quads, 6 matching quads are the minimum. If more then 500 quads are extracted except only 10 matches or more.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
To be honest, I don't like that the result is so reliant on number of chosen stars. As mentioned earlier, I want to experiment with magnitude-based selection to ensure that only stars of e.g. comparable brightness are connected rather than just nearest ones (which might be arbitrarily dim, and result in lots of mismatched polygons).
Connecting based on brightness would be closer to what humans did too with constellations, and likely much more reliable regardless of number or magnitude limit of chosen stars, but will require playing around to find good heuristics.
I'll post here if I come up with something working.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Unlike previous steps, it's neither under "image" nor under "star database". d1 is an unscaled value (longest edge of the polygon) in both cases, so it's not clear which side should be used.
It's still not clear what to use as a d1reference - is it an arbitrary polygon's edge? That doesn't seem to make sense statistically, but I'm not sure what kind of choice is expected here?
If I do choose d1 from database polygons, and d1reference as arbitrary match, it doesn't amount to the correct solve. Here's an Excel spreadsheet where I played with the data returned by a blind search: https://1drv.ms/x/s!At-E4rfQkyDyzp1n_DU7nDa0PJC-YA?e=aV0fbp
The correct solves should be around Ra +12h, Dec +15° (the area around M100), and, as you can see, there are quite a few matches there, but calculating by the method above points to row 93 (RA 09h04m07s Dec -40°26'48") as a better solution.
I suspect the issue is one of the 2 in the list above, but I'm not sure what it is...
Last edit: Ingvar Stepanyan 2021-03-06
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Instead of d1_found/d1_reference it would be clearer to write d1_image/d1_database.
d1_image will be d1 length in in pixels
d1_database the d1 could be expressed in arcsecs.
So if the image pixels scale is 2"/pixel, and d1_image is 10 pixels then d1_image/d1_database ratio for all matching quads should be around 10 pixels/20" so 0.5. So maybe you find for the matches the following ratios
0.5, 0.51, 0.49, 0.505, 3.1 , 0.499
The mean will be about 0.5. The standard deviation maybe 0.1 and the outlier 3.1. So throw the match with ratio 3.1 away.
I will look later to your spreadsheet.
Han
Last edit: han.k 2021-03-06
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
In my program the ratio is d1_database/ d1_image, so in the example you get as median ratio 2.0 equals the number of arc second per pixel. I also use the median rather then mean since it work better with big outliers and small number of sample (quads).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I updated the linked spreadsheet to calculate median of DB / img distances and sorted by deviation from said median again, but the results seem even more scattered.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Going ahead, I want to note that I could probably just ask user to specify a starting point for the search and limiting after certain number of found polygons like ASTAP does, which would likely help a lot with those extreme outliers (they simply wouldn't be reachable) but I'm hoping to find some robust filter that would work even for a completely blind search like in the dataset above.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Looking to the list the most common value seems around 7.74416E-11. See attached graph. Does this match with your pixel scale?
It looks like your previous filtering is not strict enough. I have the tolerance for matching quids distances at 0.007 pixel. So the 5 distances should match within 0.007 pixel
Yep, that is the right value and rows matching it are the correct solve.
I can probably make initial matching stricter - right now it's considering any polygons within 1% of difference along every edge as equal - but, regardless, it seems surprising that median is not choosing the correct value that is "visually" the most frequent already. Technically that's not what the median does and maybe I just need to try other functions for selecting a peak, it's just frustrating when it's so close and you can already see the right result as the most common in the dataset 😅
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The median will only work if the correct values are in the majority. In your list they are a minority.
I found the 7.74416E-11 by just looking to the list. You could find the most common value by rounding the values and just count how often this rounded value exist. E.g. multiply with 1E12 such that round(7.74416E-11 * 1E12) => 77 Then you will find 77 is most common value. But it is not so robust I guess. You could experiment.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yeah that's how I initially verified that the correct value is most frequent - by using Excel's histogram and gradually reducing buckets - but I feel the added complexity in actual code will bring more problems than it solves.
I'll experiment with other methods.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ok reduced to 0.5% for now. Lost some valid matches, but at least median matches the right value now and overall "SNR" improved a lot - 74% of correct matches vs previous 29% for this image.
Last edit: Ingvar Stepanyan 2021-03-06
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I assumed the error is only caused by the star detection. So it is not depending on the distance but the same for all distances and expressed in pixel fraction.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Right, I can see the appeal of going that route (and I saw it described in the algorithm), but so far staying "pixel-less" and using relative values on the DB side seems to have more pros than cons - in particular, it allows me not to care about input FOV during matching altogether, and only calculate it now, during this step 7.
The only downside is that I can't as quickly throw away obvious mismatches where absolute sizes vary wildly between the image and the polygons, but so far it seems worth it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Update: got to the point where I have a transformation matrix, and can use it to get (almost) correct celestial coordinates out of image coords - e.g. for M100 it returns 12h22m55s +15°49'17" which is pretty close given all the inaccuracies in the test image and the process overall.
In theory, if I operated on FITS, the next step would be calculating position & orientation and encoding them in the FITS file, but perhaps for regular images I'd want to instead output an annotated version with notable objects, Astrometry-style. Just something to think about UX-wise till the next time :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Update: got to the point where I have a transformation matrix, and can use it to get (almost) correct celestial coordinates out of image coords - e.g. for M100 it returns 12h22m55s +15°49'17" which is pretty close given all the inaccuracies in the test image and the process overall.
Congrats! Nice to see so much progress. Are you releasing the code?
I think you've asked before, but yeah - I'm still tinkling with it and doing some major & dirty changes all the time, but I'm planning on releasing once it's ready.
For now, it appears to work on the sample image with hardcoded consts, but doesn't work on other images yet :D
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For the math you need 3 quads minimum. But if you have 500 stars and find only three matching quads, it is likely a false detection. So if you extract less then 200 quads, 3 matching quads or more are acceptable. If you extract from the image between 200 and 500 quads, 6 matching quads are the minimum. If more then 500 quads are extracted except only 10 matches or more.
Ah, that makes sense, thanks!
There are empirical values. Maybe I should put it in as a function.
Maybe something like minimum := 3 + nr_quads div 67, but I like to test that. This was put in very early in the code.
To be honest, I don't like that the result is so reliant on number of chosen stars. As mentioned earlier, I want to experiment with magnitude-based selection to ensure that only stars of e.g. comparable brightness are connected rather than just nearest ones (which might be arbitrarily dim, and result in lots of mismatched polygons).
Connecting based on brightness would be closer to what humans did too with constellations, and likely much more reliable regardless of number or magnitude limit of chosen stars, but will require playing around to find good heuristics.
I'll post here if I come up with something working.
That will be an interesting experiment.
I will fix my code as follows:
minimum_quads:=3 + nr_quads div 100;
Okay, I'm still struggling with the same step 7 from https://www.hnsky.org/astap_astrometric_solving.htm.
d1
is an unscaled value (longest edge of the polygon) in both cases, so it's not clear which side should be used.d1reference
- is it an arbitrary polygon's edge? That doesn't seem to make sense statistically, but I'm not sure what kind of choice is expected here?If I do choose
d1
from database polygons, andd1reference
as arbitrary match, it doesn't amount to the correct solve. Here's an Excel spreadsheet where I played with the data returned by a blind search: https://1drv.ms/x/s!At-E4rfQkyDyzp1n_DU7nDa0PJC-YA?e=aV0fbpThe correct solves should be around Ra +12h, Dec +15° (the area around M100), and, as you can see, there are quite a few matches there, but calculating by the method above points to row 93 (RA 09h04m07s Dec -40°26'48") as a better solution.
I suspect the issue is one of the 2 in the list above, but I'm not sure what it is...
Last edit: Ingvar Stepanyan 2021-03-06
Instead of d1_found/d1_reference it would be clearer to write d1_image/d1_database.
d1_image will be d1 length in in pixels
d1_database the d1 could be expressed in arcsecs.
So if the image pixels scale is 2"/pixel, and d1_image is 10 pixels then d1_image/d1_database ratio for all matching quads should be around 10 pixels/20" so 0.5. So maybe you find for the matches the following ratios
0.5, 0.51, 0.49, 0.505, 3.1 , 0.499
The mean will be about 0.5. The standard deviation maybe 0.1 and the outlier 3.1. So throw the match with ratio 3.1 away.
I will look later to your spreadsheet.
Han
Last edit: han.k 2021-03-06
Ah yes, that makes more sense and I actually tried it too, but it didn't result in the valid selection either.
In my program the ratio is d1_database/ d1_image, so in the example you get as median ratio 2.0 equals the number of arc second per pixel. I also use the median rather then mean since it work better with big outliers and small number of sample (quads).
I updated the linked spreadsheet to calculate median of DB / img distances and sorted by deviation from said median again, but the results seem even more scattered.
Going ahead, I want to note that I could probably just ask user to specify a starting point for the search and limiting after certain number of found polygons like ASTAP does, which would likely help a lot with those extreme outliers (they simply wouldn't be reachable) but I'm hoping to find some robust filter that would work even for a completely blind search like in the dataset above.
Looking to the list the most common value seems around 7.74416E-11. See attached graph. Does this match with your pixel scale?
It looks like your previous filtering is not strict enough. I have the tolerance for matching quids distances at 0.007 pixel. So the 5 distances should match within 0.007 pixel
Yep, that is the right value and rows matching it are the correct solve.
I can probably make initial matching stricter - right now it's considering any polygons within 1% of difference along every edge as equal - but, regardless, it seems surprising that median is not choosing the correct value that is "visually" the most frequent already. Technically that's not what the median does and maybe I just need to try other functions for selecting a peak, it's just frustrating when it's so close and you can already see the right result as the most common in the dataset 😅
The median will only work if the correct values are in the majority. In your list they are a minority.
I found the 7.74416E-11 by just looking to the list. You could find the most common value by rounding the values and just count how often this rounded value exist. E.g. multiply with 1E12 such that round(7.74416E-11 * 1E12) => 77 Then you will find 77 is most common value. But it is not so robust I guess. You could experiment.
Yeah that's how I initially verified that the correct value is most frequent - by using Excel's histogram and gradually reducing buckets - but I feel the added complexity in actual code will bring more problems than it solves.
I'll experiment with other methods.
Ok reduced to 0.5% for now. Lost some valid matches, but at least median matches the right value now and overall "SNR" improved a lot - 74% of correct matches vs previous 29% for this image.
Last edit: Ingvar Stepanyan 2021-03-06
Updated the spreadsheet, the cutoff point between valid / invalid matches is looking pretty good now - seems to be at around 1%.
I assumed the error is only caused by the star detection. So it is not depending on the distance but the same for all distances and expressed in pixel fraction.
Sorry, which error are you referring to? The difference between your 0.007px threshold and my 0.5%?
To the 0.007 pixel
Right, I can see the appeal of going that route (and I saw it described in the algorithm), but so far staying "pixel-less" and using relative values on the DB side seems to have more pros than cons - in particular, it allows me not to care about input FOV during matching altogether, and only calculate it now, during this step 7.
The only downside is that I can't as quickly throw away obvious mismatches where absolute sizes vary wildly between the image and the polygons, but so far it seems worth it.
Update: got to the point where I have a transformation matrix, and can use it to get (almost) correct celestial coordinates out of image coords - e.g. for M100 it returns 12h22m55s +15°49'17" which is pretty close given all the inaccuracies in the test image and the process overall.
In theory, if I operated on FITS, the next step would be calculating position & orientation and encoding them in the FITS file, but perhaps for regular images I'd want to instead output an annotated version with notable objects, Astrometry-style. Just something to think about UX-wise till the next time :)
Hi Ingvar:
El sáb, 13 mar 2021 a las 21:27, Ingvar Stepanyan
(rreverser@users.sourceforge.net) escribió:
Congrats! Nice to see so much progress. Are you releasing the code?
Greetings,
--
Víctor R. Ruiz
http://rvr.linotipo.es/
I think you've asked before, but yeah - I'm still tinkling with it and doing some major & dirty changes all the time, but I'm planning on releasing once it's ready.
For now, it appears to work on the sample image with hardcoded consts, but doesn't work on other images yet :D