Canon CR2 > ASTAP > FITS conversion – FITS Header errors
Stacking program, astrometric (plate) solver, fits file viewer
Brought to you by:
han59
Menu
▾
▴
Hi Han,
when having trouble stacking my data with ASTAP and getting unexpected (bad) results, I usually look into my data – because often enough the error sits in front of the screen ...
Yesterday I discovered some wrong FITS header data after ASTAP's conversion from Canon CR2 raw to FITS:
FOCALLEN = "000...2159" >>> should be "135"
CAMERA = "000...135" >>> should probably be "Canon"
BAYERPAT = "GRBG" >>> should be "RGGB"
These are important values and I fear wrong setting irritate ASTAP when processing!?
Best regards,
too
Hello Tom,
The keyword FILT-PAT comes from the included RAW conversion program unprocessed_raw. It is a modified version of the original Libraw unprocessed_raw. The modified version can create directly a FITS file.
BAYERPAT is not written by ASTAP if I remember well. But it is the same as FILT-PAT and wrong for Cannon.
To check what wrong I would need one of your .CR2 files. Can you attach one? Then I can trace what went wrong and if it is in the Libraw code or in my unprocessed_raw modification.
I have tested a .CR2 file here. And Bayer pattern looks okay. See attached.
Regards, Han
Sorry for the confusion concerning CFA pattern – actually the Canon EOS 7D used for this image set has a GBRG CFA. Obviously, I never looked it up for this body an my two other Canon DLSRs have what I had in mind: an RGGB CFA.
But what about the false FOCALLEN setting – it is correct in the RAWs (135mm f/2.0)? Seems that FOCALLEN's true value is stored as CAMERA value (000...135).
Last edit: Tom Oskar Ortleb 2021-10-21
The focallen is just coming out of the Libraw routine. I have no idea how the DSLR knows it but I assume it only works with smart Cannon lens.
Best is to check one of your raws with a utility raw-identify of Libraw and check if it reports the same. If this utility is reporting correctly, I assume I can correct it in the modified conversion program
Usage:
raw-identify -v 1.cr2
Then you get a lot of data like:
https://www.libraw.org/download
Or just give me a raw and I will check it here with the Windows version of Libraw
Regards, Han
Didn't know I had raw-identify already on board – thanks Linux! – and thus learned something new today. Here is the output of one of the Canon 7D RAWs in question:
Strangely raw-identify core dumped (last line) ....
Regards,
Tom
**EDIT: ** Sent you Download link to this exact same RAW (IMG_7151.CR2)
Last edit: Tom Oskar Ortleb 2021-10-21
Hello Tom,
Okay Libraw works :)
Now is the question why the modified unprocessed_raw conversion is making 2159 from 135 mm.
Maybe I have to update it to the latest Libraw code. Can you attach one raw here or upload it somewhere like
https://ufile.io/
and give me the link? Then I can use it for testing.
Han
Ok have received the link to the CR2. Thanks.
Last edit: han.k 2021-10-21
Fixed. I have uploaded to Github the Linux executable. Will add it the next days to ASTAP.
https://github.com/han-k59/LibRaw-with-16-bit-FITS-support/tree/master/bin
download
https://github.com/han-k59/LibRaw-with-16-bit-FITS-support/blob/master/bin/unprocessed_raw-astap%20linux%20amd64.tar.gz
Cheers, Han
Here is the Linux package updated:
http://www.hnsky.org/astap_amd64.deb
or
http://www.hnsky.org/astap_amd64.tar.gz
Hello Han,
thanks for your great and fast support once more!
I installed the updated the ASTAP deb package and put unprocessed_raw-astap in /usr/local/bin. First I tried 4 of the Canon 7D RAW files, one of which you got from me yesterday. After "Analyze and organise images" the image dimensions differ from the true dimensions:
In ASTAP the Width and Height columns report: 5202x3464 ... which is a bit more.
Don't know if this might have side effects but I would feel safer when these values were identical.
Best regards,
Tom
chronological overlap, this belongs here >> https://sourceforge.net/p/astap-program/discussion/general/thread/b0b2e045c1/#e20e/759f
Last edit: Tom Oskar Ortleb 2021-10-22
Yes the dimensions reported are pretty weird. In the Windows photo-viewer I get also 5184x3456, but comparing with the ASTAP result their about 9 pixels missing on both sides. But these pixels are fully functional and contain imaged stars. So it would be a waste not to use them. :) You can even extract more pixels but they are black and used for black level reporting but they confuse ASTAP background detection.
You can extract all sensor pixels as follows:
unprocessed_raw-astap ./IMG_7151.CR2
or
unprocessed_raw ./IMG_7151.CR2
This creates a .pgm file you can open in ASTAP. See attached. A assume/guess the bright pixels on the top contain some secret magic binary code.
Only the unprocessed_raw-astap modified can generate fits files. That avoids the intermediate .pgm conversion step.
Regards, Han
Thumb size: 5184 x 3456
Full size: 5360 x 3516
Raw inset, width x height: 5184 x 3456 left: 168 top: 56
Aspect: Other
Image size: 5202 x 3464
Output size: 5202 x 3464
Image flip: 0
Canon record mode: 6, CR2
SensorWidth = 5360
SensorHeight = 3516
SensorLeftBorder = 168
SensorTopBorder = 56
SensorRightBorder = 5351
SensorBottomBorder = 3511
Seems raw-identify interpretation is irritating or accessing the imaging dimensions via an unsuitable tag:
I believe the tag Raw inset describes the imaging area best: 5184 x 3456. This is the reported dimension all image processing software (darktable, Gimp, XnView, EOM, gwenview, ...) show as well as exiftool and others. This is also Canon's official technical information for the Canon EOS 7D.
The tag Thumb size irritates as a thumb[nail] image is usually the small JPEG preview image embedded in the RAWs. raw-identify reports it the same size as the full imaging dimension.
So, the cleanest (and clearest) access would probably be via the Raw inset tag – which already and correctly implies what it is referring to.
Hope this helps,
Tom
Last edit: Tom Oskar Ortleb 2021-10-22
I updated the LibRaw library to the master. It was a little struggle with GitHub. It doesn't change anything in unprocessed_raw for you.
It was also a struggle to change the C language code. At the moment I don't want to change it. All the pixels in range 5202x3464 are working and functional and I hadn't any problem with unprocessed_raw program since March. So I want to keep it for the moment. Other things to do.
Regards, Han
I certainly can live with the bigger pixel dimensions. Just wanted to make sure no side effects (or worse) could appear due to different dimensions.
In ASTAP, the extended pixel dimension (seem to) produce transparent red and blue borders around the stacked final image. How can we get rid of it and use the pixels covered by these borders?
Sidenote: Today I learned something new: raw-identify and unprocessed_raw-astap – thanks for your explanation.
Regards,
too
As a last action I will add this:
Canon makernotes, ChannelBlackLevel: 2047 2047 2048 2048
Highlight linearity limits: 15094 15094 15094 15094
Good for SQM measurement and bitdepth=14
I looked to the following formats:
.CR3
. ARW
. NEF
.PEF
Pentax doesn't support Raw inset. This makes it more complicated.
That should be due stacking and small difference in position and not due to the extra pixels. If not please tell me. To remove I normally use the popup menu option "remove borders".
I have added to unprocessed_raw-astap the pedestal and maximum values so this exercise has a positive result. It will be released with next version Now I will focus on some other improvements.
Regards, Han