MTFMapper GUI problems depending on filetype and settings

[Tim]

Hi,

I'm just getting my hands on the MTF-Mapper. Great tool first of all!

During testing I encountered some strange issues. As I hope they are not my bad, I would like to report those here as they may help possible bugfixing.

1)
For testing I have the same slanted edge images saved by the Win 7 Snipping Tool in .jpg and .png. Loading the .jpg works flawlessly using the default settings ("focus position" deactivated). But the .png fails with the error message below.
For sure I would be glad to provide the images if needed. Am I doing something wrong or is that an issue with the GUI?

2)
Furthermore when I additionally select "focus position" in the settings then my jpg-image is loaded but the annotation does not work. The same applies for the example image 'lensgrid_tilt_iso400.png'.
Any ideas how to solve that issue?

3)
By the way: I have seen in the documentation that it is possible to export the MTF50 values using the command line tool --> "-r". Is that somehow possible in the GUI as well?

I'm using MTF Mapper version 0.6.13 on Win7 64 Bit.

Thanks and all the best
Tim

"Error Message:
Problemereignisname: APPCRASH
Anwendungsname: mtf_mapper.exe
Anwendungsversion: 0.0.0.0
Anwendungszeitstempel: 5a130334
Fehlermodulname: MSVCR120.dll
Fehlermodulversion: 12.0.21005.1
Fehlermodulzeitstempel: 524f83ff
Ausnahmecode: 40000015
Ausnahmeoffset: 0000000000074a46
Betriebsystemversion: 6.1.7601.2.1.0.256.4
Gebietsschema-ID: 1031
Zusatzinformation 1: d216
Zusatzinformation 2: d2162884373f88a80f58fb3b85609c83
Zusatzinformation 3: d16a
Zusatzinformation 4: d16a3ae3e8dc526e749ae32b82962261"

[Frans van den Bergh]

Hi Tim,

1) Please send me both the .jpg and the .png so that I can see what is happening here. Normally MTF Mapper should work with .png files. Although I usually use only 24-bit RGB, 8-bit grayscale or 16-bit grayscale PNGs. I just checked, and 24-bit RGBA .png files with a transparency/alpha channel do not seem to work.
2) When "focus position" is selected as an output type, then most of the other outputs will not work. I should do something about this in the GUI, i.e., locking out the other output types when "focus position" is selected, or something. For now, what you are seeing is actually the expected behaviour (horrible as it may be).
3) Sort of. The GUI has no way to display this information, but you can add "-r" to the "Arguments" field in the Settings/Preferences dialog, which will cause MTF Mapper to produce the output file "raw_mtf_values.txt" in the usual location where it creates temporary files. On Win7, this is something like c:\users\<username>\AppData\temp\mtfmappertemp_0, where the "0" corresponds to the first file that was processed (and shown in the Data set list in the GUI), and "1" would be the second, etc. Anyhow, you can open or copy these output files while the MTF Mapper GUI is still open; once you close it, it will delete those temporary files. The main problem with the "raw_mtf_values.txt" file is that it does not tell you which MTF50 value corresponds to which edge. The "edge_mtf_values.txt" file (also in the temp directory) will give you that information, and that file will be created even if you do not use the "-r" option. See Section 4.6 (page 30) of the just-released MTF Mapper user guide for a description of the file format.</username>

[Tim]

Hi Frans,

thanks a lot for your detailed answer. This helps a lot.

I'm having a few more questions.

1) The output to the text files is exacly what I'm looking for. Is there any way to define the output directory? I tried it with the --logfile option (p.40 UserGuide), but when I just type --logfile D:...\NewFolder or --logfile "D:...\NewFolder" or something in the arguments field, the MTF Mapper did not produces an output. Am I using an incorrect format or something? (/ \ ' " ?)

2) What is the exact difference between SFR and MTF as you refer to them in your guide ("plot the SFR curve (or the MTF curve, if you prefer)"). I found the following on http://www.imatest.com/docs/sharpness/:
"Modulation Transfer Function (MTF), which is generally identical to Spatial Frequency Response (SFR), can be...". An in your guide it says: "By default, this is an SFR curve, i.e., the DC component is always normalized to 1.0. See --absolute-sfr switch for producing true MTF curves." (p. 42). But I did not really understands what that means?

3) For a switch in units from cycles/pixel to LP/mm one can use the --pixelsize option and "Specify the sensor’s pixel size (pitch) in microns". I'm a bit confused about that: Is it the pixel pitch of the sensor or rather the one of the image? According to formula (1) on p. 36 I understand it is the resultion of the image in microns?

Thanks again and all the best
Tim

ps: I also added the two test images mentioned in my last post, just for general availability :)

btw, for question 1): In the command line tool I got it with e.g:
mtf_mapper test.JPG D:...\NewFolder -aq

[Frans van den Bergh]

Hi Tim,

Apologies for the delayed reply. I'll try to answer your questions:

1. Were you adding the "--logfile file.txt" option to the Arguments field of the Preferences dialog in the GUI? That could explain it, because the GUI already uses the --logfile option, so you cannot currently use it from within the GUI. I just tested it in the command line as

C:\workspace\test>.\mtf_mapper.exe traps4.png out -a --logfile out\testlog.txt

and that worked as expected.

1. Although I am not really an authoritative source on the exact definitions (and thus the difference), it is my understanding that you can only calculate MTF if you know (and remove) the frequency response of the input scene. The "--absolute-sfr" option does not normalize the SFR, hence the overall contrast is reflected in the output SFR with this option enabled. I will have to go back and read what I wrote in the documentation ....
   Amongst the professionals I have encountered the two terms are often used intergchangeably, although this is not strictly correct. I would recommend using the term SFR, since that is the more correct name for what "MTF" Mapper calculates :)

2. Pixel pitch is the distance between the centres of two adjacent pixels in a row. If you assume ideal 100% fill-factor square pixels, then the width of a pixel is the same as the pixel pitch, so I often use the two interchangeably. Even better would be to call it the photosite pitch, since that makes it clear you are talking about the sensor, since that is the object that has physical dimensions. However, if you divide the number of pixels (height of the image in pixels) by the height of the sensor (in mm), then you get the number-of-pixels-per-mm value. Because we have 1000 micron in a mm, the pixel pitch = 1000 / number-of-pixels-per-mm . You could, if you stretch the terminology, refer to the "sensor height" as the "image height", since it should be clear from the context. Clear as mud?

-F

[Tim]

Hi Frans,

thanks a lot, that makes it a more clear to me :).

And yes I used the Arguments field of the Preferences dialog in the GUI..so very clear now.

Just, about your number 2. (and my 3)):
I think I understood the calculation. My question was more about "sensor mm/px" vs "image mm/px". As they are different due to distance, lens and so on my gut instinct was more to use "image mm/px" (e.g.--> make a photo of a 100mm ruler and devide by the pixels within the 100mm distance).

For explanation, what I'm basically trying to do is:
Compare two images taken in the same distance by a similar system (lens and sensor are slightly different). Then I want to see/determine which system produces the best image in terms of sharpness --> its capability to resolve LP/mm (or in this case the slanted edge step response).

It is a common approach in image quality evaluation (sharpness) to use the "sensor mm/px", so the SENSORS pixel pitch, as the basis/reference? For my case (by my humble instinct) I would choose the IMAGE pixel pitch or am I wrong there or missing something? Or does using the SENSORS pixel pitch have a certain advantage which I missing at the moment?

Thanks again and all the best
Tim

[Frans van den Bergh]

Ah, ok, I think I understand what you mean by "image" --- I will call that "object space", or the "scene" here for clarity.

The difficulty with measuring resolution in the "scene" is that you can change the magnification of the scene by moving the camera closer/further. In other words, the only way to "fix" the scale in the scene is to use a test chart of a known physical size. With a bit of effort, you can then deduce the effective magnification, meaning the conversion factor to go from "length in pixels" to "length in mm" in the scene. There is no reason not to do this for your own experiments.

The advantage of using the sensor's pixel pitch is that you normally know the height or width of the sensor (or can make a good guess based on the format of the camera). This means that all resolution measurements can be expressed as, for example, line pairs per mm on the sensor, without requiring a special test chart --- by special I mean like I described above, i.e., you know the physical size of say, a square on your test chart. It also means that your resolution measurement is independent of the distance between the test chart and the camera (within reason, of course).

Both of these measurements (line pairs on sensor, or line pairs on physical test chart) are only meaningful when you consider the final display size (e.g., print size) of your images too. If you measure the resolution as lp/mm on the sensor, and your sensor is 36 mm wide, and you print your image so that it is 300 mm wide, then the effective magnification is 300/36 = 8.33x. So your resolution of, say, 30 lp/mm at the sensor becomes 30/8.33 = 3.6 lp/mm on your printed page (which may be a little bit low, since 3.6 lp/mm is 2x3.6x25.4 ~= 183 DPI). This method of scaling your measured on-sensor resolution to your final print size will work well for your example of comparing to systems.

There is a shortcut: If you express your resolution as line pairs per picture height, then you can directly compare two systems with different sensor sizes --- things get more tricky if their aspect ratios differ, though. You can obtain lp/ph as cycles/pixel X "image height in pixels", e.g., a measurement of 0.28 cycles/pixel becomes 1375 lp/ph on a Nikon D800. If you use the same lens on a Nikon D7000, and you also measure about 0.28 cycles/pixel (both sensors have roughly the same sensor pixel pitch, so this is plausible), you would only get 914 lp/ph, which clearly shows you the D800 would produce a more detailed image. As you can see, the lp/ph measurement naturally compensates for the difference in the format sensor sizes, hence it also compensates for the difference in magnification necessary to obtain the same print size.
(Just be careful with lp/ph; you also get "line widths per picture height", or lw/ph, which is by definition such that lw/ph = 2 X lp/ph)

Anyhow, I hope that helps. Do you think that this answer (about magnification and lp/ph) would be a useful addition to the MTF Mapper user guide?

-F

[Tim]

Hi Frans,

sorrry for the late answer!
Yes, that makes it really perfectly clear for me at the moment! Thank you so much for the detailed explanations and for sure this would be a good addition to the already very useful user guide :). Especially the part with the example of the magnification was very helpful for me and avoids all missunderstandings I think.

Thanks again and all the best
Tim