Menu

Compiling MTF Mapper on MacOS Mojave

2020-11-09
2020-11-15
  • Thomas Baumann

    Thomas Baumann - 2020-11-09

    I would like to run MTF Mapper on my Mac Pro 5.1 with MacOS 10.14.6 (Mojave), so I tried to compile it from source.

    Based on the Ubuntu build notes, I used MacPorts to install missing packages, and then compiled version 0.7.32 from trunk, see mojave_build_notes.txt in the linked directory below. I got many compile warnings, see mojave_make_log.txt, but nothing serious. So it seems that I succeeded, basically.

    However, there are still issues to be solved.

    • When I start mtf_mapper_gui and open e.g. the image rect.png with or without interactive edge selection (which works fine) and with annotated image selected I get the notice MTF Mapper call failed, indicating that no valid target objects were found.
      I tried different images and threshold settings which didn't help.
      When I do the same from the command line and execute mtf_mapper -a rect.png testdir, it works fine and I get the annotated image.
    • At the end of every mtf_mapper run, after the statistics lines, it always says: Illegal instruction: 4
    • When I click on 'Help - Help' in the main GUI menu I get:
      Could not launch the browser to open help file.
      Try copying the link below, and paste it in your browser.
      file://

      The help button inside manual edge selection works and opens a nice help window.

    Command line mode seems to work great, I also tested graphics output in grid mode.

    What should I try to get file input at the GUI working and how can I help to localize the issues?

    https://www.dropbox.com/sh/swq7g5wg5tz2ki5/AADvOFEskwjlUUSvb3_HGuLua?dl=0

    Regards
    Thomas

     
    • Frans van den Bergh

      Hi Thomas,

      Thanks for the detailed feedback, and the build instructions on Mojave. To answer some of your specific questions:
      1. The GUI code that reports "... indicating no valid target objects were found." is quite new, so it is possible that something else breaks and is incorrectly triggering this error message. One possible diagnostic test is to use the manual edge selection method, because this specific error message should not be possible when using manual edge selection. Could you perhaps run the GUI, once with the normal "File->Open" method, and once with "File->Open with manual ..." method, and send me the GUI log file? (The GUI log file is usually /tmp/mtfmapperlog.txt on Linux, not sure where it will be on Mac OS X, because Qt chooses the path for me. Also note that the log file is only flushed properly when you close the GUI). Anyhow, please send mtfmapperlog.txt to fvdbergh@gmail.com ; maybe I can find some additional clues.
      2. That illegal instruction message sounds rather worrying. I do not have access to a Mac of any description, so the best I could do was to build MTF Mapper with llvm / clang++ 10.0.1 on Linux. This did not reproduce the "illegal instruction" problem. Could you perhaps test on your side if this particular error disappears when you build MTF Mapper in debug mode? The simplest is to go to the "build/bin" directory, and run "make edit_cache". This should give you a text-mode interface where you can set CMAKE_BUILD_TYPE to Debug. Then type "c" to configure again, then "g" to generate new makefiles, and then "e" to exit cmake. Lastly, another "make" should rebuild the MTF Mapper binaries in Debug mode (without compiler optimization flags), which might help. I cannot think of anything else to try right now to isolate the problem.
      3. My Ubuntu build notes are not sufficient to set up the help files (which are just some HTML) correctly. You can access them under "doc/html/mtf_mapper_user_guide.html" in the source tree. Not ideal, but it is a rather fussy process to install and configure the help, and I'd rather avoid going through that process right now.

      Just out of interest, can you install "valgrind" with "port" ? That might help with the illegal instruction problem.

      Regards,
      Frans

       
  • Thomas Baumann

    Thomas Baumann - 2020-11-10

    Hi Frans,

    Thank you for your fast reply.

    1. I did as you wrote, first Open... then Open with manual edge selection.... The error message appears in both cases, in the latter as soon as I click Accept in the manual selection window. The tmp path is shown when you quit the GUI, so that was easy to find. I copied mtfmapperlog.txt and also the two mtfmappertemp_x directories to the debug subdirectory of my dropbox link. Perhaps the line Warning: QObject: Cannot create children for a parent that is in a different thread. gives a hint.
    2. Running mtf_mapper in debug mode was also diagnostically successful:
      $ mtf_mapper -a rect.png test
      8-bit nonlinear grayscale
      Thresholding image ...
      Computing gradients ...
      Component labelling ...
      Assertion failed: (fabs(magnitude[0] - 1.0) < 1e-6), function compute_mtf, file /usr/local/src/workspace/mtfmapper-code/src/mtf_core.cc, line 625.
      Abort trap: 6
    3. Agreed, help is just nice to have.
    4. I tried to install valgrind with MacPorts but did not succeed: Error: valgrind @3.14.0 is only compatible with macOS versions from 10.5 up to 10.13.
      Fortunately, High Sierra (10.13.6) is still available on another partition, and so I compiled mtf_mapper there, and also installed valgrind. The program seems to behave identically under MacOS 10.13, also the make process gave the same warnings. So, yes, valgrind is ready. ;-)

    Regards
    Thomas

     
    • Frans van den Bergh

      Hi Thomas,

      Thanks for all those output files. Judging by the fact that the "log.txt" files in the two MTF Mapper temp output directories were zero-length, it looks like the MTF Mapper CLI program crashed before it could flush/save the log file. But all the other outputs in the temp directories look fine -- I could copy your outputs over those of another GUI session I had running on my machine, and the outputs work as expected. So the crash either happens in the summary statistics code, or right thereafter.

      But there is not much code that can run after the bit that prints out the stats, other than the destructors of a bunch of objects that were still hanging around ...

      Next, some comments on your points 1 and 2:
      1. That warning about the threads is not likely to be the problem, because it has been with us since the first GUI release :) .... and I think that the CLI program crashing is a more likely explanation. But I should probably fix the code that results in that warning, although I have tried at least once in the past.
      2. That failing assertion is unfortunate, and a bit of a red herring this time around. The assertion itself was more of a sanity check that wasn't thought through properly. Unfortunately this breaks your debug build, but we can work around that by adding the "--nosmoothing" option to the CLI, or checking the "Reduced SFR smoothing" option in the GUI preferences.

      So all my hope is currently pinned on valgrind. Could you please send/post the output of:

      valgrind ./mtf_mapper rect.png out --nosmoothing -a -v 2 --esf-model loess -q
      

      With a little bit of luck valgrind will at least tell us where the illegal instruction was encountered. It helps if you run the valgrind command above on a debug build of MTF Mapper, because that should give us source code line numbers. If the debug build no longer crashes (with --nosmoothing), then there is a RelWithDebInfo build type in cmake that should give us most optimizations and source code line numbers.

      Thanking you

      Regards,
      Frans

       
  • Thomas Baumann

    Thomas Baumann - 2020-11-11

    Hi Frans,

    In debug mode on MacOS 10.13.6 and with --nosmoothing the failed assertion is gone and the Illegal instruction: 4 is back. So I ran valgrind on the command you stated. The output didn't look very helpful, therefore I ran it another time with the verbose flag set. It ends with a segmentation fault, but I don't see any line numbers. You find both log files in the debug subdirectory on Dropbox.

    Regards
    Thomas

     
  • Thomas Baumann

    Thomas Baumann - 2020-11-12

    Hi Frans,

    I just want to let you know that according to Google we are not alone with Illegal instruction: 4 on MacOS. Apparently certain combinations of OS and package versions can lead to this error. I will try to get more information and might test other MacPort packages.

    Regards
    Thomas

     
    • Frans van den Bergh

      Hi Thomas,

      Thanks for those valgrind output files. I think the illegal instruction 4 error happens when threads that were created much earlier (in the thresholding code) are destroyed when MTF Mapper starts calling object destructors right before exiting. This is why I like valgrind; it usually points to the start of the trouble, rather than the eventual failure. I am currently trying to get HS running in Virtualbox; if I can get that working I might be able to figure out what is happening here. But there are no "small problems" when threads are involved ...

      regards,
      Frans

       
  • Thomas Baumann

    Thomas Baumann - 2020-11-12

    Hi Frans,

    I have very good news! Your object destructor problem assumption is correct. Similar cases I found online also pointed in the same direction, and I remembered that the make log flagged this warning:

    In file included from /usr/local/src/workspace/mtfmapper-code/src/mtf_mapper.cc:31:
    In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/string:500:
    In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/string_view:176:
    In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/__string:56:
    In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/algorithm:644:
    /Library/Developer/CommandLineTools/usr/include/c++/v1/memory:2335:5: warning: delete called on 'Esf_model' that is abstract but has non-virtual destructor
          [-Wdelete-non-virtual-dtor]
        delete __ptr;
        ^
    

    This explanation I often read: All base classes with a virtual function should define a virtual destructor. If an application attempts to delete a derived class object through a base class pointer, the result is undefined if the base class destructor is non-virtual.

    As a quick test I edited esf_model.h and added an empty virtual destructor in line 44:

     39 class Esf_model {
     40   public:
     41     Esf_model(double alpha=13.7)
     42     : alpha(alpha), w(NYQUIST_FREQ*4, 0.0)  {}
     43 
     44     virtual ~Esf_model(){}
     45 
     46     virtual int build_esf(vector< Ordered_point  >& ordered, double* sampled,
     47         const int fft_size, double max_distance_from_edge, vector<double>& esf,
     48         Snr& snr, bool allow_peak_shift=false) = 0;
    

    and the Illegal instruction: 4 error at the end of the CLI run of mtf_mapper is gone!

    Then I did a make in Release mode and now the GUI works!!! :-)

    Regards
    Thomas

     
    • Frans van den Bergh

      Hi Thomas,

      Thanks a lot for finding the bug for me! My wife quipped that compiler warnings are more like a compilation progress bar :)

      I tried to clean up the MTF Mapper code a bit anyway. I actually found another virtual destructor bug in the chromatic aberration plotting code. The rest of the warnings that I fixed seemed harmless enough, but I still have some testing to do just to make sure I did not break anything in the process.

      I am still working on getting HS running in a virtualbox on the back-burner, mostly because I want to see what valgrind was complaining about (hopefully, a false positive).

      Anyhow, glad you are up and running with the GUI and everything.

      Regards,
      Frans

       
  • Thomas Baumann

    Thomas Baumann - 2020-11-15

    Hi Frans,

    today I compiled the current code revision 671 on MacOS 10.14 and all warnings are gone, except the harmless a2x ones - the PDFs are generated just fine! The GUI even found the three external executables. I am so glad, and your wife will be, too. Just show the nice short compiler log file to her (I moved the old one into the debug subdir).

    I adapted the build notes, feel free to use them. I could not test with MacOS 10.15 on my old hardware, perhaps someone else will give it a try.

    Anyway, thank you for this great tool and stay healthy!

    Regards
    Thomas

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.