speeding up compile time

rubenvb
2010-04-25
2013-06-06
  • rubenvb
    rubenvb
    2010-04-25

    Hi,

    this may not be the place to ask, but I have recently installed VS2010 (it comes to me for free from Microsoft Dreamspark). I noticed the compile time of Qt (when using jom, which enables a "make -j#" like behavior of nmake) is considerably shorter than with GNU Make. I do not complain about compiler speeds, because I think they may be good enough. I noticed that CPU usage is a lot lower with GCC doing a "make -j3" than with MSVC10 doing a "jom -j3". Parallellization is just better.

    Is there any way that this can be improved? Everything built with gcc uses make as a "backend", but IMHO it's slow compared to an unmaintained utility from Microsoft…

    Any thoughts? Thanks

     
  • Jonathan Yong
    Jonathan Yong
    2010-04-25

    Are you on Linux or Windows when using GNU make. If its MSYS/Cygwin, its expected to be slow, nothing much you could do there.

     
  • rubenvb
    rubenvb
    2010-04-25

    I was using the make included in with sezero's build, just as a plain native program under cmd.exe.

     
  • Jonathan Yong
    Jonathan Yong
    2010-04-26

    I have not benchmarked the compile speeds, possibly gcc is slower because it actually tries to call the backend stuff to do its work. Maybe you could use jom to drive gcc instead of gnu make?

     
  • NightStrike
    NightStrike
    2010-04-30

    Make's ability to parallize efficiently is limited to two basic things: the Makefiles, and the platform.  In the first case, the makefiles might not be very suited to safe parallelization.  Make is extremely careful, and errs on the side of serializing.  jom may not be as picky.  It's a lot easier to parallelize a flat makefile instead of a recursive makefile.  The second issue is due to the tool, how it's compiled, and how it's handling the slow process creation on windows.  It takes long to spawn a process on windows, and so however make is doing it (no idea what's going on under the hood) might not be the most efficient.

     
  • rubenvb
    rubenvb
    2010-05-02

    Yeah, I figured as much. I knew about the process spawning slowness, but figured nmake required the same (although it might use threads instead…).

    The makefiles should be ok, they're generated by qmake (and I would think they optimized the final form)… Rereading that sentence makes me doubt everything I said in it :)

    Anyway, I was just testing msvc2010 and it is PAINFULLY slow. Qt is taking ages to complete. GCC ftw!
    PS: this is not a scientific test, it is me being convinced that mingw-w64 gcc is better than msvc… which in retrospect is quite naive, as MSVC holds the original implementation in everything the compiler has to do :)