From: Arend L. <la...@us...> - 2003-11-19 08:42:32
|
> (that said, I do not want to begin each file with 100 #includes). I don't mean to be rude, but if you would need 100 includes in a C(++) file you're doing something seriously wrong. That strategy might work if you work on relatively small programs, but if you're really into something the size of Qt, you're asking for trouble if you don't manage your dependencies. And in the end that's what programming is all about: Dependency management. A good system is completetely modular and isolates dependencies. If you ever want to do something cross platform, including "windows.h" all over the place is lethal. So, for example, rather than include "windows.h" all over, you should write your own file handling classes or fuctions and make one header file that defines your file handling functions, but does not include anything but perhaps <string.h>. Then, in your implementation you include windows.h. That way, you can update the implementation of your file handlers w/o recompiling everything that depends on them. Another nice trick is to make use of forward declarations rather then include header files, so: --- class MyConstantlyChanging; void doSomething( MyConstantlyChanging& ); class yourClass{ protected MyConstantlyChanging& foo; } --- rather then --- #include "myclass.h" void doSomething( MyConstantlyChanging ); class yourClass{ protected MyConstantlyChanging foo; } --- While the first example costs you a few lines of code, because you have to explicitly construct foo in yourClass' constructor and have to delete it, you have gained a lot in your dependencies. Users of yourClass don't have to know anything about MyConstantlyChanging in order to use yourClass, so they shouldn't have to include "myclass.h" And this is not about just compile time. Writing your code like this, forces you to think about what depends on what, so that in your implementation files there's never more then a few classes that play a role. This reduces the complexity of your code and thus improves readability, maintainability and quality. If you want an example of a large system built that way, you might want to take a look at our system that can be downloaded at opendtect.org and includes the source code. (Our file handling is in filegen.{h,c}) -- Arend -- |
From: Arend L. <la...@us...> - 2003-11-20 10:21:11
|
> Well, I do not know how much you know about GUI development, but even > if this could be lethal, it is really hard to avoid, at least if you > do not want to manually partition windows.h and/or copy portions of > windows.h to your headers/sources. > (I consider both to be extremly bad idea). I know my way around in Qt, but I must admit my knowledge of the Windows API is limited. However, we went trough quite a lot of trouble in order to avoid polluting our code with Qt header files. I wrote a whole subsystem on top of Qt, which you can find in the uiBase directory. That is the only place where Qt header files are included. > Problem is in details - imagine that you e.g. have class that has > something to do with icons. That will very likely be a class included > in any > code that uses GUI. And you very likely will need HICON interface in > this class - and you have "windows.h" in any code... I would make a class myIcon that has only the constructors I'm interested in, make a forward declaration for a myIconPrivate class: class myIconPrivate; class myIcon { public: myIcon( const char* filename); protected myIconPrivate impl; } Then, in the implementation file for myIcon, you can include whatever you like, be it something from Qt or from the win32 api. We have a class named ioPixmap (pixmap.h), which can also contain icons. In that file, I did not declare a private class, but forward declared Qt's QPixmap, which is not as clean as it could be. > > Another example - any GUI application will use very large set of >widgets. So I do think it is a good idea to provide a single include that >provides classes for all often used widgets and include it at once rather >than manually select dozens of include files for concrete widgets used. Of course, it depends on your application, but my experience is that most windows/dialogs only need a few buttons and user input fields. For input fields, we have a class uiGenInput, which I think has a very interesting approach. Rather than specifying a widget that should be used, we specify _what_ we want to input. For example, if I need an integer input, I don't really care wether it is implemented using a spinbox or a line edit. I just tell the uiGenInput I need an integer input and let the uiGenInput decide what widget is best suited to do that. That way, I don't have to include the widget's header files all over the place. BTW, >such a GUI application will compile slow using both individual and >group-header approach without precompiled headers, but will compile fast >with precompiled headers AND group-header. That is why I think precompiled >headers and group headers is the best option. > > You will also need to include most template libraries in all code. (like >STL). And so on. > > >>> Another nice trick is to make use of forward declarations rather then >>> include header files, so: >>> >>> --- >>> class MyConstantlyChanging; >>> >>> void doSomething( MyConstantlyChanging& ); >> > Great, as long as you do not change doSomething declaration. Then it > might become link-time errors nightmare (or, in plain C, runtime > nightmare). > > I strongly believe there should be single declaration of entity in all > project files... Doing otherwise is dangerous, as it deffers problems > to link time - in better case. I don't really care about link-time problems. As long as the problems are not deferred to run-time, I think it's perfectly ok. How often would you have to change a forward declaration for a class anyway? > > >>> While the first example costs you a few lines of code, because you have >>> to explicitly construct foo in yourClass' constructor and have to delete >>> it, you have gained a lot in your dependencies. >>> Users of yourClass don't have to know anything about >>> MyConstantlyChanging in order to use yourClass, so they shouldn't have >>> to include "myclass.h" > > > Well, I basically regret this approach, because what you do here is > that you are doing compiler's work. I rather want machine to do work > for me. I see it as just another way of hiding the implementation details. Hiding details is one of the cornerstones of Object Oriented development. Every textbook you open on the subject tells you hiding your details is a good idea. What's the difference between declaring a member method and forward declaring a class containing implementation details? By the way, this approach is also recommended in Scott Meyer's "Effective C++", item 34 "Minimize compilation dependencies between files". Shorter, online version: http://cpptips.hyperformix.com/cpptips/min_file_dep > >>> d this is not about just compile time. Writing your code like this, >>> forces you to think about what depends on what, so that in your >>> implementation files there's never more then a few classes that play a >>> role. This reduces the complexity of your code and thus improves >>> readability, maintainability and quality. > > > Really ? How ? I think that interfaces should be simple and > dependencies too. That is I can manage dozens of dependencies between > modules, but definitely not hunderds of dependencies between files. The idea is to minimize dependencies. That is, you make _your_ interfaces simple, clean and to the point. And you minimize the dependencies in your interface by minimizing includes from your header files. So, if you include one of your own header files, there should be very few indirect includes. The other part is the modularisation. For example, you can make a Gui base module that declares classes you need in your gui, which is implemented using Qt or Windows widgets. If the only place you include Qt header files is in the implementation files of your Gui base module, there are no dependencies towards Qt in your other modules. (except towards the library, of course..) That makes your dependencies simple. From all other Gui'ish stuff, there are only dependencies towards your own base Gui module. Not towards Qt or whatever toolkit you use. > > >>> If you want an example of a large system built that way, you might want >>> to take a look at our system that can be downloaded at opendtect.org and >>> includes the source code. (Our file handling is in filegen.{h,c}) > > > Well, some files in opendect show exact symptoms I am scared of. E.g. > see file "ui3dapplmain.cpp".... Do you really know what other headers > these 50 includes include indirectly ? Does it really help you to > simplify interfaces ? Well, I checked. (our make system makes a .deps file) Indirectly, there are 209 files included, including system files (of which 51 due to including pthread.h somewhere). Which means, on average, that there are about 4 indirect includes per header file, which is not too bad, I think. Anyway, the example you show is the famous exception to the rule ;) But serious, the example you show contains the main application logic and main window. In other words, it is the file where everything comes together. That's the reason it has so many includes. > > And, BTW, using single header for all files files in that directory >would significatnly speed up compilation with precompiled headers. > It seems logical to mee that if you include less header files, you would also need to include less precompiled header files, which would speed up compilation in that case too. If you include everything trough one big header file, even the slightest change in one of the (indirect) included header files would force a complete re-pre-compile of the precompiled header file. And, since every .cc file is dependent on your big header file, you'd hace to re-compile each and every .cc file in the directory, too. While with my approach, you only need to recompile those .cc files that include the header file you changed. So, I think if you do a complete recompile then pre-compiled headers might help, but during development it helps much more to avoid recompiling as much .cc as possible by minimizing dependencies on header files. BTW, I have my system setup to generate dependencies and do a complete recompile at night, so I always have a fresh set of libraries in the morning. During the day I seldom need a complete recompile, because I can usually delay changes requiring a complete recompile to the end of the day and have the computer sweat during the night. > That said, my system works great as long as precompiled headers are > present. So the most logical thing when 'porting' to Mingw is to > search for similiar feature, is not it ? If not available, I will > have to provide workaround, which seems possible and preliminary tests > show four times speed up for shallow headers (so it will be > significantly more for deep headers). I don't know what exactly motivates your port to Mingw, but I can fully imagine you don't want to change more then absolutely necessary. In fact, the reason I'm working with MinGw rather then MSVC is basically the same. Since my code works perfectly under unix using gcc, I don't feel much like polluting my header files with all the declspec stuff MSVC seems to require in order to make dlls. I also noticed that compiling on win32 is significantly slower then under Linux, but I don't really care. I just develop under Linux and compile infrequently under win32 to check for problems. -- Arend -- |
From: Miroslav F. <cx...@nt...> - 2003-11-20 12:28:35
|
> I would make a class myIcon that has only the constructors I'm > interested in, make a forward declaration for a myIconPrivate class: > > class myIconPrivate; > > class myIcon > { > public: > myIcon( const char* filename); > protected > myIconPrivate impl; > } > > Then, in the implementation file for myIcon, you can include whatever > you like, be it something from Qt or from the win32 api. Works nice as long as you do not need HICON in public interface... That is, if you ever need to extract HICON from icon (and that is sometimes needed even if you create multiplatform apps), there is no simple and clean way how to overcome this problem (well, you can supply another header with "HICON GetHICONFromIcon(const myIcon&), but that will only increase mess). > > Well, I basically regret this approach, because what you do here is > > that you are doing compiler's work. I rather want machine to do work > > for me. > > I see it as just another way of hiding the implementation details. > Hiding details is one of the cornerstones of Object Oriented > development. That is what public is about, is not it ? > Every textbook you open on the subject tells you hiding > your details is a good idea. I agree. But hiding details is not same as obfuscation. Maybe even term "hiding details" is not accurate. What you really need is clean interface definition. That does not have too much to do with pimpl idion (which, BTW, I think is in fact bad idea anyway - it mostly wipes out inheritance as important object oriented tool). > Anyway, the example you show is the famous exception to the rule ;) > But serious, the example you show contains the main application logic > and main window. In other words, it is the file where everything comes > together. That's the reason it has so many includes. Yes, I noticed that too. OTOH, I have seen a bunch of other files with likewise heavy includes... Mirek |
From: Wu Y. <ad...@ne...> - 2003-12-05 08:46:49
|
I was really wondering how did you achieve the 375% speed boost of building. Will you share it here? Best regards, Wu Yongwei --- Original Message from Miroslav Fidler --- Anyway, I definitely WAS succesful changing our GCC build process so that it compiles a whole bunch of files in single go - and it resulted of real 4 times speedup in large rebuilds. It is still not as fast as MSC is - MSC can use the same technology too, giving it another 25% advantage. Anyway, total compile time dropped from 45 minutes to about 12 minutes, which is something I can live with. (MSC dropped from 4 minutes to 3). |
From: Miroslav F. <cx...@nt...> - 2003-12-05 09:26:40
|
> I was really wondering how did you achieve the 375% speed boost of > building. Will you share it here? In the end, it is rather primitive. The problem of build speed is caused by long headers. GCC parsing header is faster than compiler parsing implementation + generating debug code, but still pretty slow. Now if average file size is about 1000 lines, then average number of header lines is at least 10000 lines (some of out high-level files include as much as 100000 lines of headers). So it is pretty clear that headers make build slow. OTOH, most files include pretty much same headers. And most headers are prevented against multiple includes by #ifdef/#define guards. So what I do is to determine whether all headers included by specific file in group of files are "guard locked" (by scaning headers - I need to scan them in our builders code anyway to determine header dependencies). For files that are approved (for staying at simple level, let us consider 'clean' build for now) I form something I call 'BLITZ' batch, that is file that looks #include "x:/path/sourcefile1.cpp" ...... #include "x:/path/sourcefilen.cpp" and compile it instead of individual files. Now compiler has to parse headers just once - and awesome speed gain is here... There are of course caveats. Some of them are serious (like defining static function with same name in two files), some can be cured (like #define macros in file - I scan files for such macros and place #undefs to BLITZ batch). To solve more complicated cases, I have also introduced #pragma BLITZ_PROHIBIT and #pragma BLITZ_APPROVE.... Another problem is of course unclean build - that is, rebuilding just one file. For that I have added some heurestics which basicaly work by excluding files which you work on from BLITZ batch. Well, sometimes it means that when you start work on a single file, blitz batch has to be recompiled too (even if files are otherwise unchanged), but in the long run this is not a dominant problem. Mirek |
From: Miroslav F. <cx...@nt...> - 2003-11-19 11:55:26
|
> > (that said, I do not want to begin each file with 100 #includes). > > I don't mean to be rude, but if you would need 100 includes in a C(++) > file you're doing something seriously wrong. > That strategy might work if you work on relatively small programs, but > if you're really into something the size of Qt, you're asking for > trouble if you don't manage your dependencies. > > And in the end that's what programming is all about: Dependency management. > > A good system is completetely modular and isolates dependencies. If you > ever want to do something cross platform, including "windows.h" all over > the place is lethal. Well, I do not know how much you know about GUI development, but even if this could be lethal, it is really hard to avoid, at least if you do not want to manually partition windows.h and/or copy portions of windows.h to your headers/sources. (I consider both to be extremly bad idea). Problem is in details - imagine that you e.g. have class that has something to do with icons. That will very likely be a class included in any code that uses GUI. And you very likely will need HICON interface in this class - and you have "windows.h" in any code... Another example - any GUI application will use very large set of widgets. So I do think it is a good idea to provide a single include that provides classes for all often used widgets and include it at once rather than manually select dozens of include files for concrete widgets used. BTW, such a GUI application will compile slow using both individual and group-header approach without precompiled headers, but will compile fast with precompiled headers AND group-header. That is why I think precompiled headers and group headers is the best option. You will also need to include most template libraries in all code. (like STL). And so on. > Another nice trick is to make use of forward declarations rather then > include header files, so: > > --- > class MyConstantlyChanging; > > void doSomething( MyConstantlyChanging& ); Great, as long as you do not change doSomething declaration. Then it might become link-time errors nightmare (or, in plain C, runtime nightmare). I strongly believe there should be single declaration of entity in all project files... Doing otherwise is dangerous, as it deffers problems to link time - in better case. > While the first example costs you a few lines of code, because you have > to explicitly construct foo in yourClass' constructor and have to delete > it, you have gained a lot in your dependencies. > Users of yourClass don't have to know anything about > MyConstantlyChanging in order to use yourClass, so they shouldn't have > to include "myclass.h" Well, I basically regret this approach, because what you do here is that you are doing compiler's work. I rather want machine to do work for me. > d this is not about just compile time. Writing your code like this, > forces you to think about what depends on what, so that in your > implementation files there's never more then a few classes that play a > role. This reduces the complexity of your code and thus improves > readability, maintainability and quality. Really ? How ? I think that interfaces should be simple and dependencies too. That is I can manage dozens of dependencies between modules, but definitely not hunderds of dependencies between files. > If you want an example of a large system built that way, you might want > to take a look at our system that can be downloaded at opendtect.org and > includes the source code. (Our file handling is in filegen.{h,c}) Well, some files in opendect show exact symptoms I am scared of. E.g. see file "ui3dapplmain.cpp".... Do you really know what other headers these 50 includes include indirectly ? Does it really help you to simplify interfaces ? And, BTW, using single header for all files files in that directory would significatnly speed up compilation with precompiled headers. That said, my system works great as long as precompiled headers are present. So the most logical thing when 'porting' to Mingw is to search for similiar feature, is not it ? If not available, I will have to provide workaround, which seems possible and preliminary tests show four times speed up for shallow headers (so it will be significantly more for deep headers). Mirek |