#70 Support for the Clang-based Embarcadero compilers

[Vidar Hasfjord]

Support for building OWLNext with the Clang-based compilers in Embarcadero C++Builder/RAD Studio is only partly implemented. Full support should be added and backported to supported release branches.

[Vidar Hasfjord]

Since 6.40.2 the core has built using the "source/owlcore/CBXE3" project file for 64-bit C++Builder (BCC64 in XE3 or later). Similar project files exists for the extension libraries ("source/*/CBXE3"). However, compilation errors in OCFNext and OWLExt remain.

Support for 64-bit C++Builder XE8 was added in OWLMaker Build 3183, allowing 6.40 and later to be built in 64-bit mode using this compiler.

Originally, only the static-ANSI configurations Debug and Release were provided in the project files, but support for more build modes (OWL 5 compatibility mode and Unicode) has been added on the trunk [r3193] [r3191].

TODO

It would be helpful if anyone with access to C++Builder/RAD Studio could help with the following:

1. Check that the latest version of the project files still open in C++Builder/RAD Studio. I have made changes to the project files to support more build modes and automate build setup. These changes work fine when the project files are passed to MSBuild on the command-line, but it is possible that the C++Builder/RAD Studio IDE will reject them due to these changes.
2. Configure the project files so that a precompiled header (PCH) is used to speed up the build. Currently, the build is very slow. The header file "include/owl/pch.h" should be used as the source for the PCH with _OWLALLPCH defined.

[Sebastian Ledesma]

Relating point #1:
I've opened the C:\OWL643\source\owlcore\CBXE3\owl.cbproj in C++Builder XE (relased in 2010) whithout problems. The Project manager shows the 'Win64' platform and declares it 'Unknown platform'. I will repeat the test on Berlin later.

[Vidar Hasfjord]

Hi Sebastian, the 6.43 version of the MSBuild project file, "branches/643/source/owlcore/CBXE3/owl.cbproj", is the unmodified version. My modifications were made to the 7 version of the file, "trunk/source/owlcore/CB/owl.cbproj". Hence, point 1 above relates to the latter file.

In addition to be able to open the file in the IDE, it is of course interesting to know

1. whether it actually can build all configurations successfully or not from the IDE. I have not tested this (all my trials have run out).
2. whether saving the project file from the IDE, overwrites and destroys my hand-crafted changes.

[Sebastian Ledesma]

Yes, I've just noticed.
I've downloaded the https://sourceforge.net/p/owlnext/code/HEAD/tree/trunk/source/owlcore/CB/owl.cbproj
and repited the test in XE. No problem to open it.
I've just saved and the CBuilder XE deleted some nodes related Win64 and OSX.
I will repeat the test on Berlin in a few hours.

[Sebastian Ledesma]

More info:
The .cbproj file generated after opening and saving with C++Builder XE keeped the nodes:

            <OWLNextCompatibility Condition="'$(OWLNextCompatibility)'==''"/>
            <OWLNextUnicode Condition="'$(OWLNextUnicode)'==''"/>

at the <PropertyGroup> node,
but it deleted the nodes:

    <PropertyGroup Condition="'$(OWLNextCompatibility)'=='5'">
        <Defines>OWL5_COMPAT;$(Defines)</Defines>
    </PropertyGroup>
    <PropertyGroup Condition="'$(OWLNextUnicode)'=='u'">
        <_TCHARMapping>wchar_t</_TCHARMapping>
    </PropertyGroup>

[Vidar Hasfjord]

That's unfortunate, but we can alert the users about this — if you need to change the project file, then save a copy, don't overwrite it. Unless you can come up with a solution that preserves the relevant PropertyGroup nodes, of course.

Edit: On second thought, I guess a better solution is to properly define configurations within the IDE for OWL5_COMPAT mode and Unicode, and all combinations thereof. Then you can build all those configurations from the IDE too. Today, I presume, the IDE can only build with the default setting of my hand-crafted variables (OWLNextCompatibility and OWLNextUnicode). With explicit configurations defined for all combinations of build modes, OWLMaker can just pass the requested configuration to MSBuild through the Config and Platform parameters that the IDE also understands.

Of course, there are quite a few possible combinations so it will require manually setting up a whole lot of configurations in the IDE. But I think it is the way to go, if you are up to it. An alternative and perhaps more efficient way to do this would be to manually edit the project file in a text editor.

All combinations of compatibility mode (OWLNext, OWL), debug mode (debug, release), character set mode (ANSI, Unicode) and link mode (static, dynamic) comes to 16 configurations. If you want to include support for single-threaded RTL, it doubles to 32 configurations. That is for each platform. As for naming all these configurations, I suggest using the suffix in our library naming convention, e.g. "5dtui" (OWL5_COMPAT, debug, multi-threaded RTL, Unicode, dynamic linking).

Note that you can use configuration inheritance and option sets (called style sheets in Borland C++ and property sheets in Visual Studio), to combine build mode settings into configurations. Also, the common settings for all configurations can be made in the Base configuration, which is inherited by all configurations. With that, updating settings pertaining to a particular build mode is piece-of-cake, since you can edit the Base configuration or the relevant option set, rather than make the same change in all configurations manually. See Build Configurations Overview.

[Sebastian Ledesma]

Here is the owl.cbroj after just opening and saving with C++Builder Berlin.
It seems that it deletes the conditional nodes not officialy supported.
In example: I didnt install OSX32 platform and it deleted the nodes for OSX32, in other hand it added the nodes for iOS32 and Android.

[Vidar Hasfjord]

That confirms the hand-crafted changes are destroyed after an overwrite by the IDE — if not entirely, at least to the degree of being nonfunctional. So I think the best route to take is to create explicit configurations for all combinations of supported build modes, as described in my earlier post.

Still, there is one issue left unsolved, even with explicit configurations for the build modes, and that is build setup. Build setup requires a pre-build event that calls "update_version_header.bat". The problem is that OWLMaker needs to be able to turn that event on or off, depending on the setting "Perform build setup" in the wizard. Of course, you can add separate configurations with build setup and no build setup, but that doubles the number of configurations again.

Perhaps it can be solved by using a MSBuild user-defined property (variable) in a clever way, since as you have shown, the RAD Studio IDE preserves the user-defined properties (in the top PropertyGroup), as well as their usage in the defintion of other properties (e.g. IntermediateOutput uses OWLNextUnicode in its definition). So maybe this works:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
...
<OWLNextSetup Condition="'$(OWLNextSetup)'==''">true</OWLNextSetup>
</PropertyGroup>
...
<PropertyGroup Condition="'$(Base)'!=''">
<PreBuildEvent>if '$(OWLNextSetup)'=='true' ..\..\update_version_header.bat ..\..\.. ..\version.h ..\..\..\include\owl\version.h</PreBuildEvent>
...

Otherwise, you can just execute the pre-build step unconditionally, and we'll disable the option to turn off "Perform build setup" in OWLMaker for the affected toolsets. Performing unnecessary setup does no harm (it will just report that the version header is already up to date).

[Sebastian Ledesma]

I'm just catching up with the current OWLMaker and MSBuild and perhaps we can pass the OWL5_COMPAT as a command line parameter.
Something like this:
http://stackoverflow.com/questions/166474/msbuild-set-a-specific-preprocessor-define-in-the-command-line

[Sebastian Ledesma]

I was meaning to change the

    <PropertyGroup Condition="'$(Base)'!=''">
...
        <Defines>_OWLPCH;_USE_OWL_CM_EXIT;$(OWLNextCompatibility);$(Defines)</Defines>

and add the $(OWLNextCompatibility) macro to the <Defines> node.
Then invoke the MSBuild with
msbuild /p:OWLNextCompatibility="OWL5_COMPAT"
for building OWLNext with the OWL5 API.

[Vidar Hasfjord]

Yes, we can pass MSBuild properties as parameters to the MSBuild command. That's what OWLMaker already does. You can see the generated command lines on the final page in the wizard. E.g. here is the command line generated by OWLMaker for the default release build:

msbuild CB\owl.cbproj /p:Config=Release;Platform=Win32;OWLNextVersion=7.0.0;OWLNextCompilerVersion=720;OWLNextSetup=true

The two properties Config and Platform are standard for MSBuild and recognised by RAD Studio. So setting these to the requested configuration and platform will work, provided the requested configuration and platform is defined within the project file. This is my suggested ideal solution.

However, since I did not have access to RAD Studio (my trial had run out), I instead created the additional properties and hand-crafted changes to customise the existing configurations within the project file. As you can see, these additional properties are also passed the same way as the standard properties.

[Vidar Hasfjord]

Yes, your use of the OWLNextCompatibility property within the Defines tag should work, and as you have shown, this use should be preserved by RAD Studio.

However, this solution, like my current solution, is not much good for IDE users. So the likelihood is that the user will modify the project file as desired, and in the process destroy the customisations that OWLMaker depend on. So it may not be much better than the current solutions with an added warning not to overwrite the project file.

[Sebastian Ledesma]

Here is the OWL.cbproj with the 'Debug Unicode' and 'Release Unicode' build configurations added.
Both were defined as inherited from standard 'Debug' and 'Release' build configuration.

[Vidar Hasfjord]

Looks good. I recommend explicit configurations for compatibility mode as well. Then it is straight forward to correctly set the target file name for the configuration.

Your assumption that OWLNextCompatibility is defined as "OWL5_COMPAT" (or "") in the Define tag, is not compatible with its use in the IntermediateOutputDir tag, as well in the generation of the target name in the PostBuildEvent tag, where it is assumed to be "5" (or ""). Creating an explicit configuration avoids this complexity.

[Sebastian Ledesma]

Here is the version with explicit configurations:
Debug
Debug_Unicode
Debug_OWL5COMPAT
Debug_OWL5COMPAT_Unicode
Release
Release_Unicode
Release_OWL5_COMPAT
Release_OWL5_COMPAT_Unicode

[Sebastian Ledesma]

I think that by adding the Win32 platform we can use this very OWL.cbproj to support the Embarcadero Free C++ Compiler: https://sourceforge.net/p/owlnext/feature-requests/100/

[Vidar Hasfjord]

Hi Sebastian, it is encouraging to have you working on this! It is the most obvious area of OWLNext where my capacitity for support is limited, since I am not an Embarcadero customer, and hence do not have access to C++Builder/RAD Studio. I have tried to ask Embarcadero for a license in support of OWLNext, but I had no reply. And donations, although much appreciated, have been too few and small for me to feel comfortable investing in a license and the time and effort needed to support these products in OWLNext. Still, I have devoted a fair amount of time to it, as you know.

Here is my advice on how to proceed on this task:

1. Decide which build configurations you want to implement and support.
2. Decide on a reliable and consistent naming scheme for the configurations.
3. Implement the configurations and test them, making sure they work.
4. Ensure output files have the names and location as expected by OWLMaker.
5. Ensure the path for the intermediate files are as expected by OWLMaker.
6. Solve version numbering in the output file names, preferably by automation.
7. Do the same for OCFNext and the extension libraries.
8. Backport the solution to 6.43, if there is demand.

Regarding point 2, I came up with a scheme for Owlet (which uses a Visual Studio MSBuild project file to build). As mentioned in an earlier post, I simply use the suffix of our naming convention. This naming scheme is predictable and easy to use and maintain in OWLMaker since it follows our naming convention. And it allows some clever use of the Config property, as I will explain.

Regarding point 4, in Owlet, with the establised naming convention for the configurations, I can use the property Configuration to define the target name (Configuration Properties | General | Target Name). See "branches/owlet/build/Owlet.vcxproj".I did a quick search on the net, but unfortunately, it seems RAD Studio does not support setting the target name. It seems it is fixed to the project file name. If that is so, you have to use a post-build event to rename it. This is what my current solution does (see PostBuildEvent in the project file).

Regarding point 6, one solution is to rely on OWLMaker to pass the correct version of the library, like my current solution does. However that does not work when building from the IDE. You can set a default in the top PropertyGroup to alleviate the problem. I recommend using "0.0" since it is surely not going to be properly maintained to reflect the real version number. Jogy used to do that across many files, but that was a chore. Instead, we can tell the IDE user to set it. An alternative solution is to make a script that modifies the MSBuild project files as part of the build commands generated by OWLMaker, before invoking MSBuild.

Finally, since I rely on these MSBuild scripts now to do build testing, it would be very helpful if precompiled headers and parallell (multi-processor) compilation, if available, was properly enabled to speed up the build as much as possible. Users would appreciate that too, I am sure.

[Vidar Hasfjord]

I think that by adding the Win32 platform we can use this very OWL.cbproj to support the Embarcadero Free C++ Compiler

Note that the Free C++ Compiler does not come with the MSBuild tool. It only comes with "make", that's why my preliminary work on supporting this compiler provides a make file for this setup.

[Sebastian Ledesma]

HI:
1 - I think that the relevant configurations are:
Debug, Debug_Unicode, Release, Release_Unicode for OWLNext 6.4x and 7.x. Not sure in backport it also to 6.35.x
We dont use the OWL5_COMPAT mode, instead we prefer to use:

#if OWLVersion > xxxx
//code for new  API
#else
//code por old API
#endif

where it's needed. We prefer this way to do the migration path from older versions to newers.
In the other hand i will try to keep the OWL5_COMPAT for compatibility. So for me there is no problem in adding also the Debug_OWL5COMPAT, Debug_OWL5COMPAT_Unicode, Release_OWL5COMPAT, Release_OWL5COMPAT_Unicode configurations.

2 - The OWLNext naming conventions has worked well all this time. My idea is to keep using it. I'm not sure right now if it's needed a specific suffix to indicate a Clang compiled library.

3 - I need a OWLMaker updated for that :)
I think that I can recompile it using C++Builder, but for that I need to move to OWL7.

4, 5, y 6 Agree.

7 Our main application (AIRE.EXE) uses OCFNext, so support for OCFNext it's a must.
The OWLext is small compared to OWLNext so it can be included in the task

8 As we are using 6.30.12+ on production we intend to move to 6.4x. We did a migration of a small project (3 OWLNext files for the app, plus12 shared files with others modules) with sucess. In the way we learned ther process, so problably we avoid the 6.35 step to go directly to 6.4x

[Vidar Hasfjord]

Hi Sebastian, great, sounds like you are on top of this. Good!

1. What about dynamic linking and single-threaded RTL? No support for these modes? And what about the special VCL threading mode, i.e. VCL compatible RTL, and the CodeGuard debug mode? Are those modes obsolete with the Clang-based compilers?
2. Note that I meant the naming for the configurations in the IDE, i.e. the configuration name that OWLMaker needs to calculate and pass on as an argument to MSBuild. So I would prefer configurations named "t" for Release, "dt" for Debug, "dtu" for Debug Unicode, etc. as in my Owlet project file.
3. I meant build and test the configurations in the RAD Studio IDE, to make sure they work fine, before we start driving the build with OWLMaker. The project file should be fully functional stand-alone.

Regarding OCFNext and 6.43, note that there are two bug tickets resolved and pending on the trunk (7). You may also find it useful to browse all tickets with the OCF label. There are many open and unresolved issues.

[Vidar Hasfjord]

I'm not sure right now if it's needed a specific suffix to indicate a Clang compiled library.

We don't. The updated naming convention now uses the "c" prefix on the compiler part of the library name to denote that the library is built with a Clang-based Embarcadero compiler, e.g. "owl-7.0.0-c720-x86-dt".

[Vidar Hasfjord]

for that I need to move to OWL7.

Note that there is no problem in installing many working copies from different branches in the code repository. The source code, including the examples, no longer depends on the setting of environment variable OWLROOT. And OWLMaker will build any version you point it to on the initial wizard page. I have these working copies checked out with Subversion (TortoiseSVN), allowing me to work on them all concurrently, do merging, etc.

\OWLNext\trunk (trunk)
\OWLNext\branch-6.35 (branches/635)
\OWLNext\branch-6.43 (branches/643)
\OWLNext\branch-owlet (branches/owlet)

[Sebastian Ledesma]

What about dynamic linking and single-threaded RTL? No support for these modes? And what about the special VCL threading mode, i.e. VCL compatible RTL, and the CodeGuard debug mode? Are those modes obsolete with the Clang-based compilers?

As I know Codeguard it's not compatible with CLang. No problem in support single/multi/VCL threaded.

Note that I meant the naming for the configurations in the IDE, i.e. the configuration name that OWLMaker needs to calculate and pass on as an argument to MSBuild. So I would prefer configurations named "t" for Release, "dt" for Debug, "dtu" for Debug Unicode, etc. as in my Owlet project file.

Ok.

[Sebastian Ledesma]

Note that there is no problem in installing many working copies from different branches in the code repository. The source code, including the examples, no longer depends on the setting of environment variable OWLROOT. And OWLMaker will build any version you point it to on the initial wizard page. I have these working copies checked out with Subversion (TortoiseSVN), allowing me to work on them all concurrently, do merging, etc.

Yes, i've noticed. It allows to more easily try several versions.