On 21 September 2011 21:02, Dave Tapley <dukedave@...> wrote:
> On 21 September 2011 17:33, Jeremy O'Donoghue
> <jeremy.odonoghue@...> wrote:
>> Hi Dave,
>>
>> On 20 September 2011 17:58, Dave Tapley <dukedave@...> wrote:
>>>
>>> On 16 September 2011 23:30, Dave Tapley <dukedave@...> wrote:
>>>>
>>>> I presume everyone has a very long compile time when building wxcore?
>>>> Specifically rebuilding everything under src/cpp/ every time..
>>>>
>>>> Has anyone ever looked into avoiding this complete rebuild?
>>>
>>> I've just spent a few hours looking deeper in to this and come across two
>>> issues:
>>>
>>> 1. There is a very informative blog post[1] written by Jeremy, which deals
>>> with the subject of "Compiling C or C++ code from within Cabal".
>>> Unfortunately I can't find any of the code mentioned in the post in the
>>> project, specifically I tried to find a "myBuildHook" in "./wxcore/Setup.hs"
>>> (I also looked in previous revisions using a trackdown grep[2]) but I didn't
>>> find anything. Perhaps someone with better knowledge of the project can
>>> comment on if/where/when the code in the post was used?
>>
>> The code sits on my development machine. It works, but with a fairly serious
>> limitation.
>>
>> The limitation is that I do not do any dependency tracking. It works by
>> checking if an object file is older than its corresponding source file. This
>> works fine for .cpp files but breaks if you change a header. Short of
>> writing a complete dependency tracker, this is hard to fix, and in truth I
>> think it belongs in Cabal.
>>
>> If Cabal wants to say that it can compile C/C++ code, it should be the one
>> to do so correctly, especially as dependency tracking for C/C++ is vile and
>> compiler dependent(*).
>
> Firstly welcome back, and thanks for taking the time to reply :)
>
> Ah, this all makes more sense now.
> Do you know if there is any documentation on cabal's C/C++ compiling abilities?
> I still have no idea how the "confHook" stuff actually results in code
> being built, because I don't see any reference to the C++ code (i.e.
> wxcore/src/cpp/) in Setup.hs, only the calling of 'wxdirect' to
> generate them, and use of 'libBuildInfo' to set up GCC.
>
>>
>> I would be very happy to put the code out there as a GitHub gist or similar
>> - it's only in one or two files, and quite easy to follow, but I don't feel
>> it is ready for prime time due to the limitations noted above.
>
> Well, for me, the compile time of the C++ component is around four
> minutes, which proves to be a real pain when you miss one semi-colon.
> I'd certainly like to give your code a go, and I shall do so whilst
> heeding your warning :)
>
>>
>> (*) one option might be to do this only for GCC, as in practice we only
>> really support GCC anyway.
>>
>>> 2. Inspecting the wxdirect code you can see that "System.IO.writeFile" is
>>> used to write all the generated code[3], but no test is performed to see if
>>> the output file has actually changed. Thus the file is always opened for
>>> write, and so its modification time is changed, and so everything is
>>> recompiled every time wxcore is built. I have have written a local patch
>>> which replaces the "writeFile" function with one which first checks whether
>>> the string to be written differs (aside from date/time stamp) to the current
>>> one; it only performs the "writeFile" if there has been a change.
>>> Using this patch none of the Haskell code is re-built, but unfortunately
>>> all the C++ code is.
>>
>> This is a nice patch - I think we should apply it.
>
> Excellent, well at the moment it's a rather 'quick' implementation,
> comprising of the following function:
> writeFileIfRequired :: FilePath -> String -> IO ()
> writeFileIfRequired f str = readFile f >>= evaluate >>= \str' -> when
> (not $ and $ zipWith isSame (lines str) (lines str')) (writeFile f
> str)
> where isSame l1 l2 = ("UTC" `isInfixOf` l1) || ("UTC" `isInfixOf`
> l2) || (l1 == l2)
>
> You'll note the the use of ("UTC" `isInfixOf`), this is my nasty hack
> to get around the fact that the time of generation is written in to
> the file, and this obviously changes every time. Unless anyone has a
> good reason not to, I'd like to suggest we remove this "generated
> on..." lines (but leave in the "don't edit this" blurb), if someone
> really wants to know they can always check the file modification time?
>
> Dave,
>
>>
>> Jeremy
>>
>
Hmm, I was just revisiting this and happened on the following:
The Setup.hs for wxcore uses "simpleUserHooks", overriding only "confHook".
However there is also "cleanHook", which is defined by simpleUserHooks to be:
cleanHook = \p _ _ f -> clean p f,
If you consult the source for clean[1] you'll see that it tries to
"remove the whole dist/ directory rather than tracking exactly what
files we created in there". I presume that's why we have to do a full
re-build every time?
To try and circumvent this I modified the definition of "main" in
Setup.hs to this:
main = defaultMainWithHooks simpleUserHooks { confHook = myConfHook,
cleanHook = (\_ _ _ _ -> return ())}
Unfortunately it still seems to re-build all the C++ on each 'install'
from cabal.
Not sure why?
Dave,
[1] Taken from http://www.haskell.org/ghc/docs/6.10.4/html/libraries/Cabal/src/Distribution-Simple.html#simpleUserHooks
-- Cleaning
clean :: PackageDescription -> CleanFlags -> IO ()
clean pkg_descr flags = do
let distPref = fromFlag $ cleanDistPref flags
notice verbosity "cleaning..."
maybeConfig <- if fromFlag (cleanSaveConf flags)
then maybeGetPersistBuildConfig distPref
else return Nothing
-- remove the whole dist/ directory rather than tracking exactly what files
-- we created in there.
chattyTry "removing dist/" $ do
exists <- doesDirectoryExist distPref
when exists (removeDirectoryRecursive distPref)
-- these live in the top level dir so must be removed separately
removeRegScripts
-- Any extra files the user wants to remove
mapM_ removeFileOrDirectory (extraTmpFiles pkg_descr)
-- If the user wanted to save the config, write it back
maybe (return ()) (writePersistBuildConfig distPref) maybeConfig
where
removeFileOrDirectory :: FilePath -> IO ()
removeFileOrDirectory fname = do
isDir <- doesDirectoryExist fname
isFile <- doesFileExist fname
if isDir then removeDirectoryRecursive fname
else if isFile then removeFile fname
else return ()
verbosity = fromFlag (cleanVerbosity flags)
|