On 12 December 2011 17:18, Jeremy O'Donoghue <jeremy.odonoghue@...> wrote:
> Hi Dave, all,
>
> This is fantastic news - especially the bit about GHCi. I have put a couple
> of general comments inline, but it looks like you have found most of the
> issues, at least on Linux.
>
> I think that at least some of the issues would go away if, as you say,
>
> "we could have an entirely separate cabal project (perhaps "wxc") for this
> shared library, and then have wxcore depend on it?"
>
> In this case, we would be able to ensure that libwxc.so.x.y.z (or wxc.dll or
> libwxc.dylib or whatever) was in a 'normal' place before wxcore gets built.
>
> On 9 December 2011 14:13, Dave Tapley <dukedave@...> wrote:
>>
>> On 8 December 2011 22:34, Dave Tapley <dukedave@...> wrote:
>> > Hello everyone,
>> >
>> > I wrote a story, and I invite you all to comment and help me make it
>> > better.
>> > As an experiment I've decided to put it on hpaste instead of having
>> > the mail thread get out of hand (I'm also cross posting it to the
>> > cabal list and have mentioned it in #haskell, and I want to keep all
>> > correspondence in one place):
>> > http://hpaste.org/55027
>>
>> Ha, well that was good timing for hpaste to go down!
>> Here's it is:
>>
>> I've been trying to resurrect the idea of building a shared library
>> for the C++ component of the wxhaskell library.
>>
>> Jeremy (to my knowledge) first put this forward, here:
>>
>> http://wewantarock.wordpress.com/2010/11/01/working-around-the-static-libstdc-restriction/
>>
>> In addition to the advantages he lists there, I have the following reason:
>> When building wx-core (in its current incarnation) cabal will *always*
>> rebuild all the C++ code, which takes a majority of the build time.
>> This becomes very frustrating when you change one line of Haskell and
>> have to wait seven minutes to rebuild.
>> I complained about it here:
>> http://sourceforge.net/mailarchive/message.php?msg_id=28099997
>>
>> So, I decided to try and stop this complete C++ rebuild, I partially
>> succeed, but I eventually got stuck for reasons I complained here:
>> http://www.haskell.org/pipermail/cabal-devel/2011-October/007816.html
>>
>> Side note: I did eventually discover why (or, where) the c-sources
>> (cSources) list is used to compile and link, and it is here:
>>
>> http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/src/Distribution-Simple-GHC.html#buildLib
>> You can see:
>> "| filename <- cSources libBi]" under "-- build any C sources", and
>> you can see:
>> "cSharedObjs = map (`replaceExtension` ("dyn_" ++ objExtension))
>> (cSources libBi)"
>> under "-- link:".
>> Is this assumption correct?
>
>
> This is correct - it's the standard way Cabal builds C/C++ code. To be
> honest, this is really a consequence of the fact that Cabal is more of a
> tool for simple distribution than a Make replacement, and the developers
> probably didn't expect Cabal would be used to build large bodies of C++
> code.
>
> My approach is far from perfect, too - in fact it is incorrect because I
> don't do dependency tracking on header files, so if you edit a header, it
> (incorrectly) fails to rebuild. This is because I wasn't keen on rewriting
> 'make' in the wxHaskell build system - it seems to me like something Cabal
> could usefully support. I did play with some very hacky code which
> automatically rebuilt all of the C++ code if any header is newer than any
> C++ source. This is aesthetically dreadful, but it somewhat captures the
> reality, since in practice, pretty much all of the C++ files depend on all
> of the headers (and it's easy to write).
>
> On reflection, we should probably do this as what exists at present could
> lead to unexpectedly incorrect code being generated if someone is in full
> flow of development. I'll see if I can dig up some code - I think I have it
> somewhere.
You know, I'm surprised this didn't occur to me until now:
Does the C++ code actually need to use cabal at all?
If it's a separate project, perhaps it would be more sensible to use a
C++ oriented build system?
Then, unless I'm missing something, it can install the headers
somewhere, and wxcore can run wxdirect on the headers are currently.
The big disadvantage I can see with moving the C++ out of the wxcore
project is lots of pain for users when they 'cabal update' and
suddenly wxcore fails because they need to update the C++ headers /
library.
Another disadvantage is that this will make development a little more
painful (although the time saved not having to rebuild all the C++ is
a huge bonus) because you wouldn't (without some hackery) be able to
use cabal-dev as described here:
http://haskell.org/haskellwiki/WxHaskell/Development/Environment#Building_and_testing_wxHaskell
I happened upon this (dead) project, which I seems to have the goal I
am outlining:
http://wxc.sourceforge.net/
>
>> At this point I thought about either:
>> 1. Getting the cabal source and starting to write code to give
>> BuildInfo a "cObjs" in addition to "cSources".
>> 2. Picking up Jeremy's shared library code, so wxhaskell would have
>> its own code to build the library, in which I could do sensible
>> re-compilation.
>>
>> Given that there were other advantages to 2, I went with that.
>>
>> Jeremy had written one blog post on building a such a shared library,
>> here:
>>
>> http://wewantarock.wordpress.com/2010/11/03/building-a-shared-library-in-cabal/
>> In response to an email on the wxhaskell-devel list he also kindly put
>> up this gist, with the code he'd been working on:
>> https://gist.github.com/1301115
>>
>> I dutifully took this gist, and have now attempted to integrate in to
>> my wxhaskell-dev branch, which you can find here:
>> http://darcsden.com/DukeDave/wxhaskell-dev
>> (note: I haven't pushed any of the shared library code to darcsden
>> yet, for reasons I'm about to explain)
>>
>> This is where things got interesting, after a few hours of hacking I
>> have built a shared library, but in this very back-handed way:
>>
>> Firstly, in Jeremy's code the to-be-compiled shared library is added
>> to the wxcore BuildInfo through a custom hook.
>> We can see this because:
>> > let all_dlls = parseDLLs ["x-dll-name", "x-dll-extra-libraries"]
>> > custom_bi
>> (the modified wxcore.cabal contains the line: "x-dll-name: wxc")
>>
>> However on a clean build of wxcore we get the following error:
>>
>> setup: Missing dependency on a foreign library:
>> * Missing C library: wxc
>> This problem can usually be solved by installing the system package that
>> provides this library (you may need the "-dev" version). If the library is
>> already installed but in a non-standard location then you can use the
>> flags
>> --extra-include-dirs= and --extra-lib-dirs= to specify where it is.
>
>
> My code worked for Windows - but this is a better cross-platform solution.
Interesting.
I found this fixed feature on cabal:
http://hackage.haskell.org/trac/hackage/ticket/262
It suggests that cabal uses Autoconf to resolve dependencies,
specifically the AC_CHECK_LIB macro.
I have zero knowledge of build systems in Windows, but is it
reasonable to suggest that Autoconf could be more relaxed?
I really should get a Windows build environment set up..
>
>>
>> The error is dumped *before* cabal calls myBuildHook, and since this
>> actually builds the library the error makes sense; cabal is looking
>> for the shared library before we've built it.
>>
>> To get around this I modified the offending line, thus:
>> > let all_dlls = parseDLLs ["x-dll-extra-libraries"] custom_bi
>>
>> With that modification cabal would now get to myBuildHook, but then
>> another curious error arose:
>> We see that the actual linking is called here:
>> > runProgram verbose gcc (opts' ++ objs' ++ lib_dirs' ++ libs')
>>
>> But on hitting that line the following error is spat out:
>> /usr/bin/ld: cannot open output file dist/build/libwxc.so.0.13.1: No
>> such file or directory
>
>
> Windows doesn't behave this way - this part worked for me as well.
>
>>
>> I checked all my permissions and couldn't see anything wrong, I could
>> touch the file.
>> Conveniently I noticed that, if the verbosity is set high enough,
>> runProgram will call printRawCommandAndArgs:
>>
>> http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/src/Distribution-Simple-Utils.html#printRawCommandAndArgs
>>
>> The output (i.e. the linker invocation) looks like this:
>> /usr/bin/gcc -fno-stack-protector -shared -Wl,-soname,libwxc.so.0 -o
>> dist/build/libwxc.so.0.13.1 dist/build/src/cpp/apppath.o
>> dist/build/src/cpp/dragimage.o dist/build/src/cpp/eljaccelerator.o
>> [snip-rest-of-.o-files] -L/usr/local/lib -lstdc++ -lwx_baseu-2.9
>> [snip-rest-of-wx-libs]
>>
>> Now if I cd into the wxcore and paste the command *verbatim* then gcc
>> works and generates libwxc.so.0.13.1 as expected.
>> You can see in Jeremy's code the linkShareLib function contains:
>> > cwd <- getCurrentDirectory
>>
>> I used this to confirm that the we were in ./wxcore and we are, even
>> making the path for -o absolute didn't sovle the issue.
>> I ended up replacing runProgram line with this, less satisfactory line:
>> > system $ (unwords ([show . locationPath . programLocation $ gcc] ++
>> > opts' ++ objs' ++ lib_dirs' ++ libs'))
>>
>> Which works, but still doesn't explain why runProgram doesn't.
>> Any suggestions?
>
>
> I've tried this on Linux. You're absolutely right and I'm stumped as to why.
> I think we should pull this specific part of the thread out and post to the
> Haskell list. Maybe someone who knows Cabal and/or GHC would have an idea
> why this is the case.
Ah, good to know it works on Windows.
I'll try and replicate it in isolation and send a mail to the cabal list.
>
>>
>> So with that complete I was able to link a shared library to:
>> wxcore/dist/build/libwxc.so.0.13.1
>>
>> Of course, at this point we've generated the shared library, but
>> wxcore still isn't aware of it, so the build completes successfully,
>> but any attempt to compile against it results (understandably) in
>> hundreds of linker errors.
>> To resolve this I had to add "x-dll-name" (read in from wxcore.cabal
>> as "wxc") back to parseDLLs:
>> > let all_dlls = parseDLLs ["x-dll-name", "x-dll-extra-libraries"]
>> > custom_bi
>>
>> Well, not quite, attempt to build wxcore again and we're back to this
>> error:
>> * Missing C library: wxc
>>
>> Of course, the .so isn't in a 'normal' place, so add its location as
>> an extra lib directory:
>> > { extraLibDirs = extraLibDirs libbi ++ extraLibDirs wx ++
>> > ["/full/path/to/wxcore/dist/build"]
>> (note: I tried using just "dist/build", but cabal said: "library-dirs:
>> dist/build is a relative path")
>
>
> I think we need a more robust solution in the longer run, as
> <wxhaskell>/wcore/dirt/build is really a temporary location.
I guess the 'correct' solution would be to 'install' the library to
somewhere like LD_LIBRARY_PATH in Linux.
Again I don't know how this fits in with Windows.
It's also a pain for people developing wxhaskell, because as I
mentioned before, you'd have to 'install' a potentially unstable,
development version of the shared library in order to be able to
reference it.
>
>>
>> This still isn't quite enough, it took one more kludge:
>> /full/path/to/wxcore/dist/build/$ ln -s
>> /full/path/to/wxcore/dist/build/libwxc.so.0.13.1 libwxc.so
>
>
> Earlier you mentioned that, at least on Linux, the build command is:
>
>
> /usr/bin/gcc -fno-stack-protector -shared -Wl,-soname,libwxc.so.0 -o
> dist/build/libwxc.so.0.13.1 dist/build/src/cpp/apppath.o
> dist/build/src/cpp/dragimage.o dist/build/src/cpp/eljaccelerator.o
> [snip-rest-of-.o-files] -L/usr/local/lib -lstdc++ -lwx_baseu-2.9
> [snip-rest-of-wx-libs]
>
> I know GCC generated this and not you, but it looks like the root of the
> issue. I am slightly confused though. ld.so is supposed to *know* that
> libwxc.so.0.13.1 is a valid instance of libwxc.so.0
Firstly, I'm a little confused by your statement "GCC generated this
and not you", that build command is generated by linkCxxOpts, not GCC?
This is curious: The name of the symlink has to be "libwxc.so", not
even "libwxc.so.0".
I think this makes sense though, the only "information" the simple
build system gets about the dependency is the parsing of this line in
wxcore.cabal by parseDLLs:
x-dll-name: wxc
I assume that the "wxc" is some how turned in to "libwxc.so", I'd like
to know how, and if it is possible to specify the major revision of a
shared library passed; this would have to be some convention in the
naming, because extraLibs is just [String]:
http://hackage.haskell.org/packages/archive/Cabal/latest/doc/html/Distribution-PackageDescription.html#v:extraLibs
>
> In normal Linux-land, you run /sbin/ldconfig after installing a new library,
> and it creates these symlinks automatically You could do this by executing
> something like (not tested)
>
> /sbin/ldconfig -n <path-to-directory>
Thanks for the tip, that makes more sense.
I can now see that the symlink which "ldconfig -n" generates is
extracted from the -soname given at link time.
So, by modifying the -soname to *not* specify the major revision, i.e:
This:
> "-Wl,-soname,lib" ++ basename ++ ".so",
Instead of this:
> "-Wl,-soname,lib" ++ basename ++ ".so" ++ maj_ver,
I can now use "ldconfig -n" in dist/build/ to create the symlink (to
libwxc.so), instead of doing it by hand, I suppose this is a little
less broken.
>
>> I'd like to think there's a more elegant solution than this and I'm
>> open to suggestions.
>> I should note that at this point I became aware that we could have an
>> entirely separate cabal project (perhaps "wxc") for this shared
>> library, and then have wxcore depend on it?
>>
>> Running one more time I *thought* I was finally there, until I noticed
>> this line, hidden in cabal's output:
>> /full/path/to/wxcore/dist/build/libwxc.so: file not recognized: File
>> truncated
>>
>> As far as I can tell, this occurs because we now load(?)
>> "libwxc.so.0.13.1" in myConfHook, but then in myBuildHook we want to
>> use it as the destination for the linker. Worse still it seems that
>> 'truncated' actually means 'deleted'. To get around this I had to one
>> again remove "x-dll-name" from the parseDLLs call (so I could get to
>> myBuildHook and create "libwxc.so.0.13.1" again), build wxcore, then
>> add "x-dll-name" back again, but this time also comment out the call
>> to linkSharedLib (and so stop cabal attempting to open
>> "libwxc.so.0.13.1") before building wxcore one last time.
>>
>> Finally everything seemed to work.
>>
>> Now, I should note that I have been using cabal-dev, and so I now have
>> a package-conf I need to reference when building with my new shared
>> library wxhaskell. To test it I built the wxhaskell HelloWorld sample
>> thus:
>> /full/path/to/samples/wxcore$ ghc --make -package-conf
>> ../../cabal-dev/packages-7.0.3.conf HelloWorld.hs
>>
>> It worked, but then when I try to run HelloWorld I get:
>> ./HelloWorld: error while loading shared libraries: libwxc.so.0:
>> cannot open shared object file: No such file or directory
>>
>> Okay, again, libwxc.so.0 isn't in a 'normal' place, so I kludged it
>> with another symlink:
>> /full/path/to/samples/wxcore$ ln -s
>> /full/path/to/wxcore/dist/build/libwxc.so libwxc.so.0
>>
>> And now, I can report that HelloWorld runs.
>> [imagine a screen shot of the HelloWorld sample application running here]
>> Rejoice.
>>
>> But wait, there's more:
>> One of the promises of the shared library approach is that we'd all be
>> able to use wxhaskell with ghci again, and to my utter disbelief, it's
>> true:
>> /full/path/to/samples/wxcore$ ghci -package-conf
>> ../../cabal-dev/packages-7.0.3.conf HelloWorld.hs
>> Ok, modules loaded: Main.
>> Prelude Main> main
>>
>> [imagine a screen shot of the HelloWorld sample application running here]
>> Double rejoice.
>>
>>
>>
>> >
>> > Dave,
>>
>>
>> ------------------------------------------------------------------------------
>> Cloud Services Checklist: Pricing and Packaging Optimization
>> This white paper is intended to serve as a reference, checklist and point
>> of
>> discussion for anyone considering optimizing the pricing and packaging
>> model
>> of a cloud services business. Read Now!
>> http://www.accelacomm.com/jaw/sfnl/114/51491232/
>> _______________________________________________
>> wxhaskell-devel mailing list
>> wxhaskell-devel@...
>> https://lists.sourceforge.net/lists/listinfo/wxhaskell-devel
>
>
|