Menu
▾
▴
#92 Unable to build Webkit-600
In the absence of recent WebKit-600.x.x builds for Snow Leopard (10.6) I have attempted to build it myself without success.
Following the instructions located https://sourceforge.net/p/leopard-webkit/wiki/BuildInstructions/ results in a build failure. Failure begins at the "svn patch" stage with numerous missing files. The first missing file is "WTF/wtf/EnumClass.h" and I could provide a complete build log if helpful.
I AM able to successfully build WebKit-537.78.2 following these instructions and with these patches, though several additional patches were needed in my case.
Are there updated build instructions available for the WebKit-600.x.x series?
Related
You are the first one to try to build the 600 series for Snow Leopard.
We could manage to get this working but it would take some weeks of trying, reporting, modifying and retrying. If you'd like to accept that challenge please respond here.
Hi Tobias,
I have made some progress and gotten it to compile. It crashes in initNSURLResponse while trying to copy mime-type string, I suspect something is freeing the string before its used. I have been working on it for a few weeks now and I have a compiler setup that I know works fine for your set of patches to WebKit-537.78.2 (it's clang-3.3, but Apple's version that they call "5.0") on 10.6
I'm definitely up for the challenge!
Impressive work!
So I guess it was a benefit that I have been on vacation for 5 weeks.
However I don't think clang would be the right compiler choice for the current patches as since the 600 series they are mostly a GCC 4.2 compatibility backport.
In order to go on with with clang 3.3 you should indeed start at 537.78.2, porting the patches over to the safari-600.8-branch.
But first I should look into what you needed to do in order to get that far, like for example your current patchset or your build command line.
I use to produce my patchsets using the following commands at the source root:
Not that impressive - it doesn't actually work! :-P
I could use your build script... Currently I'm building with this:
and linking with this
which I am sure isn't the best way to do it... It results in a link success and no unresolved symbols, though.
as far as patches, my source base is a mess since I've edited dozens of files inserting debug code to try and figure out where things are going awry without success. I can try to put a patch together without the edits, but they are mostly the same changes you made in your patch in these sections:
Last edit: not available 2015-08-10
So you didn't start with my patches applied but just patched it until it did build?
Originally I tried to apply your patches - that worked fine for 537.78.2 so that's where I started. There were a bunch of missing files and the patches didn't apply cleanly, so I tried to find the right versions of the missing files and apply but I could never get it to compile.
So I started with your patches and tried going through applying them manually 1 by 1, but a lot of the changes weren't required to compile with clang, they were gcc specific as you've pointed out. So I applied all the ones that looked relevant to 10.6... And that's where I'm left now.
I solved the crashing problem due to a null string, it was a combination of WKSI updates to allow null pointers to be returned as well as having CACHE_PARTITIONING still active. I can now launch the browser and it actually renders a page in Safari...
However, it crashes almost immediately. I haven't dug into it too deeply yet, but here's a partial backtrace just in case it's something you've already tracked down:
0 com.apple.JavaScriptCore 0x0000000100aa0214 WTF::KeyValuePair<jsc::regexpkey, jsc::weak\<jsc::regexp=""> > WTF::HashTable<jsc::regexpkey, wtf::keyvaluepair\<jsc::regexpkey,="" jsc::weak\<jsc::regexp=""> >, WTF::KeyValuePairKeyExtractor\<wtf::keyvaluepair\<jsc::regexpkey, jsc::weak\<jsc::regexp=""> > >, JSC::RegExpKey::Hash, WTF::HashMap\<jsc::regexpkey, jsc::weak\<jsc::regexp="">, JSC::RegExpKey::Hash, WTF::HashTraits\<jsc::regexpkey>, WTF::HashTraits\<jsc::weak\<jsc::regexp> > >::KeyValuePairTraits, WTF::HashTraits\<jsc::regexpkey> >::lookup<wtf::identityhashtranslator\<jsc::regexpkey::hash>, JSC::RegExpKey>(JSC::RegExpKey const&) + 148 (RegExpKey.h:88)
1 com.apple.JavaScriptCore 0x0000000100a9f8bf JSC::RegExpCache::lookupOrCreate(WTF::String const&, JSC::RegExpFlags) + 63 (HashTable.h:404)
2 com.apple.JavaScriptCore 0x00000001009730fa JSC::JSGlobalObject::reset(JSC::JSValue) + 10122 (JSGlobalObject.cpp:349)
3 com.apple.WebCore 0x00000001018b3d8e JSC::JSGlobalObject::finishCreation(JSC::VM&, JSC::JSObject</wtf::identityhashtranslator\<jsc::regexpkey::hash></jsc::regexpkey></jsc::weak\<jsc::regexp></jsc::regexpkey></jsc::regexpkey,></wtf::keyvaluepair\<jsc::regexpkey,></jsc::regexpkey,>) + 142 (JSGlobalObject.h:313)
4 com.apple.WebCore 0x00000001018fd626 WebCore::JSDOMWindowBase::finishCreation(JSC::VM&, WebCore::JSDOMWindowShell) + 38 (JSDOMWindowBase.cpp:77)
5 com.apple.WebCore 0x0000000101901ca0 WebCore::JSDOMWindow::create(JSC::VM&, JSC::Structure, WTF::PassRefPtr<webcore::domwindow>, WebCore::JSDOMWindowShell) + 160 (JSDOMWindow.h:40)
6 com.apple.WebCore 0x00000001019018b3 WebCore::JSDOMWindowShell::setWindow(WTF::PassRefPtr<webcore::domwindow>) + 483 (JSDOMWindowShell.cpp:86)
7 com.apple.WebCore 0x000000010190169f WebCore::JSDOMWindowShell::finishCreation(JSC::VM&, WTF::PassRefPtr<webcore::domwindow>) + 31 (PassRefPtr.h:52)
8 com.apple.WebCore 0x0000000101dd76b7 WebCore::JSDOMWindowShell::create(JSC::VM&, WTF::PassRefPtr<webcore::domwindow>, JSC::Structure</webcore::domwindow></webcore::domwindow></webcore::domwindow>, WebCore::DOMWrapperWorld&) + 103 (PassRefPtr.h:52)
9 com.apple.WebCore 0x0000000101dd54f9 WebCore::ScriptController::createWindowShell(WebCore::DOMWrapperWorld&) + 169 (ScriptController.cpp:120)
10 com.apple.WebCore 0x0000000101dd5ef3 WebCore::ScriptController::initScript(WebCore::DOMWrapperWorld&) + 51 (ScriptController.cpp:240)
11 com.apple.WebKitLegacy 0x00000001010592f0 -[WebFrame(WebPrivate) _globalContextForScriptWorld:] + 160 (WriteBarrier.h:92)</webcore::domwindow></jsc::regexpkey,>
Ok that didn't format correctly, and now it won't allow me to edit it. Hopefully this is right:
Great that's it's almost working now!
I don't think that any of the files shown in the backtrace should need 10.6 specific changes - and some of those are actually generated code.
But if you could post your current patchset I might be able to tell more.
I managed to get it working more or less!
YouTube and other HTML5 media doesn't work quite right though, the controller never updates the progress and some videos never start playing. Full screen video actually works though but doesn't always start playing, though you can see the whole video is loaded by moving the time slider around. Is that something you have seen?
Sunspider 1.0.2 comes in < 140ms, a huge improvement over Safari 5.0.5! :-D
I also found and fixed a couple of other issues that I thought were specific to my build but they happen in the 537.78.2 build also, some additional null ptr deref's. I'm still debugging but I hope to have my patchset cleaned up soon for review.
That sounds really great!
There are indeed some issues with HTML5 playback, see ticket [#84].
So do you full JavaScript JIT enabled (JIT and DFG JIT)? (in case you didn't change the corresponding flags in wtf/Platform.h It should be.)
You should also run the JavaScript test suite if you don't have yet - or at least part of it; Tools/Script/run-javascriptcore-tests will do that for you; there are a couple of options available (don't forget to also specify --debug or --release in order to test the correct build), I do always pass --no-build so it doesn't re-run the build process before testing and --no-jsc-stress because the stress tests take much time on my single core G4/G5 machines.
I'm curious to see those additional fixes you needed!
What version tag/branch are your sources based upon? The current, and maybe final branch for the 600 series, is safari-600.8-branch with Safari-600.8.10 being the latest tag until now.
Once you've got the 600 series working you should update your sources to the branch safari-601.1-branch which is probably going to be released with 10.11; I'm currently undecided whether or not I'll continue the Leopard port - backporting to gcc 4.2 is so much work. Btw the fact that you're using clang has the consequence that you'll not be able to build for PowerPC which would be necessary for Rosetta support.
Related
Tickets:
#84Yes working with JIT and DFG, not FTL but I did build LLVM from scratch so I should have the required bits. It hasn't been a priority.
Thanks for the tip on the jsc testing, I haven't run it yet relying instead on web-based javascript tests. Promises work!
I'm still working with 600.7.12, once I fix the last couple of issues I will resume code cleanup and get a patch out hopefully. I'm hoping it applies relatively cleanly to 600.8.
Re: clang for PPC, if there was a solution to this then maintaining the PPC/Leopard port would be a lot easier! Have you seen:
http://www.csl.cornell.edu/~fang/sw/llvm/
Do you have a patch available for the security framework changes? Many sites no longer support the old SSL protocols and are locked out despite having the new WebKit code.
You should simply do an svn merge like this
or this
That might leave you with unresolved conflicts which you'd have to resolve and mark them as resolved afterwards using
For resolving conflicts you should definitely use something like this: http://www.defraine.net/~brunod/fmdiff/
That is what I've been using ever since I started to work with Leopard WebKit - although I had to modify it because it doesn't work with localized versions of svn.
I do observe that project but IMO chances aren't very high that it will ever reach a code quality nearly as good as gcc 4.2.4 - and I'm not yet a compiler developer so that I could fix all those bugs that would appear for sure when trying to build a project as large as WebKit using it.
Unfortunately I haven't done that yet - I guess that should be my priority now. But I can tell you: the code quality of the SSL library is quite bad in comparison with WebKit - no fun at all.
I will make a couple copies of the source and try that - my plan was to make a standalone diff from 600.7.12 and then see how it applies to 600.8, just to avoid the issues with svn. I had never seen FileMerge before, I've been manually merging the patches with sdiff and a couple different text windows. That will make it much easier going forward!
I fixed almost all of the crashes and I got HTML 5 video working on all the tests I can find, including full screen.
The build works for Safari, but not the as a replacement for /System/Library/Frameworks/WebKit.framework - there are complaints about the binary being found but incompatible GC options, I'm not sure what the cause is there? Since Safari is the main goal, I modified Safari to use a different path for WebKit and I'll have to check out the GC issue later. Next step is removing all the debug code.
For anyone who finds this in the future, the built-in WebKit debug logging is an excellent resource for understanding what's going on. I didn't find any documentation on enabling it for OSX, but you can enable it in the build options and activate logging with
defaults write com.apple.Safari WebCoreLogging -string Media
for example to enable all logging all media events
Until you have far less changes than 100 you might consider doing it manually, but in the future you should really let svn try the merge and what it can't resolve automatically you can resolve with FileMerge or other merge tools during the svn merge or postponed - it took me some hours to configure it correctly and to learn how to use but it already saved me some days of merging manually.
Yes, I've discovered only recently that I had to rebuild whole WebKit because in the files Configurations/Base.xcconfig of each framework projects GCC_ENABLE_OBJC_GC at some point was disabled by default; make sure your compiler gets passed the option "-fobjc-gc" and if not look at what I did to the Base.xcconfig files.
Why modify Safari when you could place the frameworks in the WebKit application bundle and launch that?
You should add a wiki page for things like this. And there's not only WebCoreLogging but also WebKitLogging.
Thanks for the tips! I did discover GCC_ENABLE_OBJC_GC eventually, and this build now works with System Preferences and other apps on 10.6.
I will attach my very rough build patches, which is a diff from straight WebKit-600.7.12 from svn.
Some of my "fixes" are hacks, but I now have a stable browser that plays video and animations. My next goals are to get to 600.8.10, and get Security built for 10.6 also.
My build environment is Apple clang-500.2.76 with libc++ and libcppabi-26. I also hope to release binaries of the build environment at some point, this is a real sore spot for 10.6
Modifying the Safari binary with install_name_tool took less time than making the WebKit app bundle
:-)
I will add a Wiki page for some of the notes I've made while working on getting it running.
Some quick remarks and questions:
It will take me some days to review the changes in detail..
And please wait with trying to build the Security framework until I complete the instructions!
I was getting random crashes before defining ENABLE_LLINT_C_LOOP=1, same with parallel GC. It's possible that something else fixed the crashes and these were unrelated, I will try another build without those changes and see.
Based on my notes IOSurface on 10.6.8 is missing IOSurfaceIsInUse() export even though it exists in the framework binary. This can be worked around, but other symbols could be missing also. I marked it for revisiting at a later time.
I rebuilt without ENABLE_LLINT_C_LOOP in Source/WebCore/Configurations/FeatureDefines.xcconfig and it results in a repeatable crash:
Oddly, leaving it defined in JavaScriptCore seems to be fine, which is the only place I thought it was being used.
ENABLE_LLINT_C_LOOP must always be defined the same in all FeatureDefines.xcconfig or you'll get problems.
In the backtrace you see that WebCore includes JSGlobalObject.h which includes tons of other headers and there's probably at least some header that generates different code depending on ENABLE_LLINT_C_LOOP.
And most importantly the definition in wtf/Platform.h must match the ones in FeatureDefines.xcconfig - in fact I don't know why it's needed in the xcconfig at all.
I get that same crash if it's undefined in all of the FeatureDefines and Platform.h, but no crash if it's undefined everywhere except the WebCore FeatureDefines. It seems that the code generation pieces use the define in the FeatureDefines.xcconfig file but NOT in Platform.h based on what I'm seeing.
I revisited the IOSurface issue, and while that define is missing in the SDK headers, that isn't the real problem. There are other supporting functions missing (wkIOSurfaceContextCreate which invokes a missing CGIOSurfaceContextCreate, wkIOSurfaceContextCreateImage which invokes a missing CGIOSurfaceContextCreateImage). With the single-process Safari 5.0 I don't know if there's really any benefit to making this work performance-wise.
I just found out why you need to enable ENABLE_LLINT_C_LOOP in the WebCore FeatureDefines.xcconfig - it's because you forgot to disable ENABLE_FTL_JIT in all FeatureDefines.xcconfig files except JavaScriptCore. And by defining ENABLE_LLINT_C_LOOP the Platform.h global header (included by config.h from the compiler command line) will define ENABLE_JIT as well as ENABLE_DFG_JIT and ENABLE_FTL_JIT to 0.
So what you've got to do is to ALWAYS have IDENTICAL FeatureDefines.xcconfig files in ALL the WebKit projects. I've already had inexplicable trouble with not paying 100% attention to this. So if you change one of those FeatureDefines.xcconfig files always copy the changed one over the other ones - in fact they should by symlinked but that's difficult to manage with version control systems.
In your case copy JavaScriptCore.xcconfig over the one in WebCore and WebKit/mac and recompile the whole thing.
You might also want to try reenabling ENABLE_PARALLEL_GC if it works without defining ENABLE_LLINT_C_LOOP for WebCore.
I hope to be able to complete the Security framework build instructions this weekend.
This was exactly it! It builds and runs fine now.
I have left PARALLEL_GC disabled as I am still trying to track down the real source of a couple other bugs that I just worked around (like popupbutton being NULL) but it works very well for the most part, and hasn't crashed in a long time.
I noticed that by only applying the patches needed for getting WebKit built you missed (all?) the little (but very important) bug fixes that I've also done - I think those should be found and applied first.
I visually scanned my whole patch file in order to find the small but very important patches hidden in there
In the attached file you find my results.
However MediaPlayerQT needs some more investigation..