From: Charles W. <cwi...@us...> - 2010-01-25 01:02:31
|
So, I've been trying to rebuild msys-perl for a few days now. I started with msysgit's 5.8.8 version, but when that failed I just tried to rebuild our existing perl-5.6.1 using the new tools (msys-gcc-3.4.4-2, msys-binutils-2.19.51-2, msysCORE-1.0.13-{bin|dev}. There were a number of (expected) snags, such as: gcc -fnative-struct vs gcc -mms-bitfields and the like. However, after I worked through those, I ended up with a build that coredumps if -O3, but works (mostly) at -O0. I haven't yet tried -O1/-O2. (I haven't updated to the -3 release of the new msys toolchain, but AFAICT the differences between Cesar's -2 and -3 releases are only cosmetic.) Now, I could live with -O0 (or -O1/-O2 if we're lucky) -- and I'd just chalk it up to an optimization bug and say "don't do that" until we have time to track it down in gcc. But... Even my -O0 build fails some of the tests, which the 2009/05 version (built using msys-gcc-2.95.3, old msys-binutils, and msysCORE-1.0.11) did not. I was able to boil one issue down to a simple <perl> testcase: PATH="/usr/src/perl/OLD/perl-5.6.1-2:$PATH" ./perl.exe -I../lib -e \ 'use warnings; $b=123456.789; $a=sprintf(">%E<", $b); print "$a\n";' >-1.234568E+05< Note the minus sign. Now, if I remove 'use warnings;' then it works as expected. If I replace the msys-1.0.dll with the msys-1.0-debug.dll, then it works as expected. Debugging, I get the following. Buried down deep in perl, PERL_do_sprintf eventually calls Perl_sv_vcatpvfn which, in turn, calls the MSYS sprintf. Breakpoint 3, Perl_sv_vcatpvfn (sv=0xa03e9d0, pat=0xa0420e8 ">%E<", patlen=4, args=0x0, svargs=0xa034010, svmax=1, maybe_tainted=0x22fa9f "") at sv.c:6561 6561 (void)sprintf(PL_efloatbuf, eptr, nv); Nothing up my sleeve: (PL_efloatbuf is a local, dynamically allocated buffer of length 46). (gdb) p PL_efloatbuf $3 = 0xa033e88 "" Notice that the floating point variable has the correct (positive) sign: (gdb) p nv $4 = 123456.789 (gdb) p eptr $5 = 0x22f9cc "%E" The following is the "perl" return value location, from the $a = sprintf(">%E<", $b) statement. As you can see, it already has the leading '>' character, and is waiting for msys sprintf to fill in the %E bit. (gdb) p *((XPV *) sv->sv_any) $6 = {xpv_pv = 0xa032b58 ">", xpv_cur = 1, xpv_len = 4} Now, I simply step over the MSYS sprintf function, and... (gdb) s 6563 eptr = PL_efloatbuf; (gdb) p PL_efloatbuf $6 = 0xa033e88 "-1.234568E+05" Err, say what? From this, the rest flows: PL_efloatbuf is copied into sv->sv_any, and eventually shows up in $a. Oddly, if the problem is *INSIDE* the new MSYS dll, then you'd expect our existing perl to also fail in this way, but no...even if I force the old perl to use all of the new lib files: OLD PERL, NEW DLL (non debug): WORKS. $ PATH="/usr/src/perl/OLD/perl-5.6.1-2:$PATH" /usr/bin/perl.exe -I../lib -e 'use warnings; $b=123456.789; $a=sprintf(">%E<", $b); print "$a\n";' >1.234568E+05< NEW PERL, NEW DLL (non debug): BROKEN. $ PATH="/usr/src/perl/OLD/perl-5.6.1-2:$PATH" ./perl.exe -I../lib -e 'use warnings; $b=123456.789; $a=sprintf(">%E<", $b); print "$a\n";' >-1.234568E+05< But, if I simply replace the MSYS dll with its debug version, but do not rebuild perl, then it works! NEW PERL, NEW DLL (*): WORKS. debug version: msys-1.0-debug.dll copied to msys-1.0.dll $ PATH="/usr/src/perl/OLD/perl-5.6.1-2:$PATH" ./perl.exe -I../lib -e 'use warnings; $b=123456.789; $a=sprintf(">%E<", $b); print "$a\n";' >1.234568E+05< Naturally I can't reproduce this in a simple C test case. This combination of things tells me that the issue, somehow, is in how the linker/compiler is setting up the CALL into the msys dll: with one DLL, good input values --> char* buffer is properly populated. With the other DLL, using the perl just compiled with the NEW toolchain --> char* buffer is IMproperly populated. So, the problem is in the msys DLL? But...with the OLD perl, and the NEW dll, it works. So, the difference is in which toolchain was used to compile the client (and, technically, which import lib's static objects and thunk functions were linked into that client when it was built). But if the msys DLL is ok, that doesn't explain why the NEW perl, built using the same import lib and toolchain -- heck, not even relinked -- works peachy if you simply swap the msys-1.0-debug.dll for the non debug one. And don't even get me started on WTH 'use warnings;' could have to do with the whole thing. I'm stumped. Unless it's just plain old stack corruption, and the analysis ^^^^ above is just chasing shadows. But...I'm using the SAME perl source code as the 2009/05 version, and it didn't have this problem. (Furthermore, when debugging perl (but regular msys dll), I see: <before the call to msys sprintf: everything looks fine> <call sprintf> <after the call to msys sprintf: the buffer has the @#!!#$ minus sign> So...if it's stack corruption, it's happening INSIDE msys. But only when MSYS is called in certain configurations, and you're not using the debug version of the dll, blah blah... Which brings me back to "what's different": msys-1.0.13, and the new msys toolchain. Interestingly, I tried replacing the msys-1.0.dll alone with the 1.0.12 version and the 1.0.11 version, without rebuilding (new) perl at all. 1.0.13 + new perl: FAIL. 1.0.12 + new perl: FAIL. 1.0.11 + new perl: WORKS. 1.0.13 + old perl: WORKS. 1.0.13.debug + new perl: WORKS. So...what do 1.0.12 and 1.0.13 have in common, that 1.0.11 did not? Oddly, not the toolchain. 1.0.12 was built using gcc-2.95.3/old-binutils from MsysDVLPR, as far as I can tell, while 1.0.13 was the first one to be built using the new msys-gcc3.4.4. Any ideas, such as what could have changed in msys or its build procedure --excluding which toolchain was used-- between 1.0.11 and 1.0.12 that could explain this wacky behavior? -- Chuck |
From: Cesar S. <ces...@gm...> - 2010-01-25 14:30:18
|
Charles Wilson wrote: > However, after I worked through those, I ended up with a > build that coredumps if -O3, but works (mostly) at -O0. Intriguing. I will investigate when I get a chance. First thing is to reproduce your results. I will report back when it's done. Cesar |
From: Charles W. <cwi...@us...> - 2010-01-26 02:54:56
|
Cesar Strauss wrote: > Charles Wilson wrote: >> However, after I worked through those, I ended up with a >> build that coredumps if -O3, but works (mostly) at -O0. > > Intriguing. I will investigate when I get a chance. > > First thing is to reproduce your results. I will report back when it's done. Well, you'll need a few additional packages: I've rebuilt all the prereqs using the new toolchain -- including gdbm (*). Plus, there are some changes to the buildscript/patchset even for old perl-5.6.1 (the -mms-bitfields thing and similar stuff) as I mentioned before. I can tar all that up and post it somewhere for you, but it'll be a day or two. OTOH, some news: it appears to be an optimizer bug in building *msys-1.0.dll*. When I rebuild msys using the 3.4.4-3 toolchain, at -O3 or -O2, then I see the behavior described. When I build it at -O0 or -O1, I do not (but note, in these tests I do NOT rebuild perl. I simply drop in the new msys DLL). This explains the "but when I use msys-1.0-debug.dll, it works" -- the -debug DLL is always built at -O0. So now the question becomes, what is OLD perl doing differently, such that it somehow can call the same function exported from the "broken" -O3/-O2 dll, and have it work correctly? And, what is -O2/-O3 doing that breaks this function in such an interesting way? Some possibilities (fixed in 3.4.5 upstream): http://gcc.gnu.org/gcc-3.4/changes.html#3.4.5 Optimizations issues * 17810 ICE in verify_local_live_at_start * 17860 Wrong generated code for loop with varying bound * 21709 ICE on compile-time complex NaN * 21964 broken tail call at -O2 or more * 22167 Strange optimization bug when using -Os * 22619 Compilation failure for real_const_1.f and real_const_2.f90 * 23241 Invalid code generated for comparison of uchar to 255 * 23478 Miscompilation due to reloading of a var that is also used in EH pad * 24470 segmentation fault in cc1plus when compiling with -O * 24950 ICE in operand_subword_force I'm thinking that 17860 (-fstrength-reduce) http://gcc.gnu.org/bugzilla/show_bug.cgi?id=17860 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=2.2326.2.906&r2=2.2326.2.907 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/loop.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.488.2.8&r2=1.488.2.9 21964 (tail recursion) http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21964 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=2.2326.2.904&r2=2.2326.2.905 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/stmt.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.342.2.3&r2=1.342.2.4 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.3389.2.419&r2=1.3389.2.420 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/gcc.c-torture/execute/pr21964-1.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=NONE&r2=1.1.4.1 22167 (-Os problem) http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22167 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=2.2326.2.882&r2=2.2326.2.883 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/gcse.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.288.2.9&r2=1.288.2.10 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.3389.2.406&r2=1.3389.2.407 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/g++.dg/opt/pr22167.C.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=NONE&r2=1.1.4.1 23241 ("uchar vs. 255") http://gcc.gnu.org/bugzilla/show_bug.cgi?id=23241 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=2.2326.2.905&r2=2.2326.2.906 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/combine.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.400.4.14&r2=1.400.4.15 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.3389.2.420&r2=1.3389.2.421 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/gcc.dg/char-compare.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=NONE&r2=1.1.4.1 23478 ("caller-save bug") http://gcc.gnu.org/bugzilla/show_bug.cgi?id=23478 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=2.2326.2.912&r2=2.2326.2.913 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/regs.h.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.31.4.1&r2=1.31.4.2 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/local-alloc.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.126.4.1&r2=1.126.4.2 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/flow.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.572.4.4&r2=1.572.4.5 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/global.c.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.98&r2=1.98.4.1 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=1.3389.2.426&r2=1.3389.2.427 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/testsuite/g++.dg/opt/pr23478.C.diff?cvsroot=gcc&only_with_tag=gcc-3_4-branch&r1=NONE&r2=1.1.4.1 might bear investigation. I'll try turning on the individual optimizations in -O2 one at a time -- starting with -fstrength-reduce -- until I find the one that causes this bug to appear (I hope it's not a /combination/ of optimizations...) Here's the list of optimizations that /this version/ of gcc enables at -O2 but not at -O1 on /this target/: $ echo 'int c;' > test.c $ gcc -fverbose-asm -save-temps -O2 test.c -S -o - | tr ' ' '\n' |\ sed -n '/^-/p' | sort > testO2.l $ gcc -fverbose-asm -save-temps -O1 test.c -S -o - | tr ' ' '\n' |\ sed -n '/^-/p' | sort > testO1.l $ diff testO1.l testO2.l | sed -n '/^> /s/^> //p' -fcaller-saves -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdelete-null-pointer-checks -fexpensive-optimizations -fforce-mem -fgcse -foptimize-register-move -foptimize-sibling-calls -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fschedule-insns2 -fstrength-reduce <<< bug #17860 -fstrict-aliasing -funit-at-a-time -- Chuck |
From: Charles W. <cwi...@us...> - 2010-01-26 02:58:58
|
Charles Wilson wrote: > Well, you'll need a few additional packages: I've rebuilt all the > prereqs using the new toolchain -- including gdbm (*). Plus, there are (*) I forgot to explain the gdbm thing. Here's a clip from the new README: About libgdbm_compat: From time immemorial, libgdbm provided emulation of so-called "system" database interfaces. There were two flavors of these: the 'dbm' interface, and the 'ndbm' interface. libgdbm supports both, as well as its own more flexible interface. Sometime after 2002, the libgdbm maintainers separated the emulation code into a separate library: libgdbm_compat. However, the separation was not pretty: data items which are conceptually private to libgdbm are shared between libgdbm and libgdbm_compat. Stir in Win32 DLLs. Data items are notoriously difficult to deal with, and (normally) require that various compile- time options when compiling clients (such as libgdbm_compat) that wish to access data items exported by the DLL (libgdbm). This is a lot of bother, and on regular mingw and cygwin, gcc and ld provide support for bypassing it all: see 'info ld auto-import'. However, in that past that doesn't work for msys: While gcc-2.95.3 and ld 2.11.90 do support auto-import, they do NOT support the runtime-pseudo-reloc behavior, needed when your DLL exports complicated structs. And wouldn't you know it: one of the data items "shared" between libgdbm and libgdbm_compat is a gigantic struct. But now that we have (msys) gcc 3.4.4 and (msys) binutils 2.19.51.1, plus pseudo-reloc support in the msys DLL, we can do it just like the other platforms. Yay! Short version: I used to hack up gdbm to put the _compat stuff into the main DLL, and then provide two copies of the import lib, libgdbm.dll.a and libgdbm_compat.dll.a. Now, I provide two separate DLLs and two distinct import libraries. -- Chuck |
From: Charles W. <cwi...@us...> - 2010-01-26 09:55:14
|
Charles Wilson wrote: > -fcaller-saves > -fstrength-reduce > -fcrossjumping > -fcse-follow-jumps > -fcse-skip-blocks > -fdelete-null-pointer-checks > -fexpensive-optimizations > -fforce-mem > -fgcse > -foptimize-register-move > -foptimize-sibling-calls > -fpeephole2 > -fregmove > -freorder-blocks > -freorder-functions > -frerun-cse-after-loop > -frerun-loop-opt > -fschedule-insns2 > -fstrict-aliasing > -funit-at-a-time Well, it seems that -funit-at-a-time is the culprit, but only when combined with other flags. Compiling msys DLL using the following flags, and then dropping the new msys DLL into place and running my perl test case (new perl, but without recompiling or relinking perl itself): The following all work: -O1 -O1,+ -fcaller-saves -O1,+ -fstrength-reduce -O1,+ the first 11 switches in the list above -O1,+ the first 16 switches in the list above -O1,+ the first 18 switches in the list above -O1,+ all but -funit-at-a-time but then, explicitly adding all of the switches in the list above, which should be equivalent to -O2: -O1,+all the switches above --- FAILS. (which is expected, I guess, as -O2 by itself fails. But it's interesting that the addition of -funit-at-a-time was the "deciding factor") Next, try just the "offending" switch: -O1,-funit-at-a-time --- WORKS. Hmmm...so, -funit-at-a-time is not enough, by itself, to trigger this bug. However, in combination with (some undetermined set of additional optimizations), it does trigger the bug. Happily: -O2,+ -fno-unit-at-a-time -O3,+ -fno-unit-at-a-time both WORK. -- Chuck |
From: Cesar S. <ces...@gm...> - 2010-01-26 11:37:03
|
Charles Wilson wrote: > > 1.0.13 + new perl: FAIL. > 1.0.12 + new perl: FAIL. > 1.0.11 + new perl: WORKS. > > 1.0.13 + old perl: WORKS. > 1.0.13.debug + new perl: WORKS. > > > So...what do 1.0.12 and 1.0.13 have in common, that 1.0.11 did not? > Oddly, not the toolchain. 1.0.12 was built using > gcc-2.95.3/old-binutils from MsysDVLPR, as far as I can tell, while > 1.0.13 was the first one to be built using the new msys-gcc3.4.4. Just to clarify: 1.0.12 was built using gcc-2.95.3, but I was already using the new binutils (compiled with gcc 3.4.4). An idea is to try rebuilding 1.0.12 without updating binutils, and rebuild 1.0.11 with updated binutils. Cesar |
From: Charles W. <cwi...@us...> - 2010-01-27 02:17:00
|
Cesar Strauss wrote: > Charles Wilson wrote: >> 1.0.13 + new perl: FAIL. >> 1.0.12 + new perl: FAIL. >> 1.0.11 + new perl: WORKS. >> >> 1.0.13 + old perl: WORKS. >> 1.0.13.debug + new perl: WORKS. >> >> >> So...what do 1.0.12 and 1.0.13 have in common, that 1.0.11 did not? >> Oddly, not the toolchain. 1.0.12 was built using >> gcc-2.95.3/old-binutils from MsysDVLPR, as far as I can tell, while >> 1.0.13 was the first one to be built using the new msys-gcc3.4.4. > > Just to clarify: 1.0.12 was built using gcc-2.95.3, but I was already > using the new binutils (compiled with gcc 3.4.4). An idea is to try > rebuilding 1.0.12 without updating binutils, and rebuild 1.0.11 with > updated binutils. Well, I don't think it's necessary to go that far: my other message pinpointed the problem, and the workaround. Also, this thread: http://thread.gmane.org/gmane.comp.gnu.mingw.msys/4712 shows another manifestation of the incorrectly-compiled msys DLL, but in a much simpler fashion (no newly-compiled perl needed). Using the same old gawk-3.1.7 binary from last fall, without recompiling it, you can get broken/working behavior by simply changing MSYS DLLs. With the currently-distributed msys DLL (1.0.13-1): $ gawk '{printf "%g %g\n", -1, 2}' 1 -2 -1 2 1 -2 -1 2 is very very wrong -- and again, is related to (s)printf function calls. However, drop in a newly compiled msys DLL, compiled with -fno-unit-at-a-time, and: $ gawk '{printf "%g %g\n", -1, 2}' -1 2 -1 2 -1 2 -1 2 as expected. Cesar -- it seems pretty clear that there is an optimization bug in the current msys toolchain, which can be avoided by adding -fno-unit-at-a-time. But, the current official MSYS DLL is broken because of this bug. Certainly it would be a good idea to track down and fix the compiler bug -- or perhaps update to gcc-3.4.6 and hope for the best -- but in the interim, can we get a new MSYS dll recompiled with the appropriate flags to work around the issue? I've already got a set of packages (1.0.13-2) that I could upload, if you like. They were built from your original 1.0.13-1-src tarball, and I just updated the .RELEASE_NOTES and msysrlsbld.ini slightly. Because the source code is unchanged, I didn't bump the dll version to 1.0.14; I only incremented the bld number. -- Chuck |
From: Cesar S. <ces...@gm...> - 2010-01-27 11:43:30
|
Charles Wilson wrote: > Cesar -- it seems pretty clear that there is an optimization bug in the > current msys toolchain, which can be avoided by adding > -fno-unit-at-a-time. Thanks for tracking this down, great detective work! > But, the current official MSYS DLL is broken > because of this bug. > Technically, I'd consider 1.0.12 to be the current official MSYS DLL, as 1.0.13 is still in Proposed. I'm glad I waited long enough for issues to crop up before making it official. A big thank you to all the testers. > Certainly it would be a good idea to track down and fix the compiler bug > -- or perhaps update to gcc-3.4.6 and hope for the best I wonder why it doesn't show on Cygwin, tough. > -- but in the > interim, can we get a new MSYS dll recompiled with the appropriate flags > to work around the issue? Sure. > > I've already got a set of packages (1.0.13-2) that I could upload, if > you like. They were built from your original 1.0.13-1-src tarball, and I > just updated the .RELEASE_NOTES and msysrlsbld.ini slightly. Great, feel free to upload as you see fit. I'm interested to know if it solves a failure I was having when building gcc 4.4.3 for mingw, also related to gawk. I had to fall back to 1.0.12 for the build to continue. Perhaps it would be wise to rebuild MSYS gcc and binutils afterwards, just in case. > Because > the source code is unchanged, I didn't bump the dll version to 1.0.14; I > only incremented the bld number. Sounds reasonable. Cesar |
From: Charles W. <cwi...@us...> - 2010-01-28 05:57:40
|
Cesar Strauss wrote: > Technically, I'd consider 1.0.12 to be the current official MSYS DLL, as > 1.0.13 is still in Proposed. I'm glad I waited long enough for issues to > crop up before making it official. A big thank you to all the testers. Ah, but apparently 1.0.12 is also broken in this way, and it, too, exhibits the problematic behavior in both my perl test case and the gawk test case. But 1.0.12 was compiled using old-as-dirt gcc-2.95.3! Curiouser and curiouser. >> Certainly it would be a good idea to track down and fix the compiler bug >> -- or perhaps update to gcc-3.4.6 and hope for the best > > I wonder why it doesn't show on Cygwin, tough. Yeah, that's odd. I've also been thinking a bit about these problems, and how we can move forward with MSYS. I don't think it will be possible to just dive in and build say msys-gcc-4.5.0. There has been some synergistic development over the last few years with cygwin+binutils+gcc. That is, certain features wouldn't work properly in gcc unless support was added or behavior was corrected in cygwin. Then, once that occurred, gcc moved on *assuming the required functionality was present and/or fixed* for host cygwin. Yet, here we are with msys=cygwin-1.3.3, without any of those fixes, blithely trying to add *-*-msys* and __MSYS__ every time we see *-*-cygwin* and __CYGWIN__. But...that won't work, given the synergistic gcc development. I'm starting to think that Earnie has the right idea (more below). Certainly, right now, getting a decent modularized installer like mingw-get up and running, and ironing out all of those bugs will be a big improvement to the mingw/msys user experience. The new msys-gcc/msys-binutils will allow us to make some significant improvements in the msys tools that we ship, over the current set -- esp. with c99 support. But...I'm not sure how much effort we should put into advancing the current MSYS source code. Earnie's suggestion that, perhaps, taking the cygwin-1.7 code and integrating "just enough" changes to implement traditional MSYS behavior -- and thus, inheriting all of the goodness and improvements in the six years of cygwin development since cygwin-1.3.3 -- might be the "Next Big Thing". (OTOH: I'd sure like to see msys-1.0.14 incorporate working stdint.h and inttypes.h headers, which would really ease some of the porting issues in some packages). >> -- but in the >> interim, can we get a new MSYS dll recompiled with the appropriate flags >> to work around the issue? > > Sure. > >> I've already got a set of packages (1.0.13-2) that I could upload, if >> you like. They were built from your original 1.0.13-1-src tarball, and I >> just updated the .RELEASE_NOTES and msysrlsbld.ini slightly. > > Great, feel free to upload as you see fit. Done. > I'm interested to know if it solves a failure I was having when building > gcc 4.4.3 for mingw, also related to gawk. I had to fall back to > 1.0.12 for the build to continue. Odd. Again, I see the "bug" with both 1.0.13-1 and 1.0.12, even though 1.0.12 was compiled with very old gcc. > Perhaps it would be wise to rebuild MSYS gcc and binutils afterwards, > just in case. Couldn't hurt. I'd probably make both BOOT_CFLAGS and CFLAGS include -fno-unit-at-a-time. -- Chuck P.S. While trying to get perl-5.8.8 to work before falling back to perl-5.6.1, I ran into the dreaded "unable to remap" error (actually, it's the older wording of that error -- which I can't recall at the moment) -- but anyway, it's the hoary "you need to rebase your DLLs" problem. So, I ported the dash shell and cygwin's rebase package to msys. They work fine, even tho they did not solve my problem. Should these be added to the MSYS package set? |
From: Cesar S. <ces...@gm...> - 2010-01-29 00:10:29
|
Charles Wilson wrote: > Cesar Strauss wrote: > Ah, but apparently 1.0.12 is also broken in this way, and it, too, > exhibits the problematic behavior in both my perl test case and the gawk > test case. Could you check the gawk test case again? It ran fine for me on 1.0.12. > But...I'm not sure how much effort we should put into advancing the > current MSYS source code. Earnie's suggestion that, perhaps, taking the > cygwin-1.7 code and integrating "just enough" changes to implement > traditional MSYS behavior -- and thus, inheriting all of the goodness > and improvements in the six years of cygwin development since > cygwin-1.3.3 -- might be the "Next Big Thing". > Yes, we do not have the resources to keep the same development pace as Cygwin. I think at least the following changes to cygwin 1.7 code are needed: 1) Change the DLL name. 2) Change the string returned by uname. 3) Path translation of command-line arguments and environment variables for native programs. 4) Replace symlink creation by a copy operation. Fortunately, with Cygwin 1.7 we no longer have to worry about: 5) Mount table in /etc/fstab. 6) Support for multiple parallel installations. But first we need a MSYS port of gcc 4.x. Maybe, to bootstrap the whole thing, we will need to build initially a MSYS-targeted cross-compiler hosted on MinGW or Cygwin (or even Linux). > (OTOH: I'd sure like to see msys-1.0.14 incorporate working stdint.h and > inttypes.h headers, which would really ease some of the porting issues > in some packages). > On my wishlist is a MSYS port of mintty, to replace rxvt. >> I'm interested to know if it solves a failure I was having when building >> gcc 4.4.3 for mingw, also related to gawk. I had to fall back to >> 1.0.12 for the build to continue. 1.0.13-2 solved this issue as well. The symptom was double-negative signs, as in --1. Although the printf format was %d in this case. I didn't bothered to reduce it to a simple test case. > > P.S. While trying to get perl-5.8.8 to work before falling back to > perl-5.6.1, I ran into the dreaded "unable to remap" error (actually, > it's the older wording of that error -- which I can't recall at the > moment) -- but anyway, it's the hoary "you need to rebase your DLLs" > problem. > > So, I ported the dash shell and cygwin's rebase package to msys. They > work fine, even tho they did not solve my problem. Should these be added > to the MSYS package set? Sure, they would be useful tools to have. Cesar |
From: Charles W. <cwi...@us...> - 2010-01-29 02:12:18
|
Cesar Strauss wrote: > Charles Wilson wrote: >> Cesar Strauss wrote: >> Ah, but apparently 1.0.12 is also broken in this way, and it, too, >> exhibits the problematic behavior in both my perl test case and the gawk >> test case. > > Could you check the gawk test case again? It ran fine for me on 1.0.12. $ uname -a MINGW32_NT-6.0 KHELDAR 1.0.12(0.47/3/2) 2010-01-13 00:27 i686 Msys $ gawk '{printf "%g %g\n", -1, 2}' 1 -2 -1 2 1 -2 -1 2 > Yes, we do not have the resources to keep the same development pace as > Cygwin. > > I think at least the following changes to cygwin 1.7 code are needed: > 1) Change the DLL name. It's not just the DLL name. There are a lot of "global" data structures (global in the sense that they are objects in Window's global namespace, where any process can see them). These are usually named "cyg*XXXXX" where -- now -- XXXXX is some hash based on the installation path. Since it is unlikely that someone will install the msys DLL and a cygwin DLL in the same directory, that's probably OK. But for utmost isolation, we'd want to change those names to msys*XXXXXX. > 2) Change the string returned by uname. > 3) Path translation of command-line arguments and environment variables > for native programs. This will be fun. The entire innards of cygwin have been rewritten to use wide char functions for all path translation logic. And, there's ongoing debate about how to handle $LANG and national character sets in the console/mintty. > 4) Replace symlink creation by a copy operation. 4a) cygserver -> msys-server (or kill it; at the cost of removing support for IPC; which might be OK given MSYS's limited goals) 4b) port and distribute some of the winsup/utils -- I'd really like an msys version of cygwin's new ldd program. 4c) UID/GID handling, and security attribute stuff. MSYS ripped most of that out from 1.3.3; but now there's a lot more of it in cygwin-1.7. Ripping it out will be commensurately more difficult, and will have -- I believe -- some effects both far-reaching and difficult to correct. Not impossible, just hard. > Fortunately, with Cygwin 1.7 we no longer have to worry about: > 5) Mount table in /etc/fstab. > 6) Support for multiple parallel installations. Well, REALLY, first we need mingw-get (or a cookbook installation method like your scripts, or mine (mine are based on a modified version of the mingw-get *script* somebody posted to the list a while back) published for general use. Right now, it's too difficult for folks to actually install an up-to-date MSYS/MinGW environment, in a repeatable, documentable way such that *we* can be sure of what *they* have on their system, when *they* reporting bugs or test results with any new software. > But first we need a MSYS port of gcc 4.x. Maybe, to bootstrap the whole > thing, we will need to build initially a MSYS-targeted cross-compiler > hosted on MinGW or Cygwin (or even Linux). It's possible. I'm pretty sure that's the route Earnie took the first time around, when he initially released the original MSYS-1.0.x. >> (OTOH: I'd sure like to see msys-1.0.14 incorporate working stdint.h and >> inttypes.h headers, which would really ease some of the porting issues >> in some packages). >> > > On my wishlist is a MSYS port of mintty, to replace rxvt. Yes, rxvt should die. I want to kill it on cygwin, too... I use console2 nowadays; it's too bad that product is tied to MSVC. >>> I'm interested to know if it solves a failure I was having when building >>> gcc 4.4.3 for mingw, also related to gawk. I had to fall back to >>> 1.0.12 for the build to continue. > > 1.0.13-2 solved this issue as well. The symptom was double-negative > signs, as in --1. Although the printf format was %d in this case. I > didn't bothered to reduce it to a simple test case. Good news. >> So, I ported the dash shell and cygwin's rebase package to msys. They >> work fine, even tho they did not solve my problem. Should these be added >> to the MSYS package set? > > Sure, they would be useful tools to have. OK. I'll upload them at some point. -- Chuck |
From: Keith M. <kei...@us...> - 2010-01-29 21:33:59
|
On Friday 29 January 2010 01:36:10 Charles Wilson wrote: > I use console2 nowadays; You and me both. I did find some of its default key bindings to be unsatisfactory -- e.g. Ctrl-S to activate the configuration dialogue kills info's incremental search -- but that's easy to fix. I even have it configured to start MSYS directly, without using msys.bat > it's too bad that product is tied to MSVC. Indeed. But for that, I would have suggested bundling it as the standard MSYS console -- it claims GPL-2 licensing, in spite of its closed source dependencies. Maybe one day, when I've got some of the more important stuff -- e.g. mingw-get, some enhancements and improved documentation for pdfroff/pdfmark.tmac -- out of the way, I might look into porting it to a more open compiler platform. -- Regards, Keith. |
From: Charles W. <cwi...@us...> - 2010-01-30 20:12:55
|
Charles Wilson wrote: > So, I've been trying to rebuild msys-perl for a few days now. I started > with msysgit's 5.8.8 version, but when that failed I just tried to > rebuild our existing perl-5.6.1 using the new tools (msys-gcc-3.4.4-2, > msys-binutils-2.19.51-2, msysCORE-1.0.13-{bin|dev}. There were a number > of (expected) snags, such as: > > gcc -fnative-struct vs gcc -mms-bitfields > > and the like. However, after I worked through those, I ended up with a > build that coredumps if -O3, but works (mostly) at -O0. I haven't yet > tried -O1/-O2. (I haven't updated to the -3 release of the new msys > toolchain, but AFAICT the differences between Cesar's -2 and -3 releases > are only cosmetic.) FYI, with the latest (1.0.13-2) msys DLL and dev files, and using msys-gcc-3.4.4-3 + msys-binutils2.19.51-3, I was able to successfully rebuild perl-5.6.1 and it had approximately equivalent test results as the one built six months ago using msysDVLPR (e.g. old-msys-binutils and msys-gcc-2.95.3). I used "-O2 -fno-unit-at-a-time" optimization flags, and actually the new perl's test results were somewhat better than before: there were zero regressions compared to msys perl-5.6.1_2-1, and the new perl passed a few additional tests which old perl failed. I plan to update msys-perl to perl-5.8.8 eventually, but for sanity I wanted to ensure that the *existing* tools can all be rebuilt using the new msys toolchain. I'll keep plugging away at rebuilding the MSYS tools using the new toolchain. Once I've rebuilt ALL of them, then I'll upload them all at once. -- Chuck |
From: Vincent R. <fo...@sm...> - 2010-02-02 11:33:17
|
On Sat, 30 Jan 2010 15:12:33 -0500, Charles Wilson <cwi...@us...> wrote: > Charles Wilson wrote: >> So, I've been trying to rebuild msys-perl for a few days now. I started >> with msysgit's 5.8.8 version, but when that failed I just tried to >> rebuild our existing perl-5.6.1 using the new tools (msys-gcc-3.4.4-2, >> msys-binutils-2.19.51-2, msysCORE-1.0.13-{bin|dev}. There were a number >> of (expected) snags, such as: >> >> gcc -fnative-struct vs gcc -mms-bitfields >> >> and the like. However, after I worked through those, I ended up with a >> build that coredumps if -O3, but works (mostly) at -O0. I haven't yet >> tried -O1/-O2. (I haven't updated to the -3 release of the new msys >> toolchain, but AFAICT the differences between Cesar's -2 and -3 releases >> are only cosmetic.) > > FYI, with the latest (1.0.13-2) msys DLL and dev files, and using > msys-gcc-3.4.4-3 + msys-binutils2.19.51-3, I was able to successfully > rebuild perl-5.6.1 and it had approximately equivalent test results as > the one built six months ago using msysDVLPR (e.g. old-msys-binutils and > msys-gcc-2.95.3). > > I used "-O2 -fno-unit-at-a-time" optimization flags, and actually the > new perl's test results were somewhat better than before: there were > zero regressions compared to msys perl-5.6.1_2-1, and the new perl > passed a few additional tests which old perl failed. > > I plan to update msys-perl to perl-5.8.8 eventually, but for sanity I > wanted to ensure that the *existing* tools can all be rebuilt using the > new msys toolchain. > > I'll keep plugging away at rebuilding the MSYS tools using the new > toolchain. Once I've rebuilt ALL of them, then I'll upload them all at > once. > Hi, What is the status of your work ? Thanks |
From: Charles W. <cwi...@us...> - 2010-02-02 14:07:30
|
Vincent Richomme wrote: > On Sat, 30 Jan 2010 15:12:33 -0500, Charles Wilson >> I'll keep plugging away at rebuilding the MSYS tools using the new >> toolchain. Once I've rebuilt ALL of them, then I'll upload them all at >> once. > > What is the status of your work ? Completed rebuilds (incl, in most cases and where applicable, updates to most recent upstream version): ============================ lndir zlib bzip2 crypt gdbm perl autoconf automake libtool termcap libiconv(*) gettext(*) xz regex minires New packages: ============================ dash rebase expat(**) libxml2(**) Todo: ============================ flex bison inetutils openssl openssh libarchive sed m4 mktemp file grep gzip gawk texinfo less rxvt tar diffutils patch findutils groff man cvs vim gmp guile autogen popt cygutils coreutils bash make (*) Now, built as DLLs (**) Needed for future perl update Later: update perl to newer version (5.8.8? 5.10.x?) ditto autogen (5.9.8?) ditto openssh (5.3p1-1) Note -- not even contemplating updating the following packages to newer versions: inetutils coreutils bash Like I said, I'm still plugging away. It'll be another week or two before it's complete. Some might be wondering why I'm bothering, since the current packages, built with the older msys gcc, work fine. Answer: (1) if we're going to be able to use the new msys gcc in the future, then all existing static libraries must be rebuilt since there were a few ABI changes between gcc 2.95.3 and 3.4.x (2) work out the kinks in the new msys gcc itself and related basic components, since this rebuild project is the ultimate stress test. Already found a few, which have led to the msys-1.0.13-2 kernel release and the -3 gcc release. (3) shared DLL versions of libiconv and gettext mean that when other i18n-capable binaries are rebuilt they will be much smaller (1.2MB -> 26kB, times 100 or so executables...it's a significant savings). -- Chuck |
From: Vincent R. <fo...@sm...> - 2010-02-03 09:57:40
|
On Tue, 02 Feb 2010 09:06:56 -0500, Charles Wilson <cwi...@us...> wrote: > Vincent Richomme wrote: >> On Sat, 30 Jan 2010 15:12:33 -0500, Charles Wilson >>> I'll keep plugging away at rebuilding the MSYS tools using the new >>> toolchain. Once I've rebuilt ALL of them, then I'll upload them all at >>> once. >> >> What is the status of your work ? > > Completed rebuilds (incl, in most cases and where applicable, updates to > most recent upstream version): > ============================ > lndir zlib bzip2 crypt gdbm > perl autoconf automake libtool > termcap libiconv(*) gettext(*) > xz regex minires > > New packages: > ============================ > dash rebase expat(**) libxml2(**) > > Todo: > ============================ > flex bison inetutils openssl > openssh libarchive sed m4 > mktemp file grep gzip gawk > texinfo less rxvt tar diffutils > patch findutils groff man cvs > vim gmp guile autogen popt > cygutils coreutils bash make > > (*) Now, built as DLLs > (**) Needed for future perl update > > Later: > update perl to newer version (5.8.8? 5.10.x?) > ditto autogen (5.9.8?) > ditto openssh (5.3p1-1) > > Note -- not even contemplating updating the following packages to newer > versions: > inetutils > coreutils > bash > > Like I said, I'm still plugging away. It'll be another week or two > before it's complete. Some might be wondering why I'm bothering, since > the current packages, built with the older msys gcc, work fine. Answer: > (1) if we're going to be able to use the new msys gcc in the future, > then all existing static libraries must be rebuilt since there were a > few ABI changes between gcc 2.95.3 and 3.4.x (2) work out the kinks in > the new msys gcc itself and related basic components, since this rebuild > project is the ultimate stress test. Already found a few, which have led > to the msys-1.0.13-2 kernel release and the -3 gcc release. (3) shared > DLL versions of libiconv and gettext mean that when other i18n-capable > binaries are rebuilt they will be much smaller (1.2MB -> 26kB, times 100 > or so executables...it's a significant savings). > Ok thanks. Another question: why are you using gcc 3.4.x and not gcc-4.4.x or even better gcc-4.5 ? Same question about binutils because next release(shouldn 't be too long now) add support for more PE-COFF features (last one is possibility to specify section alignement). |
From: Charles W. <cwi...@us...> - 2010-02-03 13:59:02
|
Vincent Richomme wrote: > Another question: why are you using gcc 3.4.x and not gcc-4.4.x or even > better gcc-4.5 ? Because 3.4.4 is the most recent version of gcc that has been ported to MSYS. Until three weeks ago, we were using 2.95.3 and a binutils from 2003. You have to walk before you can run. You can't even compile gcc-4.x with a gcc older than 3.2, so we had to start somewhere, with some intermediate 3.x release before we can even think about updating the *MSYS* compiler to 4.x. Not that I'm in any hurry to do that. > Same question about binutils because next release(shouldn 't be too long > now) add support > for more PE-COFF features (last one is possibility to specify section > alignement). Uhh...why should I care? The new tools are light years better than what we had; there's no need to be greedy. Now we can compile C99 code (like xz) without massive patching; now we can use auto-import + pseudo-relocation, which allows shared versions of libiconv and gettext (among other things). This already will enable a tremendous advance in the MSYS tools we can build and deliver. We have no need for more, right now. I doubt we're going to bother updating MSYS gcc until msys-2.0 is released, which may not occur for a year of two. And...why should YOU care? There are about five people in the world who need to worry about what MSYS compiler is used. The VAST majority of people should care about the *MinGW* compiler -- which is currently at version 4.4.0. -- Chuck |
From: Vincent R. <fo...@sm...> - 2010-02-03 14:04:03
|
On Wed, 03 Feb 2010 08:38:28 -0500, Charles Wilson <cwi...@us...> wrote: > Vincent Richomme wrote: >> Another question: why are you using gcc 3.4.x and not gcc-4.4.x or even >> better gcc-4.5 ? > > Because 3.4.4 is the most recent version of gcc that has been ported to > MSYS. Until three weeks ago, we were using 2.95.3 and a binutils from 2003. > > You have to walk before you can run. You can't even compile gcc-4.x with > a gcc older than 3.2, so we had to start somewhere, with some > intermediate 3.x release before we can even think about updating the > *MSYS* compiler to 4.x. > That makes sense. > Not that I'm in any hurry to do that. > >> Same question about binutils because next release(shouldn 't be too long >> now) add support >> for more PE-COFF features (last one is possibility to specify section >> alignement). > > Uhh...why should I care? The new tools are light years better than what > we had; there's no need to be greedy. Now we can compile C99 code (like > xz) without massive patching; now we can use auto-import + > pseudo-relocation, which allows shared versions of libiconv and gettext > (among other things). This already will enable a tremendous advance in > the MSYS tools we can build and deliver. We have no need for more, right > now. > > I doubt we're going to bother updating MSYS gcc until msys-2.0 is > released, which may not occur for a year of two. > > And...why should YOU care? There are about five people in the world who > need to worry about what MSYS compiler is used. The VAST majority of > people should care about the *MinGW* compiler -- which is currently at > version 4.4.0. > Yes you are right ;-) |