#1196 x86-mingw32-build.sh: Dry run may break subsequent build

closed-fixed
MinGW (57)
2014-08-17
2008-07-29
Dave Korn
No

Trying to run the build script on a Linux 2.6 x86_64-unknown-linux-gnu host. Download stage goes fine, stage 1 binutils builds and installs ok, headers build ok, gcc builds and installs ok, then we get to w32api but the script bombs out with an error:

------------------<snip>------------------
make[2]: Nothing to be done for `install'.
make[2]: Leaving directory
`/home/dk/mingw/tmp/mingw-3.4.5/build-gcc/libiberty/testsuite'
make[1]: Leaving directory
`/home/dk/mingw/tmp/mingw-3.4.5/build-gcc/libiberty'

x86-mingw32-build.sh: stage 1: build w32api ...
x86-mingw32-build.sh: line 139: ../config.guess: No such file or directory
x86-mingw32-build.sh: line 139: ../configure: No such file or directory
x86-mingw32-build.sh: unrecoverable error configuring w32api
------------------<snip>------------------

Without having figured it out yet, there's something wrong in the way that wildcards are used in e.g. setbuilddir:-

------------------<snip>------------------
137 case $COMPONENT in mingw-runtime | w32api)
138 setbuilddir ${COMPONENT}-*
139 $RUN ../configure --prefix="$INSTALL_DIR" --host="$TARGET" \ 140 --build=${BUILD_PLATFORM=`../config.guess`} || die $? \ 141 "$unrecoverable configuring $COMPONENT"
------------------<snip>------------------

... because looking in the build temp dir:

------------------<snip>------------------
[dk@quattro x86-mingw32-build.sh-0.0-20061107-1]$ ls -la ../tmp/mingw-3.4.5/
total 32
drwxr-xr-x 8 dk eng 4096 Jul 29 16:57 .
drwxr-xr-x 3 dk eng 4096 Jul 29 16:52 ..
drwx------ 18 dk eng 4096 Jul 29 16:55 binutils-2.17.50-20060716-1-src
drwxr-xr-x 5 dk eng 4096 Jul 29 16:58 build-gcc
drwxr-xr-x 10 dk eng 4096 Dec 12 2005 gcc-3.4.5-20060117-1
drwxr-xr-x 5 dk eng 4096 Aug 30 2006 mingw-runtime-3.10
drwxr-xr-x 4 dk eng 4096 Jul 29 17:00 w32api-*
drwxr-xr-x 4 dk eng 4096 Apr 14 2006 w32api-3.7
[dk@quattro x86-mingw32-build.sh-0.0-20061107-1]$
------------------<snip>------------------

it becomes clear that the wildcard isn't actually globbing, it's been used in an mkdir invocation. Sure enough, config.guess et. al. live in the real w32api-3.7 dir, not the bogus w32api-\* dir, but, since it got created, the cd in setbuilddir finds the matching name and doesn't glob it or go into the right directory.

I'll try "set -x" and run the whole thing again from scratch to narrow down exactly how/where the dir gets created. I'm not entirely clear how this ever worked though.

setbuilddir()
# usage: setbuilddir parent [subdir]
#
# Ensure that the directory parent/subdir exists, creating it if
# necessary, then make it the current directory.
{
mkdir -p $1/${2-build} && cd $1/${2-build}
}

OK, that makes a bit more sense: if COMPONENT=w32api, then

setbuilddir ${COMPONENT}-*

becomes

mkdir -p w32api-*/ && cd w32api/

Hmm, pardon the rather wordy bug report, but I'm thinking aloud here. It looks to me like the invocation is just plain wrong. Why only 1 arg when two are expected?

Here's my build log for the settings: shouldn't be anything surprising or relevant among them.

x86-mingw32-build.sh: building i586-pc-mingw32 ...
selected components: binutils headers gcc w32api mingw-runtime
selected languages: c,c++

x86-mingw32-build.sh: required packages ...
gcc-core-3.4.5-20060117-1-src.tar.gz
binutils-2.17.50-20060716-1-src.tar.gz
mingw-runtime-3.10-20060909-1-src.tar.gz
w32api-3.7-src.tar.gz
gcc-g++-3.4.5-20060117-1-src.tar.gz

x86-mingw32-build.sh: general build options ...
--with-gcc
--with-gnu-as
--with-gnu-ld
--disable-nls
--disable-debug

x86-mingw32-build.sh: GCC specific build options ...
--enable-threads=win32
--disable-win32-registry
--enable-sjlj-exceptions

Discussion

  • Keith Marshall
    Keith Marshall
    2008-07-30

    Logged In: YES
    user_id=823908
    Originator: NO

    > I'm not entirely clear how this ever worked though.
    >
    > setbuilddir()
    > # usage: setbuilddir parent [subdir]
    > #
    > # Ensure that the directory parent/subdir exists, creating it if
    > # necessary, then make it the current directory.
    > {
    > mkdir -p $1/${2-build} && cd $1/${2-build}
    > }
    >
    > OK, that makes a bit more sense: if COMPONENT=w32api, then
    >
    > setbuilddir ${COMPONENT}-*
    >
    > becomes
    >
    > mkdir -p w32api-*/ && cd w32api/

    Wrong! It becomes

    mkdir -p w32api-*/build && cd w32api-*/build

    A more pertinent question to ask would be, "why, when w32api-3.7/ exists, does `setbuilddir w32api-*' not expand to

    mkdir -p w32api-3.7/build && cd w32api-3.7/build

    as it should"? That's how this has worked, flawlessly, for me and for everyone else who has successfully built a MinGW cross-gcc on their GNU/Linux host, using this very script. Your shell should have globbed this when it parsed the `setbuilddir ${COMPONENT}-*' command line -- it failed. In this respect, this is not a bug in the script; it is your broken shell which is at fault, and you should direct your bug report to the correct quarter.

    However, I do see a possible problem with this function -- if the temporary build tree is not clean, (in respect of having subdirs for only one version each individual component present), then the behaviour of this function call may be undefined. IIRC, that shouldn't be allowed to happen, but this much does merit follow-up.

    Having said the above, what happens if you modify line 138 to read

    setbuilddir `echo ${COMPONENT}-*`

    or

    eval setbuilddir ${COMPONENT}-*

    or the command line within the body of `setbuilddir()' to read

    mkdir -p `echo $1`/${2-build} && cd $1/${2-build}

    ?

    One of those workarounds may make the script more robust, when invoked with a broken shell, such as you appear to be using.

     
  • Dave Korn
    Dave Korn
    2008-07-30

    Logged In: YES
    user_id=2162291
    Originator: YES

    Broken shell? LOL, well, if you say so!

    [dk@ori ~]$ bash --version
    GNU bash, version 3.00.15(1)-release (x86_64-redhat-linux-gnu)
    Copyright (C) 2004 Free Software Foundation, Inc.
    [dk@ori ~]$ sh --version
    GNU bash, version 3.00.15(1)-release (x86_64-redhat-linux-gnu)
    Copyright (C) 2004 Free Software Foundation, Inc.
    [dk@ori ~]$

    Anyway, I'm doing that run with -x to see what's exactly happening. It could conceivably be some interaction with NFS and the NetApp filer I'm using.

    The -x run finished while I was writing this. Without any error this time.

    Ah, <sfx:lightbulb>, I just had a thought: maybe the dry-run I did first created some build dirs and left them lying around and that's what confused it second time round when I did it for real; I remember reading somewhere in the docs it said that --dry-run might do that (but not that it might have adverse consequences).

    I deleted all the stuff in the temp dir before I tried the -x run, so that's probably why it worked this time. I'll try the dry-run-followed-by-plain sequence again and see if I can reproduce it. If that's the case, then it's a real bug: either dry-run should be properly dry, or the build should be robust against having old build dirs lying around, or both. I'll let you know if it repro's.

     
  • Dave Korn
    Dave Korn
    2008-07-30

    Logged In: YES
    user_id=2162291
    Originator: YES

    ROFL, gottit.

    It's a logic bug in --dry-run, plain and simple.

    How is the call to setbuilddir supposed to glob the version out of the dir name when there's nothing to glob because there's no dir because we haven't unpacked the tarball because we're only really doing a dry run?

    It can't, of course, is the answer. So the asterisk is treated as a literal. Here's what ends up in the build tree after a dry-run from clean:

    [dk@ori x86-mingw32-build.sh-0.0-20061107-1]$ find ../tmp/
    ../tmp/
    ../tmp/mingw-3.4.5
    ../tmp/mingw-3.4.5/build-gcc
    ../tmp/mingw-3.4.5/w32api-*
    ../tmp/mingw-3.4.5/w32api-*/build
    ../tmp/mingw-3.4.5/w32api-*/build/mingw-runtime-*
    ../tmp/mingw-3.4.5/w32api-*/build/mingw-runtime-*/build

    Heh, broken shell indeed. All those folks who report it working flawlessly - has nobody ever done a dry run before? Or maybe the normal mode or operation is to do a real run first and only do a dry run afterward - that'd work!

    Or perhaps more plausibly, the only-a-few people who have tested that option knew (whether explicitly or implicitly) to delete the build dir after doing that, but it's not documented anywhere. Unwritten 'cultural' knowledge.

    When someone new does something for the first time and it fails when the same thing works for everyone else, you should consider that they might just have a different workflow from the convention, or that everyone else might have the same culturally-acquired habitual way of doing things, rather than leaping too rapidly at the conclusion that the guy reporting the problem must be doing something stupid or broken! :-)

     
  • Dave Korn
    Dave Korn
    2008-07-30

    bug fix --dry-run creating problematic directories.

     
    Attachments
  • Dave Korn
    Dave Korn
    2008-07-30

    Logged In: YES
    user_id=2162291
    Originator: YES

    Well, adding a couple of $RUN invocations in places where they look missing fixes the problem in as much as no directories are created any more. We still get one error message:

    x86-mingw32-build.sh: line 139: ../config.guess: No such file or directory

    and this is because even if RUN=echo, we still have to try and evaluate the backticks where it says "BUILD_PLATFORM=`../config.guess`" on that line.

    File Added: pr2031691.diff

     
  • Keith Marshall
    Keith Marshall
    2008-07-30

    Logged In: YES
    user_id=823908
    Originator: NO

    Ok Dave, peace.

    You pissed me off with what you probably thought was a witty quip, in the intro to your e-mail relating to this; I thought it was unnecessary, leaving me predisposed to repost "maybe, one rainy day, following a night when the moon turns blue, and I find a couple of idle round tuits lounging about...". You also armed me with just enough uninformed diagnosis to lead me to a wrong conclusion, and so I didn't consider the likely issues seriously enough.

    Thanks for the patch; I will follow it up, but not as a priority, unless someone else picks it up first.

     
  • Keith Marshall
    Keith Marshall
    2009-02-19

    • milestone: --> IINR_-_Include_In_Next_Release
    • assigned_to: nobody --> keithmarshall
    • summary: x86-mingw32-build.sh "config.guess: No such file or dir..." --> x86-mingw32-build.sh: Dry run may break subsequent build
    • status: open --> closed-fixed
     
  • Keith Marshall
    Keith Marshall
    2009-02-19

    Pending check-in later tonight, I've now fixed this; applied your patch, thanks, plus additional pre-definition of BUILD_PLATFORM, in x86-mingw32-build.sh.getopts, to avoid running config.guess in dry-run case.

    Just FTR, if you'd allowed the real run to clean the build directory, at start of build, (as it does by default), this problem wouldn't have arisen. I guess that's why the majority of users, who have successfully used this set of scripts, didn't notice.