mingw-users

[Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

Hi all,

I haven't posted to this list before and I'm not sure if it's the
right place to ask this question. If not then please let me know. I'm
looking for some help in order to resolve a long-standing issue in
using mingw with Python.

Building extension modules for Python with recent versions of mingw
has been broken for some time now. I first encountered the problem
about two years ago and found this SO question
http://stackoverflow.com/questions/6034390/compiling-with-cython-and-mingw-produces-gcc-error-unrecognized-command-line-o
Shortly after that an issue was opened on the Python tracker:
http://bugs.python.org/issue12641
The issue has languished on the tracker for two years since then. As
someone who builds Python extension modules with mingw I have been
manually patching distutils in my own Python installations for some
time now. While this is acceptable for me I'd really like to get this
fixed so that mingw and Python can work together out of the box.

The issue is (as I understand it) caused by the fact that at some
point a mingw release removed the deprecated '-mno-cygwin' command
line option. I believe the option was a relic from when mingw forked
from cygwin's gcc and that it had at some point been turned into a
deprecated no-op before at some other point being removed entirely.
With mingw releases since then it has been necessary to patch
distutils removing the option in order to build Python extension
modules with mingw. That much has been understood since two years ago.
However the patch to simply remove the option has not been accepted. I
think this is because the core Python devs are unsure who is using
mingw with Python, what versions of mingw they are using and what
exactly they are doing with them. This makes it difficult to establish
whether or not the change could break anyone's current workflow.
(distutils in particular is a delicate area in Python since it is used
entirely for installation of third party packages of which there are
many.)

A number of possibilities have been considered (see the issue link
above) but a new one has occurred to me. Official Python releases for
Windows are compiled with MSVC. Currently maintained CPython versions
(2.6+) are compiled with MSVC 2008 or 2010 and link to msvcr90.dll.
This places a constraint on which versions of mingw can be used to
compile extension modules (I remember needing to update my mingw
because of this). This constraint precludes older versions of mingw
and the '-mno-cygwin' constraint precludes newer versions. If support
for linking against msvcr90.dll was added after the '-mno-cygwin'
option was removed (or after it was made a no-op) then it can be
proven that anyone using mingw with official CPython binaries is not
depending on this option. I think that this would help to make the
case for reviving the issue and getting the patch in.

So my question is: Does anyone know if there ever were any versions of
mingw that accepted the '-mno-cygwin' option and were also capable of
linking against msvcr90.dll?

Or even better, can anyone say in which releases of mingw the
following occurred:
1) The '-mno-cygwin' option was made a no-op.
2) The '-mno-cygwin' option stopped being accepted.
3) Support for msvcr90.dll was added.


Thanks in advance,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[LRN <lr...@gm...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 20.05.2013 02:37, Oscar Benjamin wrote:
> Hi all,
> 
> I haven't posted to this list before and I'm not sure if it's the
> right place to ask this question. If not then please let me know. I'm
> looking for some help in order to resolve a long-standing issue in
> using mingw with Python.
> 
> Building extension modules for Python with recent versions of mingw
> has been broken for some time now. I first encountered the problem
> about two years ago and found this SO question
> http://stackoverflow.com/questions/6034390/compiling-with-cython-and-mingw-produces-gcc-error-unrecognized-command-line-o
> Shortly after that an issue was opened on the Python tracker:
> http://bugs.python.org/issue12641
> The issue has languished on the tracker for two years since then.
This is MUCH older than that, see [1].

[1] http://bugs.python.org/issue3871

- -- 
O< ascii ribbon - stop html email! - www.asciiribbon.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (MingW32)

iQEcBAEBAgAGBQJRmVwmAAoJEOs4Jb6SI2CwPhoIAOLGcnUE4DJZJUsNNYPh9raq
2tEIa9ezdQh91ltoq0prR8pYHJCNgKKW6hlyOdMtDB/p8AWTTrucrEgOcEAe2Vac
UMp3hYCBg6w1TpVgSBjcCkhsmMrYE6m9IpKTrpy7mWfYtkNeD9F/Nfw/cDmhLOuZ
FUDeqkqTJ4Vb7v8JRtM4h6HrXSeDesEznIbQmmyT4o4toPJakVy8di06MD9sMzrv
EniIZ/eWGQNCkCpMhu3BiuVosSQCiTnK/xHC4KpTwG++RfxMcJcjeVbcmA0AwAoz
CoAnpk/+y9RDfsHJCmLycU6UbfTNyKboKPgUPvsb/W90Fw6kSfik/8SnLwbAWbQ=
=3TnK
-----END PGP SIGNATURE-----

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 00:11, LRN <lr...@gm...> wrote:
> On 20.05.2013 02:37, Oscar Benjamin wrote:
>>
>> Building extension modules for Python with recent versions of mingw
>> has been broken for some time now. I first encountered the problem
>> about two years ago and found this SO question
>> http://stackoverflow.com/questions/6034390/compiling-with-cython-and-mingw-produces-gcc-error-unrecognized-command-line-o
>> Shortly after that an issue was opened on the Python tracker:
>> http://bugs.python.org/issue12641
>> The issue has languished on the tracker for two years since then.
> This is MUCH older than that, see [1].
>
> [1] http://bugs.python.org/issue3871

Thanks for responding.

There are many issues with mingw and Python but the key one that I am
trying to resolve and the only one that I have directly experienced is
the '-mno-cygwin' issue. I see no reference to it there until 6 months
ago:
http://bugs.python.org/issue3871#msg178631

In general I think that in Python there will be a move away from using
distutils that will in the long term provide a better fix for most of
these problems. Until that happens we're stuck using distutils which
has a lot of inertia around applying fixes (probably the reason those
patches were not accepted). However I would like to try and push for a
fix for this particular problem as it thwarts *any* attempt to build
using mingw without (monkey-)patching the Python standard library.


Thanks,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Renato S. <br....@gm...>]

2013/5/19 Oscar Benjamin <osc...@gm...>

>
> Or even better, can anyone say in which releases of mingw the
> following occurred:
> 1) The '-mno-cygwin' option was made a no-op.
> 2) The '-mno-cygwin' option stopped being accepted.
> 3) Support for msvcr90.dll was added.
>


2) I think it has been removed in GCC 4.6.

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 05:19, Renato Silva <br....@gm...> wrote:
>
> 2) I think it has been removed in GCC 4.6.

Thanks.

Is there a web accessible changelog or similar that I could use to confirm this?


Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Keith M. <kei...@us...>]

On 19 May 2013 23:37, Oscar Benjamin wrote:

>
> So my question is: Does anyone know if there ever were any versions of
> mingw that accepted the '-mno-cygwin' option and were also capable of
> linking against msvcr90.dll?
>

Any version which does accept -mno-cygwin should be capable of linking
against MSVCR90.DLL, provided you have the appropriate import lib, and
adjust the specs file accordingly; (certainly true for GCC-3.x onward,
until the cygwin folks declared -mno-cygwin officially dead).  However,
such linking capability is not supported by default; *you* must tweak
your MinGW configuration to support it.

Or even better, can anyone say in which releases of mingw the
> following occurred:
> 1) The '-mno-cygwin' option was made a no-op.
>

 MinGW has *never*, AFAIK, *required* -mno-cygwin; it has *always* been a
no-op, (certainly throughout the ten or more years I've been using, and
have been associated with MinGW; I have *never* used -mno-cygwin).

2) The '-mno-cygwin' option stopped being accepted.
>

You'll have to ask the cygwin folks; they are the architects of the
-mno-cygwin madness, and it is only *their* native GCC which has ever
implemented it as anything other than a no-op.  I believe that they
began the deprecation process during the transition from GCC-3.x to
GCC-4.x, but I've no idea how far inertia carried support into 4.x.


> 3) Support for msvcr90.dll was added.
>

I don't know which version of GCC was prevalent at the time, but the
ChangeLog indicates that it first appeared in mingwrt-3.14, towards the
end of 2007; (this may have been during the interregnum, when we lacked
an active maintainer to bring us forward from GCC-3.x to GCC-4.x).

-- 
Regards,
Keith.

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 09:50, Keith Marshall
<kei...@us...> wrote:
> On 19 May 2013 23:37, Oscar Benjamin wrote:
>>
>>
>> So my question is: Does anyone know if there ever were any versions of
>> mingw that accepted the '-mno-cygwin' option and were also capable of
>> linking against msvcr90.dll?
>
> Any version which does accept -mno-cygwin should be capable of linking
> against MSVCR90.DLL, provided you have the appropriate import lib, and
> adjust the specs file accordingly; (certainly true for GCC-3.x onward,
> until the cygwin folks declared -mno-cygwin officially dead).  However,
> such linking capability is not supported by default; *you* must tweak
> your MinGW configuration to support it.

Ah, I wondered if that would be the case.

>
>> Or even better, can anyone say in which releases of mingw the
>> following occurred:
>> 1) The '-mno-cygwin' option was made a no-op.
>
>  MinGW has *never*, AFAIK, *required* -mno-cygwin; it has *always* been a
> no-op, (certainly throughout the ten or more years I've been using, and
> have been associated with MinGW; I have *never* used -mno-cygwin).

Well this is better than I thought. If the '-mno-cygwin' option was
*never* meaningful for mingw then this is a very simple problem. I
wonder why this wasn't spotted before. Is there a way for me to
confirm this by e.g. having a link to the source of an old revision of
mingw?


>> 2) The '-mno-cygwin' option stopped being accepted.
>
> You'll have to ask the cygwin folks; they are the architects of the
> -mno-cygwin madness, and it is only *their* native GCC which has ever
> implemented it as anything other than a no-op.  I believe that they
> began the deprecation process during the transition from GCC-3.x to
> GCC-4.x, but I've no idea how far inertia carried support into 4.x.

If the option was never meaningful for mingw then I think that
whatever cygwin were doing with it is thankfully irrelevant for this
problem. CPython's distutils mixes up the code for cygwin and mingw a
little but the '-mno-cygwin' option is only used for mingw:
http://hg.python.org/cpython/file/42dda22a6f8c/Lib/distutils/cygwinccompiler.py#l274

>> 3) Support for msvcr90.dll was added.
>
> I don't know which version of GCC was prevalent at the time, but the
> ChangeLog indicates that it first appeared in mingwrt-3.14, towards the
> end of 2007; (this may have been during the interregnum, when we lacked
> an active maintainer to bring us forward from GCC-3.x to GCC-4.x).

So that would probably be before the option was removed. Perhaps the
msvcr90.dll question is not so helpful for this.

BTW where is the changelog and can it be viewed from the web? I'm
having trouble locating it.


Thanks,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Keith M. <kei...@us...>]

On 20 May 2013 12:35, Oscar Benjamin wrote:

> On 20 May 2013 09:50, Keith Marshall wrote:
> > On 19 May 2013 23:37, Oscar Benjamin wrote:
> >>
> >>
> >> Or even better, can anyone say in which releases of mingw the
> >> following occurred:
> >> 1) The '-mno-cygwin' option was made a no-op.
> >
> > MinGW has *never*, AFAIK, *required* -mno-cygwin; it has *always* been a
> > no-op, (certainly throughout the ten or more years I've been using, and
> > have been associated with MinGW; I have *never* used -mno-cygwin).
>
> Well this is better than I thought. If the '-mno-cygwin' option was
> *never* meaningful for mingw then this is a very simple problem. I
> wonder why this wasn't spotted before. Is there a way for me to
> confirm this by e.g. having a link to the source of an old revision of
> mingw?
>

The sources for *all* previous MinGW releases are available from SF, but
you may need to search the directory trees on one of their mirror sites
to find them; they aren't readily visible on the browseable download
pages.

>> 2) The '-mno-cygwin' option stopped being accepted.
> >
> > You'll have to ask the cygwin folks; they are the architects of the
> > -mno-cygwin madness, and it is only *their* native GCC which has ever
> > implemented it as anything other than a no-op.  I believe that they
> > began the deprecation process during the transition from GCC-3.x to
> > GCC-4.x, but I've no idea how far inertia carried support into 4.x.
>
> If the option was never meaningful for mingw then I think that
> whatever cygwin were doing with it is thankfully irrelevant for this
> problem.


The option was only ever meaningful for *cygwin* native GCC; it was a
nasty kludge they implemented to hide the mechanics of cross-compiling
from their users, while actually invoking mingw32 GCC to cross compile.
Why would native mingw32 GCC have needed such a kludge?

BTW, your earlier allusion to MinGW having forked from cygwin is flawed;
MinGW never did fork from cygwin, (other than in the creation of MSYS);
the only reason mingw32 GCC accepted -mno-cygwin was because the cygwin
folks had it adopted upstream, as a standard option for WIN32 targets.

CPython's distutils mixes up the code for cygwin and mingw a
> little but the '-mno-cygwin' option is only used for mingw:
>

This makes no sense whatsoever; -mno-cygwin is only meaningful for
*cygwin*, to force its native GCC to redirect its activity so as to
behave as a wrapper for a mingw32 cross-GCC.

BTW where is the changelog ...


<sigh> I shouldn't need to say this; you can find it in the source
tarball for the current MinGW runtime (mingwrt) release, which *is*
available from our SF download pages, (and visible there).
Alternatively, you may just:

  mingw-get source mingw32-mingwrt

to download it via mingw-get, or:

  mingw-get source mingw32-mingwrt --print-uri

to see the download reference.

... and can it be viewed from the web?


Yes.


> I'm having trouble locating it.
>

Since we stopped hosting alongside cygwin, and merged mingwrt and w32api
into a common Windows System Libraries package, it has been moved to the
doc/historical tree of our WSL git repository:
https://sourceforge.net/p/mingw/mingw-org-wsl/ci/master/tree/doc/historical/mingw/ChangeLog

-- 
Regards,
Keith.

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 13:30, Keith Marshall
<kei...@us...> wrote:
> On 20 May 2013 12:35, Oscar Benjamin wrote:
>>
>> On 20 May 2013 09:50, Keith Marshall wrote:
>> > On 19 May 2013 23:37, Oscar Benjamin wrote:
>> >>
>> >>
>> >> Or even better, can anyone say in which releases of mingw the
>> >> following occurred:
>> >> 1) The '-mno-cygwin' option was made a no-op.
>> >
>> > MinGW has *never*, AFAIK, *required* -mno-cygwin; it has *always* been a
>> > no-op, (certainly throughout the ten or more years I've been using, and
>> > have been associated with MinGW; I have *never* used -mno-cygwin).
>>
>> Well this is better than I thought. If the '-mno-cygwin' option was
>> *never* meaningful for mingw then this is a very simple problem. I
>> wonder why this wasn't spotted before. Is there a way for me to
>> confirm this by e.g. having a link to the source of an old revision of
>> mingw?
>
> The sources for *all* previous MinGW releases are available from SF, but
> you may need to search the directory trees on one of their mirror sites
> to find them; they aren't readily visible on the browseable download
> pages.

Okay, thanks. I guess given your comments below this is unnecessary anyway.

>> >> 2) The '-mno-cygwin' option stopped being accepted.
>> >
>> > You'll have to ask the cygwin folks; they are the architects of the
>> > -mno-cygwin madness, and it is only *their* native GCC which has ever
>> > implemented it as anything other than a no-op.  I believe that they
>> > began the deprecation process during the transition from GCC-3.x to
>> > GCC-4.x, but I've no idea how far inertia carried support into 4.x.
>>
>> If the option was never meaningful for mingw then I think that
>> whatever cygwin were doing with it is thankfully irrelevant for this
>> problem.
>
>
> The option was only ever meaningful for *cygwin* native GCC; it was a
> nasty kludge they implemented to hide the mechanics of cross-compiling
> from their users, while actually invoking mingw32 GCC to cross compile.
> Why would native mingw32 GCC have needed such a kludge?

Brilliant.

>
> BTW, your earlier allusion to MinGW having forked from cygwin is flawed;
> MinGW never did fork from cygwin, (other than in the creation of MSYS);
> the only reason mingw32 GCC accepted -mno-cygwin was because the cygwin
> folks had it adopted upstream, as a standard option for WIN32 targets.
Ah, okay.

>> CPython's distutils mixes up the code for cygwin and mingw a
>> little but the '-mno-cygwin' option is only used for mingw:
>
> This makes no sense whatsoever; -mno-cygwin is only meaningful for
> *cygwin*, to force its native GCC to redirect its activity so as to
> behave as a wrapper for a mingw32 cross-GCC.

Perfect. Thanks for confirming this.

>> BTW where is the changelog ...
>
> <sigh> I shouldn't need to say this; you can find it in the source
> tarball for the current MinGW runtime (mingwrt) release, which *is*
> available from our SF download pages, (and visible there).
> Alternatively, you may just:

Sorry I'm unfamiliar with the way that sourceforge hosts things and
also unsure exactly which project's changelog I should be looking at.
The link you posted was the one I was after as it shows where the
msvcr90.dll target was added.


Thanks,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Earnie B. <ea...@us...>]

On Mon, May 20, 2013 at 8:30 AM, Keith Marshall wrote:
> On 20 May 2013 12:35, Oscar Benjamin wrote:
>> BTW where is the changelog ...
>
>
> <sigh> I shouldn't need to say this; you can find it in the source
> tarball for the current MinGW runtime (mingwrt) release, which *is*
> available from our SF download pages, (and visible there).
> Alternatively, you may just:
>

I think he meant for GCC.

>   mingw-get source mingw32-mingwrt
>

mingw-get source mingw32-gcc

> to download it via mingw-get, or:
>
>   mingw-get source mingw32-mingwrt --print-uri
>
> to see the download reference.
>
>> ... and can it be viewed from the web?
>
>
> Yes.
>
>>
>> I'm having trouble locating it.
>
>
> Since we stopped hosting alongside cygwin, and merged mingwrt and w32api
> into a common Windows System Libraries package, it has been moved to the
> doc/historical tree of our WSL git repository:
> https://sourceforge.net/p/mingw/mingw-org-wsl/ci/master/tree/doc/historical/mingw/ChangeLog
>

For the GCC repository you would need to go look at http://gcc.gnu.org.

-- 
Earnie
-- https://sites.google.com/site/earnieboyd

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Keith M. <kei...@us...>]

On 20 May 2013 13:44, Earnie Boyd wrote:

> On Mon, May 20, 2013 at 8:30 AM, Keith Marshall wrote:
> > On 20 May 2013 12:35, Oscar Benjamin wrote:
> >> BTW where is the changelog ...
> >
> > <sigh> I shouldn't need to say this; you can find it in the source
> > tarball for the current MinGW runtime (mingwrt) release, which *is*
> > available from our SF download pages, (and visible there).
> > Alternatively, you may just:
>
> I think he meant for GCC.
>

Nope.  It's mingwrt which provides support for linking to MSVCR90.DLL.
I'd previously referred him to the ChangeLog for that; he was asking
where it might be found.

-- 
Regards,
Keith.

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Earnie B. <ea...@us...>]

On Mon, May 20, 2013 at 7:35 AM, Oscar Benjamin wrote:
> On 20 May 2013 09:50, Keith Marshall wrote:
>>
>>> Or even better, can anyone say in which releases of mingw the
>>> following occurred:
>>> 1) The '-mno-cygwin' option was made a no-op.
>>
>>  MinGW has *never*, AFAIK, *required* -mno-cygwin; it has *always* been a
>> no-op, (certainly throughout the ten or more years I've been using, and
>> have been associated with MinGW; I have *never* used -mno-cygwin).
>
> Well this is better than I thought. If the '-mno-cygwin' option was
> *never* meaningful for mingw then this is a very simple problem. I
> wonder why this wasn't spotted before. Is there a way for me to
> confirm this by e.g. having a link to the source of an old revision of
> mingw?

There is no need to confirm if removing -mno-cygwin will break
anything, we can assure you that it should not at least as far as
MinGW GCC is concerned.  We do not know if Python did anything special
because of it but I would doubt it.

As for support of when it was removed, you'll need to go look at the
http://gcc.gnu.org ChangeLog to see if you can find it.  For Cygwin it
was a decision made for the 1.7.x release because it became a
nightmare for them to support.  It was much easier for them to
actually use a cross compiler method.

-- 
Earnie
-- https://sites.google.com/site/earnieboyd

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 13:50, Earnie Boyd <ea...@us...> wrote:
> On Mon, May 20, 2013 at 7:35 AM, Oscar Benjamin wrote:
>> On 20 May 2013 09:50, Keith Marshall wrote:
>>>
>>>> Or even better, can anyone say in which releases of mingw the
>>>> following occurred:
>>>> 1) The '-mno-cygwin' option was made a no-op.
>>>
>>>  MinGW has *never*, AFAIK, *required* -mno-cygwin; it has *always* been a
>>> no-op, (certainly throughout the ten or more years I've been using, and
>>> have been associated with MinGW; I have *never* used -mno-cygwin).
>>
>> Well this is better than I thought. If the '-mno-cygwin' option was
>> *never* meaningful for mingw then this is a very simple problem. I
>> wonder why this wasn't spotted before. Is there a way for me to
>> confirm this by e.g. having a link to the source of an old revision of
>> mingw?
>
> There is no need to confirm if removing -mno-cygwin will break
> anything, we can assure you that it should not at least as far as
> MinGW GCC is concerned.  We do not know if Python did anything special
> because of it but I would doubt it.
>
> As for support of when it was removed, you'll need to go look at the
> http://gcc.gnu.org ChangeLog to see if you can find it.  For Cygwin it
> was a decision made for the 1.7.x release because it became a
> nightmare for them to support.  It was much easier for them to
> actually use a cross compiler method.

Thanks for this. That's what I wanted to know. I'll go look there.


Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Paul M. <p.f...@gm...>]

On 20 May 2013 13:50, Earnie Boyd <ea...@us...> wrote:

> There is no need to confirm if removing -mno-cygwin will break
> anything, we can assure you that it should not at least as far as
> MinGW GCC is concerned.  We do not know if Python did anything special
> because of it but I would doubt it.
>

I think that the issue for Python (and if true, this is OT for the mingw
list, obviously, I only post here to close the loop for people following
the thread here) is that the distutils support for "building Python
extensions using mingw" did not have a very clear picture of what
constituted the mingw compiler and what constituted the cygwin one. (I may
be wrong, as I wasn't around when the code was written, but that's how it
looks to me). Keith alluded to the confusing cygwin situation earlier in
the thread, so I guess you're aware of how people can get things muddled :-)

So building an extension using --compiler=mingw in Python could pick up a
cygwin gcc if that was on PATH, and this will work as long as -mno-cygwin
is passed on the command line. But it won't work (it will build a DLL with
a dependency on the cygwin DLL) if -mno-cygwin is omitted. I'd argue that
people should just install and use mingw rather than cygwin, but that may
not be what everyone does in practice.

The real problem here is that no-one knows if people actually try to build
(native windows) Python extensions by installing Cygwin and using distutils
with Cygwin's gcc on PATH. I suspect that some do, but they may not have
been able to do so successfully for some years because of the runtime
compatibility issues. That's essentially the OP's question, but maybe it
needs directing at the cygwin folks.

Paul

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 14:13, Paul Moore <p.f...@gm...> wrote:
> On 20 May 2013 13:50, Earnie Boyd <ea...@us...> wrote:
>>
>> There is no need to confirm if removing -mno-cygwin will break
>> anything, we can assure you that it should not at least as far as
>> MinGW GCC is concerned.  We do not know if Python did anything special
>> because of it but I would doubt it.
>
>
> I think that the issue for Python (and if true, this is OT for the mingw
> list, obviously, I only post here to close the loop for people following the
> thread here) is that the distutils support for "building Python extensions
> using mingw" did not have a very clear picture of what constituted the mingw
> compiler and what constituted the cygwin one. (I may be wrong, as I wasn't
> around when the code was written, but that's how it looks to me). Keith
> alluded to the confusing cygwin situation earlier in the thread, so I guess
> you're aware of how people can get things muddled :-)
>
> So building an extension using --compiler=mingw in Python could pick up a
> cygwin gcc if that was on PATH, and this will work as long as -mno-cygwin is
> passed on the command line. But it won't work (it will build a DLL with a
> dependency on the cygwin DLL) if -mno-cygwin is omitted. I'd argue that
> people should just install and use mingw rather than cygwin, but that may
> not be what everyone does in practice.

Oh, okay. That seems a plausible explanation for the origin of this bug.

> The real problem here is that no-one knows if people actually try to build
> (native windows) Python extensions by installing Cygwin and using distutils
> with Cygwin's gcc on PATH. I suspect that some do, but they may not have
> been able to do so successfully for some years because of the runtime
> compatibility issues. That's essentially the OP's question, but maybe it
> needs directing at the cygwin folks.

I guess I'll still need go check what cygwin did with the option then.

Thanks to everyone here. I think I have all the information that is
needed concerning mingw.


Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Earnie B. <ea...@us...>]

On Mon, May 20, 2013 at 9:13 AM, Paul Moore wrote:
> So building an extension using --compiler=mingw in Python could pick up a
> cygwin gcc if that was on PATH, and this will work as long as -mno-cygwin is
> passed on the command line. But it won't work (it will build a DLL with a
> dependency on the cygwin DLL) if -mno-cygwin is omitted. I'd argue that
> people should just install and use mingw rather than cygwin, but that may
> not be what everyone does in practice.

No!!! The -mno-cygwin abomination is dead.  If you want to build a
native Python using Cygwin you would do it the cross compiler way and
state the --host you're configuring for.  Python's distutil needs to
remove the -mno-cygwin option.

>
> The real problem here is that no-one knows if people actually try to build
> (native windows) Python extensions by installing Cygwin and using distutils
> with Cygwin's gcc on PATH. I suspect that some do, but they may not have
> been able to do so successfully for some years because of the runtime
> compatibility issues. That's essentially the OP's question, but maybe it
> needs directing at the cygwin folks.
>

If they do they'll need to use the cross compilers.

-- 
Earnie
-- https://sites.google.com/site/earnieboyd

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 14:54, Earnie Boyd <ea...@us...> wrote:
> On Mon, May 20, 2013 at 9:13 AM, Paul Moore wrote:
>> So building an extension using --compiler=mingw in Python could pick up a
>> cygwin gcc if that was on PATH, and this will work as long as -mno-cygwin is
>> passed on the command line. But it won't work (it will build a DLL with a
>> dependency on the cygwin DLL) if -mno-cygwin is omitted. I'd argue that
>> people should just install and use mingw rather than cygwin, but that may
>> not be what everyone does in practice.
>
> No!!! The -mno-cygwin abomination is dead.  If you want to build a
> native Python using Cygwin you would do it the cross compiler way and
> state the --host you're configuring for.  Python's distutil needs to
> remove the -mno-cygwin option.

I think given what's been said in this thread that it's a no-brainer
that this patch needs to go into the next Python version. However,
there is also the issue of older releases. If this is a bug and not a
feature then it can possibly be fixed in already released Python
versions.

However a change of that sort needs to be demonstrably
backward-compatible. If someone was using an older version of cygwin
in the way that Paul describes then fixing distutils this way in a
bugfix release could break some system that they have and this may not
be considered acceptable. This is why I was trying to prove that no
version of mingw could exist to satisfy the constraints imposed by
Python/distutils.

I see now that there is no problem with mingw and that cygwin is in
fact the issue and I'm trying to pin down when the change occurred in
cygwin. So far grepping the gcc change logs has been inconclusive:
Documentation of the deprecation of -mno-cygwin was added in 2009
which I guess is the patch discussed in these threads:
http://gcc.gnu.org/ml/gcc-patches/2009-03/msg01051.html
http://cygwin.com/ml/cygwin/2009-03/msg00772.html
Before that the next reference to -mno-cygwin is in 2003. The
changelog entry "Disable spec when -mno-cygwin" from 2010-04-01 is the
closest I can find to the option being removed from gcc:
http://gcc.gnu.org/viewcvs/gcc/trunk/gcc/ChangeLog-2010?view=co

So I can't find when it was deprecated, it was not previously a no-op
in cygwin, and it was only removed three years ago. This means that
there could be people using -mno-cygwin option in the way that Paul
describes (although the scenario does sound a little absurd).

>>
>> The real problem here is that no-one knows if people actually try to build
>> (native windows) Python extensions by installing Cygwin and using distutils
>> with Cygwin's gcc on PATH. I suspect that some do, but they may not have
>> been able to do so successfully for some years because of the runtime
>> compatibility issues. That's essentially the OP's question, but maybe it
>> needs directing at the cygwin folks.
>
> If they do they'll need to use the cross compilers.

The msvcr90 target was added to mingw in Nov 2007 according to
http://sourceforge.net/p/mingw/mingw-org-wsl/ci/master/tree/doc/historical/mingw/ChangeLog
This is 2-3 years before the -mno-cygwin option was disabled in gcc.
However, it seems that what I really need to find is when the msvcr90
target was usable from cygwin which I am trying to do now.


Thanks again for your help,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Paul M. <p.f...@gm...>]

On 20 May 2013 14:54, Earnie Boyd <ea...@us...> wrote:

> On Mon, May 20, 2013 at 9:13 AM, Paul Moore wrote:
> > So building an extension using --compiler=mingw in Python could pick up a
> > cygwin gcc if that was on PATH, and this will work as long as
> -mno-cygwin is
> > passed on the command line. But it won't work (it will build a DLL with a
> > dependency on the cygwin DLL) if -mno-cygwin is omitted. I'd argue that
> > people should just install and use mingw rather than cygwin, but that may
> > not be what everyone does in practice.
>
> No!!! The -mno-cygwin abomination is dead.  If you want to build a
> native Python using Cygwin you would do it the cross compiler way and
> state the --host you're configuring for.  Python's distutil needs to
> remove the -mno-cygwin option.
>

Sorry - I gave the wrong impression. Yes, this is a completely wrong
approach. And no, nobody should be doing this. The Python fix (to get rid
of -mno-cygwin from the flags distutils supplies) is held up by backward
compatibility concerns that people *are* doing this and making them change
their process is a compatibility breakage (for better or worse, distutils
is in a "somewhat frozen" state where all compatibility breakage, no matter
how justified, is essentially impossible to get approved).

The OP is trying to gather evidence that it is *not possible* for people to
be relying on the existing broken behaviour, because it cannot produce
viable executables (the only versions of gcc that could exist anywhere
which accept -mno-cygwin, do not support building with the version of
MSVCRT that Python needs). If he can *prove* that there are no users who
will be adversely impacted by the change, it might be possible to get the
change moving at last.

The change the OP is pursuing is precisely what you suggest - removing
-mno-cygwin from Python's distutils.

Paul

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Greg C. <gch...@sb...>]

On 2013-05-20 14:54Z, Paul Moore wrote:
[...]
> The OP is trying to gather evidence that it is *not possible* for people to
> be relying on the existing broken behaviour, because it cannot produce
> viable executables (the only versions of gcc that could exist anywhere
> which accept -mno-cygwin, do not support building with the version of
> MSVCRT that Python needs). If he can *prove* that there are no users who
> will be adversely impacted by the change, it might be possible to get the
> change moving at last.

http://cygwin.com/ml/cygwin/2009-03/msg00802.html
|  gcc-3 -mno-cygwin still works just as well (or badly!) as it ever has done,
| and will be retained for ever.

Some people may still be using Cygwin gcc version 3, which needs
'-mno-cygwin'. Those are the only people who might be affected.

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 20 May 2013 16:58, Greg Chicares <gch...@sb...> wrote:
> On 2013-05-20 14:54Z, Paul Moore wrote:
> [...]
>> The OP is trying to gather evidence that it is *not possible* for people to
>> be relying on the existing broken behaviour, because it cannot produce
>> viable executables (the only versions of gcc that could exist anywhere
>> which accept -mno-cygwin, do not support building with the version of
>> MSVCRT that Python needs). If he can *prove* that there are no users who
>> will be adversely impacted by the change, it might be possible to get the
>> change moving at last.
>
> http://cygwin.com/ml/cygwin/2009-03/msg00802.html
> |  gcc-3 -mno-cygwin still works just as well (or badly!) as it ever has done,
> | and will be retained for ever.

Brilliant. Thanks for that. The next part of that message is also very
interesting:

 | gcc-4 series releases will not support it at all.  As the whole thing is
 | (still) experimental and explicitly warned to be unstable I don't
see the need
 | to go for a deprecation period.

I guess that means that cygwin users of the arrangement that Paul
described should also see this bug when using any gcc 4.x. This is
certainly true of gcc 4.5.3 in my own old and unused cygwin
installation:

$ gcc --version
gcc (GCC) 4.5.3
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ gcc hello.c
$ ./a.exe
Hello World
$ gcc -mno-cygwin hello.c
gcc: The -mno-cygwin flag has been removed; use a mingw-targeted cross-compiler.

If there were people using distutils' mingw32 mode with cygwin's gcc I
would have expected this to have been reported on the Python issue
tracker but it is not. This seems like reasonable evidence that people
are not actually using this setup even if I can't prove that it is
impossible.

> Some people may still be using Cygwin gcc version 3, which needs
> '-mno-cygwin'. Those are the only people who might be affected.

This is the only possible breakage and it concerns a possibly
non-existent group of users who are abusing distutils' mingw32 mode to
compile with a now-abandoned, experimental feature of a legacy cygwin
compiler. Hopefully this obscure, possible breakage can be outweighed
by the definite, known breakage in distutils' support for using any
recent and future mingw compilers in the appropriate way as described
in the distutils docs.

Thanks everyone. I think I have more than enough information now. I'll
take this over to the Python issue tracker.


Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Keith M. <kei...@us...>]

On 21 May 2013 12:34, Oscar Benjamin wrote:

> On 20 May 2013 16:58, Greg Chicares wrote:
>
> Some people may still be using Cygwin gcc version 3, which needs
> > '-mno-cygwin'. Those are the only people who might be affected.
>
> This is the only possible breakage and it concerns a possibly
> non-existent group of users who are abusing distutils' mingw32 mode to
> compile with a now-abandoned, experimental feature of a legacy cygwin
> compiler.
>

One related point you may wish to consider: even if such users of legacy
cygwin do exist, there's nothing to prevent them from setting up more
conventional mingw32-gcc cross-compiler configurations, within their
existing cygwin installations.  They should be strongly encouraged to do
so, thus allowing -mno-cygwin to be consigned, forevermore, to the scrap
heap of history, where it rightfully belongs.

-- 
Regards,
Keith.

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Earnie B. <ea...@us...>]

On Tue, May 21, 2013 at 8:25 AM, Keith Marshall wrote:
> On 21 May 2013 12:34, Oscar Benjamin wrote:
>>
>> On 20 May 2013 16:58, Greg Chicares wrote:
>>
>> > Some people may still be using Cygwin gcc version 3, which needs
>> > '-mno-cygwin'. Those are the only people who might be affected.
>>
>> This is the only possible breakage and it concerns a possibly
>> non-existent group of users who are abusing distutils' mingw32 mode to
>> compile with a now-abandoned, experimental feature of a legacy cygwin
>> compiler.
>
>
> One related point you may wish to consider: even if such users of legacy
> cygwin do exist, there's nothing to prevent them from setting up more
> conventional mingw32-gcc cross-compiler configurations, within their
> existing cygwin installations.  They should be strongly encouraged to do
> so, thus allowing -mno-cygwin to be consigned, forevermore, to the scrap
> heap of history, where it rightfully belongs.

Setting a requirement for a specific version of GCC within the
distutils package should be easy enough.  Those with older versions of
GCC would just need to use the older distutils package.

-- 
Earnie
-- https://sites.google.com/site/earnieboyd

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 21 May 2013 14:27, Earnie Boyd <ea...@us...> wrote:
> On Tue, May 21, 2013 at 8:25 AM, Keith Marshall wrote:
>> On 21 May 2013 12:34, Oscar Benjamin wrote:
>>>
>>> On 20 May 2013 16:58, Greg Chicares wrote:
>>>
>>> > Some people may still be using Cygwin gcc version 3, which needs
>>> > '-mno-cygwin'. Those are the only people who might be affected.
>>>
>>> This is the only possible breakage and it concerns a possibly
>>> non-existent group of users who are abusing distutils' mingw32 mode to
>>> compile with a now-abandoned, experimental feature of a legacy cygwin
>>> compiler.
>>
>>
>> One related point you may wish to consider: even if such users of legacy
>> cygwin do exist, there's nothing to prevent them from setting up more
>> conventional mingw32-gcc cross-compiler configurations, within their
>> existing cygwin installations.  They should be strongly encouraged to do
>> so, thus allowing -mno-cygwin to be consigned, forevermore, to the scrap
>> heap of history, where it rightfully belongs.

I'm not sure but I think that further modifications to distutils would
be necessary to support cygwin's cross-compilers:
http://bugs.python.org/issue12641#msg165182

> Setting a requirement for a specific version of GCC within the
> distutils package should be easy enough.  Those with older versions of
> GCC would just need to use the older distutils package.

This is what I suggested on the issue tracker:
http://bugs.python.org/issue12641#msg189748

My proposal is to remove '-mno-cygwin' from distutils in Python 3.4
and to only use it for gcc 3.x in Python 2.7, 3.2 and 3.3. If anyone
sees a problem with that approach I would be very grateful if you
could let me know.


Thanks again,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Oscar B. <osc...@gm...>]

On 21 May 2013 14:45, Oscar Benjamin <osc...@gm...> wrote:
> My proposal is to remove '-mno-cygwin' from distutils in Python 3.4
> and to only use it for gcc 3.x in Python 2.7, 3.2 and 3.3. If anyone
> sees a problem with that approach I would be very grateful if you
> could let me know.

Hi all, and sorry to both you again but I have one more question.

I think I can get the above changes into distutils but there is one
last snag [1]. I can address it if I have a reliable way of querying
whether or not a given gcc is from cygwin or from mingw. I have found
a possible way to do this this by testing the output of 'gcc
-dumpmachine':

$ /q/tools/cygwin/bin/gcc-4.exe -dumpmachine
i686-pc-cygwin

$ /q/tools/cygwin/bin/gcc-3.exe -dumpmachine
i686-pc-cygwin

$ /q/tools/MinGW/bin/gcc.exe -dumpmachine
mingw32

So I thought that I could do something like:

  if dumpmachine == "mingw32":
    don't use -mno-cygwin

or perhaps

  if dumpmachine ends with 'cygwin':
    use -mno-cygwin

Can anyone confirm/deny (or improve) the reliability of either of the
above methods to distinguish between cygwin and mingw gcc?


[1] http://bugs.python.org/issue12641#msg189770


Thanks,
Oscar

Re: [Mingw-users] Fixing mingw support in Python's distutils

[Earnie B. <ea...@us...>]

On Thu, May 23, 2013 at 11:46 AM, Oscar Benjamin wrote:
>
> $ /q/tools/cygwin/bin/gcc-4.exe -dumpmachine
> i686-pc-cygwin
>
> $ /q/tools/cygwin/bin/gcc-3.exe -dumpmachine
> i686-pc-cygwin
>
> $ /q/tools/MinGW/bin/gcc.exe -dumpmachine
> mingw32
>
> So I thought that I could do something like:
>
>   if dumpmachine == "mingw32":
>     don't use -mno-cygwin
>
> or perhaps
>
>   if dumpmachine ends with 'cygwin':
>     use -mno-cygwin
>
> Can anyone confirm/deny (or improve) the reliability of either of the
> above methods to distinguish between cygwin and mingw gcc?

The dumpmachine idea isn't a bad one but Cygwin itself no longer uses
-mno-cygwin.  If you want a native build using Cygwin tools you'll
need to use a cross compiler and specify a --host=i686-pc-mingw32 or
whatever appropriate triplet.

-- 
Earnie
-- https://sites.google.com/site/earnieboyd