msys2-users

[Msys2-users] python and directory separators

[Stephen L. <ste...@st...>]

I'm building Botan (http://botan.randombit.net/download.html). It uses a
Python script for configre. The configure script generates paths with Python's
os.path.join(...); in msys Python this uses '\' as the path separator.
This causes problems in many places in the Makefile, giving errors like:

install: cannot stat ‘.srcutilstypes.h

when it should be ./src/utils/types.h

I had the same problem in msys; I solved it with a sed script:

   $ sed -i 's#\\\(.\)#/\1#g' Makefile

But that is broken in msys2:

/usr/bin/sed: -e expression #1, char 11: Unmatched ( or \(

We can't simply replace _all_ \; some are line continuations.


The best fix would seem to be to fix msys2 Python os.path.join to use /.

Can I do that?

Short of that, any ideas?

-- 
-- Stephe

Re: [Msys2-users] python and directory separators

[Alexpux <al...@gm...>]

29 апр. 2014 г., в 22:36, Stephen Leake <ste...@st...> написал(а):

> I'm building Botan (http://botan.randombit.net/download.html). It uses a
> Python script for configre. The configure script generates paths with Python's
> os.path.join(...); in msys Python this uses '\' as the path separator.
> This causes problems in many places in the Makefile, giving errors like:
> 
> install: cannot stat ‘.srcutilstypes.h
> 
> when it should be ./src/utils/types.h
> 
> I had the same problem in msys; I solved it with a sed script:
> 
>   $ sed -i 's#\\\(.\)#/\1#g' Makefile
> 
> But that is broken in msys2:
> 
> /usr/bin/sed: -e expression #1, char 11: Unmatched ( or \(
> 
> We can't simply replace _all_ \; some are line continuations.
> 
> 
> The best fix would seem to be to fix msys2 Python os.path.join to use /.
> 
> Can I do that?
> 
> Short of that, any ideas?
> 
Maybe you use mingw-python?
> -- 
> -- Stephe
> 
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Msys2-users mailing list
> Msy...@li...
> https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] python and directory separators

[Stephen L. <ste...@st...>]

Stephen Leake <ste...@st...> writes:

> I had the same problem in msys; I solved it with a sed script:
>
>    $ sed -i 's#\\\(.\)#/\1#g' Makefile
>
> But that is broken in msys2:
>
> /usr/bin/sed: -e expression #1, char 11: Unmatched ( or \(

I found a workaround; use Cygwin bash, Cygwin sed to run this command.
Then the msys64 'make install' succeeds.


-- 
-- Stephe

Re: [Msys2-users] python and directory separators

[Alexpux <al...@gm...>]

29 апр. 2014 г., в 22:43, Stephen Leake <ste...@st...> написал(а):

> Stephen Leake <ste...@st...> writes:
> 
>> I had the same problem in msys; I solved it with a sed script:
>> 
>>   $ sed -i 's#\\\(.\)#/\1#g' Makefile
>> 
>> But that is broken in msys2:
>> 
>> /usr/bin/sed: -e expression #1, char 11: Unmatched ( or \(
> 
> I found a workaround; use Cygwin bash, Cygwin sed to run this command.
> Then the msys64 'make install' succeeds.
> 
> 
Can you give me small test case for this issue?

> -- 
> -- Stephe
> 
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Msys2-users mailing list
> Msy...@li...
> https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] python and directory separators

[Ray D. <min...@gm...>]

Which python do you mean here? There are many:

pacman -Ss "A high-level scripting language"
mingw32/mingw-w64-i686-python2 2.7.6-10 [installed]
    A high-level scripting language (mingw-w64)
mingw32/mingw-w64-i686-python3 3.4.0-3 [installed]
    A high-level scripting language (mingw-w64)
mingw64/mingw-w64-x86_64-python2 2.7.6-10 [installed]
    A high-level scripting language (mingw-w64)
mingw64/mingw-w64-x86_64-python3 3.4.0-3 [installed]
    A high-level scripting language (mingw-w64)
msys/python2 2.7.6-1 [installed]
    A high-level scripting language

msys/python2 should never use '\'.
mingw*/python* should check for env. variable MSYSTEM being set
(indicating it is running under the MSYS2 bash shell), and if it is
set then '/' is used as os.sep, otherwise '\' is used (in-case you run
it from cmd.exe with scripts that assume os.sep on Windows to be '\').

Can you show the result of:
echo $MSYSTEM
.. from bash please?


On Tue, Apr 29, 2014 at 7:54 PM, Alexpux <al...@gm...> wrote:
>
> 29 апр. 2014 г., в 22:43, Stephen Leake <ste...@st...> написал(а):
>
>> Stephen Leake <ste...@st...> writes:
>>
>>> I had the same problem in msys; I solved it with a sed script:
>>>
>>>   $ sed -i 's#\\\(.\)#/\1#g' Makefile
>>>
>>> But that is broken in msys2:
>>>
>>> /usr/bin/sed: -e expression #1, char 11: Unmatched ( or \(
>>
>> I found a workaround; use Cygwin bash, Cygwin sed to run this command.
>> Then the msys64 'make install' succeeds.
>>
>>
> Can you give me small test case for this issue?
>
>> --
>> -- Stephe
>>
>> ------------------------------------------------------------------------------
>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
>> unparalleled scalability from the best Selenium testing platform available.
>> Simple to use. Nothing to install. Get started now for free."
>> http://p.sf.net/sfu/SauceLabs
>> _______________________________________________
>> Msys2-users mailing list
>> Msy...@li...
>> https://lists.sourceforge.net/lists/listinfo/msys2-users
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.  Get
> unparalleled scalability from the best Selenium testing platform available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Msys2-users mailing list
> Msy...@li...
> https://lists.sourceforge.net/lists/listinfo/msys2-users

Re: [Msys2-users] python and directory separators

[Stephen L. <ste...@st...>]

Ray Donnelly <min...@gm...> writes:

> Which python do you mean here? There are many:
>
> mingw64/mingw-w64-x86_64-python2 2.7.6-10 [installed]
>     A high-level scripting language (mingw-w64)

This one; it was installed by mingw-w64-x86_64-toolchain.

> msys/python2 should never use '\'.

Ah, yes.

I keep forgetting when I need to use msys tools, and when I need to use
mingw. I'll get used to it about the time I'm done building monotone :).

> mingw*/python* should check for env. variable MSYSTEM being set
> (indicating it is running under the MSYS2 bash shell), and if it is
> set then '/' is used as os.sep, otherwise '\' is used (in-case you run
> it from cmd.exe with scripts that assume os.sep on Windows to be '\').

export MSYSTEM=MSYS

fixes my problem, using mingw/python

Thanks.

-- 
-- Stephe

[Msys2-users] python and directory separators

[Stephan P. <ste...@ge...>]

I forgot:

Both os.sep and os.path.sep: returns the correct separator:  /

So, this is for me definitely an issue worth for a fix by the maintainer

cheers
Stephan


-- 


Stephan Petzchen

Gesius GmbH
gesius. engineering excellence

Sitz der Gesellschaft/Registered office: Hamburg

Handelsregister/Commercial register:  HRB 117558

Geschäftsführung/executive board: Stephan Petzchen

---
This communication contains confidential information. If you are not the
intended recipient please return this email to the sender and delete it
from your records.
Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht der
beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an den
Absender zurück und löschen Sie die E-mail aus Ihrem System.

Re: [Msys2-users] python and directory separators

[Ray D. <min...@gm...>]

2015-06-05 8:53 GMT+01:00 Stephan Petzchen <ste...@ge...>:
>
> I forgot:
>
> Both os.sep and os.path.sep: returns the correct separator:  /
>
> So, this is for me definitely an issue worth for a fix by the maintainer
>

You are right, both os.getcwd() and os.path.expanduser() need to be
fixed for Python 2, and os.path.expanduser() needs to be fixed for
Python 3 still. I filed an issue for this:
https://github.com/Alexpux/MINGW-packages/issues/657 and will get
round to it when I can (not soon unfortunately ..)

> cheers
> Stephan
>
>
> --
>
>
> Stephan Petzchen
>
> Gesius GmbH
> gesius. engineering excellence
>
> Sitz der Gesellschaft/Registered office: Hamburg
>
> Handelsregister/Commercial register:  HRB 117558
>
> Geschäftsführung/executive board: Stephan Petzchen
>
> ---
> This communication contains confidential information. If you are not the
> intended recipient please return this email to the sender and delete it from
> your records.
> Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht der
> beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an den
> Absender zurück und löschen Sie die E-mail aus Ihrem System.
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Msys2-users mailing list
> Msy...@li...
> https://lists.sourceforge.net/lists/listinfo/msys2-users
>

Re: [Msys2-users] python and directory separators

[Ray D. <min...@gm...>]

On Fri, Jun 5, 2015 at 1:31 PM, Stephan Petzchen
<ste...@ge...> wrote:
> Thats cool. Thanks for the feedback and issueing.
> I am just working on googles multi git repo tool to get it runnable for
> msys2. The normpath workaround is doing a good job so far.

AFAIR, Google's repo needs the fcntl module:

https://docs.python.org/3.4/library/fcntl.html

.. which is Unix only. I recommend you use the msys2 python for repo
instead, as that does support fcntl. I think there may be some parital
implementations for native Windows python, but I've no idea how well
they work.

>
> Besides, many thanks for this msys2 toolchain. I loved it to leave the messy
> raw mingw & cygwin path. Beside the honor they all deserve, a devops
> nightmare. With msys2 I see again some light at the end of the tunnel.
> Many thanks to all achiever.

Thanks.

>
> On Fri, Jun 5, 2015 at 11:13 AM, Ray Donnelly <min...@gm...>
> wrote:
>>
>> 2015-06-05 8:53 GMT+01:00 Stephan Petzchen <ste...@ge...>:
>> >
>> > I forgot:
>> >
>> > Both os.sep and os.path.sep: returns the correct separator:  /
>> >
>> > So, this is for me definitely an issue worth for a fix by the maintainer
>> >
>>
>> You are right, both os.getcwd() and os.path.expanduser() need to be
>> fixed for Python 2, and os.path.expanduser() needs to be fixed for
>> Python 3 still. I filed an issue for this:
>> https://github.com/Alexpux/MINGW-packages/issues/657 and will get
>> round to it when I can (not soon unfortunately ..)
>>
>> > cheers
>> > Stephan
>> >
>> >
>> > --
>> >
>> >
>> > Stephan Petzchen
>> >
>> > Gesius GmbH
>> > gesius. engineering excellence
>> >
>> > Sitz der Gesellschaft/Registered office: Hamburg
>> >
>> > Handelsregister/Commercial register:  HRB 117558
>> >
>> > Geschäftsführung/executive board: Stephan Petzchen
>> >
>> > ---
>> > This communication contains confidential information. If you are not the
>> > intended recipient please return this email to the sender and delete it
>> > from
>> > your records.
>> > Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht
>> > der
>> > beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an
>> > den
>> > Absender zurück und löschen Sie die E-mail aus Ihrem System.
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> >
>> > _______________________________________________
>> > Msys2-users mailing list
>> > Msy...@li...
>> > https://lists.sourceforge.net/lists/listinfo/msys2-users
>> >
>
>
>
>
> --
>
> Stephan Petzchen
>
> Gesius GmbH
>
> gesius. engineering excellence
>
> Sitz der Gesellschaft/Registered office: Hamburg
>
> Handelsregister/Commercial register:  HRB 117558
>
> Geschäftsführung/executive board: Stephan Petzchen
>
> ---
> This communication contains confidential information. If you are not the
> intended recipient please return this email to the sender and delete it from
> your records.
> Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht der
> beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an den
> Absender zurück und löschen Sie die E-mail aus Ihrem System.

Re: [Msys2-users] python and directory separators

[Stephan P. <ste...@ge...>]

yeah...I know...especially used for non-blocking output capturing
(O_NONBLOCK) but this can be replaced by the pythons subprocess
communicate() func. At least this how I did it so far...

On Sun, Jun 7, 2015 at 11:04 AM, Ray Donnelly <min...@gm...>
wrote:

> On Fri, Jun 5, 2015 at 1:31 PM, Stephan Petzchen
> <ste...@ge...> wrote:
> > Thats cool. Thanks for the feedback and issueing.
> > I am just working on googles multi git repo tool to get it runnable for
> > msys2. The normpath workaround is doing a good job so far.
>
> AFAIR, Google's repo needs the fcntl module:
>
> https://docs.python.org/3.4/library/fcntl.html
>
> .. which is Unix only. I recommend you use the msys2 python for repo
> instead, as that does support fcntl. I think there may be some parital
> implementations for native Windows python, but I've no idea how well
> they work.
>
> >
> > Besides, many thanks for this msys2 toolchain. I loved it to leave the
> messy
> > raw mingw & cygwin path. Beside the honor they all deserve, a devops
> > nightmare. With msys2 I see again some light at the end of the tunnel.
> > Many thanks to all achiever.
>
> Thanks.
>
> >
> > On Fri, Jun 5, 2015 at 11:13 AM, Ray Donnelly <min...@gm...>
> > wrote:
> >>
> >> 2015-06-05 8:53 GMT+01:00 Stephan Petzchen <ste...@ge...
> >:
> >> >
> >> > I forgot:
> >> >
> >> > Both os.sep and os.path.sep: returns the correct separator:  /
> >> >
> >> > So, this is for me definitely an issue worth for a fix by the
> maintainer
> >> >
> >>
> >> You are right, both os.getcwd() and os.path.expanduser() need to be
> >> fixed for Python 2, and os.path.expanduser() needs to be fixed for
> >> Python 3 still. I filed an issue for this:
> >> https://github.com/Alexpux/MINGW-packages/issues/657 and will get
> >> round to it when I can (not soon unfortunately ..)
> >>
> >> > cheers
> >> > Stephan
> >> >
> >> >
> >> > --
> >> >
> >> >
> >> > Stephan Petzchen
> >> >
> >> > Gesius GmbH
> >> > gesius. engineering excellence
> >> >
> >> > Sitz der Gesellschaft/Registered office: Hamburg
> >> >
> >> > Handelsregister/Commercial register:  HRB 117558
> >> >
> >> > Geschäftsführung/executive board: Stephan Petzchen
> >> >
> >> > ---
> >> > This communication contains confidential information. If you are not
> the
> >> > intended recipient please return this email to the sender and delete
> it
> >> > from
> >> > your records.
> >> > Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht
> >> > der
> >> > beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an
> >> > den
> >> > Absender zurück und löschen Sie die E-mail aus Ihrem System.
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> >
> >> > _______________________________________________
> >> > Msys2-users mailing list
> >> > Msy...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/msys2-users
> >> >
> >
> >
> >
> >
> > --
> >
> > Stephan Petzchen
> >
> > Gesius GmbH
> >
> > gesius. engineering excellence
> >
> > Sitz der Gesellschaft/Registered office: Hamburg
> >
> > Handelsregister/Commercial register:  HRB 117558
> >
> > Geschäftsführung/executive board: Stephan Petzchen
> >
> > ---
> > This communication contains confidential information. If you are not the
> > intended recipient please return this email to the sender and delete it
> from
> > your records.
> > Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht der
> > beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an den
> > Absender zurück und löschen Sie die E-mail aus Ihrem System.
>



-- 

Mit freundlichen Grüßen

Stephan Petzchen


Gesius GmbH
Beim Strohhause 31
20097 Hamburg, Germany
Tel.    +49 (40) 6094673-10
Mobil +49 (172) 1977014
Fax   +49 (40) 6094673-99
email: ste...@ge... <ste...@ge...>
www.gesius.de <http://gesius.de>

gesius. engineering excellence

Sitz der Gesellschaft/Registered office: Hamburg

Handelsregister/Commercial register:  HRB 117558

Geschäftsführung/executive board: Stephan Petzchen

---
This communication contains confidential information. If you are not the
intended recipient please return this email to the sender and delete it
from your records.
Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht der
beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an den
Absender zurück und löschen Sie die E-mail aus Ihrem System.

[Msys2-users] python and directory separators

[Stephan P. <ste...@ge...>]

Hi guys,

do someone know the current status of this issue?

Because i am running the same problem with the python path separator for

mingw64/mingw-w64-x86_64-python2 2.7.10-1

i.e.
 os.getcwd()
os.path.expanduser('~')

still return the windows path separator '\' in pathes. Is this intended
(WHY?) or are there any plans to fix that.

In the meanwhile the os.path.normpath function do the trick and converts
any path to the msys2 proper format, well only according the separator but
thats fine.


-- 

Stephan Petzchen

Gesius GmbH

gesius. engineering excellence

Sitz der Gesellschaft/Registered office: Hamburg

Handelsregister/Commercial register:  HRB 117558

Geschäftsführung/executive board: Stephan Petzchen

---
This communication contains confidential information. If you are not the
intended recipient please return this email to the sender and delete it
from your records.
Diese Nachricht enthält vertrauliche Informationen. Sollten Sie nicht der
beabsichtigte Empfänger dieser E-mail sein, senden Sie bitte diese an den
Absender zurück und löschen Sie die E-mail aus Ihrem System.