Donate Share

Python for Windows extensions

Email Archive: pywin32-checkins (read-only)

[pywin32-checkins] py3k status
From: Mark Hammond <mhammond@sk...> - 2008-11-14 01:37
> Argh! I'm still struggling with version control -- learning three at
> once. My personal & commercial stuff is on Bazaar so I only use CVS for
> pywin32 and am not too good with it. I intended that this checkin be
> applied to the py3k fork only.

My apologies for the short and cryptic message. You did check in on the
py3k branch - I just wasn't clear.

[BTW - how are you finding bzr? I'm working with them on a tortoise
implementation, but find windows support is a little lacking and they aren't
that motivated to help fix the niggles - eg, EOL support]

> I used python 2.6 for testing because I needed to import pythoncom and
> win32com, which work on 2.6 and I assumed were not yet ready on py3k.
> (If I had compiled them would they have worked?)

Let me explain what is happening:

Roger started the py3k work a long time ago, well before '2to3' was much use
at all. The only practical way to proceed was to "fork" the .py code by
running it through 2to3 then fixing whatever broke. (The C++ code is much
less of a problem - the pre-processor helps a lot there). As a result, the
py3k branch has all .py code in py3k syntax.

I'm personally willing to take a lot of pain in the interests of *not*
forking the .py code. The 2to3 tool has improved a lot and is now available
as a library. Thus, the strategy I'm aiming for is to maintain all the code
in Python 2.x syntax, and use the 2to3 tool to convert the .py code at build
time. This is slow, but still workable.

However, we aren't *quite* at the point where this works for Roger yet. He
doesn't use 'setup.py' but still builds with the Visual Studio projects. As
a result, his build process doesn't support the 2to3 conversion. Further,
this means you simply can *not* have the pywin32 source tree on your
sys.path in py3k - you must convert to an intermediate directory (setup.py
handles this by converting to a temp directory, and these are written to
site-packages at install time)

So, I'm working towards my goal on a bzr branch -
lp:~mhammond/pywin32/py3k-integration - and this (mostly) works with *both*
2.3->2.7 and 3.0 from the same source tree. Most of my work these days is
just merging and testing - the merging is working towards the following
goal:

* The py3k branch and the trunk should become identical in everything other
than .py code - but all such differences should be limited to 3.x specific
changes only (ie, all other 'modernizations' that work OK in 2.3 will be
merged to the trunk, leaving only the py3.x specifics)
* The bzr integration branch and the trunk should become identical in .py
code (implying all .py code is capable os successfully passing through 2to3)
* Once done, this implies all 3 will be identical, except for the .py code,
and those differences will be only in ways 2to3 can automatically handle.

Once we get to this point, hopefully Roger will be able to work with a 2to3
based build and the py3k branch will be able to be retired. There is still
a fair bit to go to get there though - specifically in bytes and buffer
handling.

Getting back specifically to ado: this means that while making changes in a
2.6-only syntax is fine for the py3k branch, it will still leave me/us with
a future merge problem. As we should be able to continue supporting py2.x
versions for a while, it makes the most sense for new versions to continue
to be developed in a Python 2.x syntax, and using 2to3, even manually in the
short term, to test on py3k.

I hope that clarifies things - please let me know if you have any concerns
or comments. I'd be more than welcome to accept bundles/patches against the
bzr branch - even though that branch will also end up being thrown away, it
is currently useful in helping me merge.

Cheers,

Mark

 

Thread View

Thread Author Date
[pywin32-checkins] pywin32/adodbapi adodbapi.py,1.2.2.1,1.2.2.2 Vernon Cole <kf7xm@us...>
From: Mark Hammond <mhammond@sk...> - 2008-11-13 04:47
> + version 2.2.3 update for Python 3, require python 2.6 or later

Ack - the rest of pywin32 will support 2.3 and up, and will stick with a 2.x
syntax, in the interests of avoid a "fork" into 2 incompatible code-bases.
I'm slowly going through the py3k branch and reverting all the 3k specific
things - is there any chance we can revert all non 2.4 compatible changes
and apply them only to the trunk?

Thanks,

Mark

From: Vern Cole <kf7xm@ya...> - 2008-11-13 13:02
Argh! I'm still struggling with version control -- learning three at once. My personal & commercial stuff is on Bazaar so I only use CVS for pywin32 and am not too good with it. I intended that this checkin be applied to the py3k fork only. It was supposed to update that branch with the same changes I already applied to the trunk as version 2.2.2. (Please tell me that one made it.)

I used python 2.6 for testing because I needed to import pythoncom and win32com, which work on 2.6 and I assumed were not yet ready on py3k. (If I had compiled them would they have worked?)

This code merges your changes and mine from the trunk, and Roger Upole's from the py3k fork, and a bit of new work for compatibility. I updated the version number to 2.2.3 to reflect the changes that let it work with either 2.6 or 3.0. (It has to use "buffer" in 2.6 and "memoryview" in 3.0) I dropped the 2.3 specific code as superfluous in that fork.

So these changes are known to work on 2.6 and may (hopefully) work in py3k and some future IronPython 3.0. Would you be so kind as to make sure that this update is in the py3k fork where it belongs? I don't want to make things worse by fumbling with it.
--
Vernon

--- On Wed, 11/12/08, Mark Hammond <mhammond@sk...> wrote:

> From: Mark Hammond <mhammond@sk...>
> Subject: RE: [pywin32-checkins] pywin32/adodbapi adodbapi.py,1.2.2.1,1.2.2.2
> To: "'Vernon Cole'" <kf7xm@us...>, pywin32-checkins@li...
> Date: Wednesday, November 12, 2008, 9:46 PM
> > + version 2.2.3 update for Python 3, require python
> 2.6 or later
>
> Ack - the rest of pywin32 will support 2.3 and up, and will
> stick with a 2.x
> syntax, in the interests of avoid a "fork" into 2
> incompatible code-bases.
> I'm slowly going through the py3k branch and reverting
> all the 3k specific
> things - is there any chance we can revert all non 2.4
> compatible changes
> and apply them only to the trunk?
>
> Thanks,
>
> Mark

From: Mark Hammond <mhammond@sk...> - 2008-11-14 01:37
> Argh! I'm still struggling with version control -- learning three at
> once. My personal & commercial stuff is on Bazaar so I only use CVS for
> pywin32 and am not too good with it. I intended that this checkin be
> applied to the py3k fork only.

My apologies for the short and cryptic message. You did check in on the
py3k branch - I just wasn't clear.

[BTW - how are you finding bzr? I'm working with them on a tortoise
implementation, but find windows support is a little lacking and they aren't
that motivated to help fix the niggles - eg, EOL support]

> I used python 2.6 for testing because I needed to import pythoncom and
> win32com, which work on 2.6 and I assumed were not yet ready on py3k.
> (If I had compiled them would they have worked?)

Let me explain what is happening:

Roger started the py3k work a long time ago, well before '2to3' was much use
at all. The only practical way to proceed was to "fork" the .py code by
running it through 2to3 then fixing whatever broke. (The C++ code is much
less of a problem - the pre-processor helps a lot there). As a result, the
py3k branch has all .py code in py3k syntax.

I'm personally willing to take a lot of pain in the interests of *not*
forking the .py code. The 2to3 tool has improved a lot and is now available
as a library. Thus, the strategy I'm aiming for is to maintain all the code
in Python 2.x syntax, and use the 2to3 tool to convert the .py code at build
time. This is slow, but still workable.

However, we aren't *quite* at the point where this works for Roger yet. He
doesn't use 'setup.py' but still builds with the Visual Studio projects. As
a result, his build process doesn't support the 2to3 conversion. Further,
this means you simply can *not* have the pywin32 source tree on your
sys.path in py3k - you must convert to an intermediate directory (setup.py
handles this by converting to a temp directory, and these are written to
site-packages at install time)

So, I'm working towards my goal on a bzr branch -
lp:~mhammond/pywin32/py3k-integration - and this (mostly) works with *both*
2.3->2.7 and 3.0 from the same source tree. Most of my work these days is
just merging and testing - the merging is working towards the following
goal:

* The py3k branch and the trunk should become identical in everything other
than .py code - but all such differences should be limited to 3.x specific
changes only (ie, all other 'modernizations' that work OK in 2.3 will be
merged to the trunk, leaving only the py3.x specifics)
* The bzr integration branch and the trunk should become identical in .py
code (implying all .py code is capable os successfully passing through 2to3)
* Once done, this implies all 3 will be identical, except for the .py code,
and those differences will be only in ways 2to3 can automatically handle.

Once we get to this point, hopefully Roger will be able to work with a 2to3
based build and the py3k branch will be able to be retired. There is still
a fair bit to go to get there though - specifically in bytes and buffer
handling.

Getting back specifically to ado: this means that while making changes in a
2.6-only syntax is fine for the py3k branch, it will still leave me/us with
a future merge problem. As we should be able to continue supporting py2.x
versions for a while, it makes the most sense for new versions to continue
to be developed in a Python 2.x syntax, and using 2to3, even manually in the
short term, to test on py3k.

I hope that clarifies things - please let me know if you have any concerns
or comments. I'd be more than welcome to accept bundles/patches against the
bzr branch - even though that branch will also end up being thrown away, it
is currently useful in helping me merge.

Cheers,

Mark

From: Vern Cole <kf7xm@ya...> - 2008-11-14 19:23
Mark:
You have no idea how much you just made my day! Mother used to tell me that "great minds run in the same channels."

--- On Thu, 11/13/08, Mark Hammond <mhammond@sk...> wrote:
> To: "'Vern Cole'" <kf7xm@ya...>, "'Vernon Cole'" <kf7xm@us...>, pywin32-checkins@li...
> Date: Thursday, November 13, 2008, 6:33 PM
> > Argh! I'm still struggling with version control -- learning three at
> > once. My personal & commercial stuff is on Bazaar so I only use CVS for
> > pywin32 and am not too good with it. I intended that this checkin be
> > applied to the py3k fork only.
>
> My apologies for the short and cryptic message. You did check in on the
> py3k branch - I just wasn't clear.
>
> [BTW - how are you finding bzr? I'm working with them on a tortoise
> implementation, but find windows support is a little lacking and they aren't
> that motivated to help fix the niggles - eg, EOL support]

I installed Bazaar 1.9 yesterday (that was an adventure in itself, https://bugs.launchpad.net/bugs/238869) so I am just seeing the tortoise for the first time. It may not be "just what the doctor ordered" yet, but I am confident that it will be.

> > I used python 2.6 for testing because I needed to import pythoncom and
> > win32com, which work on 2.6 and I assumed were not yet ready on py3k.
> > (If I had compiled them would they have worked?)

> Let me explain what is happening:
>
> Roger started the py3k work a long time ago, well before '2to3' was much use
> at all. The only practical way to proceed was to "fork" the .py code
[... snip ...]
> So, I'm working towards my goal on a bzr branch -
> lp:~mhammond/pywin32/py3k-integration - and this (mostly) works with *both*
> 2.3->2.7 and 3.0 from the same source tree. Most of my work these days is
> just merging and testing
[... snip ...] and the py3k branch will be able to be retired.
> There is still a fair bit to go to get there though - specifically in bytes and buffer handling.

Here's what I used in my py3k attempt:
try:
memoryViewType = memoryview # will work in p3k
except NameError:
memoryViewType = types.BufferType # will work in 2.6
memoryview = buffer

> Getting back specifically to ado: this means that while making changes in a
> 2.6-only syntax is fine for the py3k branch, it will still leave me/us with
> a future merge problem. As we should be able to continue supporting py2.x
> versions for a while, it makes the most sense for new versions to continue
> to be developed in a Python 2.x syntax, and using 2to3, even manually in the
> short term, to test on py3k.

That fits with the recommendations from BDFL.

> I hope that clarifies things - please let me know if you have any concerns
> or comments. I'd be more than welcome to accept bundles/patches against the
> bzr branch - even though that branch will also end up being thrown away, it
> is currently useful in helping me merge.

Okay: I have killed off my crufty CVS checkout and pulled a bzr branch.
(Wow, that was easy!)
I will re-merge my newer stuff from py3k into that trunk. I want to do more of the version-specific stuff at import time, so I want to do some more refactoring anyway. In reviewing the merge changes, I see many things like:
if win32:
f = rs.Fields[i]
else: # Iron Python
f = rs.Fields.Item[i]
which I'ld like to re-code, if I can, to get rid of the "if" at run time.
Would something like:
f = getItemValue(rs.Fields,i)
be too confusing?
--
Vernon

From: Mark Hammond <mhammond@sk...> - 2008-11-14 22:47
> [... snip ...] and the py3k branch will be able to be retired.
> > There is still a fair bit to go to get there though - specifically
> in bytes and buffer handling.
>
> Here's what I used in my py3k attempt:
> try:
> memoryViewType = memoryview # will work in p3k
> except NameError:
> memoryViewType = types.BufferType # will work in 2.6
> memoryview = buffer

The rest of pywin32 needs some help for byte literals etc. I've an idea
I've run past Roger that might work but I'm yet to see how practical it is.
Certainly the above is good to get going!

> Okay: I have killed off my crufty CVS checkout and pulled a bzr branch.
> (Wow, that was easy!)
> I will re-merge my newer stuff from py3k into that trunk.

Just to be clear - launchpad hosts a "trunk mirror" and a "py3k integration
branch" - please send patches against the latter, not the former.

> I want to do
> more of the version-specific stuff at import time, so I want to do some
> more refactoring anyway. In reviewing the merge changes, I see many
> things like:
> if win32:
> f = rs.Fields[i]
> else: # Iron Python
> f = rs.Fields.Item[i]
> which I'ld like to re-code, if I can, to get rid of the "if" at run
> time.
> Would something like:
> f = getItemValue(rs.Fields,i)
> be too confusing?

Actually, I think you will find that 'rs.Fields[i]' is just a "shortcut" for
'rs.Fields.Item[i]' by making use of the "default property". So I suspect
that you could use the ironpython version in win32 too. If so a simple
comment would then do.

Cheers,

Mark