visualpython-users

[Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Guilherme B. T. <gui...@gm...>]

Hello,

Just wanted to report how I compiled Visual Python 5.74 from source on
OS X. The dependencies were supplied by Homebrew [1] (alternative to
MacPorts and Fink). More importantly, I also drafted a Homebrew formula
which automates the build on OS X.

My environment is as follows:
- OS X 10.6.8 (Snow Leopard)
- XCode 3.2.6
- Python 2.7.3 (built with Homebrew [2])

All other dependencies are being provided by Homebrew.

I haven't tested against the Python shipped with OSX, nor custom Python
builds installed in custom prefixes. The formula tries to take care of
finding the proper python library path.

The draft Homebrew formula is temporarily on the link below. After a bit
of test and cleanup we can ask to have it included into Homebrew.
https://gist.github.com/3340522

If you have a working Homebrew installation, you might try to install
Visual Python from the above formula (see instructions below for other
Python dependencies). It might take a while, since Homebrew will trigger
a download and compile chain for a whole bunch of dependencies (20+
other packages).

$ brew install
https://gist.github.com/raw/3340522/630f49b0f9c91b80c2ad652e0ee5def455116f62/visualpython.rb

I picked the name 'visualpython' for the package, let me know if I
should change it.

Issues:
1) Down below I suggest a one-line patch to the build system, to remove
a Fink hardcoded path. Please keep it in mind for the next release.
2) I am getting a warning (see below) from the X11 window which runs the
visual objects. Is that relevant?


Ideally, one should be able to install Visual Python with just 2
commands. One for Homebrew to compile and install Visual. Another for
pip to fetch the other Python modules from PyPI (currently two modules
have broken links or broken builds at PyPI).

This was the interesting part of this message.

--

Below I gathered some notes on how I compiled Visual Python manually
before creating the Homebrew formula.

I have installed Python 2.7 via Homebrew [2], but perhaps the following
will work also with the Python shipped with OS X, and other versions of
OS X.

Let's get the dependencies. It might take a while, since Homebrew will
download and compile a whole bunch of dependencies (20+ other packages)
to satisfy these three main dependencies.

$ brew install boost
$ brew install gtkglextmm
$ brew install libglademm

We need to 'link' some packages in order to get them ahead of the OSX
older versions.
Apparently visual needs autoconf >= 2.68. Homebrew has 2.69 which we
link to put it on the path. gktglextmm needs cairo linked and cairo
needs pixman linked.

$ brew link autoconf cairo pixman

Ready to move on to Visual Python source.

$ wget http://www.vpython.org/contents/download/visual-5.74_release.tar.bz2
$ tar xvfz visual-5.74_release.tar.bz2
$ cd visual-5.74_release
$ autoconf

Need to edit or patch a Makefile which is hardcoded to Fink. Remove
hardcoded path, switch to dynamic boost-python, switch to multi-threaded
boost-signal

[edit | patch] src/Makefile.in
203  -    $(GTHREAD_LIBS) /sw/lib/libboost_python-mt.a -lboost_thread-mt
-lboost_signals)
203  +    $(GTHREAD_LIBS) -lboost_python-mt -lboost_thread-mt
-lboost_signals-mt)
[end edit]

Now we can configure and build.

$ mkdir build; cd build

We instruct pkg-config to look for .pc files into Homebrew and X11
(xrender.pc).

$ export PKG_CONFIG_PATH="/usr/local/lib/pkgconfig:/usr/X11/lib/pkgconfig"

The `brew --prefix` is typically /usr/local. The install might break if
you haven't installed anything into
/usr/local/local/python2.7/site-packages yet.

During build it will pick some libs from /usr/X11/lib (libGLU, libXmu,
...). The '-framework OpenGL' gets around 'Undefined symbols:
"_CGLGetCurrentContext" '. By pointing to libpython2.7 it gets around
'Undefined symbols: "_PyExc_RuntimeError" '.

Using the LDFLAGS was the less disruptive way I found to succeed with
the build.

$ ../configure --prefix=`brew --prefix` LDFLAGS="-L/usr/X11/lib
-framework OpenGL
-L/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib
-lpython2.7"

Finally...

$ make install

------
Now we move on to the Python dependencies (ttfquery, fonttools, polygon).

$ pip install ttfquery

Fonttools link is broken on PyPI, pointing to the tarball allows you
continue.

$ pip install
http://sourceforge.net/projects/fonttools/files/2.3/fonttools-2.3.tar.gz

Polygon is also broken on PyPI. It also has a small NumPy issue on its
setup.py. I already informed upstream [3].

$ wget https://github.com/downloads/jraedler/Polygon2/Polygon-2.0.5.zip
$ unzip Polygon-2.0.5.zip

[edit | patch] Polygon-2.0.5/setup.py
29  if withNumPy:
30      try:
31          import numpy
32          print "Using NumPy extension"
33          mac.append(('WITH_NUMPY', 1))
34  +       numPyIncludePath=numpy.get_include() #add this
[end edit]

$ pip install  ./Polygon-2.0.5

------
Assuming that your /usr/local/lib/python2.7/site-packages is present on
PYTHONPATH, you should now be good to run visual.

I tested the installation with the following script:

   # testvisual.py
   from visual import *
   redbox=box(pos=vector(4,2,3),size=(8,4,6),color=color.red)
   ball=sphere(pos=vector(4,7,3),radius=2,color=color.green)

Both objects show up on a X11 window, but the following warning is also
issued:
   Xlib:  extension "RANDR" missing on display "/tmp/launch-Nowttg/org.x:0".
   warning: get proc address request is not for a gl or glX function


----

That was it.

With best regards,
Guilherme



[1] http://mxcl.github.com/homebrew/
[2]
http://www.thisisthegreenroom.com/2011/installing-python-numpy-scipy-matplotlib-and-ipython-on-lion/
[3] https://github.com/jraedler/Polygon2/pull/1

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Bruce S. <Bru...@nc...>]

Many thanks for this! Are you able to build for 64-bit Python?

For Mac users who may not be familiar with the X11 window system, it
should be explained that this procedure produces a Unix/Linux-like
version of VPython, not a "native-mode" version. I'm hoping to produce
a native-mode version based on Cocoa, using wxPython, which would be
usable with 64-bit Python.

Bruce Sherwood

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Guilherme B. T. <gui...@gm...>]

On 13/08/12 21:04, Bruce Sherwood wrote:
> Many thanks for this! Are you able to build for 64-bit Python?

My Python was built as universal (32 and 64-bit), however the 
cvisualmodule.so ended as 64-bit. See below:

$ file /usr/local/lib/python2.7/site-packages/vis/cvisualmodule.so
/usr/local/lib/python2.7/site-packages/vis/cvisualmodule.so: Mach-O 
64-bit bundle x86_64

In necessary, with some compiler flags tweaking perhaps VPython can be 
also build as fat universal.

> For Mac users who may not be familiar with the X11 window system, it
> should be explained that this procedure produces a Unix/Linux-like
> version of VPython, not a "native-mode" version. I'm hoping to produce
> a native-mode version based on Cocoa, using wxPython, which would be
> usable with 64-bit Python.

Yes, it should be made clear the build yields a X11 interface.

I currently do my small GUIs with Qt/PyQt. It should not be that far 
away from wxPython. Once you get the wxPython running I try to have a 
look on how to have it on PyQt. It would be nice if we could embed 
VPython into other applications.

Are you going to provide this "native-mode" as the default or as optional?

Best,
Guilherme

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Bruce S. <Bru...@nc...>]

Currently there are many novice programmers using VPython, including
several thousand physics students being introduced to computational
modeling in the "Matter & Interactions" curriculum
(matterandinteractions.org). These users expect a standard install
procedure and a native look and feel. At the moment this is impossible
on the Mac because VPython is built on Carbon but 64-bit Python
requires a VPython built on Cocoa.

The problem is that Cocoa is required to be the primary thread,
whereas the VPython architecture requires the GUI component to be a
secondary thread. Until very recently there didn't seem to be any way
around this. Recently however, with lots of excellent help and advice
from the wxPython community, I've found a solution in principle,
though I haven't yet implemented it.

The scheme is bizarre but it works in a simple (non-VPython) test case
on all platforms, including Mac Cocoa. Suppose your program is in a
file that contains an import of visual and a rotating box in a
rate-limited while loop. The import of visual sets up the wxPython GUI
machinery but does NOT start the interact loop. The visual module next
imports your program (!) and after this import has its own infinite
while loop. Here's a sketch of the visual module:

* Set up GUI machinery
* Import user program
* while true: rate(30)

Note that if your program ends, execution falls through to the visual
module's while loop, which leaves the graphics window still active,
with zoom and rotate still available.

In your animation loop you call the rate function, which at
appropriate times calls a one-shot interaction with Cocoa, developed
by Robin Dunn, the creator of wxPython. This processes events and
renders the current scene. There are no threads! It's all done by
polling, and requires that animation loops contain rate statements,
something that's pretty universal anyway, and has seemed okay in the
new GlowScript project, too (glowscript.org). Moreover, it addresses
the current problem that a long scene setup may display a partial
scene due to interrupts by the secondary GUI thread.

(Minor point: any code in your program that precedes the import of
visual will be executed twice, which needs to be explained. This is
pretty innocent if the repeated code consists just of imports of
Python modules, because no calculations in those modules will be
repeated.)

What remains is a fair amount of work to replace the current GUI
machinery with this scheme, which has the added advantage of being
based on wxPython which will provide such features as pull-down menus,
etc.

Bruce Sherwood

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Guilherme B. T. <gui...@gm...>]

On 14/08/12 00:19, Bruce Sherwood wrote:
> What remains is a fair amount of work to replace the current GUI
> machinery with this scheme, which has the added advantage of being
> based on wxPython which will provide such features as pull-down menus,
> etc.

In case deep changes are needed, perhaps it is worth investigation how 
other projects handle the GUI part.
For instance, Matplotlib[1] and Chaco[2] can handle several backends 
(WX, Qt, GTK, Tk, Cocoa/Mac OSX, ...). Backends can be changed at will 
(provided dependencies are available during install). The default is 
decided during installation or via a variable stated on a 
configuration/profile file. Therefore a 'import visual' will know how to 
handle itself on the target system, hopefully without breaking existing 
code. One can also switch backends by importing the desired backend 
module or via a 'select' method on the beginning of the program.
The backend encapsulates the user interaction and the rendering, letting 
the lower level modules be as generic as possible (in terms of GUI 
management). This would allow easy embedding whenever a backend is 
available.
Users should be able to follow templates and create new backends as they 
do for Matplotlib [3],[4].

I must say that I'm not yet familiar with the VPython code. I also don't 
know how it all fit together with the conflict between VPython and 
Cocoa, but the suggestion remains.

Best,
Guilherme

[1] http://matplotlib.sourceforge.net/faq/installing_faq.html#backends
[2] 
http://code.enthought.com/projects/chaco/docs/html/architecture_overview.html
[3] 
https://github.com/matplotlib/matplotlib/tree/master/lib/matplotlib/backends
[4] https://github.com/matplotlib/matplotlib/tree/master/src

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Martin C. <cos...@wa...>]

On 13/08/12 17:21, Guilherme Brondani Torri wrote:
[]
> Need to edit or patch a Makefile which is hardcoded to Fink. Remove
> hardcoded path, switch to dynamic boost-python, switch to multi-threaded
> boost-signal
>
> [edit | patch] src/Makefile.in
> 203  -    $(GTHREAD_LIBS) /sw/lib/libboost_python-mt.a -lboost_thread-mt
> -lboost_signals)
> 203  +    $(GTHREAD_LIBS) -lboost_python-mt -lboost_thread-mt
> -lboost_signals-mt)
> [end edit]

In fact, for the Fink package I am patching this line, too. Fink doesn't 
necessarily live in /sw, so hardcoding this isn't as helpful for Fink as 
one could assume.

For Fink on OSX 10.7/10.8, I haven't yet made a visualpython package. I 
am running into compilation errors of the type

./core/util/gl_extensions.cpp:50:17: error: assigning to 
'PFNGLTEXIMAGE3DEXTPROC' (aka 'void (*)(GLenum, GLint, GLenum, GLsizei, 
GLsizei, GLsizei, GLint, GLenum, GLenum, const GLvoid *)') from 
incompatible type 'void (GLenum, GLint, GLint, GLsizei, GLsizei, 
GLsizei, GLint, GLenum, GLenum, const GLvoid *)': type mismatch at 3rd 
parameter ('GLenum' (aka 'unsigned int') vs 'GLint' (aka 'int'))
                         glTexImage3D = ::glTexImage3D;
                                      ^ ~~~~~~~~~~~~~~

There are lots of header files that might be responsible for such 
incompatibilities, and I am not willing to spend hours chasing down such 
errors.

-- 
Martin

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Martin C. <cos...@wa...>]

On 14/08/12 16:27, Martin Costabel wrote:
[]
> For Fink on OSX 10.7/10.8, I haven't yet made a visualpython package. I
> am running into compilation errors of the type
>
> ./core/util/gl_extensions.cpp:50:17: error: assigning to
> 'PFNGLTEXIMAGE3DEXTPROC' (aka 'void (*)(GLenum, GLint, GLenum, GLsizei,
> GLsizei, GLsizei, GLint, GLenum, GLenum, const GLvoid *)') from
> incompatible type 'void (GLenum, GLint, GLint, GLsizei, GLsizei,
> GLsizei, GLint, GLenum, GLenum, const GLvoid *)': type mismatch at 3rd
> parameter ('GLenum' (aka 'unsigned int') vs 'GLint' (aka 'int'))
>                           glTexImage3D = ::glTexImage3D;
>                                        ^ ~~~~~~~~~~~~~~
>
> There are lots of header files that might be responsible for such
> incompatibilities, and I am not willing to spend hours chasing down such
> errors.

Finally, I did spend those hours and found a workaround. It turns out 
that there are two incompatible declarations of glTexImage3D floating 
around in the various OpenGL/gl.h or GL/glext.h headers. Apple changed 
the OpenGL/gl.h header between OSX 10.6 and 10.7 from one version to the 
other, and vpython is now mixing the two, which does not compile.

Patching the definition of PFNGLTEXIMAGE3DEXTPROC in include/GL/glext.h 
allows visual to compile, and it runs OK.

Version 5.74 is now in Fink.

-- 
Martin

Re: [Visualpython-users] Visual Python, OS X source build, Homebrew dependencies

[Bruce S. <Bru...@nc...>]

Thanks much, Martin. What would be the most useful link to your
version to place on the vpython.org Mac download page?

Bruce Sherwood

On Thu, Aug 16, 2012 at 6:27 PM, Martin Costabel <cos...@wa...> wrote:
> On 14/08/12 16:27, Martin Costabel wrote:
> []
>> For Fink on OSX 10.7/10.8, I haven't yet made a visualpython package. I
>> am running into compilation errors of the type
>>
>> ./core/util/gl_extensions.cpp:50:17: error: assigning to
>> 'PFNGLTEXIMAGE3DEXTPROC' (aka 'void (*)(GLenum, GLint, GLenum, GLsizei,
>> GLsizei, GLsizei, GLint, GLenum, GLenum, const GLvoid *)') from
>> incompatible type 'void (GLenum, GLint, GLint, GLsizei, GLsizei,
>> GLsizei, GLint, GLenum, GLenum, const GLvoid *)': type mismatch at 3rd
>> parameter ('GLenum' (aka 'unsigned int') vs 'GLint' (aka 'int'))
>>                           glTexImage3D = ::glTexImage3D;
>>                                        ^ ~~~~~~~~~~~~~~
>>
>> There are lots of header files that might be responsible for such
>> incompatibilities, and I am not willing to spend hours chasing down such
>> errors.
>
> Finally, I did spend those hours and found a workaround. It turns out
> that there are two incompatible declarations of glTexImage3D floating
> around in the various OpenGL/gl.h or GL/glext.h headers. Apple changed
> the OpenGL/gl.h header between OSX 10.6 and 10.7 from one version to the
> other, and vpython is now mixing the two, which does not compile.
>
> Patching the definition of PFNGLTEXIMAGE3DEXTPROC in include/GL/glext.h
> allows visual to compile, and it runs OK.
>
> Version 5.74 is now in Fink.
>
> --
> Martin
>
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Visualpython-users mailing list
> Vis...@li...
> https://lists.sourceforge.net/lists/listinfo/visualpython-users