numpy-discussion — Discussion list for all users of Numerical Python

RE: Re: [Numpy-discussion] taking on new EM simulator project- ASAP

[<ro...@py...>]

Then it wouldn't be an educational tool, for me or anyone else.  It would just look like gibberish/Fortran.   Rob.

>----- Original Message -----
>From: Pearu Peterson <pe...@ce...>
>To: Rob <ro...@py...>
>Sent: Thu, 10 Jan 2002 +0200 (EET) 17:07:48
>
>
>On Thu, 10 Jan 2002, Rob wrote:
>
>> This one will be interesting.  I'm going to
>convert the Fortran ASAP MoM
>> simulator to Numpy.	I've already started and its
>bringing back bad
>> memories of punch cards, etc.  ASAP is a viable
>alternative to NEC2 for
>> wire antennas, as it uses the Sommerfield
>conditions for imperfect
>> ground.  And its not too long of a program to
>port.	The heavy stuff
>> will be I/O.  Fortunately, I can use f2c to
>decode the format statements
>> :)
>
>But why don't you use Fortran/Python binding tools
>like pyfort or f2py?
>
>Pearu

Re: [Numpy-discussion] taking on new EM simulator project- ASAP

[Pearu P. <pe...@ce...>]

On Thu, 10 Jan 2002, Rob wrote:

> This one will be interesting.  I'm going to convert the Fortran ASAP MoM
> simulator to Numpy.  I've already started and its bringing back bad
> memories of punch cards, etc.  ASAP is a viable alternative to NEC2 for
> wire antennas, as it uses the Sommerfield conditions for imperfect
> ground.  And its not too long of a program to port.  The heavy stuff
> will be I/O.  Fortunately, I can use f2c to decode the format statements
> :)

But why don't you use Fortran/Python binding tools like pyfort or f2py?

Pearu

[Numpy-discussion] taking on new EM simulator project- ASAP

[Rob <ro...@py...>]

This one will be interesting.  I'm going to convert the Fortran ASAP MoM
simulator to Numpy.  I've already started and its bringing back bad
memories of punch cards, etc.  ASAP is a viable alternative to NEC2 for
wire antennas, as it uses the Sommerfield conditions for imperfect
ground.  And its not too long of a program to port.  The heavy stuff
will be I/O.  Fortunately, I can use f2c to decode the format statements
:)

Rob.


-- 
The Numeric Python EM Project

www.pythonemproject.com

Re: [Numpy-discussion] Sparse matricies

[Andrew P. L. <bs...@al...>]

Travis,

Here is some of the most recent stuff I've found on sparse matricies.

http://buffy.EECS.Berkeley.EDU/IRO/Summary/01abstracts/chapter18index.html

http://buffy.EECS.Berkeley.EDU/IRO/Summary/01abstracts/ejr.1.html

Andy L.

Re: [Numpy-discussion] Sparse matricies

[Andrew P. L. <bs...@al...>]

Perhaps next time I should dig on the web a little more ...

On Tue, 8 Jan 2002, Travis Oliphant wrote:

> I'd be interested in the package you are talking about.

Also available as a separate package,

http://buffy.eecs.berkeley.edu/IRO/Software/Catalog/Description/sparse1.3.html

-a

Re: [Numpy-discussion] Sparse matricies

[Andrew P. L. <bs...@al...>]

On Tue, 8 Jan 2002, Travis Oliphant wrote:

> SciPy has a subpackage for Sparse matrices, but it is not ready yet.

Well, I'm in the process of setting up the simulator now.  I don't need
the sparse matrix package immediately, but if there isn't anything useful
in a week or so, I'll have to figure out a workaround.  I'd *really*
rather not.  Even something which isn't quite ready would be better than
what I would write (and then have to throw away).

> I'd be interested in the package you are talking about.

http://www-cad.eecs.berkeley.edu:80/Software/software.html

The packge is SPICE, which is an electronic circuit simulator.  Spice3f4
in particular.  Download the Spice3f4 kit.

Unpack it.  The entire package is in src/lib/sparse.

Andy L.

Re: [Numpy-discussion] Sparse matricies

[Travis O. <oli...@ee...>]

>
> Is there anything current for Python which handles sparse matrix codes?
>
> If someone is looking for a C implementation of sparse matrix handlers, I
> would refer to the sparse matrix package developed at Berekeley which
> is part of the Spice 3f4 circuit simulator (released under a modified
> old-style BSD license).  I can provide pointers if they are desired.

I've done about three Sparse matrix packages built on SPARSEKIT, UMFPACK,
and some of my own rolled stuff.

SciPy has a subpackage for Sparse matrices, but it is not ready yet.

I'd be interested in the package you are talking about.

-Travis Oliphant

[Numpy-discussion] Sparse matricies

[Andrew P. L. <bs...@al...>]

I just looked through the archives and the web searches and didn't really
find anything current on sparse matricies.

Is there anything current for Python which handles sparse matrix codes?

If someone is looking for a C implementation of sparse matrix handlers, I
would refer to the sparse matrix package developed at Berekeley which
is part of the Spice 3f4 circuit simulator (released under a modified
old-style BSD license).  I can provide pointers if they are desired.

My goal is to actually create a useful circuit simulator wholly in Python
so that it would be useful for both teaching and modification in special
cases.

Andy L.

Re: distutils for scientific stuff, was RE: [Numpy-discussion] RPMsout of date, have problems

[Rob <ro...@py...>]

Please when writing these utilities keep us BSD users in mind.  I had to
severely hack SciPy to get it to compile on FreeBSD, part of which was
the Fortran utility.  Rob.

Pearu Peterson wrote:
> 
> On Sun, 6 Jan 2002, Paul F. Dubois wrote:
> 
> > Adding Fortran support is quite a bit more difficult because the whole
> > idea of distutils is to piggy-back off the Python configure, which
> > doesn't configure Fortran compiler options or paths. I don't think
> > distutils ought to try, really. You just compile the Fortran into a
> > library and then use that in your setup.py.
> 
> Saying that "just compile the Fortran into .." sounds like a common step
> for using Fortran, then I don't see why distutils shouldn't try to support
> this. As Eric mentioned in the previous message, scipy_distutils already
> does this step for an intermediate user. We have tested this to work for
> different compilers like Gnu,Intel F90,VAST,NAG F95,Absoft,etc and it
> works like a charm.
> 
> Regards,
>         Pearu
> 
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

-- 
The Numeric Python EM Project

www.pythonemproject.com

Re: distutils for scientific stuff, was RE: [Numpy-discussion] RPMs out of date, have problems

[Pearu P. <pe...@ce...>]

On Sun, 6 Jan 2002, Paul F. Dubois wrote:

> Adding Fortran support is quite a bit more difficult because the whole
> idea of distutils is to piggy-back off the Python configure, which
> doesn't configure Fortran compiler options or paths. I don't think
> distutils ought to try, really. You just compile the Fortran into a
> library and then use that in your setup.py. 

Saying that "just compile the Fortran into .." sounds like a common step
for using Fortran, then I don't see why distutils shouldn't try to support
this. As Eric mentioned in the previous message, scipy_distutils already
does this step for an intermediate user. We have tested this to work for
different compilers like Gnu,Intel F90,VAST,NAG F95,Absoft,etc and it
works like a charm.

Regards,
	Pearu

distutils for scientific stuff, was RE: [Numpy-discussion] RPMs out of date, have problems

[Paul F. D. <pa...@pf...>]

About doing different things in distutils for different platforms, that
is easy. It is a Python script so you can set things depending on
sys.platform.
E.g. from Numerical,

# You might need to add a case here for your system
if sys.platform == 'win32':
    mathlibs = []
    define_macros = []
    undef_macros = ['HAVE_INVERSE_HYPERBOLIC']
elif sys.platform == 'beos5':
    mathlibs = []

Later the setup call uses these variables in its argument list.

Adding Fortran support is quite a bit more difficult because the whole
idea of distutils is to piggy-back off the Python configure, which
doesn't configure Fortran compiler options or paths. I don't think
distutils ought to try, really. You just compile the Fortran into a
library and then use that in your setup.py. 

I think a possible way out is scons, but that is just a preliminary
impression. It bears a strong resemblance to the system I was working on
in 1998 and abandoned when I changed jobs.

My theory was that the build should be rule-based, with finer and finer
rules for special cases or platforms. The highest priority rule that
governs a particular file, wins.

E.g., 
Rule 1: To compile a .c file, do cc -O ...
Rule 2: To compile foo.c, do cc -O3 ...
Rule 3: To compile foo.c on platform win32, do cc -O1 ...
Rule 4: compile bar.c but only on platform win32

There was more to it, because one of the tricky points is that nowadays
files produce multiple outputs per execution and may need to be
processed in more than one way. 

Note that if you add a new .c file or a new platform, you're covered
unless it needs special treatment.

Anyway, this is actually not a design area the numerical community ought
to try to get into; there are people who have spent a lot of time on
this already. Given that sentence, I can't explain why I was doing it,
other than that I hate make so badly I was driven to it. The project was
called MMD: Make Must Die.

Re: [Numpy-discussion] RPMs out of date, have problems

[eric <er...@en...>]

> > (1) Extension( ..., extra_compile_args = [ "-O17" ], ...)
> > This will append the optimization level 17 to the compiler flag
> > (the last -O option counts).
>
> But that will add the argument for all platforms. I need to specify
> compilation flags separately for each platform/compiler. I might even
> want to compile some modules with the new Intel compiler for Linux.
>
> > (2) export CFLAGS=-O3; python setup.py build
> > This will append -O3
>
> For all extension modules - I need to specify different flags for each
> extension module, in one case even for a single source code file.

This seems to be one of the big complaints about distutils for scientist
(and maybe others).  I think it is also one of Paul Dubois' major reasons
for not liking distutils for Fortran files.  I've run into this occasionally
too and wished for a dictionary argument, maybe file_special_instructions,
that had file names as keys and a dictionary of options for that file as the
value.  I think this might go some distance to solving the problem.

file_special_instructions = {
                                           'really_fast.c': { compiler =
'icc',

override_compiler_flags = ['O2', 'tpp7']
                                                                   }
                                           'sorta_fast.c':  {
extra_compile_args = ['-O2']}
                                           }

So an extension module that used this and had 'slow.c', 'really_fast.c', and
'kinda_fast.c' would use the standard compile tools for 'slow.c', use the
specified compiler with the specified flags for 'really_fast.c', and add the
specified compiler flags to the standard flags for 'sorta_fast.c'.  This
approach would provide quite a bit of flexiblity.  To handle platform
specific versions of file_special_instructions, I think you'd have to use
if/thens within the setup.py file to build a separate set of instructions
for each platform.

There is one other major issue I've run into with distutils that I haven't
found a way around.  Sometimes you need compiler/linker flags inter-mingled
with source or library files in a certain way.  This can occur on SunOS when
you want to link statically to some libraries and dynamically to others.
distutils just doesn't provide a way of doing this that I have found.

SciPy now has a package of extensions/changes to distutils called
scipy_distutils helpful for building fortran based extension modules and
other things needed by SciPy.  We could experiment with adding something
like the file_special_instructions flag if others thought it useful.  Later,
it could be folded back into distutils if the rest of the community wanted
it.

Thoughts?

eric

Re: [Numpy-discussion] RPMs out of date, have problems

[Konrad H. <hi...@cn...>]

> Can you check which version of Python/distutils. If it is a version python
> that does not come with distutils, we can count it out.

I don't remember, and I don't have any of that stuff left.

> Yes, and they will complain to JH if they are failing to install JH's
> binary RPMs. The best we can do is explain the motivation of NOT
> providing binary RPMs. If that is not sufficient for some, we can

Somebody else will provide them, and you will still get complaints. I
also get them. Fortunately there is a standard answer to send out.

> There are two ways to do this:
> 
> (1) Extension( ..., extra_compile_args = [ "-O17" ], ...)
> This will append the optimization level 17 to the compiler flag
> (the last -O option counts).

But that will add the argument for all platforms. I need to specify
compilation flags separately for each platform/compiler. I might even
want to compile some modules with the new Intel compiler for Linux.

> (2) export CFLAGS=-O3; python setup.py build
> This will append -O3

For all extension modules - I need to specify different flags for each
extension module, in one case even for a single source code file.

My current solution is to build everything with distutils and then
recompile the critical modules manually in the spec file.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] RPMs out of date, have problems

[<gve...@la...>]

> 
> > python setup.py bdist_rpm needs only write access to the user's home
> > directory.
> 
> Everywhere? It didn't when I first tried (that was an early distutils
> release under RedHat 6.2).
>
Can you check which version of Python/distutils. If it is a version python
that does not come with distutils, we can count it out.
> 
> > Of course, adding a generic spec file is the easy 2 minutes solution,
> > but I have choosen not to do that because it will invite bad policy
> > (providing binary RPMs is asking for trouble).
> 
> And yet many people will not accept anything but binary RPMs, out of
> ignorance or fear. We won't make everyone happy...
>
Yes, and they will complain to JH if they are failing to install JH's
binary RPMs. The best we can do is explain the motivation of NOT
providing binary RPMs. If that is not sufficient for some, we can
only advise that they switch to Windows or pester their distro
providers for RPMs of the latest NumPy.
> 
> I admit that I never looked into fully automatic RPM generation by
> distutils as I usually need to add some manual steps to the
> build/install process. For example, distutils doesn't let me specify
> compiler options, but I want time-critical code to be compiled with
> the highest optimization level. Hopefully this will be addressed in
> distutils one day.
>
There are two ways to do this:

(1) Extension( ..., extra_compile_args = [ "-O17" ], ...)
This will append the optimization level 17 to the compiler flag
(the last -O option counts).

(2) export CFLAGS=-O3; python setup.py build
This will append -O3

The result of (1) and (2) is gcc .... -017 -03


Gerard

PS: distutils may look intimidating at first, but it is really
simple compared to automake/autoconf. The disadvantage is that
it is not very well tested.


---------------------------------------------
This message was sent using Endymion MailMan.
http://www.endymion.com/products/mailman/

Re: [Numpy-discussion] RPMs out of date, have problems

[Konrad H. <hi...@cn...>]

> Disadvantage: by default, the package will go into a distro dependent place
> (Mandrake, Red Hat and Suse are all different) and the package builder
> has to have write access to that place (Suse-7.3 grants it, Mandrake not,
> Red Hat don't know). This invites to building packages as root (DON'T).

RedHat doesn't by default (I changed that on my system).

> python setup.py bdist_rpm needs only write access to the user's home
> directory.

Everywhere? It didn't when I first tried (that was an early distutils
release under RedHat 6.2).

> Of course, adding a generic spec file is the easy 2 minutes solution,
> but I have choosen not to do that because it will invite bad policy
> (providing binary RPMs is asking for trouble).

And yet many people will not accept anything but binary RPMs, out of
ignorance or fear. We won't make everyone happy...

I admit that I never looked into fully automatic RPM generation by
distutils as I usually need to add some manual steps to the
build/install process. For example, distutils doesn't let me specify
compiler options, but I want time-critical code to be compiled with
the highest optimization level. Hopefully this will be addressed in
distutils one day.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] RPMs out of date, have problems

[<gve...@la...>]

> gve...@la... writes:
> 
> > Caveats:
> > (1) renaming Numeric to python-numeric has as consequence that
> > the header files go into .../include/python2.1/python-numeric
> > instead of .../include/python2.1/Numeric. Personally I don't
> > like it (being author of a package that builds on Numeric).
> 
> Me neither. But the fix is simple: take the distutils-generated spec
> file, edit the package name, and put it into the tar ball. The added
> advantage is that the binary RPM can be generated by rpm --rebuild
> numpy_xxx.tar.gz.
> 
You mean rpm -ta numpy_xxx.tar.gz

Disadvantage: by default, the package will go into a distro dependent place
(Mandrake, Red Hat and Suse are all different) and the package builder
has to have write access to that place (Suse-7.3 grants it, Mandrake not,
Red Hat don't know). This invites to building packages as root (DON'T).
Building packages as a normal user protects you from erratic
writes in /usr, /, ...

Remember, we want to satisfy the needs of astronomer newbies. Users like
you will allways find a solution.

python setup.py bdist_rpm needs only write access to the user's home
directory.

Of course, adding a generic spec file is the easy 2 minutes solution,
but I have choosen not to do that because it will invite bad policy
(providing binary RPMs is asking for trouble).

IMHO, the current setup.py/setup_all.py are not permitting distutils's
way of 'RPM building'. It is better to fix that.

I will try to see renaming the package from Numeric to python-numeric
allows to install the headers in ../python2.1/Numeric instead of
./python2.1/python-numeric by changing my setup.py file.

Gerard



---------------------------------------------
This message was sent using Endymion MailMan.
http://www.endymion.com/products/mailman/

Re: [Numpy-discussion] RPMs out of date, have problems

[Konrad H. <hi...@cn...>]

gve...@la... writes:

> Caveats:
> (1) renaming Numeric to python-numeric has as consequence that
> the header files go into .../include/python2.1/python-numeric
> instead of .../include/python2.1/Numeric. Personally I don't
> like it (being author of a package that builds on Numeric).

Me neither. But the fix is simple: take the distutils-generated spec
file, edit the package name, and put it into the tar ball. The added
advantage is that the binary RPM can be generated by rpm --rebuild
numpy_xxx.tar.gz.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] RPMs out of date, have problems

[<gve...@la...>]

> 
> > I continue to be less than impressed by RPMs. Every time I try one I get
> > version conflicts with other software.
>
Agreed, I always use RPMs because they help me to keep my systems
in a reversible state. BUT, I never install binary RPMs that are
not taylored to the Distro-Version that I am running (Try to install
RH-7.2 packages on a RH-7.0). I may take source RPMs from other
distributions as a starting point for my own tweaks, but I NEVER
install 'foreign' RPMs.


Attached you'll find a replacement for the setup.py script allowing
to build a single binary and source RPM for Numeric + subpackages.

The README.RPM for a normal user would be something like:
(1) unpack python-numeric-XXX.YY.Z.tar.gz
(2) cd python-numeric-XXX.YY.Z
(3) python setup.py bdist_rpm
This will build a binary and source RPM adapted to YOUR system
(even yours, Paul, with your non-standard Python location) in
the subdirectory dist (to be created automatically)
(4) install with rpm -Uvh python-numeric-XXX.YY.Z-1.i?86.rpm
You can always uninstall with rpm -e python-numeric.

IMHO, this is a better way than trying to distibute binary RPMs.


It happens that my favorite distribution is not on JH's list :-(

But also, http://www.mandrakesoft.com/products/81/gaming-edition ,
the favorite toy of 11 years old whizkids, doing pygame-NumPy
programming and who are certainly future students of astronomy.


If you want to test the script:
(1) replace the old setup.py with mine
(2) rename Lib/Numeric.py to Lib/__init__.py (Numeric will become
a real Python package).
(3) python setup.py bdist
(4) and install the RPM

Caveats:
(1) renaming Numeric to python-numeric has as consequence that
the header files go into .../include/python2.1/python-numeric
instead of .../include/python2.1/Numeric. Personally I don't
like it (being author of a package that builds on Numeric).
(I think this can be fixed, if needed).

(2) I tried to acknowledge Paul Dubois' contributions in a
long_description (the subpackages do not build to separate
packages anymore).

(3) The setup.py script shows how to add documentation
to an RPM (requires to work around a bug in distutils).
Personnally, I think that HTML and PDF documentation
should also be added. In the current state of distutils
this requires inclusion of the documentation in the
python-numeric-XXX.YY.Z.tar.gz.


Gerard


---------------------------------------------
This message was sent using Endymion MailMan.
http://www.endymion.com/products/mailman/

Re: [Numpy-discussion] Obscure setup question

[<gve...@la...>]

Hi,

This is very strange. I am also making packages (RPMs in my case).
I do this as a normal user without write access to /usr/.....

If running setup.py would have as a consequence that 
/usr/lib/....../install_headers.pyc would get overwritten,
then setup.py would die.

Can you tell if your python-2.1 is really 2.1 or 2.1.1?
It happens that I am tweaking the setup.py script of
Numeric and testing with python-2.1.1 and I am sure
that distutils works.

I suggest the following test:

login as a normal user, unpack Numeric, cd Numeric and do

python2.1 setup.py install --root=~/tmp

This will install Numeric in ~/tmp/usr/lib/python2.1 and
you will see if setup.py still tries to overwrite
install_headers.pyc

If so, your python2.1 is broken for some reason, if not
your build system and/or sandbox are broken. 

Gerard

> Hi!
> 
> I have noticed an interesting difference when installing Numeric-20.3
> against python-2.1 vs. python-2.2.
> 
> When installing against python-2.1, setup trys to write to
> /usr/lib/python2.1/distutils/command/install_headers.pyc
> 
> But when installing against python-2.2, this doesn't happen.
> 
> Note:  this question came up because I am making an ebuild (basically a
> compile and install script) for Gentoo linux, http://www.gentoo.org. 
> The attempted write to install_headers.pyc with python-2.1 violates
> Gentoo's sandboxed build environment causing the merge to fail.  If you
> are interested see 
> 
> http://bugs.gentoo.org/show_bug.cgi?id=21
> 
> Just curious if there is a known explanation for this.  The simple fix
> on my end, is to set the dependency >=python-2.2.
> 
> Thanks for the great program!
> 
> tod
> 
> 
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 


---------------------------------------------
This message was sent using Endymion MailMan.
http://www.endymion.com/products/mailman/

[Numpy-discussion] Obscure setup question

[Tod M. N. <tn...@fi...>]

Hi!

I have noticed an interesting difference when installing Numeric-20.3
against python-2.1 vs. python-2.2.

When installing against python-2.1, setup trys to write to
/usr/lib/python2.1/distutils/command/install_headers.pyc

But when installing against python-2.2, this doesn't happen.

Note:  this question came up because I am making an ebuild (basically a
compile and install script) for Gentoo linux, http://www.gentoo.org. 
The attempted write to install_headers.pyc with python-2.1 violates
Gentoo's sandboxed build environment causing the merge to fail.  If you
are interested see 

http://bugs.gentoo.org/show_bug.cgi?id=21

Just curious if there is a known explanation for this.  The simple fix
on my end, is to set the dependency >=python-2.2.

Thanks for the great program!

tod

Re: [Numpy-discussion] RPMs out of date, have problems

[Konrad H. <hi...@cn...>]

Joe Harrington <jh...@oo...> writes:

> RPM.  Plus, I can uninstall!  What matters is what the general users
> want, and they have spoken loudly in favor of package managers.

I couldn't agree more. I tend to make many of my own RPMs, and still
I prefer RPMs to straightforward installation in /usr/local. Clean
updating and uninstalling is a big advantage.

> /usr/bin/python2.1 exists under 7.2, and claims in its startup that it
> existed in 7.1.  You get 1.5.2 if you just type "python", but you get
> 2.1.1 if you type "python2.1".  alias python python2.1...

Definitely not in 7.1, there was not Python 2.1 at all (before I
installed it by hand).

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------

Re: [Numpy-discussion] RPMs out of date, have problems

[Joe H. <jh...@oo...>]

> I *never* build into /usr or /usr/local.

Good!  You should leave /usr for your package manager to deal with.
However, you should *release* software to install there, which you can
with RPM without having to build it in /usr.  /usr/local is yours to
play with.  The only thing in mine is IDL.  (...which is why I'd like
to delete it!)

> I suppose
> what was bothering people was having to use --prefix because the default
> will not work because my Python is not in some standard place (said
> standard place seeming to me to be a mythical location given how I see
> people actually working).

There was no documentation anywhere in the download area suggesting
the use of --prefix.  Even if there were, RPMs get collected and put
in archives (rpmfind.net) without any other associated docs.  They
should always install into the system following the Linux file system
standards.  Nobody expects to have to configure an RPM.  They're just
supposed to work, and the vast majority just do.

> There are lots of people like me
> that have multiple pythons around with other stuff added in.

There are lots if *developers* like you.  The current group of Numpy
users is demographically distinct from the future user base, if Numpy
becomes a player for numerical work in science and engineering.  We
will have many users who have never written a line of C or C++ or
FORTRAN and who will run screaming at the mention of the word "tar".
We will have high school teachers and students.  We will have lots of
people who have zero time and zero tolerance for learning unrelated
topics like system administration or software tweaking.  If we don't
care about them, they won't use our software.

> I continue to be less than impressed by RPMs. Every time I try one I get
> version conflicts with other software.

I understand your frustration with the bad RPMs, but it's not the
fault of RPM, it's the fault of either the package developer or the
person installing the package.  If the package developer links against
a library that is itself under rapid development, there is a problem
waiting to happen.  It's worse if that library is included in some or
all distributions.  The simple solution is to statically link that
library, or rename it and include it in the package.  If the user has
an out-of-date system, it's also trouble.

If we build against the C library and python that are in the OS
releases, we're in good shape.  I don't think we need to bend over
backwards to provide RPMs for people who install their own Python,
unless a new version of Numeric requires use of an updated Python RPM.
Very few (maybe 0.5%?) "mass-market" users will install their own
Python.  The experts who do can handle a tarball install, or one will
build an RPM and distribute it if it's a particularly important
release.

The conflict problems pale in comparison to the problems with manually
building all packages you want to install, particularly if you don't
know how to fix makefile problems or don't have a compiler.  If you're
only installing a few packages, it's not that big a deal.  If you're
installing 90, it's a month's work by hand or a day with RPM, if you
don't know the install procedure for the software beforehand.  See
ftp://oobleck.astro.cornell.edu/pub/clues.tar.gz for my attempt to
tame this problem.  As you can see from the dates of the files
therein, I abandonned maintaining each package page as it came out
under RPM.  It was just impossible to keep up.  It's trivial under
RPM.  Plus, I can uninstall!  What matters is what the general users
want, and they have spoken loudly in favor of package managers.

Konrad writes:
> RedHat
> is rather conservative among the Linux distributors, but that is also
> the reason for still using Python 1.5 in their latest versions.

/usr/bin/python2.1 exists under 7.2, and claims in its startup that it
existed in 7.1.  You get 1.5.2 if you just type "python", but you get
2.1.1 if you type "python2.1".  alias python python2.1...

--jh--

Re: [Numpy-discussion] RPMs out of date, have problems

[<ro...@bl...>]

>>>>> "SR" == Scott Ransom <ra...@ph...> writes:

    SR> Hello All, Debian (unstable -- which is actually quite stable)
    SR> provides slightly more up-to-date packages of numeric (20.2)
    SR> and its extensions.  You can find them here:

    SR> http://packages.debian.org/unstable/math/python-numeric.html
    SR> http://packages.debian.org/unstable/interpreters/python-numeric-ext.html

Nothing like 2 days to make it now 20.3 in Debian unstable/Sid.

best,
-tony

-- 
A.J. Rossini				Rsrch. Asst. Prof. of Biostatistics
U. of Washington Biostatistics		rossini@u.washington.edu	
FHCRC/SCHARP/HIV Vaccine Trials Net	ro...@sc...
-------------- http://software.biostat.washington.edu/ --------------
FHCRC: M-W: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email
UW:   T-Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX
Rosen: (Mullins' Lab) Fridays, and I'm unreachable except by email.

RE: [Numpy-discussion] RPMs out of date, have problems

[Paul F. D. <pa...@pf...>]

I can confirm that the Python I use is in /pcmdi/dubois/python. I
*never* build into /usr or /usr/local. There are lots of people like me
that have multiple pythons around with other stuff added in. I suppose
what was bothering people was having to use --prefix because the default
will not work because my Python is not in some standard place (said
standard place seeming to me to be a mythical location given how I see
people actually working).

I continue to be less than impressed by RPMs. Every time I try one I get
version conflicts with other software.

Re: [Numpy-discussion] RPMs out of date, have problems

[Joe H. <jh...@oo...>]

In the exchange below, my query is >>, Gerard's response is >, and my
present response to that is unquoted.

>>From: Gerard Vermeulen <gve...@la...>
>>Subject: Re: [Numpy-discussion] RPMs out of date, have problems
>>Date: Fri, 4 Jan 2002 09:44:21 +0100

>> Could you check that it produces an RPM with files that go in
>> /usr/lib/python2.1     rather than /pcmdi/dubois/linux/lib/python2.1 and
>> /usr/include/python2.1 rather than /pcmdi/dubois/linux/include/python2.1?

>Yes, mine go into /usr/lib/python2.1

>> Paul D. isn't sure why /pcmdi is being created.  It's probably that
>> way on his machine and he doesn't know how to tell distutils to use a
>> different default prefix.  Neither do I.  That is the issue.  Do you?

>Well, the paths are figured out in /usr/lib/python2.1/distutils/sysconfig.py,
>using sys.prefix or sys.exec_prefix.

>So, I think that Paul's python has been build with
>./configure --prefix=/pcmdi/dubois/linux
>and that it still lives there, or has been moved to /usr afterwards.

>Or that it resides on an NFS mounted volume???
>Or some python startup file that messes sys.(exec_)prefix.

Ok, Paul, Konrad, is this what you need?

>Anyhow, I think that with respect to binary packages
>Windows has the edge over Linux (I know from experience,
>because I am author of a Python wrapper to Qwt - a Qt library
>allowing to do interactive plotting and it is SO easy to
>produce a Windows).

>Every combination of Distro-Version-Python-Version would
>require its own binary RPM.

If you consider Microsoft Windows one OS, Red Hat Linux another,
Debian GNU/Linux a third, and so on, the distinction goes away.  The
fact that Microsoft has a monopoly is definitely convenient to
producers.  However, as users, we lose and shouldn't support it.

>(A) IMHO, it is better to add a README.DIY.RPM:
>(I am going through the steps based on Numeric-20.2.1)
>(1) unpack Numeric-20.2.1.tar.gz (or whatever)
>(2) cd Numeric-20.2.1
>(2) python setup.py bdist_rpm
>(3) cd dist
>(4) rpm -Uvh Numeric-20.2.1-1.i?86.rpm (as root, i?86 could be another
>architecture)

>But it is not so easy to build the extension packages (FFT & friends) like 
>this, without knowledge of RPM. I could try to hack setup.py to make it 
>compatible with RPM (means making 1 big package without subpackages)

>(B) The other possibility is to include a generic python-numeric.spec file
>(you build an RPM with rpm -ba python-numeric.spec). I can adapt mine.

>Gerard

>PS: do you have a RPM based Linux, yourself?

Yes, I run Red Hat.  So do the vast majority of US astronomers I have
spoken to.  I understand that SuSE is equally popular in Europe.  A
few other dists just copy one of them and add a few packages, and so
don't need separate RPMs of their own.  If we configure RPMs with the
version of python included in these dists, we're not talking about a
large number of packages, after all, to get 98% of the users or
better.  I think you are onto a solution:

We build RPMs (and .deb files for Debian, if we think there are enough
Debian users to make it worthwhile, and the same for Solaris if it
uses a different package manager) for the current release of each
popular distribution, with that release's python.  That's maybe 4
binary packages per Numeric release (Red Hat, SuSE, Debian, Solaris).
We also make a source RPM and distribute with it instructions for
building a new binary RPM with appropriately-named version ID (e.g.,
python-numeric-20.4py2.2glibc6-1).  We solicit users of unsupported
combinations of OS and Python version to contribute those RPMs along
with a simple ASCII form specifying what kind of system made it,
contact info, etc.  I predict we will not get a huge number of such
submissions from outside this list, as the vast majority of Numeric
users who upgrade their Python when a new Python release comes out (as
opposed to a new OS release) are developers who don't care about the
RPM anyway.

The ultimate solution is to get this thing incorporated into the
python core.

--jh--