numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Re: Purchasing Documentation

[Tim C. <tc...@op...>]

Paul F. Dubois wrote:
> The original sources were in Framemaker. I am not positive where they
> are. In any case they are copyrighted by the Regents of the University
> of California. I am not a lawyer and don't know what the consequences of
> that are. LLNL granted free distribution of the printed document with
> the Numeric source code but I don't know what their position would be on
> using their copyrighted text in a new document or on giving away the
> sources.

OK, thanks Paul. That may have implications for you, Travis, if you are
planning to base your SciPy Core book on the existing NumPy documentation.

Given the circumstance which Paul describes, perhaps the only way
forward is to create a freely available addendum to the freely
available, existing NumPy documentation which describes how and where
SciPy Core differs or extends NumPy? Or to write entirely new
documentation from scratch.

> I believe that under U.S. copyright law, a person may lend their copy of
> a book to anyone but not reproduce it and give them a copy.

Yes, likewise here in Oz, and in most countries. Of course, the
publisher of a book can impose additional restrictions to which the
purchaser of the book must agree, such as 'You may not show or lend your
copy of this book to your colleagues, or to anyone else.' However, who
in their right mind would buy a book which was sold with such
restrictions attached to it?

> Given the way open source capitalism works, I would not be surprised if
> someone produces a 'quick reference guide' that they give away, or
> writes a book on 'Using scipy.core in biology' that they sell for $88.
> Let a thousand flowers bloom.

Yes, I agree entirely. Travis, you are at perfect liberty to create
commercial documentation for SciPy Core, but please don't object if
others try to organise to create free open source documentation as well.

Tim C

> Tim Churches wrote:
> 
>> Travis Oliphant wrote:
>>
>>> Tim Churches wrote:
>>>
>>>
>>>> Travis Oliphant wrote:
>>>>
>>>>
>>>>
>>>>> Tim Churches wrote:
>>>>>
>>>>>  
>>>>
>>>>
>>>> Travis, are you saying that this agreement only allows a single person
>>>> to read the single printed copy? If so, I think you need a formally
>>>> worded legal license to make that stick - certainly Australian
>>>> copyright
>>>> law (nor copyright law in other countries, I suspect) does not provide
>>>> any support for such severe restrictions in the use of a printed
>>>> document. Under copyright law, you may not make unauthorised copies
>>>> of a
>>>> printed document, but you can certainly lend that copy to others, or
>>>> sell or give it to them, or share it as a bench manual.
>>>>
>>>>
>>>
>>> I'm just stating what I think is fair.  I'm not going to try and use the
>>> power of the Australian state or any other to try and enforce it. 
>>
>>
>>
>> My point was that you won't get any help from the State if what you
>> think is fair extends to restricting who can read a single printed copy
>> of your documentation. You will need to resort to tort law to enforce
>> such restrictions, and thus you really need a formal documentation
>> license, if that is your aim.
>>
>>
>>>> Many of those downloads are for "t[y|i]re-kicking" - potential users
>>>> determining whether it meets their needs. If those potential users have
>>>> to pay for documentation up-front, then a large proportion will go
>>>> elsewhere. The rest are probably existing NumPy users upgrading (or
>>>> testing the upgrade waters). The latter group may well pay for
>>>> documentation, but how large is that group? For example, how many
>>>> people
>>>> are subscribed to the numpy-discussion mailing list?
>>>>
>>>>
>>>
>>> Don't know.  But, there is enough free documentation to get started, I
>>> think.    Lack of documentation has hampered more people than cost of
>>> existing documentation, I think.
>>
>>
>>
>> I have to disagree on this point. Granted the existing documentation for
>> NumPy is not fantastic - some aspects of it are positively obscure - but
>> overall it is better than good-enough, we have found, and was not a
>> barrier to our decision to use NumPy. Having to cough up $50 to even
>> peek at the documentation would be a far, far greater barrier to uptake
>> than the occasional obscure wording of some of the function
>> descriptions, IMHO.
>>
>>
>>>> Travis, although many of us are grateful for your efforts on SciPy
>>>> Core,
>>>> no-one made you do it. If you wanted to earn (more) money by doing
>>>> other
>>>> things, you should have done them.
>>>>
>>>
>>> I'm really thinking about the future here.  If I can succesfully raise
>>> money to support work on this using a simple time-delay documentation
>>> encouragement, then perhaps others will do the same, and we can all have
>>> more.    Waiting for people to "donate" their time to develop scientific
>>> python is rather slow....  I've been freely donating time for a long
>>> time, and there are few who help.   So, we'll try a slightly different
>>> route and see where that goes.
>>
>>
>>
>> Yes, fair enough. My point was that 7 years is far too long a time
>> delay. Given that SciPy Core will probably take another year to become
>> rock-solid, I would happily buy several copies of your documentation at
>> $50 per copy if I knew that in a year the documentation could be freely
>> distributed to all God's children. But in 7 years? Nope, too long to
>> wait, not interested in supporting that.
>>
>>
>>>> I think there needs to be some community debate about this. Is there
>>>> sufficient interest for people other than Travis to start with the
>>>> Numeric documentation and update it as necessary to become a free SciPy
>>>> Core documentation? The NumPy documentation is available in HTML format
>>>> as the basis of this - perhaps the original source (LaTeX?) for the
>>>> Numeric docs is also available?
>>>
>>>
>>> I don't know why you would want to undermine my efforts in this way by
>>> duplicating effort. 
>>
>>
>>
>> Travis, I don't want to undermine your efforts, but you must understand
>> that one of the attractions of NumPy and thus Scipy Core is that they
>> are free, open source software, which implies that there exists adequate
>> free, open source documentation for them, or if such free, open source
>> documentation does not exist, then taht users are collectively at
>> liberty to create it. That's a basic FOSS tenet. What you are are
>> proposing is the creation of proprietary documentation for SciPy Core.
>> No-one objects to this, but only if it is supplementary to free, open
>> source "core" documentation, not instead of it.
>>
>>
>>> Perhaps, instead you could have people donate $$
>>> instead of time to releasing the documentation.    I give away copies of
>>> the documentation to people who participate in the development process
>>> all the time (and that comes off the total price --- though I haven't
>>> advertised this as of yet).   So, why don't you encourage people who
>>> don't have the money to contribute to the project instead.
>>
>>
>>
>> I understand that open source software and its documentation don't just
>> appear out of thin air, and I fully support the idea of funded or
>> collectively-commissioned open source development. But only if the
>> results of that funded or commissioned development is, in fact, free and
>> open sourced. By preventing free distribution of the documentation you
>> are creating for 7 years, the end product cannot even remotely be
>> described as open source, thus personally I am disinterested in funding
>> it. If the delay until open sourcing was just one year, then that's a
>> different matter, but you do not seem to be proposing that.
>>
>> Hence I think the NumPy/SciPy Core user community should establish a
>> SciPy Core documentation project, with the aim of updating and enhancing
>> the existing NumPy documentation so that it covers the new and changed
>> features of SciPy Core, and making that documentation available on teh
>> same free, open source basis as NumPy/SciPy Core itself.
>>
>> To that end, are the original LaTeX or whatever sources for the currenet
>> NumPy documenattion available somewhere (apart from the HTML and PDF
>> versions, that is).
>>
>> Tim C
>>
>>
>> -------------------------------------------------------
>> This SF.Net email is sponsored by:
>> Power Architecture Resource Center: Free content, downloads, discussions,
>> and more. http://solutions.newsforge.com/ibmarch.tmpl
>> _______________________________________________
>> Numpy-discussion mailing list
>> Num...@li...
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
> 

Re: [Numpy-discussion] Re: Purchasing Documentation

[Tim C. <tc...@op...>]

Chris Barker wrote:
> Tim Churches wrote:
> 
>> I think there needs to be some community debate about this.
> 
> Well, no, there doesn't. Travis is doing two things:
> 
> 1) Spending a great deal of his time producing some wonderful open
> source software.
> 
> 2) Embarking on a small business venture, trying to sell books.
> 
> There is no need for community debate about an individual's business
> venture. It might benefit Travis to do a bit more market research, but
> as was pointed out, there's no reason he couldn't lower those limits and
> open source the book sooner if he chooses to.

Chris, I think you misunderstand what I meant - whic is clarified in
another message I just posted a minute ago. I meant that there needs to
be community debate over whether there is a requirement for a project to
create free, open source documentation for SciPy Core, not over whether
Travis does or doesn't have the right to, as you put it "...embark on a
small business venture, trying to sell books.". I completely agree that
he has every legal and moral right in the world to do that.

> Travis Oliphant wrote:
> 
>> I am interested in feedback.  If  you don't buy the book because you
>> think I'm asking too much money, then let me know.
> 
> Since you ask, I think $39.99 is a bit steep for an electronic copy. As
> a quick comparison, I just bought the new wxWidgets book, documenting
> another open source project. You can get a printed copy from Amazon for
> $32.99.
> 
> I also do think Tim has a point. Usually the open source model is that
> the basic reference docs are freely available, and there are nice user
> and newbie friendly books for sale. It will be much harder for
> scipy.base to catch on if there are no freely available docs.
> 
> However, having looked at the little bit of the book that is now
> available, the quality and detail are far beyond what one usually finds
> in open-source references (with the possible exception of the python
> references). It certainly looks better than the old NumPy docs, which
> were still adequate.
> 
> I have no doubt that the NumPy community will produce some free "getting
> started" docs anyway, so we can all be happy. Maybe, as Tim suggests, we
> could fill a Wiki with the existing Numpy docs, and all start editing away!

Yup, that's all I had in mind. If, in fact we are allowed to edit the
existing NumPy docs. If not, then a wiki to produce an addendum or set
of annotations to them (like a Biblical concordance, perhaps) is the
best that can be done.

Tim C

Re: [Numpy-discussion] Re: Purchasing Documentation

[Chris B. <Chr...@no...>]

Tim Churches wrote:

> I think there needs to be some community debate about this.

Well, no, there doesn't. Travis is doing two things:

1) Spending a great deal of his time producing some wonderful open 
source software.

2) Embarking on a small business venture, trying to sell books.

There is no need for community debate about an individual's business 
venture. It might benefit Travis to do a bit more market research, but 
as was pointed out, there's no reason he couldn't lower those limits and 
open source the book sooner if he chooses to.

Travis Oliphant wrote:
> I am interested in feedback.  If  you don't buy the book because you 
> think I'm asking too much money, then let me know.

Since you ask, I think $39.99 is a bit steep for an electronic copy. As 
a quick comparison, I just bought the new wxWidgets book, documenting 
another open source project. You can get a printed copy from Amazon for 
$32.99.

I also do think Tim has a point. Usually the open source model is that 
the basic reference docs are freely available, and there are nice user 
and newbie friendly books for sale. It will be much harder for 
scipy.base to catch on if there are no freely available docs.

However, having looked at the little bit of the book that is now 
available, the quality and detail are far beyond what one usually finds 
in open-source references (with the possible exception of the python 
references). It certainly looks better than the old NumPy docs, which 
were still adequate.

I have no doubt that the NumPy community will produce some free "getting 
started" docs anyway, so we can all be happy. Maybe, as Tim suggests, we 
could fill a Wiki with the existing Numpy docs, and all start editing away!

-Chris






-- 
Christopher Barker, Ph.D.
Oceanographer
                                      		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] Re: Purchasing Documentation

[Tim C. <tc...@op...>]

Travis Oliphant wrote:
> Tim Churches wrote:
> 
>> Travis Oliphant wrote:
>>  
>>
>>> Tim Churches wrote:
>>>
>>>   
>>
>> Travis, are you saying that this agreement only allows a single person
>> to read the single printed copy? If so, I think you need a formally
>> worded legal license to make that stick - certainly Australian copyright
>> law (nor copyright law in other countries, I suspect) does not provide
>> any support for such severe restrictions in the use of a printed
>> document. Under copyright law, you may not make unauthorised copies of a
>> printed document, but you can certainly lend that copy to others, or
>> sell or give it to them, or share it as a bench manual.
>>  
>>
> I'm just stating what I think is fair.  I'm not going to try and use the
> power of the Australian state or any other to try and enforce it. 

My point was that you won't get any help from the State if what you
think is fair extends to restricting who can read a single printed copy
of your documentation. You will need to resort to tort law to enforce
such restrictions, and thus you really need a formal documentation
license, if that is your aim.

>> Many of those downloads are for "t[y|i]re-kicking" - potential users
>> determining whether it meets their needs. If those potential users have
>> to pay for documentation up-front, then a large proportion will go
>> elsewhere. The rest are probably existing NumPy users upgrading (or
>> testing the upgrade waters). The latter group may well pay for
>> documentation, but how large is that group? For example, how many people
>> are subscribed to the numpy-discussion mailing list?
>>  
>>
> Don't know.  But, there is enough free documentation to get started, I
> think.    Lack of documentation has hampered more people than cost of
> existing documentation, I think.

I have to disagree on this point. Granted the existing documentation for
NumPy is not fantastic - some aspects of it are positively obscure - but
overall it is better than good-enough, we have found, and was not a
barrier to our decision to use NumPy. Having to cough up $50 to even
peek at the documentation would be a far, far greater barrier to uptake
than the occasional obscure wording of some of the function
descriptions, IMHO.

>> Travis, although many of us are grateful for your efforts on SciPy Core,
>> no-one made you do it. If you wanted to earn (more) money by doing other
>> things, you should have done them.
>>  
> I'm really thinking about the future here.  If I can succesfully raise
> money to support work on this using a simple time-delay documentation
> encouragement, then perhaps others will do the same, and we can all have
> more.    Waiting for people to "donate" their time to develop scientific
> python is rather slow....  I've been freely donating time for a long
> time, and there are few who help.   So, we'll try a slightly different
> route and see where that goes.

Yes, fair enough. My point was that 7 years is far too long a time
delay. Given that SciPy Core will probably take another year to become
rock-solid, I would happily buy several copies of your documentation at
$50 per copy if I knew that in a year the documentation could be freely
distributed to all God's children. But in 7 years? Nope, too long to
wait, not interested in supporting that.

>> I think there needs to be some community debate about this. Is there
>> sufficient interest for people other than Travis to start with the
>> Numeric documentation and update it as necessary to become a free SciPy
>> Core documentation? The NumPy documentation is available in HTML format
>> as the basis of this - perhaps the original source (LaTeX?) for the
>> Numeric docs is also available?
>
> I don't know why you would want to undermine my efforts in this way by
> duplicating effort. 

Travis, I don't want to undermine your efforts, but you must understand
that one of the attractions of NumPy and thus Scipy Core is that they
are free, open source software, which implies that there exists adequate
free, open source documentation for them, or if such free, open source
documentation does not exist, then taht users are collectively at
liberty to create it. That's a basic FOSS tenet. What you are are
proposing is the creation of proprietary documentation for SciPy Core.
No-one objects to this, but only if it is supplementary to free, open
source "core" documentation, not instead of it.

>  Perhaps, instead you could have people donate $$
> instead of time to releasing the documentation.    I give away copies of
> the documentation to people who participate in the development process
> all the time (and that comes off the total price --- though I haven't
> advertised this as of yet).   So, why don't you encourage people who
> don't have the money to contribute to the project instead.

I understand that open source software and its documentation don't just
appear out of thin air, and I fully support the idea of funded or
collectively-commissioned open source development. But only if the
results of that funded or commissioned development is, in fact, free and
open sourced. By preventing free distribution of the documentation you
are creating for 7 years, the end product cannot even remotely be
described as open source, thus personally I am disinterested in funding
it. If the delay until open sourcing was just one year, then that's a
different matter, but you do not seem to be proposing that.

Hence I think the NumPy/SciPy Core user community should establish a
SciPy Core documentation project, with the aim of updating and enhancing
the existing NumPy documentation so that it covers the new and changed
features of SciPy Core, and making that documentation available on teh
same free, open source basis as NumPy/SciPy Core itself.

To that end, are the original LaTeX or whatever sources for the currenet
NumPy documenattion available somewhere (apart from the HTML and PDF
versions, that is).

Tim C

[Numpy-discussion] Contributions to development is one way to get documentation

[Travis O. <oli...@ee...>]

I have not made this explicitly clear,  but if somebody cannot afford 
the documentation, I will give it to them in return for help with 
development (significant bug-fixes, optimizations, improved docstrings,  
new features etc.).  Any copy I give away, counts against the total 
price as well...

I've already given away several copies.  If you feel like you should 
receive one, and I've forgotten about you,  let me know.  

Best,

-Travis

[Numpy-discussion] Re: Purchasing Documentation

[Tim C. <tc...@op...>]

Travis Oliphant wrote:
> Tim Churches wrote:
> 
>> Eric Firing wrote:
>>  
>>
>>>> OK, thanks. In the absence of documentation, I just looked for an MA
>>>> subdirectory, couldn't find one and assumed that it wasn't (yet)
>>>> supported.
>>>>     
>>>
>>> Tim,
>>>
>>> Documentation is coming along, but being made available in an unusual
>>> way: http://www.tramy.us/
>>>   
>>
>>
>> copy of the documentation. Most likely, one copy of the documentation
>> will be purchased to be shared between several (or many) users in a
>> workgroup within an institution.
> 
> 
> Note that it is expressly against the agreement for one copy to be
> shared between multiple users at the same institution.  I hope this is
> clear....  Of course you can let somebody else look at it a couple of
> times, but if they will use it regularly, they need to get their own copy. 
> Prices are always a matter of supply and demand.   The whole point of
> the system is to allow the price system to help coordinate what people
> think is valuable and what developers spend time doing.    What you see
> currently is the maximum price (and time) I could possibly set as per
> the agreement with Trelgol.   These things can always come down,
> however, as time proceeds, and the market responds.

The only agreement I can find is these words on the above site:

"When you receive a copy of content under an MBTDR, you implicitly agree
to make only back-up copies of the content and to not make a copy for
others during the restriction period.  If additional copies of the
content are needed (for multiple users at a company, for example) you
must purchase them. You may make exactly one "printed" or hard-copy for
each copy of the content you purchase.   For books, Trelgol may make
available volume printing options at cost for those who have purchased a
copy of the electronic content."

Travis, are you saying that this agreement only allows a single person
to read the single printed copy? If so, I think you need a formally
worded legal license to make that stick - certainly Australian copyright
law (nor copyright law in other countries, I suspect) does not provide
any support for such severe restrictions in the use of a printed
document. Under copyright law, you may not make unauthorised copies of a
printed document, but you can certainly lend that copy to others, or
sell or give it to them, or share it as a bench manual.

> Now, obviously the cost of the documentation includes something of the
> cost of producing the actual code.  Of course, you may disagree, but I
> did choose the numbers based on a little bit of market research.   I
> don't think that 7000 copies of the documentation or 7 years is all that
> ridiculous given that there have been over 12000 downloads of the
> Numeric 24.0b2 source code since April and Numeric has been in stable
> use for nearly 10 years.

Many of those downloads are for "t[y|i]re-kicking" - potential users
determining whether it meets their needs. If those potential users have
to pay for documentation up-front, then a large proportion will go
elsewhere. The rest are probably existing NumPy users upgrading (or
testing the upgrade waters). The latter group may well pay for
documentation, but how large is that group? For example, how many people
are subscribed to the numpy-discussion mailing list?

> If scipy does it's job correctly,  then a user-base needing
> documentation of 7000 is rather low-balling it I would say.   I want
> scipy to surpass the number of users of Numeric.     I'm trying to make
> scipy core so that everybody can convert to it, eventually.  The old
> Numeric manual still provides documentation, and the source is still
> available.  I think you are still getting a great deal.   Unless there
> is another majore re-write, the documentation will be updated as it goes
> (and you get the updates). 

Yes, maybe all that is needed is a (free) SciPy Core update of the
existing (free) NumPy documentation. The (non-free) SciPy Core book
which Travis is writing can complement that, just as the dozens of
commercial Python books complement the free Python documentation. But
imagine if everyone had to pay $50 to access the basic Python
documentation - the number of Python users worldwide would be very much
smaller than it is is now, I dare say.

>> I would say that perhaps $30k or one year (after completion of the
>> documentation) would be more reasonable criteria for making the
>> documentation freely available (but then I am not writing it).
>>  
>>
> Well, given the time I had to spend on this, that is quite a bit less
> than the market will bear for my services elsewhere.  I suppose if I
> were rich, I could donate more of my time.  But, I'm not....

Travis, although many of us are grateful for your efforts on SciPy Core,
no-one made you do it. If you wanted to earn (more) money by doing other
things, you should have done them.

> I'm really not trying to make people upset.  I'm really a huge fan of
> open information, and would love to see the documentation completely
> free.  It's just that I cannot afford to create it for nothing.   I have
> lots of demands on my time.  Spending it doing scientific python has to
> be justified, somehow.   I did not start the creation of a hybrid
> Numeric / numarray with the hope of making any money.  I started it
> because there was a need. I thought I would get more help with its
> implementation.  But, given the reality of people's scarce time (they
> need to make money...), nobody was able to help me.  Out of this, the
> documentation idea was born to try and help fund development of scipy core.
> I hope people can understand that the reality of scarcity dictates that
> we coordinate efforts through some mechanism.  The price mechanism has
> been the most succesful large-scale mechanism yet developed.
> 
> I am interested in feedback.  If  you don't buy the book because you
> think I'm asking too much money, then let me know, as Tim has done.  You
> can email me directly, as well.

I think there needs to be some community debate about this. Is there
sufficient interest for people other than Travis to start with the
Numeric documentation and update it as necessary to become a free SciPy
Core documentation? The NumPy documentation is available in HTML format
as the basis of this - perhaps the original source (LaTeX?) for the
Numeric docs is also available?

Tim C

[Numpy-discussion] Purchasing Documentation

[Travis O. <oli...@ee...>]

Tim Churches wrote:

>Eric Firing wrote:
>  
>
>>>OK, thanks. In the absence of documentation, I just looked for an MA
>>>subdirectory, couldn't find one and assumed that it wasn't (yet)
>>>supported.
>>>      
>>>
>>Tim,
>>
>>Documentation is coming along, but being made available in an unusual
>>way: http://www.tramy.us/
>>    
>>
>
>copy of the documentation. Most likely, one copy of the documentation
>will be purchased to be shared between several (or many) users in a
>workgroup within an institution. 
>

Note that it is expressly against the agreement for one copy to be 
shared between multiple users at the same institution.  I hope this is 
clear....  Of course you can let somebody else look at it a couple of 
times, but if they will use it regularly, they need to get their own 
copy.  

Prices are always a matter of supply and demand.   The whole point of 
the system is to allow the price system to help coordinate what people 
think is valuable and what developers spend time doing.    What you see 
currently is the maximum price (and time) I could possibly set as per 
the agreement with Trelgol.   These things can always come down, 
however, as time proceeds, and the market responds.

Now, obviously the cost of the documentation includes something of the 
cost of producing the actual code.  Of course, you may disagree, but I 
did choose the numbers based on a little bit of market research.   I 
don't think that 7000 copies of the documentation or 7 years is all that 
ridiculous given that there have been over 12000 downloads of the 
Numeric 24.0b2 source code since April and Numeric has been in stable 
use for nearly 10 years.

If scipy does it's job correctly,  then a user-base needing 
documentation of 7000 is rather low-balling it I would say.   I want 
scipy to surpass the number of users of Numeric.     I'm trying to make 
scipy core so that everybody can convert to it, eventually.  The old 
Numeric manual still provides documentation, and the source is still 
available.  I think you are still getting a great deal.   Unless there 
is another majore re-write, the documentation will be updated as it goes 
(and you get the updates).  

>I would say that perhaps $30k or one year (after completion of the
>documentation) would be more reasonable criteria for making the
>documentation freely available (but then I am not writing it).
>  
>
Well, given the time I had to spend on this, that is quite a bit less 
than the market will bear for my services elsewhere.  I suppose if I 
were rich, I could donate more of my time.  But, I'm not....

I'm really not trying to make people upset.  I'm really a huge fan of 
open information, and would love to see the documentation completely 
free.  It's just that I cannot afford to create it for nothing.   I have 
lots of demands on my time.  Spending it doing scientific python has to 
be justified, somehow.   I did not start the creation of a hybrid 
Numeric / numarray with the hope of making any money.  I started it 
because there was a need. I thought I would get more help with its 
implementation.  But, given the reality of people's scarce time (they 
need to make money...), nobody was able to help me.  Out of this, the 
documentation idea was born to try and help fund development of scipy 
core. 

I hope people can understand that the reality of scarcity dictates that 
we coordinate efforts through some mechanism.  The price mechanism has 
been the most succesful large-scale mechanism yet developed.

I am interested in feedback.  If  you don't buy the book because you 
think I'm asking too much money, then let me know, as Tim has done.  You 
can email me directly, as well.

Best regards,

-Travis

Re: [Numpy-discussion] ma is in scipy_core

[Tim C. <tc...@op...>]

Eric Firing wrote:
> 
>> OK, thanks. In the absence of documentation, I just looked for an MA
>> subdirectory, couldn't find one and assumed that it wasn't (yet)
>> supported.
> 
> 
> Tim,
> 
> Documentation is coming along, but being made available in an unusual
> way: http://www.tramy.us/

Yes. I don't mind paying about AUD$50 for a copy of the SciPy Core
documentation in order to expedite its production, but the Total Price
and Total Time parameters as they currently stand are ridiculous: $300k
represents about 7 or 8 thousand copies - and although there may be that
many users of NumPy worldwide, realistically only a proportion of them
will migrate to SciPy Core and of those, only a proportion will buy a
copy of the documentation. Most likely, one copy of the documentation
will be purchased to be shared between several (or many) users in a
workgroup within an institution. Thus, the $300k mark is unlikely to be
reached and it will be 7 years before the documentation is freely
available. Given that the lifecycle of projects like NumPy or SciPy Core
is of that same order of magnitude, it effectively means that the
documentation for SciPy Core will not be free for most or all of its
lifespan. That is a pretty severe limitation and one which end users
should carefully consider before converting to SciPy Core.

I would say that perhaps $30k or one year (after completion of the
documentation) would be more reasonable criteria for making the
documentation freely available (but then I am not writing it).

Tim C

Re: [Numpy-discussion] ma is in scipy_core

[Eric F. <ef...@ha...>]

> OK, thanks. In the absence of documentation, I just looked for an MA
> subdirectory, couldn't find one and assumed that it wasn't (yet) supported.

Tim,

Documentation is coming along, but being made available in an unusual 
way: http://www.tramy.us/

Eric

Re: [Numpy-discussion] ma is in scipy_core

[Eric F. <ef...@ha...>]

Tim Churches wrote:
> Eric Firing wrote:
> 
>>Paul, Tim,
>>
>>Masked arrays *are* in scipy_core.
>>
>>import scipy.base.ma as ma
> 
> 
> OK, thanks. In the absence of documentation, I just looked for an MA
> subdirectory, couldn't find one and assumed that it wasn't (yet) supported.
> 
> 
>>In addition, a quick look indicates that NaN-handling is in good shape.
> 
> 
> How about other IEEE special values?
> 
> Tim C

Tim,

It has inf:

 >>> import scipy.base as nx
 >>> nx.inf
inf
 >>> nx.isposinf(nx.inf)
True
 >>> nx.isposinf(-nx.inf)
False
 >>> nx.isneginf(-nx.inf)
True
 >>>

I don't know about signed floating point zero; I have never used such a 
beast, and am aware of it only because it is in numarray's ieeespecial 
module.

See http://numeric.scipy.org/new_features.html; it includes:

Errors are handled through the IEEE floating point status flags and 
there is flexibility on a per function / module / builtin level for 
handling these errors.

I haven't looked into how this is done.

Eric

Re: [Numpy-discussion] ma is in scipy_core

[Tim C. <tc...@op...>]

Eric Firing wrote:
> Paul, Tim,
> 
> Masked arrays *are* in scipy_core.
> 
> import scipy.base.ma as ma

OK, thanks. In the absence of documentation, I just looked for an MA
subdirectory, couldn't find one and assumed that it wasn't (yet) supported.

> In addition, a quick look indicates that NaN-handling is in good shape.

How about other IEEE special values?

Tim C

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Tim C. <tc...@op...>]

Chris Barker wrote:
>> Tim Churches wrote:
>>
>>>  missing values are
>>> ubiquitous in the biological sciences and any package which can't handle
>>> them isn't, to put it bluntly, of much use.
> 
> 
> MA is great, but I wonder if many of the simple "missing value" use
> cases could be covered by robust handling of NaNs.

Most.

> Which brings up the question: How does scipy_core handle NaN and the
> other IEEE special values? This was a major weakness in Numeric for me.

MA is indeed very flexible, well-designed and easy to use, but its
weakness is that it is slow - necessarily so due to its "add-on" design
- every operation is at least twice as slow as the equivalent oepration
on a Numeric array. The mask arrays also eat some additional memory,
which is sometimes an issue (untill everyone moves to 64 bit systems).

Tim C

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Tim C. <tc...@op...>]

Stephen Walton wrote:
> Tim Churches wrote:
> 
>> That would eb a most welcome development. If you are in the mood for
>> some minor extensions to MA (you probably aren't, but just on the
>> off-chance...), then support for multiple masking values would be great.
>>  
>>
> This sounds like an overly complicated addition to MA.  If I may be so
> bold, it seems to me that what you might want is your own object which
> maintains parallel arrays of values and "mask reasons," if you will, and
> generates a mask array accordingly

Yes, but the wish is that NumPy, or rather, SciPy Core, treats any mask
value other than zero the same - that is, that the mask value doesn't
have to be one. Teh machinery to attach meaning to mask values is indeed
situation-specific and shouldn't be part of SciPy Core (what is the
handy abbreviation of SciPy Core, BTW?).

Tim C

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Travis O. <oli...@ee...>]

Stephen Walton wrote:

> Tim Churches wrote:
>
>> However, can I ask if there are plans to add Masked Arrays to SciPy
>> Core?
>>
> Eric Firing pointed out on the matplotlib mailing list that they're 
> already there:
>
> import scipy.base.ma as ma


from scipy import ma  

also works

(everything under scipy.base is also under scipy alone).

-Travis

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Chris B. <Chr...@no...>]

Travis Oliphant wrote:
> Chris Barker wrote:
>> Which brings up the question: How does scipy_core handle NaN and the 
>> other IEEE special values? This was a major weakness in Numeric for me.
> 
> They can exist in your arrays and  isnan, isinf, isposinf, isneginf  
> detects them.

Wonderful! I'm really looking forward to this being stable.

thanks, Travis!

-CHB


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Stephen W. <ste...@cs...>]

Tim Churches wrote:

>However, can I ask if there are plans to add Masked Arrays to SciPy
>Core?
>
Eric Firing pointed out on the matplotlib mailing list that they're 
already there:

import scipy.base.ma as ma

[Numpy-discussion] ma is in scipy_core

[Eric F. <ef...@ha...>]

Paul, Tim,

Masked arrays *are* in scipy_core.

import scipy.base.ma as ma

In addition, a quick look indicates that NaN-handling is in good shape.

Eric

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Chris B. <Chr...@no...>]

> Tim Churches wrote:
>>  missing values are
>> ubiquitous in the biological sciences and any package which can't handle
>> them isn't, to put it bluntly, of much use.

MA is great, but I wonder if many of the simple "missing value" use 
cases could be covered by robust handling of NaNs.

Which brings up the question: How does scipy_core handle NaN and the 
other IEEE special values? This was a major weakness in Numeric for me.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Stephen W. <ste...@cs...>]

Tim Churches wrote:

>That would eb a most welcome development. If you are in the mood for
>some minor extensions to MA (you probably aren't, but just on the
>off-chance...), then support for multiple masking values would be great.
>  
>
This sounds like an overly complicated addition to MA.  If I may be so 
bold, it seems to me that what you might want is your own object which 
maintains parallel arrays of values and "mask reasons," if you will, and 
generates a mask array accordingly

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Paul F. D. <pa...@pf...>]

I was going to see if I could port MA. It is pure Python so shouldn't be 
too hard.

Tim Churches wrote:
> Travis Oliphant wrote:
> 
>>This is to announce the release of SciPy Core 0.4.X (beta)
>>
>>It is available at sourceforge which you can access by going to
>>
>>http://numeric.scipy.org
>>
>>Thanks to all who helped me with this release, especially
>>
>>Robert Kern
>>Pearu Peterson
>>
>>Now, let's start getting this stable....
> 
> 
> Great work, and we are much relieved that NumPy has a firm future
> (albeit under a different name).
> 
> However, can I ask if there are plans to add Masked Arrays to SciPy
> Core? If not, we'll have to stick with NumPy - missing values are
> ubiquitous in the biological sciences and any package which can't handle
> them isn't, to put it bluntly, of much use. We are happy to help test MA
> for SciPy Core, and our project (www.netepi.org) has over a thousand
> unit tests which give the basics of NumPy and MA a good work-out - so
> if/when we convert to SciPy Core the unit tests will be helpful. However
> we are not familiar enough with how SciPy Core differs from NumPy to
> help with the porting, I'm afraid.
> 
> Tim C
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Tim C. <tc...@op...>]

Paul F. Dubois wrote:
> I was going to see if I could port MA. It is pure Python so shouldn't be
> too hard.

That would eb a most welcome development. If you are in the mood for
some minor extensions to MA (you probably aren't, but just on the
off-chance...), then support for multiple masking values would be great.
In other words, instead of the mask just taking the values zero or one,
it can also take 2, 3, etc (within the constraints of the Int8 data
type), where any non-zero value means masked. That would allow the
reason why a value in an array is masked to be neatly encoded, not just
the fact that it is masked (missing). For example, in (human) survey
data, a scalar value (eg age at menopasue) may be missing because the
respondent did not meet the precondition(s) for the question (eg sex not
female), or because they have not yet undergone menopause, or because
they refuse to answer, or because they don't know or don't recall.
That's at least 4 different reasons why the value may be missing
(masked). But don't ask me the the mask value should be when you
multiply a masked-because-not-asked value by a
masked-because-refused-to-answer value. Perhaps a reasonable rule would
be max(mask0,mask1)?

Tim C

> 
> Tim Churches wrote:
> 
>> Travis Oliphant wrote:
>>
>>> This is to announce the release of SciPy Core 0.4.X (beta)
>>>
>>> It is available at sourceforge which you can access by going to
>>>
>>> http://numeric.scipy.org
>>>
>>> Thanks to all who helped me with this release, especially
>>>
>>> Robert Kern
>>> Pearu Peterson
>>>
>>> Now, let's start getting this stable....
>>
>>
>>
>> Great work, and we are much relieved that NumPy has a firm future
>> (albeit under a different name).
>>
>> However, can I ask if there are plans to add Masked Arrays to SciPy
>> Core? If not, we'll have to stick with NumPy - missing values are
>> ubiquitous in the biological sciences and any package which can't handle
>> them isn't, to put it bluntly, of much use. We are happy to help test MA
>> for SciPy Core, and our project (www.netepi.org) has over a thousand
>> unit tests which give the basics of NumPy and MA a good work-out - so
>> if/when we convert to SciPy Core the unit tests will be helpful. However
>> we are not familiar enough with how SciPy Core differs from NumPy to
>> help with the porting, I'm afraid.
>>
>> Tim C
>>
>>
>> -------------------------------------------------------
>> This SF.Net email is sponsored by:
>> Power Architecture Resource Center: Free content, downloads, discussions,
>> and more. http://solutions.newsforge.com/ibmarch.tmpl
>> _______________________________________________
>> Numpy-discussion mailing list
>> Num...@li...
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
> 

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Tim C. <tc...@op...>]

Travis Oliphant wrote:
> 
> This is to announce the release of SciPy Core 0.4.X (beta)
> 
> It is available at sourceforge which you can access by going to
> 
> http://numeric.scipy.org
> 
> Thanks to all who helped me with this release, especially
> 
> Robert Kern
> Pearu Peterson
> 
> Now, let's start getting this stable....

Great work, and we are much relieved that NumPy has a firm future
(albeit under a different name).

However, can I ask if there are plans to add Masked Arrays to SciPy
Core? If not, we'll have to stick with NumPy - missing values are
ubiquitous in the biological sciences and any package which can't handle
them isn't, to put it bluntly, of much use. We are happy to help test MA
for SciPy Core, and our project (www.netepi.org) has over a thousand
unit tests which give the basics of NumPy and MA a good work-out - so
if/when we convert to SciPy Core the unit tests will be helpful. However
we are not familiar enough with how SciPy Core differs from NumPy to
help with the porting, I'm afraid.

Tim C

[Numpy-discussion] Re: Release of SciPy Core 0.4 (Beta)

[Eric F. <ef...@ha...>]

Travis,

> This is to announce the release of SciPy Core 0.4.X (beta)

Wonderful progress--I can't thank you enough for the work you are doing 
on this.

I initially tried to build from the tarball, but ran into missing pieces:

compile options: '-Iscipy/base/include -Ibuild/src/scipy/base 
-Iscipy/base/src -I/usr/include/python2.4 -c'
gcc: lapack_lite/blas_lite.c
gcc: lapack_lite/blas_lite.c: No such file or directory
gcc: no input files

I then installed from the rpm (on a Mandriva 2005 LE system) with no 
problems, and proceeded to poke around.  It looks great!

> Now, let's start getting this stable....

Bug: in function_base.py, there are four places where "nx" should be 
"_nx"; see attached diff.

Also attached is a diff for arraymethods, to fix what look to me like 
erroneous docstrings for the reshape and resize methods.  The present 
docstrings indicate that reshape makes the change in place, and resize 
returns a new array, but in both cases the opposite is true--reshape 
returns a reshaped copy, and resize returns None.

Eric

[Numpy-discussion] Re: [SciPy-user] Release of SciPy Core 0.4 (Beta)

[Rob M. <ma...@ll...>]

OK, I am giving this a go.

I seem to be not setting a flag about some math functions availability.

Here are the details (aorry about the length of it but I snipped a lot!):

OSX 10.3.9, Python 2.4.1

Installed g77v3.4-bin.tar.gz from http://hpc.sourceforge.net/ as 
described into /usr/local/...
Installed fftw-3.0.1-fma.tar.gz from 
http://www.fftw.org/download.html into ~/Documents/local/... where I 
have other libraries I have installed.

Got scipy_core-0.4.1.tar.gz from 
http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=6222, 
wanted to try out the new replacement for Numeric

tar -zxvf scipy_core-0.4.1.tar.gz
cd scipy_core-0.4.1
python setup.py build
Assuming default configuration 
(scipy/distutils/command/{setup_command,setup}.py was not found)
Appending scipy.distutils.command configuration to scipy.distutils
Assuming default configuration 
(scipy/distutils/fcompiler/{setup_fcompiler,setup}.py was not found)
...
Appending scipy.base configuration to scipy
blas_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec', 
'-I/System/Library/Frameworks/vecLib.framework/Headers']

lapack_opt_info:
   FOUND:
     extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
     define_macros = [('NO_ATLAS_INFO', 3)]
     extra_compile_args = ['-faltivec']

Appending scipy.lib configuration to scipy
Assuming default configuration 
(scipy/fftpack/{setup_fftpack,setup}.py was not found)

...
scipy_core version 0.4.1
running build
running config_fc
running build_src
building extension "scipy.base.multiarray" sources
creating build
creating build/src
creating build/src/scipy
creating build/src/scipy/base
Generating build/src/scipy/base/config.h
customize NAGFCompiler
customize AbsoftFCompiler
customize IbmFCompiler
Could not locate executable f77
customize GnuFCompiler
customize GnuFCompiler
...
compiling C sources
gcc options: '-fno-strict-aliasing -Wno-long-double -no-cpp-precomp 
-mno-fused-madd -fno-common -dynamic
-DNDEBUG -g -O3 -Wall -Wstrict-prototypes'
creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build
creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src
creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy
creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base
creating build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base/src
compile options: '-Ibuild/src/scipy/base/src -Iscipy/base/include 
-Ibuild/src/scipy/base -Iscipy/base/src
  -I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 -c'
gcc: build/src/scipy/base/src/umathmodule.c
In file included from build/src/scipy/base/src/umathmodule.c:8:
scipy/base/include/scipy/arrayobject.h:84: warning: redefinition of `ushort'
/usr/include/sys/types.h:82: warning: `ushort' previously declared here
scipy/base/include/scipy/arrayobject.h:85: warning: redefinition of `uint'
/usr/include/sys/types.h:83: warning: `uint' previously declared here
build/src/scipy/base/src/umathmodule.c: In function `nc_floor_quotl':
build/src/scipy/base/src/umathmodule.c:1257: warning: implicit 
declaration of function `floorl'
build/src/scipy/base/src/umathmodule.c: In function `nc_sqrtl':
build/src/scipy/base/src/umathmodule.c:1269: warning: implicit 
declaration of function `sqrtl'
...
build/src/scipy/base/__umath_generated.c:171: error: initializer 
element is not constant
build/src/scipy/base/__umath_generated.c:171: error: (near 
initialization for `arccosh_data[2]')
error: Command "gcc -fno-strict-aliasing -Wno-long-double 
-no-cpp-precomp -mno-fused-madd -fno-common -dy
namic -DNDEBUG -g -O3 -Wall -Wstrict-prototypes 
-Ibuild/src/scipy/base/src -Iscipy/base/include -Ibuild/s
rc/scipy/base -Iscipy/base/src 
-I/Library/Frameworks/Python.framework/Versions/2.4/include/python2.4 
-c b
uild/src/scipy/base/src/umathmodule.c -o 
build/temp.darwin-7.9.0-Power_Macintosh-2.4/build/src/scipy/base
/src/umathmodule.o" failed with exit status 1


At 12:59 AM -0600 9/30/05, Travis Oliphant wrote:
>This is to announce the release of SciPy Core 0.4.X (beta)
>
>It is available at sourceforge which you can access by going to
>
>http://numeric.scipy.org
>
>Thanks to all who helped me with this release, especially
>
>Robert Kern
>Pearu Peterson
>
>Now, let's start getting this stable....
>
>-Travis Oliphant
>
>
>
>_______________________________________________
>SciPy-user mailing list
>Sci...@sc...
>http://www.scipy.net/mailman/listinfo/scipy-user


-- 
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-
Rob Managan               email managan at llnl.gov
LLNL                      phone: 925-423-0903
P.O. Box 808, L-095       FAX:   925-422-3389
Livermore, CA  94551-0808

Re: [Numpy-discussion] Release of SciPy Core 0.4 (Beta)

[Gerard V. <ger...@gr...>]

On Fri, 30 Sep 2005 00:59:22 -0600
Travis Oliphant <oli...@ee...> wrote:

> 
> This is to announce the release of SciPy Core 0.4.X (beta)
> 
> It is available at sourceforge which you can access by going to
> 
> http://numeric.scipy.org
> 
> Thanks to all who helped me with this release, especially
> 
> Robert Kern
> Pearu Peterson
> 
> Now, let's start getting this stable....
> 
> -Travis Oliphant
> 
I have found two problems:

(1) scipy_core-0.4.1 does not compile on this system without ATLAS,
    because there are missing files:
...
 creating build/temp.linux-i686-2.4/lapack_lite
compile options: '-Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c'
gcc: scipy/corelib/lapack_lite/lapack_litemodule.c
gcc: lapack_lite/dlapack_lite.c
gcc: lapack_lite/dlapack_lite.c: No such file or directory
gcc: no input files
gcc: lapack_lite/dlapack_lite.c: No such file or directory
gcc: no input files
error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -fomit-frame-pointer -pipe -march=i586 -mtune=pentiumpro-g -fPIC -Iscipy/base/include -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c lapack_lite/dlapack_lite.c -o build/temp.linux-i686-2.4/lapack_lite/dlapack_lite.o" failed with exit status 1
[packer@titan scipy_core-0.4.1]$  


(2) when building scipy_core from SVN the header files do not get installed.
    The install section from my RPM SPEC file reads:

%install
rm -rf %{buildroot}
python setup.py install --root=%{buildroot}

# The API headers do not get installed
mkdir -p %{buildroot}/%{_includedir}/python%{pyver}/scipy/
for header in scipy/base/include/scipy/*object.h ; do
    install -m 644 $header %{buildroot}/%{_includedir}/python%{pyver}/scipy/
done
# The generated API headers and config.h do not get installed either
for header in build/src/scipy/base/*.h ; do
    install -m 644 $header %{buildroot}/%{_includedir}/python%{pyver}/scipy/
done

# Mandrake 10.2 needs this:
%multiarch_includes %{buildroot}/%{_includedir}/python%{pyver}/scipy/config.h

%clean
#rm -rf %{buildroot}


Regards -- Gerard