Menu
▾
▴
numpy-discussion — Discussion list for all users of Numerical Python
You can subscribe to this list here.
| 2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
| 2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
| 2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
| 2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
| 2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
| 2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
|
From: Lionel R. <lro...@li...> - 2006-09-21 14:17:56
|
Hi all, Is it possible to put masked values into recarrays, I need a array with heterogenous types of datas (datetime objects in the first col, all others are float) but with missing values in some records. For the moment, I don't find any solution for that. I have tried with arrays of dtype=object, but I have problem when I want to compute min, max, ... with an error like: TypeError: function not supported for these types, and can't coerce safely to supported types. thanks -- Lionel Roubeyrie - lro...@li... LIMAIR http://www.limair.asso.fr |
|
From: Peter B. <Pet...@ug...> - 2006-09-21 11:28:40
|
Hi,
I just installed rc1 on an AMD64 machine. but I get this error message when
trying to import it:
Python 2.4.3 (#1, Sep 21 2006, 13:06:42)
[GCC 4.1.1 (Gentoo 4.1.1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib64/python2.4/site-packages/numpy/__init__.py", line 36, in ?
import core
File "/usr/lib64/python2.4/site-packages/numpy/core/__init__.py", line 7,
in ?
import numerictypes as nt
File "/usr/lib64/python2.4/site-packages/numpy/core/numerictypes.py", line
191, in ?
_add_aliases()
File "/usr/lib64/python2.4/site-packages/numpy/core/numerictypes.py", line
169, in _add_aliases
base, bit, char = bitname(typeobj)
File "/usr/lib64/python2.4/site-packages/numpy/core/numerictypes.py", line
119, in bitname
char = base[0]
IndexError: string index out of range
Thanks!
Peter
|
|
From: Martin W. <mar...@gm...> - 2006-09-21 08:13:40
|
Thanks Travis. Do I understand correctly that the only way to be really safe is to make a copy and not to export a reference to it? Because anybody having a reference to the owner of the data can override the flag? Cheers, Martin On Wednesday 20 September 2006 20:18, Travis Oliphant wrote: > Martin Wiechert wrote: > > Hi list, > > > > I just stumbled accross NPY_WRITEABLE flag. > > Now I'd like to know if there are ways either from Python or C to make an > > array temporarily immutable. > > Just setting the flag > > Python: > > make immutable: > a.flags.writeable = False > > make mutable again: > a.flags.writeable = True > > > C: > > make immutable: > a->flags &= ~NPY_WRITEABLE > > make mutable again: > a->flags |= NPY_WRITEABLE > > > In C you can play with immutability all you want. In Python you can > only make something writeable if you either 1) own the data or 2) the > object that owns the data is itself "writeable" > > > -Travis > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your opinions on IT & business topics through brief surveys -- and earn > cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion |
|
From: <gh...@to...> - 2006-09-21 07:40:20
|
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<title>无标题文档</title>
<style type="text/css">
<!--
.td {
font-size: 12px;
color: #313131;
line-height: 20px;
font-family: "Arial", "Helvetica", "sans-serif";
}
-->
</style>
</head>
<body leftmargin="0" background="http://bo.sohu.com//images/img20040502/dj_bg.gif">
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr>
<td height="31" background="http://igame.sina.com.cn/club/images/topmenu/topMenu_8.gif" class="td"><div align="center"><font color="#FFFFFF">主办单位：易腾企业管理咨询有限公司</font></div></td>
</tr>
</table>
<br>
<table width="673" border="0" align="center" cellpadding="0" cellspacing="0">
<tr>
<td height="62" bgcolor="#8C8C8C">
<div align="center">
<table width="100%" border="0" cellspacing="1" cellpadding="0">
<tr>
<td height="62" bgcolor="#F3F3F3">
<p align="center"><span style="background-color: #F3F3F3">
<font size="5"><span style="font-weight: 700"> 非</span></font></span><font size="5"><span style="font-weight: 700; background-color: #F3F3F3">财务经理的财务管理<br>
-</span></font><font size="4"><span style="background-color: #F3F3F3">沙盘模拟课程</span></font></td>
</tr>
</table>
<font color="#FF0000" size="+3" face="黑体"></font></div></td>
</tr>
</table>
<table width="673" border="0" align="center" cellpadding="0" cellspacing="0" class="td" height="1488">
<tr>
<td bgcolor="#8C8C8C" height="22"> </td>
</tr>
<tr>
<td height="1466" bgcolor="#FFFFFF"> <div align="center">
<table width="99%" border="0" cellspacing="0" cellpadding="0">
<tr>
<td width="17%" height="20" bgcolor="#BF0000" class="td">
<div align="center"><font color="#FFFFFF">[课 程 背 景]</font></div></td>
<td width="83%" class="td"> </td>
</tr>
<tr>
<td height="74" colspan="2" class="td"><font FACE="??ì?,SimSun" LANG="ZH-CN">
<p ALIGN="JUSTIFY"> </font><font lang="ZH-CN" size="2"> </font><font lang="ZH-CN" size="2">每一位管理和技术人员都清楚地懂得，单纯从技术角度衡量为合算的方案，也许却是一个财务陷阱，表面赢利而暗地里亏损，使经营者无法接受。如何将技术手段与财务运作相结合，使每位管理和技术人员都从老板的角度进行思考 , 有效地规避财务陷阱，实现管理决策与经营目标的一致性？本课程通过沙盘模拟实施体验式教学，每个小组 5-6 人模拟公司的生产、销售和财务分析过程，学员在模拟对抗中相互激发、体验决策与经营企业的乐趣，同时讲师与学员共同分享解决问题的模型与工具，使学员 " 身同体受 " ，完成从 know-what 向 know-why 转化。本课程通过二十<span style="letter-spacing: -1pt">八个实际案例分析，使企业各级管理和技术人员掌握财务管理知识，利用财务信息改进管理决策，实现管理效益最大化</span>。<br>
★ 对会计与财务管理有基本了解, 提高日常管理活动的财务可行性;<br>
★ 掌握合乎财务原则的管理决策方法 , 与老板的思维同步；<br>
★ 通过成本分析，找到有效降低成本的途径；<br>
★ 通过边际效应分析，实施正确的成本决策；<br>
★ 掌握业绩评价的依据和方法，评估经营业绩 , 实施科学的业绩考核。</font></td>
</tr>
</table>
</div>
<div align="center" style="width: 671; height: 1"> </div>
<div align="center">
<table width="99%" height="84" border="0" cellpadding="0" cellspacing="0">
<tr>
<td width="17%" height="20" bgcolor="#0080C0" class="td"> <div align="center"><font color="#FFFFFF">[课
程 大 纲]</font></div></td>
<td width="83%" class="td"> </td>
</tr>
<tr>
<td height="64" colspan="2" class="td"> <p><font lang="ZH-CN" size="2"><font color="#FF0000">一、企业财务管理的内容及管理会计的作用</font><br>
1、财务会计的职能<br>
2、财务专家的思维模式<br>
3、财务工作的基本内容<br>
4、财务管理的四个基本原理<br>
5、会计工作的十三个基本原则<br>
<font color="#FF0000">二、如何阅读和分析财务报表</font><br>
1、会计报表的构成<br>
2、资产负债表：<br>
资产负债表的逻辑结构与主要内容（透视资产负债表的各个部分）<br>
解读资产负债表（应收款、存货、应付款、固定资产、借款、股东权益）<br>
企业拥有何物，谁拥有企业<br>
财务杠杆的运用 ---- 资本结构分析<br>
3、损益表：<br>
损益表的逻辑结构与主要内容<br>
如何正确确认收入避免 ” 苹果电脑事件 ”<br>
如何计算制造成本 , 反映每种产品的实际贡献<br>
如何计算共同成本<br>
如何计算息税前利润和净利润<br>
4、现金流量表的阅读与分析<br>
现金流量表的逻辑结构与主要内容<br>
5、会计报表之间的关系（ Dupont 模型的应用）<br>
6、如何从会计报表读懂企业经营状况<br>
• 案例分析：读报表，判断企业业绩水平<br>
<font color="#FF0000">三、如何利用财务数据分析并改善经营绩效</font><br>
1、财务比率分析<br>
2、关键财务指标解析<br>
总资产收益率 / 净资产收益率<br>
营业利润率 / 总资产周转率<br>
总资产周转率与资产管理<br>
3、盈利能力分析：资产回报率、股东权益回报率、资产流动速率<br>
4、风险指数分析：流动比率、负债 / 权益比率、营运偿债能力<br>
5、财务报表综合解读：综合运用财务信息透视公司运作水平<br>
◆ 案例分析：某上市公司的财务状况分析与经营绩效评价<br>
<font color="#FF0000">四、成本分析与成本决策</font><br>
1、产品成本的概念和构成<br>
成本习性分析<br>
完全成本法<br>
变动成本法<br>
2、ＣＶＰ（本－量－利）分析与运用<br>
3、固定成本与边际效应分析<br>
4、如何运用目标成本法控制产品成本，保证利润水平<br>
5、如何运用 ABC 作业成本法进行管理分析，实施精细成本管理<br>
6、如何针对沉没成本和机会成本进行正确决策<br>
7、如何改善采购、生产等环节的运作以改良企业的整体财务状况<br>
◆ 综合案例分析<br>
<font color="#FF0000">五、投资决策</font><br>
1、管理和技术方案的可行性分析<br>
2、设备改造与更新的决策分析<br>
3、如何根据成本与市场进行合理定价<br>
4、长期投资项目的现金流分析<br>
5、资金的时间价值<br>
6、投资项目评价方法<br>
回收期法<br>
净现值法<br>
内部收益率法<br>
◆ 综合案例演练<br>
<font color="#FF0000">六、绩效考评与内部控制</font><br>
1、如何确定责任成本？<br>
2、如何实施内部控制<br>
3、如何针对成本中心进行费用控制<br>
4、如何针对利润中心进行业绩考核<br>
5、如何针对投资中心进行业绩评价<br>
6、如何运用内部市场链进行管理，使每个部门都成为利润中心<br>
• 案例研讨： GE 和海尔的绩效考评</font>
</p></td>
</tr>
</table>
<table width="99%" height="84" border="0" cellpadding="0" cellspacing="0">
<tr>
<td width="17%" height="20" bgcolor="#0080C0" class="td">
<div align="center"><font color="#FFFFFF">[导 师 简 介]</font></div></td>
<td width="83%" class="td"> </td>
</tr>
<tr>
<td height="64" colspan="2" class="td"> <p> <font color="#FF0000"> </font> Mr Wang ，管理工程硕士、高级经济师，国际职业培训师协会认证职业培训师，历任跨国公司生产负责人、工业工程经理、管理会计分析师、营运总监等高级管理职务多年，同时还担任 < 价值工程 > 杂志审稿人、辽宁省营口市商业银行独立董事等职务，对企业管理有较深入的研究。 王老师主要从事 IE 技术应用、成本控制、管理会计决策等课程的讲授，先后为 IBM 、 TDK 、松下、可口可乐、康师傅、汇源果汁、雪津啤酒、吉百利食品、冠捷电子、 INTEX 明达塑胶、正新橡胶、美国 ITT 集团、广上科技、美的空调、中兴通讯、京信通信、联想电脑，应用材料 ( 中国 ) 公司、艾克森 - 金山石化、中国化工进出口公司、正大集团大福饲料、厦华集团、灿坤股份、 NEC 东金电子、太原钢铁集团、 PHILIPS 、深圳开发科技、大冷王运输制冷、三洋华强、 TCL 、美的汽车、上海贝尔阿尔卡特、天津扎努西、上海卡博特等知名企业提供项目辅导或专题培训。王老师授课经验丰富，风格幽默诙谐、逻辑清晰、过程互动，案例生动、深受学员喜爱。
</p></td>
</tr>
</table>
</div>
<div align="center">
<table width="99%" border="0" cellpadding="0" cellspacing="0" height="80">
<tr>
<td width="17%" height="25" bgcolor="#0080C0" class="td"> <div align="center"><font color="#FFFFFF">[时间/地点/报名]</font></div></td>
<td width="83%" class="td" height="25"> </td>
</tr>
<tr>
<td height="55" colspan="2" class="td">
<p style="line-height: 200%; margin-top: 2; margin-bottom: 2"><font size="2">
<b>
公开课报名/咨询: </b> 谢小姐
021-5118 7126 <br>
</font><font size="2"><b> 时间: </b>
9月28-29日
(周四、五) <b>地点: </b> 上海
<b>费用: </b>
1980元/人(含教材,午餐等) 四人以上参加,赠予一名名额</font></p>
</td>
</tr>
</table>
</div>
<table width="99%" height="35" border="0" align="center" cellpadding="0" cellspacing="0">
<tr>
<td height="15" class="td"> <p><font color="#000000" size="2"> <b>厂内培训和咨询项目:</b></font><font color="#000000" size="1"><br>
</font><font size="2"> <span style="letter-spacing: -1pt"> 易腾公司致力于生产、质量、成本节约等各方面的课程培训与项目咨询,
欢迎您根据需要提出具体要求，我们将为您</span><span style="letter-spacing: -3pt">制
厂内</span><span style="letter-spacing: -1pt">培训或咨询项目。<br>
内训联系： 021-5118
7132 刘小姐</span> </font>
</p>
</td>
</tr>
</table>
<table width="99%" height="35" border="0" align="center" cellpadding="0" cellspacing="0">
<tr>
<td height="15" class="td"> <font size="2"><b> 注:尊敬的客户：</b><br>
您好！以下内容为商-务-广-告;如果对您的工作及生活带来不便，在此深表歉意;如您不需要,请与技术部联系</font><font size="2" color="#000000"><br>
联系电话: 021-5118 7131 张小姐 </font></td>
</tr>
</table> </td>
</tr>
</table>
<p align="center"> </p>
</body>
</html>
|
|
From: Bill B. <wb...@gm...> - 2006-09-21 04:00:37
|
Hey Andrew, point taken, but I think it would be better if someone who actually knows the full extent of the change made the edit. I know zeros and ones changed. Did anything else? Anyway, I'm surprised the release notes page is publicly editable. --bb On 9/21/06, Andrew Straw <str...@as...> wrote: > It is a wiki, and contributions are absolutely welcome, so please go > ahead and change it to be more clear. > > Bill Baxter wrote: > > >I think that's supposed to be covered by this line: > >"The default array data-type is now float64 (c double precision) > >instead of c integer." > > > >But yeh, I agree. It's definitely not obvious what that means in > >terms of concrete API changes. > > > >--bb > > > >On 9/21/06, David Grant <dav...@gm...> wrote: > > > > > >>I noticed that the default dtype for the "ones" and "zeros" functions is now > >>float, whereas it used to be int. Should this be noted at > >>http://www.scipy.org/ReleaseNotes/NumPy_1.0 since it does > >>break code (if you are using the array you creating with ones or zeros as > >>indices into another array). > >> > >>-- > >>David Grant > >>http://www.davidgrant.ca > >> > >> |
|
From: Keith G. <kwg...@gm...> - 2006-09-21 03:47:03
|
NaN adds a lot of whitespace in the representation of a matrix.
Without NaN:
matrix([[1, 2],
[3, 4]])
With NaN:
matrix([[ nan, 2. ],
[ 3. , 4. ]])
There's enough room for the wikipedia entry for nan.
|
|
From: Andrew S. <str...@as...> - 2006-09-21 00:39:43
|
It is a wiki, and contributions are absolutely welcome, so please go ahead and change it to be more clear. Bill Baxter wrote: >I think that's supposed to be covered by this line: >"The default array data-type is now float64 (c double precision) >instead of c integer." > >But yeh, I agree. It's definitely not obvious what that means in >terms of concrete API changes. > >--bb > >On 9/21/06, David Grant <dav...@gm...> wrote: > > >>I noticed that the default dtype for the "ones" and "zeros" functions is now >>float, whereas it used to be int. Should this be noted at >>http://www.scipy.org/ReleaseNotes/NumPy_1.0 since it does >>break code (if you are using the array you creating with ones or zeros as >>indices into another array). >> >>-- >>David Grant >>http://www.davidgrant.ca >>------------------------------------------------------------------------- >>Take Surveys. Earn Cash. Influence the Future of IT >>Join SourceForge.net's Techsay panel and you'll get the chance to share your >>opinions on IT & business topics through brief surveys -- and earn cash >>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> >>_______________________________________________ >>Numpy-discussion mailing list >>Num...@li... >>https://lists.sourceforge.net/lists/listinfo/numpy-discussion >> >> >> >> >> > >------------------------------------------------------------------------- >Take Surveys. Earn Cash. Influence the Future of IT >Join SourceForge.net's Techsay panel and you'll get the chance to share your >opinions on IT & business topics through brief surveys -- and earn cash >http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >_______________________________________________ >Numpy-discussion mailing list >Num...@li... >https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > |
|
From: Bill B. <wb...@gm...> - 2006-09-21 00:04:05
|
I think that's supposed to be covered by this line: "The default array data-type is now float64 (c double precision) instead of c integer." But yeh, I agree. It's definitely not obvious what that means in terms of concrete API changes. --bb On 9/21/06, David Grant <dav...@gm...> wrote: > I noticed that the default dtype for the "ones" and "zeros" functions is now > float, whereas it used to be int. Should this be noted at > http://www.scipy.org/ReleaseNotes/NumPy_1.0 since it does > break code (if you are using the array you creating with ones or zeros as > indices into another array). > > -- > David Grant > http://www.davidgrant.ca > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > > |
|
From: David G. <dav...@gm...> - 2006-09-20 23:42:01
|
I noticed that the default dtype for the "ones" and "zeros" functions is now float, whereas it used to be int. Should this be noted at http://www.scipy.org/ReleaseNotes/NumPy_1.0 since it does break code (if you are using the array you creating with ones or zeros as indices into another array). -- David Grant http://www.davidgrant.ca |
|
From: Christopher B. <Chr...@no...> - 2006-09-20 22:18:34
|
Charles R Harris wrote:
> Thinking a bit, keeping the values in place isn't easy.
Why the heck would "in place" be desirable for sorted data anyway? I
understand that it means that if there is a NaN in the nth position
before sorting, there will be one in the nth position after sorting.
However, I see absolutely no reason at all why that would be useful (or
any more useful than putting them anywhere else)
A couple years ago, there was a long debate on this list about whether
numpy should pass -inf, NaN, and +inf through all the ufuncs without
error. there were two schools of thought:
1) They indicate a problem, the programmer should know about hat problem
as soon as it occurs, not at the end of the computation, many steps
later, when they might get presented with nothing but NaNs.
2) The whole point of "vector" computation is that you can act on a
whole bunch of numbers at once. If only subset of those numbers are
invalid, why stop the process. Raising an error when a single number has
a problem defeats the purpose of vector operations.
It seems that numpy has settled on school of thought (2), at least by
default. That being the case, it should apply to sorting also. If it
does, then that means no exception will be raised, but it makes no
difference where the heck the NaNs end up in the sorted array, as long
as everything else is in order. NaN means exactly what it's called: it's
not a number, so it doesn't matter what you do with them, as long as
they are preserved and don't mess up other things. Let the coder decide
what they want to so with them, and when they want to do it. Personally,
I'd prefer that they all ended up at the beginning or end after sorting,
but it really doesn't much matter.
That being said, if it's impossible to do a efficient sort with NaNs
mixed in, then we'll just have to live with it. It really would be best
if an exception was raised if the non-NaN values are not going to be
sorted correctly -- that really would surprise people!
> It would probably also not be unreasonable to punt and document sort
> as failing in the presence of nans.
That would be one of the worst options, but may be the only one available.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Christopher B. <Chr...@no...> - 2006-09-20 22:18:11
|
Travis Oliphant wrote:
>> In [77]: arange(5, dtype=int)/0
>> Out[77]: array([0, 0, 0, 0, 0])
> It is deliberate. Numarray introduced it (the only difference being
> that by default NumPy has division-by-zero erros turned off). It's tied
> to the way floating-point division-by zero is handled. There is a
> valid argument for having a separate integer-division flag so that you
> can raise exceptions for integer-division but not for floating-point
> division. I'm open to that change for 1.0rc1
+1 (+inf)
There is a BIG difference between getting an inf with a floating point
computation and getting a 0 with an integer one!
Also, the default integer-division flag should be set to raise an exception.
It sure would be nice to have special values for integers too....
Travis Oliphant wrote:
> Simulating and "integer-division-by-zero"
> hardware flag is not trivial as we would have to manage context
> switching ourselves. So, at least for 1.0, integer and floating-point
> division by zero are going to be handled the same.
Darn. Oh well.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Keith G. <kwg...@gm...> - 2006-09-20 21:46:56
|
I like the salutation of the letter below: "Please read this letter attentively!" Why does so much spam make it through the sourceforge filters? Or maybe they only let through the very good deals. "You will earn from 1500 up to 3000 USD per week, working only some hours per day." On Wed, 20 Sep 2006 21:34:20 -0060, Katherine Goldberg <ale...@1-...> wrote: > Please read this letter attentively! > > Financial company is in search of representatives in the US. > > - Age: from 18 till 60 years > - Have a skill to communicate and access to the Internet. > - You can speak english language above average > - You need to know how to use excel and word. > > You will earn from 1500 up to 3000 USD per week, working only some hours per day. You can work part time or full time. > > If you are interested in our offer send the following data to our e-mail us...@eu... > > - Your full name > - Your contact e-mail > - Your phone number > > If you have any questions. Feel free to ask them > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
|
From: Katherine G. <ale...@1-...> - 2006-09-20 21:34:20
|
Please read this letter attentively! Financial company is in search of representatives in the US. - Age: from 18 till 60 years - Have a skill to communicate and access to the Internet. - You can speak english language above average - You need to know how to use excel and word. You will earn from 1500 up to 3000 USD per week, working only some hours per day. You can work part time or full time. If you are interested in our offer send the following data to our e-mail us...@eu... - Your full name - Your contact e-mail - Your phone number If you have any questions. Feel free to ask them |
|
From: Mathew Y. <my...@jp...> - 2006-09-20 20:42:39
|
I don't see a windows binary. Will this be added? Mathew Travis Oliphant wrote: > There is now a 1.0rc1 tag on the NumPy SVN tree. I've confirmed it > builds and passes all tests on my Linux box for Python2.3-Python2.5 > > -Travis > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > |
|
From: Tim H. <tim...@ie...> - 2006-09-20 19:41:43
|
Robert Kern wrote: > David M. Cooke wrote: > >> On Wed, Sep 20, 2006 at 03:01:18AM -0500, Robert Kern wrote: >> >>> Let me offer a third path: the algorithms used for .mean() and .var() are >>> substandard. There are much better incremental algorithms that entirely avoid >>> the need to accumulate such large (and therefore precision-losing) intermediate >>> values. The algorithms look like the following for 1D arrays in Python: >>> >>> def mean(a): >>> m = a[0] >>> for i in range(1, len(a)): >>> m += (a[i] - m) / (i + 1) >>> return m >>> >> This isn't really going to be any better than using a simple sum. >> It'll also be slower (a division per iteration). >> > > With one exception, every test that I've thrown at it shows that it's better for > float32. That exception is uniformly spaced arrays, like linspace(). > > > You do avoid > > accumulating large sums, but then doing the division a[i]/len(a) and > > adding that will do the same. > > Okay, this is true. > > >> Now, if you want to avoid losing precision, you want to use a better >> summation technique, like compensated (or Kahan) summation: >> >> def mean(a): >> s = e = a.dtype.type(0) >> for i in range(0, len(a)): >> temp = s >> y = a[i] + e >> s = temp + y >> e = (temp - s) + y >> return s / len(a) >> >> Some numerical experiments in Maple using 5-digit precision show that >> your mean is maybe a bit better in some cases, but can also be much >> worse, than sum(a)/len(a), but both are quite poor in comparision to the >> Kahan summation. >> >> (We could probably use a fast implementation of Kahan summation in >> addition to a.sum()) >> > > +1 > > >>> def var(a): >>> m = a[0] >>> t = a.dtype.type(0) >>> for i in range(1, len(a)): >>> q = a[i] - m >>> r = q / (i+1) >>> m += r >>> t += i * q * r >>> t /= len(a) >>> return t >>> >>> Alternatively, from Knuth: >>> >>> def var_knuth(a): >>> m = a.dtype.type(0) >>> variance = a.dtype.type(0) >>> for i in range(len(a)): >>> delta = a[i] - m >>> m += delta / (i+1) >>> variance += delta * (a[i] - m) >>> variance /= len(a) >>> return variance >>> >> These formulas are good when you can only do one pass over the data >> (like in a calculator where you don't store all the data points), but >> are slightly worse than doing two passes. Kahan summation would probably >> also be good here too. >> On the flip side, it doesn't take a very large array (somewhere in the vicinity of 10,000 values in my experience) before memory bandwidth becomes a limiting factor. In that region a two pass algorithm could well be twice as slow as a single pass algorithm even if the former is somewhat simpler in terms of numeric operations. -tim > > Again, my tests show otherwise for float32. I'll condense my ipython log into a > module for everyone's perusal. It's possible that the Kahan summation of the > squared residuals will work better than the current two-pass algorithm and the > implementations I give above. > |
|
From: Travis O. <oli...@ie...> - 2006-09-20 19:11:55
|
There is now a 1.0rc1 tag on the NumPy SVN tree. I've confirmed it builds and passes all tests on my Linux box for Python2.3-Python2.5 -Travis |
|
From: Tim H. <tim...@ie...> - 2006-09-20 19:03:03
|
[Sorry, this version should have less munged formatting since I clipped
the comments. Oh, and the Kahan sum algorithm was grabbed from
wikipedia, not mathworld]
Tim Hochberg wrote:
> Robert Kern wrote:
>
>> David M. Cooke wrote:
>>
>>
>>> On Wed, Sep 20, 2006 at 03:01:18AM -0500, Robert Kern wrote:
>>>
>>>
>>>> Let me offer a third path: the algorithms used for .mean() and .var() are
>>>> substandard. There are much better incremental algorithms that entirely avoid
>>>> the need to accumulate such large (and therefore precision-losing) intermediate
>>>> values. The algorithms look like the following for 1D arrays in Python:
>>>>
>>>> def mean(a):
>>>> m = a[0]
>>>> for i in range(1, len(a)):
>>>> m += (a[i] - m) / (i + 1)
>>>> return m
>>>>
>>>>
>>> This isn't really going to be any better than using a simple sum.
>>> It'll also be slower (a division per iteration).
>>>
>>>
>> With one exception, every test that I've thrown at it shows that it's better for
>> float32. That exception is uniformly spaced arrays, like linspace().
>>
>>
> Here's version of mean based on the Kahan sum, including the
> compensation term that seems to be consistently much better than builtin
> mean It seems to be typically 4 orders or magnitude closer to the
> "correct" answer than the builtin mean and 2 orders of magnitude better
> than just naively using the Kahan sum. I'm using the sum performed with
> dtype=int64 as the "correct" value.
>
>
>
def rawKahanSum(values):
if not input:
return 0.0
total = values[0]
c = type(total)(0.0)
for x in values[1:]:
y = x - c
t = total + y
c = (t - total) - y
total = t
return total, c
def kahanSum(values):
total, c = rawKahanSum(values)
return total
def mean(values):
total, c = rawKahanSum(values)
n = float(len(values))
return total / n - c / n
for i in range(100):
data = random.random([10000]).astype(float32)
baseline = data.mean(dtype=float64)
delta_builtin_mean = baseline - data.mean()
delta_compensated_mean = baseline - mean(data)
delta_kahan_mean = baseline - (kahanSum(data) / len(data))
if not abs(delta_builtin_mean) >= abs(delta_kahan_mean) >=
abs(delta_compensated_mean):
print ">>>",
print delta_builtin_mean, delta_kahan_mean, delta_compensated_mean
> -tim
>
>> > You do avoid
>> > accumulating large sums, but then doing the division a[i]/len(a) and
>> > adding that will do the same.
>>
>> Okay, this is true.
>>
>>
>>
>>> Now, if you want to avoid losing precision, you want to use a better
>>> summation technique, like compensated (or Kahan) summation:
>>>
>>> def mean(a):
>>> s = e = a.dtype.type(0)
>>> for i in range(0, len(a)):
>>> temp = s
>>> y = a[i] + e
>>> s = temp + y
>>> e = (temp - s) + y
>>> return s / len(a)
>>>
>>> Some numerical experiments in Maple using 5-digit precision show that
>>> your mean is maybe a bit better in some cases, but can also be much
>>> worse, than sum(a)/len(a), but both are quite poor in comparision to the
>>> Kahan summation.
>>>
>>> (We could probably use a fast implementation of Kahan summation in
>>> addition to a.sum())
>>>
>>>
>> +1
>>
>>
>>
>>>> def var(a):
>>>> m = a[0]
>>>> t = a.dtype.type(0)
>>>> for i in range(1, len(a)):
>>>> q = a[i] - m
>>>> r = q / (i+1)
>>>> m += r
>>>> t += i * q * r
>>>> t /= len(a)
>>>> return t
>>>>
>>>> Alternatively, from Knuth:
>>>>
>>>> def var_knuth(a):
>>>> m = a.dtype.type(0)
>>>> variance = a.dtype.type(0)
>>>> for i in range(len(a)):
>>>> delta = a[i] - m
>>>> m += delta / (i+1)
>>>> variance += delta * (a[i] - m)
>>>> variance /= len(a)
>>>> return variance
>>>>
>>>>
>>> These formulas are good when you can only do one pass over the data
>>> (like in a calculator where you don't store all the data points), but
>>> are slightly worse than doing two passes. Kahan summation would probably
>>> also be good here too.
>>>
>>>
>> Again, my tests show otherwise for float32. I'll condense my ipython log into a
>> module for everyone's perusal. It's possible that the Kahan summation of the
>> squared residuals will work better than the current two-pass algorithm and the
>> implementations I give above.
>>
>>
>>
>
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>
|
|
From: Tim H. <tim...@ie...> - 2006-09-20 18:54:04
|
Robert Kern wrote:
> David M. Cooke wrote:
>
>> On Wed, Sep 20, 2006 at 03:01:18AM -0500, Robert Kern wrote:
>>
>>> Let me offer a third path: the algorithms used for .mean() and .var() are
>>> substandard. There are much better incremental algorithms that entirely avoid
>>> the need to accumulate such large (and therefore precision-losing) intermediate
>>> values. The algorithms look like the following for 1D arrays in Python:
>>>
>>> def mean(a):
>>> m = a[0]
>>> for i in range(1, len(a)):
>>> m += (a[i] - m) / (i + 1)
>>> return m
>>>
>> This isn't really going to be any better than using a simple sum.
>> It'll also be slower (a division per iteration).
>>
>
> With one exception, every test that I've thrown at it shows that it's better for
> float32. That exception is uniformly spaced arrays, like linspace().
>
Here's version of mean based on the Kahan sum, including the
compensation term that seems to be consistently much better than builtin
mean It seems to be typically 4 orders or magnitude closer to the
"correct" answer than the builtin mean and 2 orders of magnitude better
than just naively using the Kahan sum. I'm using the sum performed with
dtype=int64 as the "correct" value.
def rawKahanSum(values): # Stolen from mathworld
if not input:
return 0.0
total = values[0]
c = type(total)(0.0) # A running compensation for lost
low-order bits.
for x in values[1:]:
y = x - c # So far, so good: c is zero.
t = total + y # Alas, total is big, y small, so
low-order digits of y are lost.
c = (t - total) - y # (t - total) recovers the high-order
part of y; subtracting y recovers -(low part of y)
total = t # Algebriacally, c should always be
zero. Beware eagerly optimising compilers!
# Next time around, the lost low part
will be added to y in a fresh attempt.
return total, c
def kahanSum(values):
total, c = rawKahanSum(values)
return total
def mean(values):
total, c = rawKahanSum(values)
n = float(len(values))
return total / n - c / n
for i in range(100):
data = random.random([10000]).astype(float32)
baseline = data.mean(dtype=float64)
delta_builtin_mean = baseline - data.mean()
delta_compensated_mean = baseline - mean(data)
delta_kahan_mean = baseline - (kahanSum(data) / len(data))
if not abs(delta_builtin_mean) >= abs(delta_kahan_mean) >=
abs(delta_compensated_mean):
print ">>>",
print delta_builtin_mean, delta_kahan_mean, delta_compensated_mean
-tim
> > You do avoid
> > accumulating large sums, but then doing the division a[i]/len(a) and
> > adding that will do the same.
>
> Okay, this is true.
>
>
>> Now, if you want to avoid losing precision, you want to use a better
>> summation technique, like compensated (or Kahan) summation:
>>
>> def mean(a):
>> s = e = a.dtype.type(0)
>> for i in range(0, len(a)):
>> temp = s
>> y = a[i] + e
>> s = temp + y
>> e = (temp - s) + y
>> return s / len(a)
>>
>> Some numerical experiments in Maple using 5-digit precision show that
>> your mean is maybe a bit better in some cases, but can also be much
>> worse, than sum(a)/len(a), but both are quite poor in comparision to the
>> Kahan summation.
>>
>> (We could probably use a fast implementation of Kahan summation in
>> addition to a.sum())
>>
>
> +1
>
>
>>> def var(a):
>>> m = a[0]
>>> t = a.dtype.type(0)
>>> for i in range(1, len(a)):
>>> q = a[i] - m
>>> r = q / (i+1)
>>> m += r
>>> t += i * q * r
>>> t /= len(a)
>>> return t
>>>
>>> Alternatively, from Knuth:
>>>
>>> def var_knuth(a):
>>> m = a.dtype.type(0)
>>> variance = a.dtype.type(0)
>>> for i in range(len(a)):
>>> delta = a[i] - m
>>> m += delta / (i+1)
>>> variance += delta * (a[i] - m)
>>> variance /= len(a)
>>> return variance
>>>
>> These formulas are good when you can only do one pass over the data
>> (like in a calculator where you don't store all the data points), but
>> are slightly worse than doing two passes. Kahan summation would probably
>> also be good here too.
>>
>
> Again, my tests show otherwise for float32. I'll condense my ipython log into a
> module for everyone's perusal. It's possible that the Kahan summation of the
> squared residuals will work better than the current two-pass algorithm and the
> implementations I give above.
>
>
|
|
From: Travis O. <oli...@ie...> - 2006-09-20 18:18:01
|
Martin Wiechert wrote: > Hi list, > > I just stumbled accross NPY_WRITEABLE flag. > Now I'd like to know if there are ways either from Python or C to make an > array temporarily immutable. > Just setting the flag Python: make immutable: a.flags.writeable = False make mutable again: a.flags.writeable = True C: make immutable: a->flags &= ~NPY_WRITEABLE make mutable again: a->flags |= NPY_WRITEABLE In C you can play with immutability all you want. In Python you can only make something writeable if you either 1) own the data or 2) the object that owns the data is itself "writeable" -Travis |
|
From: Robert K. <rob...@gm...> - 2006-09-20 17:23:49
|
David M. Cooke wrote: > On Wed, Sep 20, 2006 at 03:01:18AM -0500, Robert Kern wrote: >> Let me offer a third path: the algorithms used for .mean() and .var() are >> substandard. There are much better incremental algorithms that entirely avoid >> the need to accumulate such large (and therefore precision-losing) intermediate >> values. The algorithms look like the following for 1D arrays in Python: >> >> def mean(a): >> m = a[0] >> for i in range(1, len(a)): >> m += (a[i] - m) / (i + 1) >> return m > > This isn't really going to be any better than using a simple sum. > It'll also be slower (a division per iteration). With one exception, every test that I've thrown at it shows that it's better for float32. That exception is uniformly spaced arrays, like linspace(). > You do avoid > accumulating large sums, but then doing the division a[i]/len(a) and > adding that will do the same. Okay, this is true. > Now, if you want to avoid losing precision, you want to use a better > summation technique, like compensated (or Kahan) summation: > > def mean(a): > s = e = a.dtype.type(0) > for i in range(0, len(a)): > temp = s > y = a[i] + e > s = temp + y > e = (temp - s) + y > return s / len(a) > > Some numerical experiments in Maple using 5-digit precision show that > your mean is maybe a bit better in some cases, but can also be much > worse, than sum(a)/len(a), but both are quite poor in comparision to the > Kahan summation. > > (We could probably use a fast implementation of Kahan summation in > addition to a.sum()) +1 >> def var(a): >> m = a[0] >> t = a.dtype.type(0) >> for i in range(1, len(a)): >> q = a[i] - m >> r = q / (i+1) >> m += r >> t += i * q * r >> t /= len(a) >> return t >> >> Alternatively, from Knuth: >> >> def var_knuth(a): >> m = a.dtype.type(0) >> variance = a.dtype.type(0) >> for i in range(len(a)): >> delta = a[i] - m >> m += delta / (i+1) >> variance += delta * (a[i] - m) >> variance /= len(a) >> return variance > > These formulas are good when you can only do one pass over the data > (like in a calculator where you don't store all the data points), but > are slightly worse than doing two passes. Kahan summation would probably > also be good here too. Again, my tests show otherwise for float32. I'll condense my ipython log into a module for everyone's perusal. It's possible that the Kahan summation of the squared residuals will work better than the current two-pass algorithm and the implementations I give above. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco |
|
From: Charles R H. <cha...@gm...> - 2006-09-20 16:59:10
|
On 9/20/06, David M. Cooke <co...@ph...> wrote: > > On Wed, Sep 20, 2006 at 03:01:18AM -0500, Robert Kern wrote: > > Let me offer a third path: the algorithms used for .mean() and .var() > are > > substandard. There are much better incremental algorithms that entirely > avoid > > the need to accumulate such large (and therefore precision-losing) > intermediate > > values. The algorithms look like the following for 1D arrays in Python: > > > > def mean(a): > > m = a[0] > > for i in range(1, len(a)): > > m += (a[i] - m) / (i + 1) > > return m > > This isn't really going to be any better than using a simple sum. > It'll also be slower (a division per iteration). You do avoid > accumulating large sums, but then doing the division a[i]/len(a) and > adding that will do the same. > > Now, if you want to avoid losing precision, you want to use a better > summation technique, like compensated (or Kahan) summation: > > def mean(a): > s = e = a.dtype.type(0) > for i in range(0, len(a)): > temp = s > y = a[i] + e > s = temp + y > e = (temp - s) + y > return s / len(a) A variant of this can also be used to generate the sin/cos for fourier transforms using repeated complex multiplication. It works amazingly well. But I suspect accumulating in higher precision would be just as good and faster if the hardware supports it. Divide and conquer, like an fft where only the constant coefficient is computed, is also a good bet but requires lg(n) passes over the data and extra storage. Some numerical experiments in Maple using 5-digit precision show that > your mean is maybe a bit better in some cases, but can also be much > worse, than sum(a)/len(a), but both are quite poor in comparision to the > Kahan summation. > > (We could probably use a fast implementation of Kahan summation in > addition to a.sum()) > > > def var(a): > > m = a[0] > > t = a.dtype.type(0) > > for i in range(1, len(a)): > > q = a[i] - m > > r = q / (i+1) > > m += r > > t += i * q * r > > t /= len(a) > > return t > > > > Alternatively, from Knuth: > > > > def var_knuth(a): > > m = a.dtype.type(0) > > variance = a.dtype.type(0) > > for i in range(len(a)): > > delta = a[i] - m > > m += delta / (i+1) > > variance += delta * (a[i] - m) > > variance /= len(a) > > return variance > > These formulas are good when you can only do one pass over the data > (like in a calculator where you don't store all the data points), but > are slightly worse than doing two passes. Kahan summation would probably > also be good here too. I think we should leave things as they are. Choosing the right precision for accumulating sums is absolutely fundamental, I always think about it when programming such loops because it is always a potential gotcha. Making the defaults higher precision just kicks the can down the road where the unwary are still likely to trip over it at some point. Perhaps these special tricks for special cases could be included in scipy somewhere. Chuck |
|
From: Christopher B. <Chr...@no...> - 2006-09-20 16:57:08
|
Sebastian Haase wrote:
> The best I can hope for is a "sound" default for most (practical) cases...
> I still think that 80bit vs. 128bit vs 96bit is rather academic for most
> people ... most people seem to only use float64 and then there are some
> that use float32 (like us) ...
I fully agree with Sebastian here. As Travis pointed out, "all we are
talking about is the default". The default should follow the principle
of least surprise for the less-knowledgeable users. Personally, I try to
always use doubles, unless I have a real reason not to. The recent
change of default types for zeros et al. will help.
clearly, there is a problem to say the default accumulator for *all*
types is double, as you wouldn't want to downcast float128s, even if
they are obscure. However, is it that hard to say that the default
accumulator will have *at least* the precision of double?
Robert Kern wrote:
> Let me offer a third path: the algorithms used for .mean() and .var() are
> substandard. There are much better incremental algorithms that entirely avoid
> the need to accumulate such large (and therefore precision-losing) intermediate
> values.
This, of course, is an even better solution, unless there are
substantial performance impacts.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: David M. C. <co...@ph...> - 2006-09-20 16:35:14
|
On Wed, Sep 20, 2006 at 03:01:18AM -0500, Robert Kern wrote:
> Let me offer a third path: the algorithms used for .mean() and .var() are
> substandard. There are much better incremental algorithms that entirely avoid
> the need to accumulate such large (and therefore precision-losing) intermediate
> values. The algorithms look like the following for 1D arrays in Python:
>
> def mean(a):
> m = a[0]
> for i in range(1, len(a)):
> m += (a[i] - m) / (i + 1)
> return m
This isn't really going to be any better than using a simple sum.
It'll also be slower (a division per iteration). You do avoid
accumulating large sums, but then doing the division a[i]/len(a) and
adding that will do the same.
Now, if you want to avoid losing precision, you want to use a better
summation technique, like compensated (or Kahan) summation:
def mean(a):
s = e = a.dtype.type(0)
for i in range(0, len(a)):
temp = s
y = a[i] + e
s = temp + y
e = (temp - s) + y
return s / len(a)
Some numerical experiments in Maple using 5-digit precision show that
your mean is maybe a bit better in some cases, but can also be much
worse, than sum(a)/len(a), but both are quite poor in comparision to the
Kahan summation.
(We could probably use a fast implementation of Kahan summation in
addition to a.sum())
> def var(a):
> m = a[0]
> t = a.dtype.type(0)
> for i in range(1, len(a)):
> q = a[i] - m
> r = q / (i+1)
> m += r
> t += i * q * r
> t /= len(a)
> return t
>
> Alternatively, from Knuth:
>
> def var_knuth(a):
> m = a.dtype.type(0)
> variance = a.dtype.type(0)
> for i in range(len(a)):
> delta = a[i] - m
> m += delta / (i+1)
> variance += delta * (a[i] - m)
> variance /= len(a)
> return variance
These formulas are good when you can only do one pass over the data
(like in a calculator where you don't store all the data points), but
are slightly worse than doing two passes. Kahan summation would probably
also be good here too.
--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...
|
|
From: eric <er...@en...> - 2006-09-20 16:29:35
|
Hey group, We are growing again and have multiple positions open here at Enthought. I'd love to recruit more people out of the Scipy/numpy community. I think many of you would find the work interesting. You can look at our career page for more info: http://www.enthought.com/careers.htm <http://www.enthought.com/careers.html> thanks! eric |
|
From: Tim H. <tim...@ie...> - 2006-09-20 16:08:38
|
Robert Kern wrote: > Sebastian Haase wrote: > >> Robert Kern wrote: >> >>> Sebastian Haase wrote: >>> >>>> I know that having too much knowledge of the details often makes one >>>> forget what the "newcomers" will do and expect. >>>> >>> Please be more careful with such accusations. Repeated frequently, they can >>> become quite insulting. >>> >>> >> I did not mean to insult anyone - what I meant was, that I'm for numpy >> becoming an easy platform to use. I have spend and enjoyed part the last >> four years developing and evangelizing Python as an alternative to >> Matlab and C/Fortran based image analysis environment. I often find >> myself arguing for good support of the single precision data format. So >> I find it actually somewhat ironic to see myself arguing now for wanting >> float64 over float32 ;-) >> > > No one is doubting that you want numpy to be easy to use. Please don't doubt > that the rest of us want otherwise. However, the fact that you *want* numpy to > be easy to use does not mean that your suggestions *will* make numpy easy to use. > > We haven't forgotten what newcomers will do; to the contrary, we are quite aware > that new users need consistent behavior in order to learn how to use a system. > Adding another special case in how dtypes implicitly convert to one another will > impede new users being able to understand the whole system. See A. M. > Archibald's question in the thread "ufunc.reduce and conversion" for an example. > In our judgement this is a worse outcome than notational convenience for float32 > users, who already need to be aware of the effects of their precision choice. > Each of us can come to different conclusions in good faith without one of us > forgetting the new user experience. > > Let me offer a third path: the algorithms used for .mean() and .var() are > substandard. There are much better incremental algorithms that entirely avoid > the need to accumulate such large (and therefore precision-losing) intermediate > values. The algorithms look like the following for 1D arrays in Python: > > def mean(a): > m = a[0] > for i in range(1, len(a)): > m += (a[i] - m) / (i + 1) > return m > > def var(a): > m = a[0] > t = a.dtype.type(0) > for i in range(1, len(a)): > q = a[i] - m > r = q / (i+1) > m += r > t += i * q * r > t /= len(a) > return t > > Alternatively, from Knuth: > > def var_knuth(a): > m = a.dtype.type(0) > variance = a.dtype.type(0) > for i in range(len(a)): > delta = a[i] - m > m += delta / (i+1) > variance += delta * (a[i] - m) > variance /= len(a) > return variance > > If you will code up implementations of these for ndarray.mean() and > ndarray.var(), I will check them in and then float32 arrays will avoid most of > the catastrophes that the current implementations run into. > +1 > >>>> We are only talking >>>> about people that will a) work with single-precision data (e.g. large >>>> scale-image analysis) and who b) will tend to "just use the default" >>>> (dtype) --- How else can I say this: these people will just assume that >>>> arr.mean() *is* the mean of arr. >>>> >>> I don't understand what you mean, here. arr.mean() is almost never *the* mean of >>> arr. Double precision can't fix that. >>> >>> >> This was not supposed to be a scientific statement -- I'm (again) >> thinking of our students that not always appreciate the full complexity >> of computational numerics and data types and such. >> > > They need to appreciate the complexity of computational numerics if they are > going to do numerical computation. Double precision does not make it any simpler. > > |
258 messages has been excluded from this view by a project administrator.
![http://bo.sohu.com//images/img20040502/dj_bg.gif">](http://bo.sohu.com//images/img20040502/dj_bg.gif">){kind=link}
![http://igame.sina.com.cn/club/images/topmenu/topMenu_8.gif"](http://igame.sina.com.cn/club/images/topmenu/topMenu_8.gif"){kind=link}