Screenshot instructions:
Windows
Mac
Red Hat Linux
Ubuntu
Click URL instructions:
Rightclick on ad, choose "Copy Link", then paste here →
(This may not be possible with some types of ads)
From: Konrad Bartkowski <k.bartkowski@fz...>  20110411 09:16:46

Ok, forwarding it to the matplotlibdevel list. Best wishes, Konrad (on behalf of our workgroup)  Original Message  Subject: Source of inaccuracies in the matplotlib library Date: Fri, 8 Apr 2011 18:12:47 +0200 From: Bartkowski, Konrad <k.bartkowski@...><mailto:k.bartkowski@...> To: dd55@...<mailto:dd55@...> <dd55@...><mailto:dd55@...>, mdroe@...<mailto:mdroe@...> <mdroe@...><mailto:mdroe@...>, efiring@...<mailto:efiring@...> <efiring@...><mailto:efiring@...>, jdhunter@...<mailto:jdhunter@...> <jdhunter@...><mailto:jdhunter@...>, jdh2358@...<mailto:jdh2358@...> <jdh2358@...><mailto:jdh2358@...> CC: Bartkowski, Konrad <k.bartkowski@...><mailto:k.bartkowski@...>, elmod@...<mailto:elmod@...> <elmod@...><mailto:elmod@...>, Matthias Bolten <bolten@...><mailto:bolten@...>, Grotendorst, Johannes <j.grotendorst@...><mailto:j.grotendorst@...>, Steffen, Bernhard <b.steffen@...><mailto:b.steffen@...> Dear Matplotlib developers, I am writing about the matplotlib library with the mpl_toolkits. First of all let me emphasize how great software it is. Recently, in one of our projects we were rendering big surfaces and encountered the following problem: http://www.mailarchive.com/matplotlibdevel@.../msg06869.html It's not a bug (which all in all is a natural and unavoidable ingredient of the software, and especially in such a big and complex system like matplotlib would be fully natural), since the software does exactly the projection mathematics that it is expected to do, but a source of the inaccuracies, which is especially visible in the critical examples. For the profit of the Python community we are sending You a proposition of a modification of the surface plotting rendering system, in case You find it interesting enough to include in the consecutive version of the library. In the source code from the attachment we redesigned a little bit the computation process – since the computations are especially sensible to numerical errors, that are for example amplified while norming or processing the quaterions in the various stages (for example division over coordinate in the perspective projection). Therefore the computational focus can be shifted from the Polygon collection to the polygons itself. In the example from the above forum or the slightly modified one, one can observe a big difference in the numerical precision while the speed of the computations does not decrease (at least visibly). While instead of the surfaces from the forum, the following surfaces are rendered: u = np.linspace(0, 2 * np.pi, 100) v = np.linspace(0, np.pi, 100) x = 10 * np.outer(np.cos(u), np.sin(v)) y = 10 * np.outer(np.sin(u), np.sin(v)) z = 10 * np.outer(np.ones(np.size(u)), np.cos(v)) ax.plot_surface(x, y, z, rstride=8, cstride=8, color='y', alpha=0.5) shiftX=28 shiftY=28 X,Y=np.meshgrid(range(20+shiftX,20+shiftX),range(20+shiftY,20+shiftY)) Z=np.ones((X.shape[0], Y.shape[1])) ax.plot_surface(X, Y, Z, color='r', rstride=10, cstride=10, alpha=1.0) the issue is visible for example at the azimuth=40 , elevation=70 – with those parameters the mentioned case is visible on the red surface, while with elevation=68 not. Moreover, now also the stride is big (in the new approach the influence of increasing stride on the numerical precision grows). So again let me use this opportunity to thank You for empowering the Python community worldwide in a great, powerful scientific visualization tool. Best wishes, Konrad Bartkowski   Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt   Besuchen Sie uns auf unserem neuen Webauftritt unter http://www.fzjuelich.de 
From: Benjamin Root <ben.root@ou...>  20110414 23:14:12
Attachments:
Message as HTML

On Mon, Apr 11, 2011 at 4:16 AM, Konrad Bartkowski < k.bartkowski@...> wrote: > Ok, forwarding it to the matplotlibdevel list. > > > Best wishes, > > Konrad (on behalf of our workgroup) > > >  Original Message  Subject: Source of inaccuracies in the > matplotlib library Date: Fri, 8 Apr 2011 18:12:47 +0200 From: Bartkowski, > Konrad <k.bartkowski@...> <k.bartkowski@...> To: > dd55@... <dd55@...> <dd55@...>, mdroe@... > <mdroe@...> <mdroe@...>, efiring@... > <efiring@...> <efiring@...>, jdhunter@... > <jdhunter@...> <jdhunter@...>, > jdh2358@... <jdh2358@...> <jdh2358@...> CC: Bartkowski, > Konrad <k.bartkowski@...> <k.bartkowski@...>, > elmod@... <elmod@...> <elmod@...>, Matthias Bolten > <bolten@...> <bolten@...>, > Grotendorst, Johannes <j.grotendorst@...><j.grotendorst@...>, > Steffen, Bernhard <b.steffen@...> <b.steffen@...> > > Dear Matplotlib developers, > > I am writing about the matplotlib library with the mpl_toolkits. First > of all let me emphasize how great software it is. Recently, in one of > our projects we were rendering big surfaces and encountered the > following problem: > http://www.mailarchive.com/matplotlibdevel@.../msg06869.html > > It's not a bug (which all in all is a natural and unavoidable ingredient > of the software, and especially in such a big and complex system like > matplotlib would be fully natural), since the software does exactly the > projection mathematics that it is expected to do, but a source of the > inaccuracies, which is especially visible in the critical examples. For > the profit of the Python community we are sending You a proposition of a > modification of the surface plotting rendering system, in case You find > it interesting enough to include in the consecutive version of the > library. In the source code from the attachment we redesigned a little > bit the computation process – since the computations are especially > sensible to numerical errors, that are for example amplified while > norming or processing the quaterions in the various stages (for example > division over coordinate in the perspective projection). Therefore the > computational focus can be shifted from the Polygon collection to the > polygons itself. In the example from the above forum or the slightly > modified one, one can observe a big difference in the numerical > precision while the speed of the computations does not decrease (at > least visibly). While instead of the surfaces from the forum, the > following surfaces are rendered: > > u = np.linspace(0, 2 * np.pi, 100) > v = np.linspace(0, np.pi, 100) > > x = 10 * np.outer(np.cos(u), np.sin(v)) > y = 10 * np.outer(np.sin(u), np.sin(v)) > z = 10 * np.outer(np.ones(np.size(u)), np.cos(v)) > > ax.plot_surface(x, y, z, rstride=8, cstride=8, color='y', alpha=0.5) > shiftX=28 > shiftY=28 > X,Y=np.meshgrid(range(20+shiftX,20+shiftX),range(20+shiftY,20+shiftY)) > Z=np.ones((X.shape[0], Y.shape[1])) > ax.plot_surface(X, Y, Z, color='r', rstride=10, cstride=10, alpha=1.0) > > the issue is visible for example at the azimuth=40 , elevation=70 – with > those parameters the mentioned case is visible on the red surface, while > with elevation=68 not. Moreover, now also the stride is big (in the new > approach the influence of increasing stride on the numerical precision > grows). > So again let me use this opportunity to thank You for empowering the > Python community worldwide in a great, powerful scientific visualization > tool. > > Best wishes, > Konrad Bartkowski > > Konrad, Thank you for this contribution. It seems that I am currently the de facto maintainer of the mplot3d code, but I have been focused on my PhD for the past month. I will take a look at the code in more detail over the weekend. Thanks! Ben Root 
From: Benjamin Root <ben.root@ou...>  20110415 20:42:46
Attachments:
Message as HTML

On Mon, Apr 11, 2011 at 4:16 AM, Konrad Bartkowski < k.bartkowski@...> wrote: > Ok, forwarding it to the matplotlibdevel list. > > > Best wishes, > > Konrad (on behalf of our workgroup) > > >  Original Message  Subject: Source of inaccuracies in the > matplotlib library Date: Fri, 8 Apr 2011 18:12:47 +0200 From: Bartkowski, > Konrad <k.bartkowski@...> <k.bartkowski@...> To: > dd55@... <dd55@...> <dd55@...>, mdroe@... > <mdroe@...> <mdroe@...>, efiring@... > <efiring@...> <efiring@...>, jdhunter@... > <jdhunter@...> <jdhunter@...>, > jdh2358@... <jdh2358@...> <jdh2358@...> CC: Bartkowski, > Konrad <k.bartkowski@...> <k.bartkowski@...>, > elmod@... <elmod@...> <elmod@...>, Matthias Bolten > <bolten@...> <bolten@...>, > Grotendorst, Johannes <j.grotendorst@...><j.grotendorst@...>, > Steffen, Bernhard <b.steffen@...> <b.steffen@...> > > Dear Matplotlib developers, > > I am writing about the matplotlib library with the mpl_toolkits. First > of all let me emphasize how great software it is. Recently, in one of > our projects we were rendering big surfaces and encountered the > following problem: > http://www.mailarchive.com/matplotlibdevel@.../msg06869.html > > It's not a bug (which all in all is a natural and unavoidable ingredient > of the software, and especially in such a big and complex system like > matplotlib would be fully natural), since the software does exactly the > projection mathematics that it is expected to do, but a source of the > inaccuracies, which is especially visible in the critical examples. For > the profit of the Python community we are sending You a proposition of a > modification of the surface plotting rendering system, in case You find > it interesting enough to include in the consecutive version of the > library. In the source code from the attachment we redesigned a little > bit the computation process – since the computations are especially > sensible to numerical errors, that are for example amplified while > norming or processing the quaterions in the various stages (for example > division over coordinate in the perspective projection). Therefore the > computational focus can be shifted from the Polygon collection to the > polygons itself. In the example from the above forum or the slightly > modified one, one can observe a big difference in the numerical > precision while the speed of the computations does not decrease (at > least visibly). While instead of the surfaces from the forum, the > following surfaces are rendered: > > u = np.linspace(0, 2 * np.pi, 100) > v = np.linspace(0, np.pi, 100) > > x = 10 * np.outer(np.cos(u), np.sin(v)) > y = 10 * np.outer(np.sin(u), np.sin(v)) > z = 10 * np.outer(np.ones(np.size(u)), np.cos(v)) > > ax.plot_surface(x, y, z, rstride=8, cstride=8, color='y', alpha=0.5) > shiftX=28 > shiftY=28 > X,Y=np.meshgrid(range(20+shiftX,20+shiftX),range(20+shiftY,20+shiftY)) > Z=np.ones((X.shape[0], Y.shape[1])) > ax.plot_surface(X, Y, Z, color='r', rstride=10, cstride=10, alpha=1.0) > > the issue is visible for example at the azimuth=40 , elevation=70 – with > those parameters the mentioned case is visible on the red surface, while > with elevation=68 not. Moreover, now also the stride is big (in the new > approach the influence of increasing stride on the numerical precision > grows). > So again let me use this opportunity to thank You for empowering the > Python community worldwide in a great, powerful scientific visualization > tool. > > Best wishes, > Konrad Bartkowski > > > Konrad, Ok, I have examined the attached file, and I see what you have done. First, the shading issue has long since been resolved and is in the development branch (but probably won't be backported to v1.0.x because the changes were extensive). Second, the zorder sorting issue is a PITA to say the least. Your approach, however, only pushes the problem down to smaller parts, and still doesn't address the same problems with PatchCollection objects. That being said, I would still be inclined to have the problem solved for surfaces and leave patches to be buggy, except that this approach completely breaks the API. plot_surface returns a Poly3DCollection object, not a list of Poly3DCollection objects. The Collections object is a doubleedge sword. It allows for easy manipulation of many artist objects, but ultimately the Collection object must report a single zorder value to represent the zorder for plotting all of the artist elements. There might be a possible workaround, though. Maybe (and I am just speculating here) if we can get the core part of matplotlib to specially treat 3d collection objects in such a way that allows the collection to return provide elements and zorder pairs. It is either that, or we finally try to get OpenGL working again in matplotlib and allow ourselves to specify coordinates in 3D. Ben Root 
From: John Hunter <jdh2358@gm...>  20110415 20:54:52
Attachments:
Message as HTML

On Fri, Apr 15, 2011 at 3:42 PM, Benjamin Root <ben.root@...> wrote: > > There might be a possible workaround, though. Maybe (and I am just > speculating here) if we can get the core part of matplotlib to specially > treat 3d collection objects in such a way that allows the collection to > return provide elements and zorder pairs. It is either that, or we finally > try to get OpenGL working again in matplotlib and allow ourselves to specify > coordinates in 3D. > > It should be fairly easy to get a collections object to support multiple zorders *within* the collection. Across artists, damn near impossible. I don't think you need to provide elements and zorder pairs perse. The typical way a property is specified for a collection if you want it to vary over the elements of the collection is that the property is a sequence, and the property is accessed as prop[i%N] where i is the element number and N is the length of the property vector. So if we make zorder a len(elements) sequence of zorders, we can order the collection by the zorder at draw time. Presumably external code would modify the zorder of the collection before each draw depending on the view. 
From: Benjamin Root <ben.root@ou...>  20110415 21:03:11
Attachments:
Message as HTML

On Fri, Apr 15, 2011 at 3:54 PM, John Hunter <jdh2358@...> wrote: > > > On Fri, Apr 15, 2011 at 3:42 PM, Benjamin Root <ben.root@...> wrote: > >> >> There might be a possible workaround, though. Maybe (and I am just >> speculating here) if we can get the core part of matplotlib to specially >> treat 3d collection objects in such a way that allows the collection to >> return provide elements and zorder pairs. It is either that, or we finally >> try to get OpenGL working again in matplotlib and allow ourselves to specify >> coordinates in 3D. >> >> > It should be fairly easy to get a collections object to support multiple > zorders *within* the collection. Across artists, damn near impossible. I > don't think you need to provide elements and zorder pairs perse. The > typical way a property is specified for a collection if you want it to vary > over the elements of the collection is that the property is a sequence, and > the property is accessed as prop[i%N] where i is the element number and N is > the length of the property vector. So if we make zorder a len(elements) > sequence of zorders, we can order the collection by the zorder at draw > time. Presumably external code would modify the zorder of the collection > before each draw depending on the view. > Within a collection, this is already done correctly (or at least, as well as one can except with a 2D rendering engine). It is when you have multiple artists that exists over parts of the zorder "axis" that there are problems. Essentially, if the 3D bounding boxes overlap, then trouble ensues. Also, that speculation I had wouldn't work either, as the problem still would exist for intersecting patches. No, what we really need is a 3D rendering engine, and a logical separation between zorder for sorting/layering and zcoordinates. Ben Root 
From: Konrad Bartkowski <k.bartkowski@fz...>  20110421 09:52:38

Dear Ben, dear John, Thank You for the emails. Well, we have modified only the part influencing the modules, that we need or will need in our projects. I admit that apart from the deceptions connected with human perception (for example http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.151.2303 ) still the inaccuracy remains. Nevertheless all the surfaces we have rendered were unproblematic. Also the examples from the forum rendered correctly even for big stride. If it is unsatisfactory one can control the precision with the stride size. The modification was written under the assumption not to modify too much in the existing system. While writing the code we were naturally keeping in mind the ratio of accuracy to the computational effort. Since in our projects we didn't need the return value from the plot_surface we return the obtained list for the efficiency reasons. But if the API should remain unchanged, then one can generate also the previous collection and return it instead of the list, while the list is added. I admit, that I didn't consider the changes You mention in the consecutive emails, since it would mean redesigning Your code. From the same reason I also cannot comment them – I simply don't know the internal structure of Your code sufficienty. Change on the level of Artists would probably imply a lot of other modifications in many, many modules.  Mit freundlichen Grüssen / With friendly greetings, Konrad Bartkowski On 04/15/11 23:02, Benjamin Root wrote: > > > On Fri, Apr 15, 2011 at 3:54 PM, John Hunter <jdh2358@... > <mailto:jdh2358@...>> wrote: > > > > On Fri, Apr 15, 2011 at 3:42 PM, Benjamin Root <ben.root@... > <mailto:ben.root@...>> wrote: > > > There might be a possible workaround, though. Maybe (and I am > just speculating here) if we can get the core part of matplotlib > to specially treat 3d collection objects in such a way that > allows the collection to return provide elements and zorder > pairs. It is either that, or we finally try to get OpenGL > working again in matplotlib and allow ourselves to specify > coordinates in 3D. > > > It should be fairly easy to get a collections object to support > multiple zorders *within* the collection. Across artists, damn > near impossible. I don't think you need to provide elements and > zorder pairs perse. The typical way a property is specified for a > collection if you want it to vary over the elements of the > collection is that the property is a sequence, and the property is > accessed as prop[i%N] where i is the element number and N is the > length of the property vector. So if we make zorder a len(elements) > sequence of zorders, we can order the collection by the zorder at > draw time. Presumably external code would modify the zorder of the > collection before each draw depending on the view. > > > > Within a collection, this is already done correctly (or at least, as > well as one can except with a 2D rendering engine). It is when you have > multiple artists that exists over parts of the zorder "axis" that there > are problems. Essentially, if the 3D bounding boxes overlap, then > trouble ensues. > > Also, that speculation I had wouldn't work either, as the problem still > would exist for intersecting patches. No, what we really need is a 3D > rendering engine, and a logical separation between zorder for > sorting/layering and zcoordinates. > > Ben Root > >   Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDirig Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt   Besuchen Sie uns auf unserem neuen Webauftritt unter http://www.fzjuelich.de 
From: Benjamin Root <ben.root@ou...>  20110421 14:53:00
Attachments:
Message as HTML

On Thursday, April 21, 2011, Konrad Bartkowski <k.bartkowski@...> wrote: > Dear Ben, dear John, > > Thank You for the emails. Well, we have modified only the part > influencing the modules, that we need or will need in our projects. I > admit that apart from the deceptions connected with human perception > (for example > http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.151.2303 ) still > the inaccuracy remains. Nevertheless all the surfaces we have rendered > were unproblematic. Also the examples from the forum rendered correctly > even for big stride. If it is unsatisfactory one can control the > precision with the stride size. Just to make sure, you never really described exactly what "numerical inaccuracies" you were talking about. From the example you posted, I am assuming that it is the overlapping/zsorting issue. If it is something else, then I am going to need to see some "before / after" images showing the difference you are talking about. If it is the overlapping issue, then the strides are only incidental to the problem. The problem exists for any 3d object, with or without strides. The problem is one of dimensionality. While mplot3d does correctly calculate the 3d coordinates of each vertex in the perspective coordinate system, only one value for the depth axis can be reported for each artist. If the artist has many vertexes with different depth values, then a single value will never be sufficient. It is this reduction of the depth coordinates from many to one that is the source of the problem. What your solution does is to not reduce the depth coordinates *as much*. But it still does a reduction, therefore not solving the problem. The only way to properly solve this is with a 3D rendering engine (like OpenGL). >The modification was written under the > assumption not to modify too much in the existing system. While writing > the code we were naturally keeping in mind the ratio of accuracy to the > computational effort. > Since in our projects we didn't need the return value from the > plot_surface we return the obtained list for the efficiency reasons. But > if the API should remain unchanged, then one can generate also the > previous collection and return it instead of the list, while the list is > added. This would cause many inconsistencies within the code. For example, if a user wants to remove the collection from the axes, it won't work because the collection doesn't exist in the axes object. Also, having a collection object unconnected to any axis might cause unexpected behavior or could even throw exceptions. > I admit, that I didn't consider the changes You mention in the > consecutive emails, since it would mean redesigning Your code. From the > same reason I also cannot comment them – I simply don't know the > internal structure of Your code sufficienty. Change on the level of > Artists would probably imply a lot of other modifications in many, many > modules. > That's ok. It was actually this problem that pushed me to learn matplotlib to the point that I could become a developer. However, it took me another 6 months before I could understand the problem well enough to realize the source of the issue and what was needed to solve it properly. If this patch works well for your purposes, that's great, and I hope it continues to serve you well. However, for a broader audience, it introduces too many potential problems for us. I hope you continue to use matplotlib and find it useful. And feel free to continue commenting on the mplot3d module and how it can be improved! Cheers, Ben Root 