From: Frank W. <fwi...@gm...> - 2009-02-19 18:54:59
|
Hi all, Has anyone here found the need to replace calls to buffer() when porting CPython to Jython? Just looking for the experience of others, especially if someone has alternatives in Java or Jython libs that perform well as replacements. I'm trying to help get Mercurial running on Jython, and this is one of the bigger issues. For whatever reason, I've never before encountered the need to attempt a buffer() replacement. -Frank |
From: Alan K. <jyt...@xh...> - 2009-02-19 20:36:21
|
[Frank] > Has anyone here found the need to replace calls to buffer() when > porting CPython to Jython? Not personally, but pyftpdlib uses buffer, which I came across when trying to address socket problems with it. The author is contactable from this bug report. http://bugs.jython.com/issue1238 There's a link in there to a message on python-dev from Guido about "buffer" and why he doesn't like it. http://mail.python.org/pipermail/python-dev/2000-October/009974.html Alan. |
From: Frank W. <fwi...@gm...> - 2009-02-19 21:09:14
|
On Thu, Feb 19, 2009 at 3:36 PM, Alan Kennedy <jyt...@xh...> wrote: > Not personally, but pyftpdlib uses buffer, which I came across when > trying to address socket problems with it. The author is contactable > from this bug report. > > http://bugs.jython.com/issue1238 cool thanks for the pointer. > There's a link in there to a message on python-dev from Guido about > "buffer" and why he doesn't like it. > > http://mail.python.org/pipermail/python-dev/2000-October/009974.html Yeah, I've seen that post, it is a big part of the reason that I'm not interested in trying to get a buffer() in Jython at all. The 3.0 alternatives I might look at one day, but that's a different matter. -Frank |
From: Philip J. <pj...@un...> - 2009-02-19 21:15:41
|
On Feb 19, 2009, at 12:36 PM, Alan Kennedy wrote: > [Frank] >> Has anyone here found the need to replace calls to buffer() when >> porting CPython to Jython? > It looks like mercurial is using buffer in about 3 places, for zero- copying of strs and arrays. E.g. converting Python array to a str-like object with no copy: # zero copy representation of addlist as a buffer addbuf = buffer(addlist) Or a zero copy substring of a str: s, self.buf = self.buf[:l], buffer(self.buf, l) Seems like we could replace these with slower str() calls or self.buf[l:] If that works, we could propose a patch to mercurial to use the slower methods if not hasattr(__builtin__, 'buffer'). > Not personally, but pyftpdlib uses buffer, which I came across when > trying to address socket problems with it. The author is contactable > from this bug report. > > http://bugs.jython.com/issue1238 > > There's a link in there to a message on python-dev from Guido about > "buffer" and why he doesn't like it. > > http://mail.python.org/pipermail/python-dev/2000-October/009974.html I think eventually we'll need to implement the buffer interface/ object, the interface might subclass java.nio's Buffer, or something like that. We might *really* need it for NumJy. In CPython, mmap objects actually implement the interface, and the re module acts on the buffer interface (so then you can re mmap'd objects). It'd probably all be easier to add in Jython 3, when the str/unicode distinction will actually make sense on Java. The post from Guido is pretty old now, I think all of his and others' gripes with the buffer interface have been resolved with the new, revamped buffer interface (PEP 3118) which was added in 2.6. FYI Travis Oliphant gave a talk on it @ last years PyCon: http://www.youtube.com/watch?v=10smLBD0kXg -- Philip Jenvey |
From: Frank W. <fwi...@gm...> - 2009-02-19 23:31:10
|
On Thu, Feb 19, 2009 at 4:15 PM, Philip Jenvey <pj...@un...> wrote: > The post from Guido is pretty old now, I think all of his and others' gripes > with the buffer interface have been resolved with the new, revamped buffer > interface (PEP 3118) which was added in 2.6. Cool, I'm very much interested in getting the revamped buffer protocols in Jython - I had been under the impression that they had a new name (memoryview maybe?) -- but whatever they call it I'm up for the newer, more carefully designed version. > > FYI Travis Oliphant gave a talk on it @ last years PyCon: > > http://www.youtube.com/watch?v=10smLBD0kXg Cool, I'll take a look at that. -Frank |
From: gidiko <gi...@gm...> - 2009-03-07 12:03:31
|
Unfortunately there are some garbage collection problems for the direct buffers of java.nio according to http://mikiobraun.blogspot.com/2008/10/matrices-jni-directbuffers-and-number.html Direct buffers modulo these problems still seem to me to be the best choice for implementing the basic array object of jython numpy package (aka numjy), since they essentially offer data sharing for the java/jython and the c/fortran/native side through e.g. jna or a higher level ctypes implementation calling into a highly tuned native matrix lib. Data structures (e.g. double[][]) of existing java matrix libraries, considered during numjy development, could be mapped (wrapped) to such a uniform buffer interface. Frank Wierzbicki wrote: > > Cool, I'm very much interested in getting the revamped buffer > protocols in Jython - I had been under the impression that they had a > new name (memoryview maybe?) -- but whatever they call it I'm up for > the newer, more carefully designed version. > -- View this message in context: http://www.nabble.com/alternatives-to-buffer%28%29-tp22107341p22386945.html Sent from the jython-users mailing list archive at Nabble.com. |