pyobjc-dev — CVS commit notices go here

[Pyobjc-dev] Plain HTML DocUtils writer available

[Bill B. <bb...@co...>]

If anyone is interested, my pure HTML DocUtils writer is now complete 
to the point where it can parse/process the 'test.txt' document in the 
DocUtils cvs archive.  That is, it should not be able to handle any 
ReST source and produce legible HTML that does not use CSS.

The writer is designed to produce HTML that is compliant with 
O'Reilly's article submission guidelines [for their DevCenter, at 
least].

The output is currently not as pretty as it could be-- 
suggestion/input/patches would be most welcome-- but it should be 
'correct' in structure.

It can be found in the bbum/DocArticle/ sandbox of the DocUtils project 
and is packaged as a standard 'disutils' based module.   Install 
DocUtils, then install DocArticle.  A command line tool -- 
docarticle.py -- can used to process a ReST document.

http://docutils.sourceforge.net/

b.bum

Re: [Pyobjc-dev] Attempt at writing Cocoa services with pyobjc

[Jack J. <Jac...@cw...>]

On Monday, Jan 6, 2003, at 16:16 Europe/Amsterdam, Ronald Oussoren 
wrote:

> Dinu,
>
> The attached files implement a simple service. It implements two menu
> items, but only 'Evaluate expression' is functional at the moment.

Ronald, this looks really cool, but could you explain how things are 
tied together (or point me to the right documentation?).

For instance, in the plist file you have the NSServices dict, which 
maps menu entries to calls to be made in the service. I assume that the 
mapping from <string>doPythonExec</string> to 
doPythonExec:userData:error: is defined somewhere, right? But it looks 
as though I miss an argument? Or does the NSServiceFunc magic have 
something to do with this? And what is the NSPortName=PythonService 
used for?

--
Jack Jansen, <Jac...@cw...>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

Re: [Pyobjc-dev] Attempt at writing Cocoa services with pyobjc

[Ronald O. <ous...@ci...>]

Dinu,

The attached files implement a simple service. It implements two menu
items, but only 'Evaluate expression' is functional at the moment.

I wrote this several months ago, but never got around to finishing it.

The problem your seeing with NSAutoreleasePool is a bug in the bridge 
caused by a feature of NSInvocation. Note that you don't have to create 
an autorelease pool.

Ronald

Re: [Pyobjc-dev] More PB Python project templates?

[<bb...@ma...>]

On Monday, Jan 6, 2003, at 08:18 US/Eastern, Dinu Gherman wrote:
> Hi,
>
> I see that pyobjc 0.8 adds exactly one template "Cocoa-Python App."
> to the list of known project templates in PB. I guess it would make
> sense to add further ones, like at least:
>
>   Cocoa-Python Document-based Application

Already in the works.

>   Python Tool (need no AppKit/GUI)

Actually-- I was thinking about a PB based template that generates a 
distutils compatible project.

And another one:


Cocoa-Python Application (Embedded Interpreter)

[Pyobjc-dev] Re: [Python-Dev] PyBuffer* vs. array.array()

[Bill B. <bb...@co...>]

On Sunday, Jan 5, 2003, at 16:58 US/Eastern, Guido van Rossum wrote:
>>          singlePlane = array.array('B')
>>          singlePlane.fromlist([0 for x in  range(0, width*height*3)] )
>
> I'm not sure if you were joking, but why not write
>
>          singlePlane.fromlist([0] * (width*height*3))
>
> ???

Not joking; not thinking and haven't really done large blob 
manipulation in Python before.   That answers another question, though 
-- if I were to build an image with four channels-- red, green, blue, 
alpha-- and wanted the alpha channel to be set to 255 throughout, then 
I would do...

     singlePlane.fromlist([0, 0, 0, 255] * (width * height))

... or ...

     array.array('B', [0, 0, 0, 255])  * width * height

>> ...........
>> --
>
> I'm not sure I understand the problem.

I was hoping that there was a single object type that could easily be 
used from both the C and Python side that could contain a large buffer 
of binary/byte data.

What I really need is a fixed length buffer that supports slicing style 
assignments / getters.  The type of the elements is largely irrelevant 
save for that each element needs to be accessed as a single byte.

The fixed length requirement comes from the need to encapsulate buffers 
of memory as returned by various APIs outside of Python.   In this 
case, I'm providing access to hunks of memory controlled by the APIs 
provided by the Foundation and the AppKit within Cocoa (or GNUstep).

I also need to allocate a hunk of memory-- an array of bytes, a string, 
a buffer, whatever-- and pass it off through the AppKit/Foundation 
APIs.   Once those APIs have the address and length of the buffer, that 
address and length must remain constant over time.   I would really 
like to be able to do the allocation from the Python side of the 
fence-- allocate, initialize with a particular byte pattern, and pass 
it off to the Foundation/AppKit (while still being able to manipulate 
the contents in Python).

The PyBuffer* C API seems to be ideal in that a buffer object produced 
via the PyBuffer_New() function is read/write (unlike a buffer produced 
by buffer() in Python), contains a reference to a fixed length array at 
a fixed address, and is truly a bag o' bytes.

At this point, I'll probably add some kind of an 'allocate' function to 
the 'objc' module that simply calls PyBuffer_New().

Did that -- works except, of course, the resulting buffer is an array 
of chars such that slicing assignments have to take strings.  
Inconvenient, but workable:

 >>> import objc
 >>> b = objc.allocateBuffer(100)
 >>> type(b)
<type 'buffer'>
 >>> b[0:10] = range(0,10)
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: bad argument type for built-in operation
 >>> b[0:10] = [chr(x) for x in range(0,10)]
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: bad argument type for built-in operation
 >>> b[0:10] = "".join([chr(x) for x in range(0,10)])
 >>> b
<read-write buffer ptr 0x1ad4bc, size 100 at 0x1ad4a0>
 >>> b[0:15]
'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t\x00\x00\x00\x00\x00'

> You could use the 'c' code for creating an array instead of 'B'.

Right;  as long as it is a byte, it doesn't matter.  I chose 'B' 
because it is an unsigned numeric type.   Since I'm generating numeric 
data that is shoved into the bitmap as R,G,B triplets, a numeric type 
seemed to be the most convenient.

> Or you can use the tostring() method on the array to convert it to a
> string.
>
> Or you could use buffer() on the array.
> But why don't you just use strings for binary data, like everyone
> else?

Because strings are variable length, do not support slice style 
assignments, and require all numeric data to be converted to a string 
before being used as 'data'.

b.bum

Re: [Pyobjc-dev] Attempt at writing Cocoa services with pyobjc

[<bb...@ma...>]

On Monday, Jan 6, 2003, at 07:43 US/Eastern, Dinu Gherman wrote:
> Hi,
>
> After recently seeing the release of pyobjc 0.8 I decided it was
> high time to have some fun writing *real* Python Cocoa programs!

Good -- the more feedback, the better!

> I started using pyobjc on a little Cocoa Services tool, much in
> the spirit of URLServices for those who know it. Unfortunately,
> I can't find a URL for downloading it, but I can provide my copy
> of it to anybody interested.
>
> There are two problems I'm having so far with that first toy pro-
> ject, after just creating a "Python Cocoa App." with PB:

You are working too hard. :-)   Neither of these problems are specific 
to Cocoa-Python -- they would both be issues if you were building an 
ObjC or Java Cocoa App.

> 1. Removing the PB Resources folder containing the MainMenu.nib
> and InfoPlist.strings results in complaints like "Unable to load nib
> file: MainMenu, exiting". The NIB file isn't needed in this case as
> there is neither a GUI nor an app "owner" object, hence I also re-
> moved the MyAppDelegate.py thinking that would be ok. Apparently
> there must be something else to get rid of the need for a NIB file.
> Any ideas?

A Cocoa application generally has the concept of a 'Main NIB file'.   
That is, a NIB file that is automatically loaded at application launch 
that contains things like the main menu and the primary controller 
object.  In the case of Cocoa/Python, the default 'controller' object 
is an instance of MyAppDelegate.  The instance appears in MainMenu.nib.

If you truly want to remove MainMenu.nib, then you need to also adjust 
the configuration of the application target such that it will no longer 
attempt to load the MainMenu at app launch.  Then, remove the reference 
to MyAppDelegate from __main__.py or set the inheritance of 
MyAppDelegate to something appropriate [NSObject] and remove the 
AutoBaseClass/NibClassBuilder stuff from MyAppDelegate.py.   You will 
need to instantiate MyAppDelegate by hand-- likely in __main__.py.

But I don't think you want to to do that-- if you want a UI-less app, 
you are far better off simply removing the single window from 
MainMenu.nib.   If you don't want the dock icon [bad idea, more often 
than not], then use the NSUIElement hack [search google].

Ahhh... I see... you want to create a services engine that runs 
completely in the background.    I'll leave the above in anyway (it is 
still useful).

More below....

> 2. In the Python main file I fumbled to port the code from the
> URLServices project inside this function:
>
> 	int main(int argc, const char **argv)
> 	{
> 	  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
> 	  URLServices *serviceProvider = [[URLServices alloc] init];
> 	
> 	  NSRegisterServicesProvider(serviceProvider, @"URLServices");
> 	
> 	  NS_DURING
> 	    [[NSRunLoop currentRunLoop] configureAsServer];
> 	    [[NSRunLoop currentRunLoop] run];
> 	  NS_HANDLER
> 	    NSLog(@"%@", localException);
> 	  NS_ENDHANDLER
> 	
> 	  [serviceProvider release];
> 	  [pool release];
> 	
> 	  exit(0);
> 	  return 0;
> 	}
>
>
> to something like the following:
>
> 	# import classes required to start application
> 	import PyServices
> 	
> 	from Foundation import NSAutoreleasePool, NSRunLoop, NSLog
> 	from AppKit import NSRegisterServicesProvider
> 	
> 	# stuff ported from URLServices' main.m
> 	## pool = NSAutoreleasePool.alloc().init()
> 	serviceProvider = PyServices.PyServices.alloc().init()
> 	NSRegisterServicesProvider(serviceProvider, "PyServices")
> 	#  NS_DURING
> 	NSRunLoop.currentRunLoop().configureAsServer()
> 	NSRunLoop.currentRunLoop().run()
> 	#  NS_HANDLER
> 	NSLog("%@", localException)
> 	#  NS_ENDHANDLER
> 	serviceProvider.release()
> 	## pool.release()
> 	
> 	# pass control to the AppKit (from template __main__.py)
> 	# import sys
> 	# sys.exit(AppKit.NSApplicationMain(sys.argv))
>
> where it seems that using the autorelease pool is a bad idea (com-
> plains with "Cannot retain an autorelease pool") and replacing the
> NSApplicationMain function with an NSRunLoop does not quite seem
> to work like expected, resulting in crash reports being dumped.

Yes -- this is a bug in the bridge, sort of.   There is a category on 
NSAutoreleasePool that allows you to push and pop pools, as needed.

@interface NSAutoreleasePool (PyObjCSupport)
+ (void) pyobjcPushPool;
+ (void) pyobjcPopPool;
@end

So, just call...

     NSAutoreleasePool.pyobjcPushPool()

... instead of NSAutoreleasePool.alloc().init().

PyObjC maps Objective-C exceptions to Python and vice-versa.  That is, 
you can do:

try:
     NSRunLoop.currentRunLoop().configureAsServer()
     NSRunLoop.currentRunLoop().run()
except:
     ... handle exception as you would any other Python exception ...

> I'm barely beginning to really play with pyobjc and so I don't have
> a detailed idea yet of how it works, while at the same time I'm un-
> dusting some Cocoa knowledge... Since it is only some 15 KB I dare
> to attach my initial code here right away.
>
> If anybody can give a helping hand this could be a nice demo for
> pyobjc since text, URL and file management is *much* easier in Py-
> thon than in the Foundation Kit...

Agreed.  I haven't seen URLServices -- is there any reason not to have 
some kind of a UI?  I.e. does it need preferences, documentation or any 
kind of a UI?  If so, I would punt on creating an 'invisible' service 
-- they are handy for a lot of things, but it also means that the user 
cannot easily kill/quit the service.  If the service is every invoked 
from a disk image or other removable media, the media cannot be ejected 
until the service is killed.

b.bum

Re: [Pyobjc-dev] Standalone table app

[Just v. R. <ju...@le...>]

David Eppstein wrote:

> * The application itself is big (7.7Mb).  More than half of the bulk
> is due to including _xmlplus due to Apple's failure to include
> pyexpat in the default command-line Python.  Pyexpat itself is not so
> big (460k) so maybe it would be possible to reduce the build size.

I forgot to mention: if expat is all you need from _xmlplus, it should
suffice to just include pyexpat.so instead of all of _xmlplus.

Just

[Pyobjc-dev] More PB Python project templates?

[Dinu G. <gh...@da...>]

Hi,

I see that pyobjc 0.8 adds exactly one template "Cocoa-Python App."
to the list of known project templates in PB. I guess it would make
sense to add further ones, like at least:

   Cocoa-Python Document-based Application
   Python Tool (need no AppKit/GUI)

For multi-document apps the app delegate needs at mininimum to have
some more methods wired to entries in the main menu, I believe. And
the __main__.py file might also be different, or not...

If someone elaborates a bit on how the Cocoa-Python Application tem-
plate was done I can think about covering the other two above.

Regards,

Dinu

--
Dinu C. Gherman
......................................................................
"Consistency is the last refuge of the unimaginative." (Oscar Wilde)

[Pyobjc-dev] Attempt at writing Cocoa services with pyobjc

[Dinu G. <gh...@da...>]

Hi,

After recently seeing the release of pyobjc 0.8 I decided it was
high time to have some fun writing *real* Python Cocoa programs!

I started using pyobjc on a little Cocoa Services tool, much in
the spirit of URLServices for those who know it. Unfortunately,
I can't find a URL for downloading it, but I can provide my copy
of it to anybody interested.

There are two problems I'm having so far with that first toy pro-
ject, after just creating a "Python Cocoa App." with PB:

1. Removing the PB Resources folder containing the MainMenu.nib
and InfoPlist.strings results in complaints like "Unable to load nib
file: MainMenu, exiting". The NIB file isn't needed in this case as
there is neither a GUI nor an app "owner" object, hence I also re-
moved the MyAppDelegate.py thinking that would be ok. Apparently
there must be something else to get rid of the need for a NIB file.
Any ideas?

2. In the Python main file I fumbled to port the code from the
URLServices project inside this function:

	int main(int argc, const char **argv)
	{
	  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
	  URLServices *serviceProvider = [[URLServices alloc] init];
	
	  NSRegisterServicesProvider(serviceProvider, @"URLServices");
	
	  NS_DURING
	    [[NSRunLoop currentRunLoop] configureAsServer];
	    [[NSRunLoop currentRunLoop] run];
	  NS_HANDLER
	    NSLog(@"%@", localException);
	  NS_ENDHANDLER
	
	  [serviceProvider release];
	  [pool release];
	
	  exit(0);
	  return 0;
	}


to something like the following:

	# import classes required to start application
	import PyServices
	
	from Foundation import NSAutoreleasePool, NSRunLoop, NSLog
	from AppKit import NSRegisterServicesProvider
	
	# stuff ported from URLServices' main.m
	## pool = NSAutoreleasePool.alloc().init()
	serviceProvider = PyServices.PyServices.alloc().init()
	NSRegisterServicesProvider(serviceProvider, "PyServices")
	#  NS_DURING
	NSRunLoop.currentRunLoop().configureAsServer()
	NSRunLoop.currentRunLoop().run()
	#  NS_HANDLER
	NSLog("%@", localException)
	#  NS_ENDHANDLER
	serviceProvider.release()
	## pool.release()
	
	# pass control to the AppKit (from template __main__.py)
	# import sys
	# sys.exit(AppKit.NSApplicationMain(sys.argv))

where it seems that using the autorelease pool is a bad idea (com-
plains with "Cannot retain an autorelease pool") and replacing the
NSApplicationMain function with an NSRunLoop does not quite seem
to work like expected, resulting in crash reports being dumped.

I'm barely beginning to really play with pyobjc and so I don't have
a detailed idea yet of how it works, while at the same time I'm un-
dusting some Cocoa knowledge... Since it is only some 15 KB I dare
to attach my initial code here right away.

If anybody can give a helping hand this could be a nice demo for
pyobjc since text, URL and file management is *much* easier in Py-
thon than in the Foundation Kit...

Regards,

Dinu

PS: BTW, this is Apple's online info about how to create services:
      
http://developer.apple.com/techpubs/macosx/Cocoa/TasksAndConcepts/ 
ProgrammingTopics/SysServices/index.html

--
Dinu C. Gherman
......................................................................
"Women give to men the very gold of their lives. But they invariably
want it back in small change." (Oscar Wilde)

[Pyobjc-dev] Anyone looking for a project?

[Jack J. <Jac...@cw...>]

In case anyone is looking for a real project to do with PyObjC: have a 
look at ImgSeek, <http://imgseek.sourceforge.net>. This is a Python 
program that searches an image database based on a sketch you make of 
the sort of image you're looking for.

I haven't inspected the code, but from the screen shots and the readme 
it looks really impressive. And if you think of something like this 
with a Cocoa interface I think you have something that would 
communicate the power of Python the Apple-people and the power of Cocoa 
to Python-people (aside from being pretty darn useful:-).
--
Jack Jansen, <Jac...@cw...>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman

Re: [Pyobjc-dev] Standalone table app

[Just v. R. <ju...@le...>]

David Eppstein wrote:

> * The application itself is big (7.7Mb).  More than half of the bulk
> is due to including _xmlplus due to Apple's failure to include
> pyexpat in the default command-line Python.  Pyexpat itself is not so
> big (460k) so maybe it would be possible to reduce the build size.

bundlebuilder.py is capable of building standalone apps (provided
modulefinder.py is available). It will analyze module dependencies and
only include those modules that are needed. However, instead of
depending on an installed Python (I suspect your app uses the Python 2.2
as shipped by Apple) it will include whatever (non-Framework) Python
(and support libraries) you run bundlebuilder with. The --strip option
will strip all binaries, which can reduce the size of the .app bundle by
half. For extra compact apps, you should try CVS Python (non-framework),
where bundlebuilder.py will use zipimport for .py modules, which may
save quite a bit if you have many modules. Don't use 2.3a1 as it has
broken zipimport support (entirely due to me) :-(.

This is all pretty new stuff and I'd appreciate any feedback.

It currently doesn't work for framework builds: this is something I'll
have to work on (and could use some help with ;-).

[ ... ]
> * Startup time is slow.

That has also been my experience, even for trivial apps :-(

Just

Re: [Pyobjc-dev] libFFI

[Ronald O. <ous...@ci...>]

On Sunday, Jan 5, 2003, at 19:23 Europe/Amsterdam, Bill Bumgarner wrote:

> On Sunday, Jan 5, 2003, at 13:03 US/Eastern, Ronald Oussoren wrote in 
> a CVS log entry:
>> libFFI support now actually works (that is passes the testsuite with 
>> the
>> same failures as the non-libFFI version)
>
> Congratulations!
>
> Some questions (as I know nothing about libFFI):
>
>     - how hard is it to build libFFI such that PyObjC can be built 
> with libFFI support?
It is not that hard, it is installed using the usual 
'configure;make;make install' commands. You just have to download the 
entire GCC distribution for this. Now that it actually works I'll try 
to build an archive containing just libFFI + whatever else is needed to 
build it.

>
>     - this eliminates register.m, correct?   How much smaller is the 
> resulting binary?
It does eliminate register.m. The objc/_objc.so without libFFI is 2.6M 
on my machine, while the one with libFFI is 0.6M (and that is including 
libFFI)

>
>     - what is the performance difference when using libFFI?   (Maybe 
> wrap runalltests with calls to 'timing'?)
That is something I haven't checked yet. I expect that the difference 
won't be noticeable: The non-libFFI version uses NSInvocation, which 
probably is as (in)efficient as libFFI.

I'll create a simple performance test, that might also be usefull to 
check how much worse the performance of PyObjC is compared to native 
Objective-C (w.r.t. message dispatching).

Ronald

Re: [Pyobjc-dev] GNUstep.

[Ronald O. <ous...@ci...>]

On Sunday, Jan 5, 2003, at 19:19 Europe/Amsterdam, bb...@ma... wrote:

> Wow!  I'm really happy to see that the GNUstep effort has such 
> momentum.   Very cool!  I'm going to add a news item to that effect 
> shortly.
>
> I'm thinking that it might be wise to start a pyobjc-dev-gnustep 
> mailing list as the GNUstep effort is going to generate a lot of 
> traffic that is highly specific in nature.   It would give people new 
> to the project an easy way to scan the archives to find relevant notes 
> regarding the state of GNUstep compatibility.
I don't think this is worth the effort. Porting to GNUstep is likely to 
generate a lot of specific traffic, but only for a short time. After 
that people on GNUstep will have the same questions as those on MacOS.

BTW. Is there anyone that is still using NeXTstep and is interested in 
using PyObjC? Porting to NeXTstep shouldn't be too much of an effort 
;-) ;-)

>
> Also-- for those working on GNUstep, there are a couple (maybe only 
> one now?) of places where the PyObjC module does something like the 
> following (found in objc/__init__.py):
>
> try:
>     import _FoundationMapping
>     del _FoundationMapping
> except ImportError:
>     pass
We might as wel remove the try:except: block. I had what I thought was 
a good reason for doing it this way, but I forgot was it is.

>
> As the port progresses, it is likely that an import error will occur 
> in this context that is indicative of a real problem.
It is also likely that there is a real problem when this occurs on 
MacOS X.

Ronald

[Pyobjc-dev] Standalone table app

[David E. <epp...@ic...>]

I seem to have hit a plateau with the table editor I've been playing with 
as a test of using pyobjc for making standalone apps.  So, in the hope that 
it will be useful to someone, provide a helpful example of how to (or how 
not to) make a pyobjc app, or elicit suggestions for improvement, I've made 
it available on my web site:

http://www.ics.uci.edu/~eppstein/tabulizer/

Known issues:

* The application itself is big (7.7Mb).  More than half of the bulk is due 
to including _xmlplus due to Apple's failure to include pyexpat in the 
default command-line Python.  Pyexpat itself is not so big (460k) so maybe 
it would be possible to reduce the build size.

* There is not currently an easy way of exchanging data with other programs 
(e.g. spreadsheets, word processors, or web browsers).  I think I need more 
understanding of how the file open and save dialogs work before reading and 
writing other file formats can be supported (currently, the only format is 
xml) and I also haven't done anything about handling drag and drop or 
pasting of spreadsheet tables.

* This is a very new program, tested only by me, so is probably quite 
buggy.  In particular, I still have not resolved the problem I reported to 
pyobjc-dev on 12/11/02, that starting the app by double-clicking on its 
icon fails to open a new document.

* Startup time is slow.

* I think I've made a build that will run on default X.2 installations, 
without need for installation of pyobjc or _xmlplus.  But the only machine 
I have running X.2 is my development machine in which those packages are 
installed.  I tested that it works if I clear out .../site-packages/ from 
the Python lib, but I haven't tested it on a vanilla X.2 machine.

* There are some standard Cocoa services I'm not using and probably should, 
including undo (I have my own undo manager instead) and help browser (help 
command instead opens a web page in your default browser).

--
David Eppstein       UC Irvine Dept. of Information & Computer Science
epp...@ic... http://www.ics.uci.edu/~eppstein/

[Pyobjc-dev] Re: Slow String Repeat (was: [Python-Dev] PyBuffer* vs. array.array())

[Jack J. <Jac...@or...>]

On maandag, jan 6, 2003, at 00:46 Europe/Amsterdam, Christian Tismer 
wrote:

> The central copying code in stringobject.c is the following
> tight loop:
>
> 	for (i = 0; i < size; i += a->ob_size)
> 		memcpy(op->ob_sval+i, a->ob_sval, (int) a->ob_size);
>
> For my example, this memcpy is started for every single
> of the one million bytes. So the overhead of memcpy,
> let is be a function call or a macro, will be executed
> a million times.

Oops, I replied before seeing this message, this does sound plausible. 
But that gives an easy way to fix it: for copies larger than a certain 
factor just copy the source object, then duplicate the source object 
until you're at size/2, then duplicat the last bit.

That is, if it is worth the trouble to optimize this,
--
- Jack Jansen        <Jac...@or...>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -

[Pyobjc-dev] Slow String Repeat (was: [Python-Dev] PyBuffer* vs. array.array())

[Christian T. <ti...@ti...>]

Christian Tismer wrote:
> Guido van Rossum wrote:
...

>> Correct; then even better:
>>
>>              singlePlane = array.array('B', [0]) * (width*height*3)
>>
>> i.e. do only one sequence repeat rather than three.

Here an addition to my former note.

Doing some simple analysis of this, I found that
it is generally safer *not* to do huge repetitions
of very small objects.
If you always use intermediate steps, you are
creating some slight overhead, but you will never
step into traps like these:

>  >>> if 1:
> ...     t = time.clock()
> ...     for i in xrange(100):
> ...         s = ' ' * 1000 * 1000
> ...     print time.clock()-t
> ...
> 0.674784644417
>  >>> if 1:
> ...     t = time.clock()
> ...     for i in xrange(100):
> ...         s = ' ' * 1000000
> ...     print time.clock()-t
> ...
> 6.28695295072
>  >>>

Analysis:
The central copying code in stringobject.c is the following
tight loop:

	for (i = 0; i < size; i += a->ob_size)
		memcpy(op->ob_sval+i, a->ob_sval, (int) a->ob_size);

For my example, this memcpy is started for every single
of the one million bytes. So the overhead of memcpy,
let is be a function call or a macro, will be executed
a million times.

On the other hand, doing ' ' * 1000 * 1000 only
has to call memcpy 2000 times.

My advice: Do not go from very small to very large in
one big step, but go to reasonable chunks.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:ti...@ti...>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

[Pyobjc-dev] Re: [Python-Dev] PyBuffer* vs. array.array()

[Christian T. <ti...@ti...>]

Guido van Rossum wrote:
>>>I'm not sure if you were joking, but why not write
>>>
>>>         singlePlane.fromlist([0] * (width*height*3))
>>
>>  Or cheaper and faster for large width and height:
>>
>>           singlePlane = array.array('B', [0])*width*height*3
> 
> 
> Correct; then even better:
> 
>              singlePlane = array.array('B', [0]) * (width*height*3)
> 
> i.e. do only one sequence repeat rather than three.

For "large" widths and heights, like 1000*1000, this
effect is remarkably small: About 3 percent only.
The above is true for simple lists.

There are also counterexamples, where you are
extremely wrong (sorry), most probably due to
the mplementation, but also by the effect, that
medium sized flat objects can be copied more
efficiently than very small ones.

 >>> if 1:
... 	t = time.clock()
... 	for i in xrange(100):
... 		s = ' ' * 1000 * 1000
... 	print time.clock()-t
...
0.674784644417
 >>> if 1:
... 	t = time.clock()
... 	for i in xrange(100):
... 		s = ' ' * 1000000
... 	print time.clock()-t
...
6.28695295072
 >>>

Did I hear you head knocking on the keyborard?

ciao - chris
-- 
Christian Tismer             :^)   <mailto:ti...@ti...>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

Re: [Python-Dev] Re: [Pyobjc-dev] PyBuffer* vs. array.array()

[Jack J. <Jac...@or...>]

Note that there's the buffer *interface* and the buffer *object*. You 
want to use the buffer interface here, then on the Python side there's 
lots of objects that will work (arrays, NumPy arrays, strings for 
readonly access, etc).
--
- Jack Jansen        <Jac...@or...>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -

[Pyobjc-dev] Re: [Python-Dev] PyBuffer* vs. array.array()

[Scott G. <xs...@ya...>]

--- Guido van Rossum <gu...@py...> wrote:
> > In writing the unit tests, I came across a problematic situation that  
> > could easily arise in code (feel free to comment on the silliness of  
> > this code, if any... and note that I'm using the comprehension style  
> > even after that long rant I posted earlier :-):
> > 
> >          singlePlane = array.array('B')
> >          singlePlane.fromlist([0 for x in  range(0, width*height*3)] )
> 
> I'm not sure if you were joking, but why not write
> 
>          singlePlane.fromlist([0] * (width*height*3))
> 
> ???
> 

  Or cheaper and faster for large width and height:

           singlePlane = array.array('B', [0])*width*height*3









__________________________________________________
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com

Re: [Pyobjc-dev] PyBuffer* vs. array.array()

[Bill B. <bb...@co...>]

On Sunday, Jan 5, 2003, at 11:06 US/Eastern, Ronald Oussoren wrote:
> On Sunday, Jan 5, 2003, at 05:56 Europe/Amsterdam, Bill Bumgarner=20
> wrote:
>> The real problem is that something created via array.array() comes=20
>> back as an object that behaves differently.   Clearly, a bug in my=20
>> bridge code, yes, but raises a question:
>>
>> What should I be using?
>>
>> =46rom the python side, it appears to only be possible to create =
buffer=20
>> style objects-- big byte bags-- via the array module [yes a <str>=20
>> will work, but a byte-array seems so much more correct].  On the C=20
>> side, it seems that the only way to create a byte buffer like object=20=

>> is to use the PyBuffer* API.
> You could use the array module on the C side. That is more work than=20=

> just using the PyBuffer API but would solve your problem.

Looking at this more closely, it seems that the array module requires=20
that it 'owns' the pointer to the memory it contains.  That is, it=20
allocates/deallocate the memory and-- the killer-- resizes the hunk of=20=

memory at will.

So, it appears that the answer is that I need to figure out how to=20
create true buffer objects from the python side.

Which is trivial.  I hadn't realized that 'buffer' is a primitive type=20=

and, therefore, is effectively built in...

Not so trivial-- primitive in 2.3, built-in function in 2.2.   Need to=20=

make sure that whatever I implement works with bost.

Deal killer:  results of buffer() are read-only on 2.2 (didn't check=20
'<buffer>' type on 2.3).  I would have to expose=20
PyBuffer_FromReadWriteObject() to Python to be able to create a=20
readwrite buffer object from within Python itself.

End result:  Stick with the current dichotomy.  It is annoying, but not=20=

killer.=13

> <advocate style=3Ddevil>
> However, what if I raw strings into the API. If I get array objects=20
> back from the API there still is an inconsistency.
> </advocate>

Raw string?  As in '1230x045678'?

Since the underlying code specifically parses for objects that=20
implement the character buffer API, strings work fine-- as do basically=20=

anything else that encapsulates a single-segment buffer.

> I agree that array objects are more suitable for 'bags of bytes' than=20=

> the (builtin) alternatives. Note that some people seem to use Numeric=20=

> python for image processing (http://www.pfdubois.com/numpy/). Adding=20=

> support for using numeric arrays instead of array.array might be a=20
> usefull extension, but probably more as an item on the TODO list than=20=

> something to implement right now.

Right.   I also want the code to build/run against a stock installation=20=

of Python 2.2 or greater.  I believe that array/buffer is a part of the=20=

core?  I.e. will always be there?

In the future, the answer may be to have a 'buffer factory' type API=20
[on the C side more than the Python side] that, given a pointer and=20
length, produces an instance of the appropriate buffer class, as=20
configured by the developer.  If the developer wishes to use the=20
Numeric array classes (which I have not looked at yet), it would simply=20=

be a matter of implementing the appropriate 'delegate' method and=20
making the factory aware of it.

But, as you say, it is a TODO item as opposed to being on the immediate=20=

radar.

> All things considered I'd go for using array.array from C, with a=20
> fallback to the PyBuffer API when
> you cannot import the array module.

I was going to go down that path, but the deeper analysis of array=20
(discussed above) indicates that it isn't the right answer.   Instead,=20=

I'm going to see what it takes to create a true 'buffer' instance in=20
Python.  The answer may be that it can't be done and that the developer=20=

is simply going to have to live with the dichotomy between buffer and=20
array.

If that is the case, then this does sound like an issue truly pertinent=20=

to discussion on python-dev.  Notably, it would be useful to have a=20
buffer/array that behaves the same on both sides of the Python/C wall=20
and can encapsulate an arbitrary, fixed length, buffer of memory=20
regardless of which side created the memory.  It should likely also=20
have the notion of 'ownership' of that memory-- i.e. whether or not it=20=

should deallocate the memory when the last reference to the object is=20
destroyed.

b.bum

[Pyobjc-dev] libFFI

[Bill B. <bb...@co...>]

On Sunday, Jan 5, 2003, at 13:03 US/Eastern, Ronald Oussoren wrote in a 
CVS log entry:
> libFFI support now actually works (that is passes the testsuite with 
> the
> same failures as the non-libFFI version)

Congratulations!

Some questions (as I know nothing about libFFI):

     - how hard is it to build libFFI such that PyObjC can be built with 
libFFI support?

     - this eliminates register.m, correct?   How much smaller is the 
resulting binary?

     - what is the performance difference when using libFFI?   (Maybe 
wrap runalltests with calls to 'timing'?)

b.bum

Re: [Pyobjc-dev] GNUstep.

[<bb...@ma...>]

Wow!  I'm really happy to see that the GNUstep effort has such 
momentum.   Very cool!  I'm going to add a news item to that effect 
shortly.

I'm thinking that it might be wise to start a pyobjc-dev-gnustep 
mailing list as the GNUstep effort is going to generate a lot of 
traffic that is highly specific in nature.   It would give people new 
to the project an easy way to scan the archives to find relevant notes 
regarding the state of GNUstep compatibility.

Also-- for those working on GNUstep, there are a couple (maybe only one 
now?) of places where the PyObjC module does something like the 
following (found in objc/__init__.py):

try:
     import _FoundationMapping
     del _FoundationMapping
except ImportError:
     pass

As the port progresses, it is likely that an import error will occur in 
this context that is indicative of a real problem.

b.bum

Re: [Pyobjc-dev] GNUstep.

[Ronald O. <ous...@ci...>]

On Sunday, Jan 5, 2003, at 21:25 Europe/Amsterdam, Mirko Viviani wrote:

> In attach there is the patch against the current cvs that works with 
> GNUstep.
> 'Works' is a big word since there are dumps with some tests yet, but 
> simple
> things work great !

Thanks. I haven't checked the entire patch yet, but I expect that most 
of it will turn up in the repository later this week.

Some remarks based on a first glance at your patch:
* Could you please sent context-diffs in the future (diff -C) that 
makes it easier to grasp just what is changed.

* You changed a number of generated files in Modules/Cocoa (the .inc 
files). These files are generated using cocoa_generator.py in 
Modules/Cocoa/scripts. Could you check if that script is useable with 
GNUstep?

Ronald

Re: [Pyobjc-dev] GNUstep.

[Mirko V. <mi...@ob...>]

Mirko Viviani <mi...@ob...> ha scritto:

Unfortunately I left hardcored path in setup.py
Here the new one.

-- 
Ciao
Mirko

Re: [Pyobjc-dev] GNUstep.

[Mirko V. <mi...@ob...>]

Ronald Oussoren <ous...@ci...> ha scritto:

> This is probably a problem in 
> Modules/objc/class-list.m:ObjC_GetClassList. As the comment above the 
> GNUstep version indicates that the GNUstep is mostly a placeholder.

In fact, PyList does not work so I have replaced it with PyTuple.

In attach there is the patch against the current cvs that works with GNUstep.
'Works' is a big word since there are dumps with some tests yet, but simple
things work great !

-- 
Ciao
Mirko