[PataPata-discuss] PataPata critique: the good, the bad, the ugly

[Paul D. F. <pdf...@ku...>]

It's been about three months from my last post to the PataPata list as 
well as my last major change to the system. I have been thinking about the 
system in the intervening time, and feel ready to produce a critique of it 
as an experiment (sort of as a, sadly, "postmortem" report). Others are 
welcome to chime in.

This critique covers various good, bad, and ugly results from this 
experiment, and then outlines some thoughts on where to go next. This note 
marks the end of this phase of the PataPata experiment. I am uncertain if 
this project on SourceForge will see more development, but I am certain if 
there is more development on this particular SourceForge project, it will 
likely be in a radically different direction than the work published here 
to date.

== THE GOOD ==

Overall, I think PataPata was a success as an experiment -- meaning I 
learned something from it. That's the value of an experiment -- to test 
reality and learn something new or confirm something old. I tried to make 
a real prototype-based system on top of Python/Jython for use in 
educational software and I learned much about the strengths and weaknesses 
of such an approach. So, in that sense, PataPata was a resounding success 
and probably worth the several person-months I (and indirectly some 
others) put into it.

I think PataPata advanced some ideas of being able to write out a world of 
live objects to text files as a human-readable program to reconstruct that 
world. I'm very happy with how that works and the potential it shows. You 
can both have a live world of objects and convert it back and forth to 
human-readable Python source whenever you want it. There are some gotchas, 
but in general it works. Some of those ideas (like naming instances to 
write out) were inspired by PythonCard.

I was very pleased with how PataPata handled having multiple worlds, 
especially with how you could use a inspector tool in one world to change 
a window in another world. So, in some way, you were "safe" if you broke 
that other world, in that you could still inspect it and fix it again. 
This is unlike how in a typical Smalltalk image you can shoot yourself in 
the foot by, say, breaking the Window "open" method (so debuggers and 
error notifiers and development tools also fail to open, so you can;t fix 
the problem). It's not a new idea from PataPata (since some Smalltalks do 
this, or wanted to, and later versions of Python's IDLE IDE do something 
like this), but I did like the result in PataPata, and I think it is not 
implemented often enough.

Wrapping widgets from TK, Swing, wxWidgets worked out fairly well; the 
results were both better than I expected and easier to achieve than I 
thought. I think this shows you can both use native widgets and have a 
cross-platform codebase at least for the basic widgets to make 90% of an 
interface -- with only a few sacrifices and a little extra work. To my 
knowledge, I think PataPata is the only system that supports (to the 
limited extent it does) both Swing and TK -- the native widgets of Python 
and Jython respectively. I was very happy about that success considering 
the modest investment of time. I think other systems, perhaps PythonCard, 
could benefit from considering this approach.

I think the hierarchical browser for looking at both the data and code of 
objects in one window worked out well. I'd seen similar things before, so 
this is not a unique innovation to PataPata; this is just to say once 
again this still looks like a good idea to me.

Finally, I was very happy that Francois Schnell made a screencast of an 
early version of PataPata. I learned a lot from that -- especially in 
terms of the power of screencast movies to get a complex idea (building a 
GUI with prototypes) across in a short time. Thank you again, Francois.

== THE BAD ==

As an actual product, for the moment, PataPata has to be rated as a 
failure. It is not usable for production code because it is not complete 
or well tested or well documented or IMHO providing enough bang for the 
buck of using it. There are aspects of its overall approach I now 
question. I remain hesitant to commit to converting our three big free 
educational software packages (a garden simulator, a botanical plant 3D 
model generator, and a story authoring system) to it, which had been my 
own hope for it and motivating much of my work on it. I think I would 
still be better off converting those things to straight Jython+Swing or 
Python+wxWindows at this point than to PataPata. At the very least, it 
would be easier for others to maintain such conventional conversions. But 
there are other reasons too.

The biggest issue is that I learned that the seductive idea of prototypes 
may not be as powerful as its execution (especially as an add-on to an 
existing language). Consider what I wrote to someone else: "I've been 
repeatedly confronted with the fact that prototype languages are powerful 
and elegant but ultimately, like Forth, end up losing something related to 
documenting programmer intent. Documenting intent is one of the most 
important part of programming, and that is something that Smalltalk's 
formal class structure enforces for most code over Self, just like C's 
named variables document intent which gets lost manipulating Forth's data 
stack. Plus, it is hard for any great language feature to offset issues 
like lack of community or lack of comprehensive libraries."
And after at person suggested you usually need to name things before you 
can share them, I replied: "And I agree on the issue of "naming" something 
in order to share it. Which gets back to the issue I mentioned of 
"documenting intent". With the PataPata system I made for Python, there 
was a choice point where I decided that prototypes used by other 
prototypes as parents needed to be named (a difference from Self). And I 
liked that choice. But then, what am I really gaining over Smalltalk 
(except not having a class side, which is nice, but is it worth the extra 
trouble and confusion?)"
So, this leaves me questioning the whole move to prototypes. That person 
also pointed out previous work on "Exemplar based Smalltalk", so that is 
something I should look into further, perhaps as a compromise with 
Prototypes when I understand such previous work better.

Having said good things about the hierarchical browser above, I found 
myself editing the code by hand in a text editor a lot. That means 
something is wrong with the development GUI design. I'm not sure where to 
pinpoint the problem or whether it was due to the early nature of the 
system, its incompleteness, or my lack of confidence in it or something 
wrong with the hierarchical browser or code export functionality. But 
clearly it was not my intent in designing such a system to spend most of 
my coding time using scrolling a long file in a text editor, and I did too 
much of that.

I think the notion of putting behavior into instances of widgets was a 
misstep. It seemed like a good idea at the time, and seemed like it might 
be an advance over Delphi or PythonCard more in line with Self or 
NewtonScript. But having seen the result, I don't like it. Essentially, 
Delphi or PythonCard or similar systems (including one proprietary 
in-house one I worked on for Smalltalk) create instances of widgets which 
have actions tied to function names, where the actual functionality in 
terms of code is stored in some central object which represented the 
window or application. So a button instance knows to send 
"closeButtonClicked" to the window, but it does not actually have any 
detailed code associated with that action. PataPata stored the behavior 
most times in the actual prototype instance, so the code for 
"closeButtonClicked" would likely be right there in the specific button 
instance. In the end, this seems confusing and harder to maintain. Is code 
in the widgets or in the window or the application? It is one more layer 
to search for things. It makes the stored source harder to follow too. It 
also makes the implementation of widget wrappers more complex. It makes it 
harder to move code from PythonCard or Delphi to this newer approach. So, 
while it seemed like a good idea at the time, overall I'd say it turned 
out bad in practice for making a GUI. After trying this alternative, I 
prefer the model where widgets send messages to some more central object 
representing a window, an application, or a domain. So, this can be seen 
as a validation of the PythonCard approach in that sense.

PataPata has too much complexity both in terms of internal code complexity 
and in what people need to learn to use it. That is in part because it 
lives in two worlds -- the world of Python/Jython classes and the world of 
Prototypes. Error messages are hard to understand because it is not clear 
what level they are coming from. I have seen this before when I wrote 
layers on top of other systems. Getting past this problem will take some 
creative thinking, if it is possible to fix at all. The general issue here 
is having a clean way to pinpoint errors in code the user writes as 
opposed to reporting on errors or limitations in the underlying machinery 
that supports the user code.

There is no documentation. This is one of the problems with moving away 
from conventional Python which has lots of documentation, email lists, and 
books for it. The best documentation came from Francois Schnell who made a 
screencast about it (mentioned above). Writing good documentation is a 
non-trivial problem.

There is no user community for PataPata, and even I don't use it. Part of 
the community issue that I haven't wanted to advertise it or push it much 
since it is an early experiment. Part of it is due to perhaps alienating 
some in the conventional schooling community on the Python edusig list by 
the way I approached talk of unschooling. Though to that list's credit, I 
have received much useful comments and criticism from the edusig list, 
quite a bit of it valid. :-) Kirby Urner made many interesting points 
including general comments on the cost of a paradigm shift in terms of 
user effort needed to move to a new system without documentation and with 
most of the overall supporting code written in a different paradigm then 
the new one you want people to try (so, they have to keep switching mental 
sets between old and new in a confusing way). Ian Bicking contributed 
several important technical ideas in discussion on the edusig list 
(especially relating to using Python's metaclasses). But some of the 
limited community is likely due to my reservations in promoting PataPata 
to either the programming community or the unschooling community, based on 
the other factors above, like the complexity of a system living in two 
worlds and also not being documented and having rough edges and big gaps. 
I did not see the system as being at a point where more help would make it 
better; more I see it at the point where some of the key ideas it pursued 
are to be questioned. Unlike, say, our popular free PlantStudio software 
when it was new eight years ago, PataPata does not pass the test for a new 
thing of essentially providing 3X-10X more value overall than the existing 
common alternatives to make it tempting to try it and then switch to it. 
Switching development environments is rarely worth the effort for most 
people for most things for, say, just a 10% improvement in one area. And 
that applies even to my own thoughts about whether to use the PataPata 
framework in terms of my supporting it or my making something with it that 
others might support. Also, I feel the GPL license perhaps cut the 
potential size of the audience down some, as well as my confidence in 
promoting it; LGPL would probably find more takers, though I'm not sure it 
would have worked for me in this particular case of an early experiment 
which included putting in many ideas and some older pieces of code I had 
(an issue requiring more thought).

== THE UGLY ==

After this experiment, I am left still preferring actual Smalltalk syntax 
and mindset over Python and Jython syntax and emphasis (as much as I love 
indentation based syntax). It is easy to extend Smalltalk's core in a 
shareable way by a few lines of code (e.g. adding a "collect:except:" 
method to OrderedCollection). Extending Python's core requires a fairly 
complete understanding of a lot of code, probably significant coding 
including test case if you break anything else, and likely months or years 
of politicking before anyone can easily try your change (since few people 
rebuild the Python interpreter). I think Smalltalk keyword: syntax does a 
better job of documenting the intent of function or method calls, because 
each argument is in some sense labeled. So, I am left unhappy that this 
system has Algol-like syntax. As I wrote to someone else: "Also, I should
have mentioned in more detail, what continued to drive my interest in
Smalltalk is the keyword syntax. Code like: "MyClass newWithX: 10 y: 20"
is just easier to guess at the intent of than "MyClass(10, 20)". The sad
thing is, while there has been plenty of work on somewhat Smalltalk-like
systems in terms of OO with Python or Java or message oriented like Ruby
or prototype oriented like the IO language, they all seem to begin with
the premise that there is something wrong with the Smalltalk syntax, and
so build an entire world around some Algol like syntax. But it is
precisely the syntax of Smalltalk that does a better job of documenting
intent in the actual text of program code. Which is another reason why I
prefer Smalltalk to say Lisp (as powerful and otherwise general as Lisp
may be). You can think about putting a keyword syntax on the other
platforms, but then you end up with an ugliness of two paradigms. In the
case of Java & the JVM, that may be worth it for various benefits, for the
others it likely is not. "
I did include a compiler for Smalltalk-like syntax in a checkin of some 
mostly unrelated code in a subdirectory, but ultimately resolving that 
syntax issue in a production-oriented way is a very complex one.

I wanted to have PataPata use a network oriented system for remote 
debugging, but alas I did not have time to pursue that (in a way I would 
have thought secure enough). I did discover the wonder of TK's remote 
execution notion implemented in the "Send" command; but I did not get that 
working so that worlds could be in different processes (right now they are 
all in the same process). In general, some existing solutions for Python 
(like the RPC system IDLE uses) were not that general, and the ones which 
probably were seemed intimidating and fraught with security issues unless 
you really knew the framework. So I just never explored that. Plus, so 
far, I could not get other Python IDE developers interested in having a 
general scheme for this, so that remains an idea not explored in detail.

A parent who does unschooling wrote to me about PataPata and I had replied:
"[The] biggest issue is I wanted to experiment more with a Self-inspired 
prototype-based programming approach,
   http://research.sun.com/self/
   http://www.dekorte.com/docs/protos/
and PataPata would in that sense be different from either Python or Squeak 
as it advanced, because both of those are more class-based than prototype 
based. Having experimented more with prototypes, I'm not sure I like them 
as much as when I started, and am more questioning if right now they are 
worth the cost of being non-mainstream, or if they are indeed worth the 
cost, if the technical and semantic difficulty of putting them efficiently 
on a class-based system is worth the value of easy access to an existing 
class-based infrastructure with a different paradigm. So I'm not sure 
exactly where that effort is going anymore. :-)
...
IMHO Python or Squeak are quite usable as-is for unschooling (as are many 
other good computer languages); with PataPata I was just interested in 
trying an experiment in making Python an even better match by making it 
more transparent in the direction Squeak (and Self) has moved, and giving 
it a GUI designer that might be more usable at a younger age, and also I 
was responding specifically to a very schooling centered educational group 
which dominates any discussion of Python and education (on the EduSig 
list, or Mark Shuttleworth's school-centered initiative).
...
Over the years, what I have discovered about "educational software" is 
that most of it is junk, and the really useful things to connect kids with 
are the open-ended packages which provide an avenue for their creativity 
and sense of mastery over aspects of the real or digital world -- so, for 
example, learning to write with a word processor is much better than 
playing some silly flash-words game, and using Photoshop or the GIMP is 
probably much better than using some silly math-blaster game or even the 
award winning Oregon Trail (which is pretty good as those things go). 
There are exceptions to this, but they are rare. Generally, a parent 
working hand-in-hand with a kid to help them learn to use a software tool 
written for amateurs or professionals (Python, Blender, OpenOffice, 
GarageBand, or our own PlantStudio :-) is probably going to yield more 
value for most older kids than anything specifically labeled 
"educational". One of the bigger potential areas for exceptions is 
educational simulations like our garden simulator, but even there, kids 
usually learn more by building their own simulations than by using a 
pre-made one. Which brings us back to good programming environments for 
kids to have to make or modify such simulations (which was another ideal 
driving the PataPata experiment). "

== WHERE TO GO FROM HERE ===

Initially, this was supposed (by me) to be a project to bring the best 
ideas of Squeak and Self to Python and Jython, but in a "Pythonic" way, 
especially to support unschooling and homeschooling. As outlined above, it 
has been an experiment with successes and failures on that score. Where 
should it (or I) go from here?

Clearly, it has not set the Squeak world on fire. :-) Neither has it 
received Guido's blessing. And I don't plan on using it as-is either. So 
IMHO as of this point it's effectively: "He's dead, Jim", at least for the 
moment as it was originally conceived. I still think there is potential 
here for the good parts of PataPata to move forward in other contexts -- 
perhaps not even using prototypes.

Since I started the PataPata project six months or so ago, the free 
computing landscape has changed radically IMHO. And that change is due to 
Sun's decision to put much of Java under the GPL, removing Richard 
Stallman's "The Java trap" argument against using Java for free software.
   http://www.gnu.org/philosophy/java-trap.html
   http://blogs.zdnet.com/Burnette/?p=199
   http://blogs.zdnet.com/BTL/index.php?p=3937

As I wrote elsewhere on benefits versus drawbacks of using the JVM and 
Java libraries as platform for free software: "What would matter to me 
from the maintainer side -- as a tradeoff against [slower] speed -- would 
simply be making my life as easy as possible to spend as much time on 
tools and applications and the least time on low-level porting issues, 
while still being reasonably crossplatform (even with Swing's usual 
warts). I fought against Java at the start, and not just because 
VisualWorks consulting back then paid the rent. :-) I'd say VW of ten 
years ago is still better than the JVM of today. Certainly the first five 
years of Java was a disaster for most people -- projects after project 
abandoned and all sorts of problems with missing features, random bugs 
only on one platform, abysmal performance, and so on. Not to mention how 
mismatched static languages are as a general choice for most dynamically 
changing business and R&D applications (great for implementing a VM 
though). And then there was RMS on "the Java trap". Nonetheless, it's now 
ten years after Java got going, and things have changed bit by bit. Many 
Smalltalk ideas have migrated into Java (Swing, HotSpot), and it has so 
many libraries, and it is (supposedly) going to be GPL at the core. And 
computers are about 100X to 1000X faster than in 1996, so performance 
issues really don't matter as much for the kind of things I want to do 
(mainly educational simulations and knowledge sharing apps). So, I feel 
Java is perhaps finally at the point where it is stable enough that I can 
expect "write once, debug once, run everywhere" to finally be close to 
being true (instead of "write once, debug everywhere"). And I think even 
the best C++ programmer has to admit by now that Java (with garbage 
collection) is easier to program in and maintain than C or C++ (at least, 
for most people and most applications, special cases excepted). Obviously 
there will always be problems with incompatibilities of different Java 
versions and missing libraries and so on.  But with Sun's Java now going 
GPL, I might expect it will become even more ubiquitous and better 
supported by those who want it for other reasons. "

Now this doesn't mean only the JVM makes sense to use for free software, 
but it does suggest it may make sense for me with my particular skills, 
needs, time limits, and experiences. From what I read, and approach to 
Mac, PC, and GNU/Linux portability which writes as much code as possible 
in a platform neutral language like C and then writing say 10% of the code 
in a platform specific way for each major platform will probably look 
nicer and be better accepted. It just takes more time for learning, 
writing, and testing -- as I needed to do for PataPata, learning tricky 
details about TK, wxWidgets, Swing, and openGL, plus evaluating other 
toolkits including QT, KDE, Cairo, Gnome, and GNUStep/Cocoa. My time for 
coding free software is limited and I'd rather focus it on application 
coding and writing programming tools on top of a stable platform. I've 
also discovered that ultimately PythonCard is probably another good choice 
for Python oriented educational software. PataPata drew several good ideas 
from PythonCard, though having said that, no doubt there are some good 
ideas in PataPata which PythonCard might use, like the hierarchical 
browser perhaps, or perhaps how to wrap basic widgets from multiple 
platforms other than wxWidgets. Python + wxWidgets (which underlies 
PythonCard) remains a good crossplatform platform for coding -- although I 
think wxWidgets is a little brittle and perhaps aging in its C++ oriented 
design; I certainly know I would not want to maintain wxWidgets or extend 
it in C++, whereas I personally would consider writing native widgets 
using Java and the Swing library. Likewise, I'd rather code in Java to 
improve something like Jython than code in C to improve Python (well, 
except for the fact that much current Jython 2.2+ coding is unfortunately 
the more boring and less glamorous (but still very important) task of 
catching up with Python 2.6).

So, I am currently exploring other alternatives for how to port our 
educational software away from Delphi and to a free system. The short list 
is really four items:
* conventional Python + wxWindows,
* conventional Jython + Swing on the JVM,
* some sort of Smalltalk on top of the JVM (an existing one, or one I 
write from scratch), or
* another free Smalltalk not on the JVM.
I'm weighing the JVM alternative heavier because I see value in being able 
to write portable Java code when needed for performance. The choice 
between Python and Smalltalk is still a tossup -- I prefer Smalltalk 
syntax and mindset and long term would prefer a codebase in Smalltalk, but 
I have already written a partial Delphi->Jython+Swing translator that did 
most of the heavy lifting of such a conversion (and converting Delphi 
function names to Smalltalk keyword: syntax is likely to be tedious and 
require being done by hand). Whichever path I choose (and I may even use 
more than one for different projects), I think the good ideas tried in 
PataPata -- like storing an image of live objects as human-readable, 
having a hierarchical browser of data and code (even if for instances and 
classes and metaclasses), using a screencast to get across a new idea, or 
wrapping widgets to give a uniform feel to them -- may find a place.

I have not decide what to do with this actual SourceForge project (or 
discovered what is possible under SourceForge's terms). It is possible I 
may just leave it as is (a cautionary tale or successful software 
experiment, take your pick :-). Or, if it is possible, I may re-purpose it 
along these new lines if I need a place to put, say, code and applications 
for an educationally-oriented effort (implemented in Smalltalk or Jython 
or both) on top the JVM. I still like the name "PataPata" for a software 
environment which was inspired from a South African song by Miriam Makeba 
on a Putumayo CD of world music we have, and having now bought a bunch of 
the Miriam Makeba's CDs and listened to them while I programmed on 
PataPata, I really like her music more and more. It has an incredible 
depth of intricate rhythm and counterpoint to it, plus sometimes a 
political edge. It would be an accomplishment to someday build a software 
environment worthy of that inspiration.

(My thanks go out to my wife and family for making significant time 
available for me to do this exploratory work.)

--Paul Fernhout