On Wed, 25 Mar 2009 20:31:23 +0900, Martin Aspeli wrote:
> Balazs Ree wrote:
>> On Wed, 25 Mar 2009 11:12:47 +0900, Martin Aspeli wrote:
>>> Matthew Wilkes wrote:
>>>> Hello everyone!
>>>> It's time for us to write up our ideas list, for now, can everyone
>>>> who has any ideas please reply to this email with a title and maybe
>>>> an explanatory sentence of anything they think is a good idea.
>>> Here's a fun and experimental one:
>>> * Server side KSS compiler
>>> Ever since our adoption of jQuery, we have some overlap between the
>>> KSS core and jQuery in things like event binding, AJAX request making
>>> and DOM manipulation. KSS has a really nice programming model, and
>>> fits a certain user base (the "I'm afraid of JS and I want structure
>>> and testability on the serverside" type) well. However, this
>>> duplication has some performance implications in terms of page weight
>>> (we send a lot of JS with each request) and the fact that both jQuery
>>> and KSS need to scan the DOM, bind events and so on when the page is
>>> Martijn Pieters suggested it should be possible to parse a KSS file
>>> server-side and "compile" it to equivalent jQuery operations for
>>> binding event handlers and performing client side manipulation or AJAX
>>> requests. That is of course only half of it: we also need something to
>>> convert server-side (or parse client-side) the response payloads that
>>> come back from KSS actions so that they do something when returned.
>>> The aim here would be to find out if we can avoid sending all those
>>> kilobytes of KSS client-side code with each request, whilst still
>>> retaining the nice programming model that KSS offers. Effectively,
>>> this would be an alternative implementation of the KSS protocol on top
>>> of jQuery.
>> I was giving some thought to this and came to similar conclusions. I
>> see a first step that's feasible to do and brings an immediate benefit.
>> Therefore I propose to give focus to this. This would be the building
>> the server side parser generator for .kss files.
> Why do we need a *parser* *generator*? That sounds very recursive to me.
> Why not just a *parser* and a jQuery *generator*. :)
By "parser generator" I mean a tool that takes a formal syntax definition
(like BNC), in this case for the .kss syntax, and generates a parser
based on this. This parser then, will be able to parse and compile kss
files and generate whatever output desired.
To get a better idea of what a "parser generator" is, please look at the
There are also other approaches to build the parser. There are several
tools and libraries made for this purpose, and one of them can be chosen
to be applied - but you can do without them too, since indeed the
functionality they provide can be just programmed in plain python. The
reason why I am interested to use existing tools and libraries for this
purpose (be it a "parser generator" or not) is that I have bad experience
with "custom made" parsers, including our very own parser for kss.
>> Based on an ideally BNC notation of .kss syntax, this parser generator
>> would be capable of generating a parser in both python (server side)
> Why do we need a JS parser? Don't we have one of those already?
If we'd keep on using two parsers, it would be an impossible task to keep
then in sync. They would deviate already at the time of creation.
But if we use the parser generator approach, we get the js parser
together with the python parser at minimal cost. And the even bigger
advantage will manifest when there will be no need to maintain and
retrotest two different parser implementations (in two different
If the parser generator approach is not used, then I agree it may be more
difficult or impossible to get a python and js parser from the same
source, so then I would not consider replacing our existing js parser,
but in that case the end result would be that kss looses the ability to
work on non-python platforms, since we would eventually drop maintaining
the js lib and keeping it in sync.
>> The server side (python) parser would parse a kss resource and would
>> page as a compiled kss resource (replacing the original .kss it
>> compiled). The operation of the server side parser could be governed
>> through the portal_kss tool first. Optionally, on long term, it could
>> make a wsgi middleware that compiles the kss resources found in the
>> pages, doing the compilation and replacing the links to the compiled
>> versions transparently.
> I hadn't thought about the WSGI approach. This may in fact be easier and
> more interesting.
> I think in the short term, though, the aim should be to just make the
> generator work. Finding out where to hook it in is not such a big deal.
Agreed, actually I think making the wsgi middleware is not such a big
task (it may be easier then hook it to plone_kss, wink), but it would be
more difficult for people to deploy on today's Plone, plus it really does
not matter that much for the end goal. Could be added later.
>> When during kss bootstrap at pageload, the client observes the compiled
>> kss code in the html page (included or linked in), it simply executes
>> it. In the first version this could execute calls on the existing kss
>> api. Which would do the right thing that would have resulted if the
>> client does the compilation from the original .kss resource. This makes
>> it possible to deploy this solution with existing kss right away.
> But then what's the point? If we need the full KSS client-side library
> anyway, we don't really need it.
My observation and analysis disagrees on this point: the biggest
performance bottleneck with kss is pageload time. This is not so much a
result of the code size (with modern compression applied, kss's
part of the full js code) but the bootstrap sequence (according to my
benchmarks). For some reason (hint: no native domload event in browers)
the delay during pageload is much bigger than what should be necessary by
the amount of CPU used.
With the server-side parsing implemented in itself, calling on the
current kss, the whole problem would be circumvented and pageload time
would reduce to zero. (What would remain is what your js code would spend
by setup anyway, but kss would not add to it.) In my eyes this is a big
enough gain, it would take away the original "why not use kss" reason.
On the other hand, if we are over with the pageload, execution speed of
kss is currently acceptable since there is nothing CPU hungry it's doing
there. If not only the pageload/bind but the execution scheme of kss
additional speedup to that gained during the pageload compilation. It
would further decrease code size (to zero if you want, although see below
my note about libraries) and it would make the use of kss and native
opinion would be the biggest goal.
So while I agree on the long path, the reason why I suggested this as a
direct focus of the GSoc topic is because of the direct benefit it
I also understand that you have different opinion about the "parser
generator" approach and I accept this. Obviously it is not the only way.
I think there are many different approaches that may be equally fruitful:
my preference for solutions come from my experience (and challenges) with
our custom made parser, and research that I did for solutions.
> I think the end result should be a JS file full of jQuery, including
> event bindings and the like. This is then executed as any other,
> hand-written file full of jQuery.
> We would need jQuery equivalents of the current client-side KSS plugins,
> of course. Many of those will be available natively in jQuery though.
True, but let me rephrase in an other way: the generated output will
there will be calls to the jquery lib, from plugins you will also call
other (possibly non jquery core) libraries.
and jquery library calls. You may also need some specific libraries to
carry out a certain kss feature. You will have the choice of (a) using
a library from kss (that first needs to be factored out for the purpose),
or (b) write that functionality yourself, or (c) drop that feature
I feel that you believe that the kss libraries can go away altogether,
but it can happen only in one of the above mentioned ways. Some
functionality of kss can be dropped and simplified, some needs to be
used, and yes the code will be much much shorter in the end but there
will be some. None of the code that is in kss, is there without a reason,
it's all doing something.
On the other hand, if the parser is written in a way that the code it
emits can be customizable, we can hook up the parser to emit code
interfacing with existing kss and we immediately get the (alleged)
advantage of zero pageload time in kss and in plone with kss.
After this the same parser can have more branches where only the emitted
kss, and there can be one or more branches that aim to arrive at full
>> To summarize what this will immediately result, let me add a subtitle
>> for the idea:
>>> * Server side KSS compiler
>> (Zero pageload overhead and smaller code for KSS)
>> I would emphasize "zero pageload overhead", as this will be a
>> substantial and immediate benefit.
>> parser from the same definitions. This would replace our existing
>> parser and permit to maintain (and regression test) a single parser
>> instead of two.
> complexity sounds pretty difficult. I think that would be a distraction
> from the most immediately useful parts of this GSoC project.
That really depends on the goals set and what you consider as most
immediately useful parts.
Note that I did not once disagree with the full path, just trying to
higlight an opportunity here.
p.s. I will reply to the rest of the posts continously, sorry if my work
in current days causes a delay.