Re: [Themis-dev] Another code base

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Mark Hellegers a écrit :
> [...] I disagree with that line of thought. I think the messaging overhead is extremely minimal, even on my old computers.
> You also seem to think that there is only one path from start to wherever
> you want to go, but that is also not true. There can be quite a few parts
> in the system interested in the "next step".
> It is also very easy to just drop in a different module or leave out a
> module you don't need.
> Messaging also makes it easier to queue up on tasks.
>
>   
>
>>> In UZI application part, I'm using a thread pool.
>>> Each thread can execute jobs (like fetching documents from server).
>>> However, from the parsing to the mapping it's a single job.
>>>
>>> I'm more concerned about memory handling than making the parser run in a
>>>       
>> separate thread than the renderer.
>>     
>>> UZI memory allocation all goes to a specific interface (BaseAllocator)
>>>       
>> that can decide to reuse objects and so on.
>>     
>>> There is more performance to gain from a correct memory allocation than
>>>       
> >from multithreading something that is inherently linear and sequential.
>
> That is an interesting approach. Do you have some numbers or something
> like that to back that up ?
> I'm not saying I don't believe you. I'd like to know something more about
> that.
>
> Mark
>   
Well, here are the papers I'm referring about:
http://www.cs.umass.edu/%7Eemery/pubs/berger-oopsla2002.pdf  (about 
using an allocator can speed up the allocation time by 40%)
And the older one:
http://www.cs.umass.edu/%7Eemery/pubs/berger-pldi2001.pdf (a basic 
example on heap layers)

I have nothing about threading a parser. However, speaking about data 
flow, I would say that:
  For the rendering to be done, you need the entire DOM tree (just think 
of absolutely positioned or floating elements for one). You also need 
all the linked resources (whatever, CSS, IMG, etc...)
  For the DOM tree to be entire, you need HTML source to be entire.
  For the CSS resource, you also need HTML source to be entire (as, even 
if <style> tag is only allowed in <head>, it's very frequent to find 
them in the body, google is a nice example of that).

So yes, surely, you can start parsing an HTML stream, and start making a 
(partial) DOM tree from it, and still start rendering it, all at the 
same time.
However, if there is a table in the page, you'll break everything not 
waiting for table end (as table size is computable only when received 
entirely), and table based layout at clearly too frequent to be ignored.
For example, a <p> tag inside a <table> tag should be rendered before 
the table (it's illegal, but that's how Mozilla and IE does).
If there is some JS, then you'll also have to get the whole things 
before rendering as JS usually deals with DOM tree (so it has to be 
entire if you don't want your JS engine to fail)
Finally, rendering prematuraly usually gives the flashing effect you had 
on Firefox 1.0 when the CSS didn't loaded in time (the <entire> DOM tree 
rendered, then the CSS arrived, and the <entire> DOM tree rerendered, 
moving things around).

Anyway, I don't get the interest of multithreading the parser & renderer 
(mainly for complexifying the parser, because of the multiple locks 
required to avoid data contention) if you are not going to render the 
final page until all steps are done.
On single core CPU, you'll never get any improvment (well, dataflow 
analysis), and on multicore CPU, the locking mechanism (and bad cache 
data) might simply break any gain you're expecting (as, at best, the 
rendering time will be : max(parser time (you need the whole document 
here), fetching time (way longer that anything else), CSS-DOM mapping, 
renderer) so it's likely = to fetching time)

That why I've implemented multithreaded fetching in UZI, so that the 
rendering time will be (deterministic), max(fetching time + rendering, 
parsing + CSS mapping + rendering), and the code still remains clear of 
locking primitives.

Anyway, you might be interested in:
http://weblogs.mozillazine.org/hyatt/archives/2005_05.html
about CSS rule mapping.

Have a nice day,
Cyril