Re: [naviserver-devel] Upload Progress

[Zoran V. <zv...@ar...>]

Am 03.01.2006 um 18:50 schrieb Stephen Deasey:

> What happens to the conn thread after this? It can't wait for
> completion, that would defeat the purpose.
>
> Do traces (logging etc.) run now, in the conn thread, or later in a
> spool thread?  If logging runs now, but the upload fails, the log will
> be wrong.  If traces run in the spool threads they may block.
>
> If further processing must be done after upload, does the spool thread
> pass control back to a conn thread?  Does this look like a new
> request? The state of any running Tcl script will be lost at this
> point (the conn thread will have cleaned up after hand-off to a spool
> thread, right?).

All very good question to which I can't give any question at
this point.

Or to put it simpler: I have no damn idea!

I wanted to get mylself some bird-eye view on the matter
in order to better understand what we are after and how
we can solve it.

What I however DO now is: we (as Gustaf) have a-kind-of
extra processing built into our app which is quite similar
to spool thread we are contemplating here about.

We use the [ns_conn channel] to splice out the socket out
of the conn structure, then wrap this into a Tcl channel
and then transfer this channel to detached thread sitting
in event loop. From the client side we fake the content length
(the client is not a browser) so the driver thread does not
come in between with its greedy reading.

Then we simply finish the processing in the conn thread but
the socket still lives and is getting serviced in the "spool"
thread. The "spool" thread operates in event-loop mode and
does the rest of the processing and eventually closes the
socket to the client when done.

Now, this is *way* dirty, but it allows us to use conn thread
only for a shortest possible time and yet do long-running
processing in the "spool" thread in the event-loop mode.

I believe this is precisely what Gustaf is also doing in OACS.

Again, I DO know that this is dirty but when you have only a
hammer everything looks like a nail, doesn't it?

That was my initial idea: add event-loop-type of processing
into the current naviserver driver-thread/conn-thread paradigm.
After thinking a while, the conn thread need not be involved
at all in the long run. We can feed the conn to conn or spool
The conn thread does all blocking whereas the spool thread
does all non-blocking?
The conn thread might be used for blocking (db) access whereas
spool thread might be used to serve images, static files, uploads
and all other things requiring IO which CAN be done non-blocking.

Now, what I wanted to give here is an *idea*. Deeper bits and pieces
i.e. how all this interact with each other I haven't consider yet, as
it is too early for that. The question is: is the idea itself worth
considering or not?

Zoran