Re: [naviserver-devel] Upload Progress

[Gustaf N. <ne...@wu...>]

Zoran Vasiljevic wrote:
>
> But that would occupy the conn thread for ages, right?
> I can imagine several slow-line large-file uploads could
> consume many of those. Wasn't that the reason everything
> was moved in the driver thread for the 4.0 ?
>
> What I have/had in mind is aync writes from the driver thread.
> Most of the OS'es have this feature (kaio) so we can employ it.
> The question of locking, however still remains in such case.
> So the decision has to be made on: what is cheaper? Locking
> or spooling to disk out of the conn thread? I have no real-life
> experience but I'm inclined to believe that spooling out of the
> conn thread would be more costly.
>
> What do you think?
i have not much looked into the c-code, so please forgive, if my
mail here is obviously wrong or naive.

One thing, which i dislike about the aolserver for file uploads is that
there is no hook for early permission or quota checking; one has to
upload the whole file before aolserver is able to give some feedback.
i see this even more important than the progress bar.

What about the following model:

A) HTTP header (up to empty line) -> driver thread
 
B) Then passing control+socket to a separate receiving thread that
  1) issues a callback at the start (e.g. for permission checking) 
  2) does async i/o to receive the message body (e.g. PUT, POST request)
    here, one can issue as well callbacks per async block, and decide
    fom tcl, whether one would like to spool to memory or disc.

C) When end of message received, pass control to connection thread as usual.

The receiving thread is like a connection thread with tcl stuff; when the
receiving thread is implemented via libthtread, there won't be probably
much locking. Only one receiving thread will be necessary.
The overall model could be quite compatible with the aolserver, since the
connection tread is invoked when all data is received. It would be necessary
to pass context information from the receiving thread to the connection 
thread
to avoid double permission checking.

It could as well make sense, to pass control to the receiving thread only
for chunked content. Maybe, the receiving thread is not needed at all,
since the driver thread could handle everything as well.

-gustaf