Re: [naviserver-devel] Upload Progress

[Zoran V. <zv...@ar...>]

Am 31.12.2005 um 11:58 schrieb Stephen Deasey:

>
> I think we talked about this before, but I can't find it in the
> mailing list archive.  Anyway, the problem with recording the upload
> process is all the locking that's required.  You could minimize this,
> e.g. by only recording uploads above a certain size, or to a certain
> URL.

Hm... not very useful.

>
> It reminds me of a similar problem we had.  Spooling large uploads  
> to disk:
>
> https://sourceforge.net/mailarchive/forum.php? 
> thread_id=7524448&forum_id=43966
>
> Vlad implemented the actual spooling, but moving that work into the
> conn threads, reading lazily, is still to be done.

Yes. This is what I wanted to do but never got to it because
of the internal work... huh.

>
> Lazy uploading is exactly the hook you need to track upload progress.
> The client starts to upload a file.  Read-ahead occurs in the driver
> thread, say 8k.  Control is passed to a conn thread, which then calls
> Ns_ConnContent().  The remaining content is read from the client, in
> the context of the conn thread and so not blocking the driver thread,
> and perhaps the content is spooled to disk.

But that would occupy the conn thread for ages, right?
I can imagine several slow-line large-file uploads could
consume many of those. Wasn't that the reason everything
was moved in the driver thread for the 4.0 ?

What I have/had in mind is aync writes from the driver thread.
Most of the OS'es have this feature (kaio) so we can employ it.
The question of locking, however still remains in such case.
So the decision has to be made on: what is cheaper? Locking
or spooling to disk out of the conn thread? I have no real-life
experience but I'm inclined to believe that spooling out of the
conn thread would be more costly.

What do you think?

Zoran