[ naviserver-Feature Requests-1403933 ] Writer Thread

[SourceForge.net <no...@so...>]

Feature Requests item #1403933, was opened at 2006-01-12 14:45
Message generated for change (Settings changed) made by seryakov
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1403933&group_id=130646

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: C-API
Group: None
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Vlad Seryakov (seryakov)
Assigned to: Vlad Seryakov (seryakov)
Summary: Writer Thread

Initial Comment:
Will it be usefull to have special writer thread that
will send multiple files in async mode to multiple
clients. For example if i serve big ISO or movie files
and have many connections, currently they all use conn
thread for along time until the whole file is sent.
Instead we can mark the conn to be used in writer
thread and release conn thread for other requests and
in the meantime the writer thread will send multiple
FDs to clients in one big loop.

Currently it is possible to simply change ConnSend in
connio.c to submit open descriptor to the writer queue
and return marking the connection so  usual NsClose
will not close actual connection socket. Then write
thread will be simple loop reading small chunks from
every file and sending to corresponding socket.

Does it make sense? 

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2006-02-09 06:42

Message:
Logged In: YES 
user_id=95086

If I understand that correctly, chunked content is better done
i.e. calculation what range to return, in the connection thread.
Once the conn-thread has calculated what range of the file
should be returned, it can pass the work to the writer thread.
I'm saying this out of the blue, not looking at the sources
but it seems feasible to me.

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2006-02-08 20:46

Message:
Logged In: YES 
user_id=184124

Just to send couple of Kbytes of return data back to clinet
is easier and faster usuall way instead of using writer
thread. It's primary purpose to serve large content without
holding conn threads. Just to add ability to submit return
content or file using ns_conn spool is easy, i may extend
WriterSock to have pointer to the buffer of data and instead
of file it will return use this buffer only. 
Bigger problem i see in my implementation is that Chunked
requests will still hold conn threads and i am not sure
wether to duplicate chunking logic into WriterThread or not.

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2006-02-08 18:48

Message:
Logged In: YES 
user_id=184124

i'll see what i can do, and yes, if multiple threads
requests are round-robin

----------------------------------------------------------------------

Comment By: Zoran Vasiljevic (vasiljevic)
Date: 2006-02-08 17:53

Message:
Logged In: YES 
user_id=95086

Why don't you make a [ns_conn spool] command 
which will gracefuly close the connection, 
passing the socket to the writer thread(s)? 

This way everybody can use writer thread out of
Tcl with ease...

Oh yes... if there are >1 writer thread, how
would you distribute? Do you have one queue 
for writer threads?




----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2006-01-12 16:12

Message:
Logged In: YES 
user_id=184124

To play and test this i can add this to CVS but have it
disabled by default, so nobody will be affected but will
allow to do real testing and improvements. I do not think we
should cover too much by this, just effective downloading
huge files will be enough.

----------------------------------------------------------------------

Comment By: Vlad Seryakov (seryakov)
Date: 2006-01-12 14:46

Message:
Logged In: YES 
user_id=184124

I did some preliminary coding and it works and the speed of
downloading 2 simultaneous huge files are the same on
average with current model, test are not very usefull
because i tested on one computer only.

I am attaching 2 files that i modified but more though and
work should be done, currently it supportd only big files
not in chunked mode returned by fastpath, but i think this
is more than enough, all other dynamic content will go as usual.

The WriterThread is at te end of driver.c and connio.c was
modified in ConnSend function only.

Yes, the logging and running atclose procs is the open
question, i would run it in the conn thread.

The parts between #ifdef #endif in WriterThread is
experiments and they should be removed, it slowed down the
server significantly.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=719009&aid=1403933&group_id=130646