naviserver-devel — Developer discussion

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Andrew P. <at...@pi...>]

On Wed, Jul 12, 2006 at 03:00:17PM -0400, Mike wrote:

> Of course benchmarking would be the ideal route, however I have the
> pleasure of writing the code now, and the hardware for this project
> won't be purchased until a while from now.

Why do you need special hardware to run you benchmark?  Just run it on
your desktop or whatever other Linux box you have available to you.
That should give be enough to at least get you started.

-- 
Andrew Piskorski <at...@pi...>
http://www.piskorski.com/

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Vlad S. <vl...@cr...>]

I could be wrong, never used sendfile before.

The code you can check is driver.c, WriterThread, and bufsize parameter 
can be used to define buffer.

I was considering using mmap, but somewhere i heard that too many 
mmapped files could be a problem for the server.

Mike wrote:
>> Async IO i mean, writer thread does loop and sends as much as possible
>> chunks from the files, same way as driver thread reads multiple chunks
>> of requests from multiple clients. sendfile blocks, so you will need 10
>> threads for 10 clients at the same time, in case of writer thread, it
>> can alone transfer 10 files to 10 clients. May be it will be slower in
>> each single case comparing to sendfile but overall it does it fast enough.
> 
> Vlad,
> Sorry, but your assertion is not correct.  sendfile() is not required
> to block - as far as I am aware it works on non-blocking sockets
> without a problem.  The key advantages of using sendfile is that it is
> implemented zero-copy and allows the OS to do large-block read-ahead
> on the files.  If serving 100 clients 650MB files, using non-blocking
> IO read+write needs to allocate local buffers.  If buffers are small
> (e.g. 100KB) then the disk has to seek like crazy from file to file,
> and that will kill performance.  If the buffers are large (e.g. 16MB)
> then the disk seek will be fine but I will need to burn 1.6GB of RAM
> for the buffers in the NS process...  How does NS decide what this
> buffer size is?  Can you point me at the code that performs this
> operation?  How challenging would it be to add optional sendfile
> support here?
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Mike <nee...@gm...>]

> Async IO i mean, writer thread does loop and sends as much as possible
> chunks from the files, same way as driver thread reads multiple chunks
> of requests from multiple clients. sendfile blocks, so you will need 10
> threads for 10 clients at the same time, in case of writer thread, it
> can alone transfer 10 files to 10 clients. May be it will be slower in
> each single case comparing to sendfile but overall it does it fast enough.

Vlad,
Sorry, but your assertion is not correct.  sendfile() is not required
to block - as far as I am aware it works on non-blocking sockets
without a problem.  The key advantages of using sendfile is that it is
implemented zero-copy and allows the OS to do large-block read-ahead
on the files.  If serving 100 clients 650MB files, using non-blocking
IO read+write needs to allocate local buffers.  If buffers are small
(e.g. 100KB) then the disk has to seek like crazy from file to file,
and that will kill performance.  If the buffers are large (e.g. 16MB)
then the disk seek will be fine but I will need to burn 1.6GB of RAM
for the buffers in the NS process...  How does NS decide what this
buffer size is?  Can you point me at the code that performs this
operation?  How challenging would it be to add optional sendfile
support here?

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Vlad S. <vl...@cr...>]

> 
> what I mean by this is not the cache - but the initial tcl library
> loaded at server startup.  I would like to know if it's possible to
> somehow reload this without shutting down the entire web server.
> 

check ns_ictl command, it can do it

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Vlad S. <vl...@cr...>]

> 
> is 5 vs 2 a typo...?  Also, what constitutes "large files" in this
> situation - is it the "writersize" argument, or is there something
> else?  I am nor sure how "async IO" compares to sendfile(2) - can you
> please comment on this?

Typo,

Yes, writersize define the size of the file should be greater than this, 
then writer thread will use it for sending.
Async IO i mean, writer thread does loop and sends as much as possible 
chunks from the files, same way as driver thread reads multiple chunks 
of requests from multiple clients. sendfile blocks, so you will need 10 
threads for 10 clients at the same time, in case of writer thread, it 
can alone transfer 10 files to 10 clients. May be it will be slower in 
each single case comparing to sendfile but overall it does it fast enough.


>>> Does a mechanism exist to "reload" the tcl libraries on a live server?
> 
> what I mean by this is not the cache - but the initial tcl library
> loaded at server startup.  I would like to know if it's possible to
> somehow reload this without shutting down the entire web server.
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Mike <nee...@gm...>]

On 7/12/06, Vlad Seryakov <vl...@cr...> wrote:
> Some time ago i added writer feature to NS, when configured in nssock
> section as
> ns_param writerthreads 2
>
> NS will create 5 threads that will perform serving large files to the
> clients leaving connection threads to do dynamic parts. Those writer
> threads are serving multiple connections at the same time using aync IO,
> so no big overhead.

is 5 vs 2 a typo...?  Also, what constitutes "large files" in this
situation - is it the "writersize" argument, or is there something
else?  I am nor sure how "async IO" compares to sendfile(2) - can you
please comment on this?

> > Does a mechanism exist to "reload" the tcl libraries on a live server?

what I mean by this is not the cache - but the initial tcl library
loaded at server startup.  I would like to know if it's possible to
somehow reload this without shutting down the entire web server.

Re: [naviserver-devel] Questions about NS details... (scalability, lib changes)

[Vlad S. <vl...@cr...>]

Some time ago i added writer feature to NS, when configured in nssock 
section as
ns_param writerthreads 2

NS will create 5 threads that will perform serving large files to the 
clients leaving connection threads to do dynamic parts. Those writer 
threads are serving multiple connections at the same time using aync IO, 
so no big overhead.

As for caching for adp, tcl, NS and AS do caching compiled ADP pages as 
well as compiled Tcl pages internally, you do not have to do anything 
except playing with config parameters that define cache size, see

cachesize in ns/server/$server/adp section

filecachesize in ns/server/$server section

Mike wrote:
> Two unrelated questions...
> 
> The project for which I am considering NS will have a dynamic and very
> large static part.  There will be many very large (650MB) static files
> that people will download.  It is hard for me to judge what the
> performance of serving these files with NS will be.  Of course
> benchmarking would be the ideal route, however I have the pleasure of
> writing the code now, and the hardware for this project won't be
> purchased until a while from now.  Should I use NS for the dynamic
> part and redirect the static requests to something like lighttpd, or
> can I rely on NS to handle such load without problems?
> 
> Does there exist a caching mechanism for dynamically generated tcl/adp
> pages?  If it does, where can I find docs for how it works/when the
> cache is invalidated?
> 
> Does a mechanism exist to "reload" the tcl libraries on a live server?
> 
> (please pardon all the questions, I'm just trying to get up to speed
> with what's in there)
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

[naviserver-devel] Questions about NS details... (scalability, lib changes)

[Mike <nee...@gm...>]

Two unrelated questions...

The project for which I am considering NS will have a dynamic and very
large static part.  There will be many very large (650MB) static files
that people will download.  It is hard for me to judge what the
performance of serving these files with NS will be.  Of course
benchmarking would be the ideal route, however I have the pleasure of
writing the code now, and the hardware for this project won't be
purchased until a while from now.  Should I use NS for the dynamic
part and redirect the static requests to something like lighttpd, or
can I rely on NS to handle such load without problems?

Does there exist a caching mechanism for dynamically generated tcl/adp
pages?  If it does, where can I find docs for how it works/when the
cache is invalidated?

Does a mechanism exist to "reload" the tcl libraries on a live server?

(please pardon all the questions, I'm just trying to get up to speed
with what's in there)

Re: [naviserver-devel] SSL support

[Vlad S. <vl...@cr...>]

I am using nsopenssl from ALserver CVS, works and compiles fine.

Mike wrote:
> All,
> 
> I am unable to find SSL support in the code.  There is a #define
> NS_DRIVER_SSL , however it does not appear to be providing the TLS/SSL
> capabilities, just changing the default parameter values for the
> driver.  Am I looking in the wrong place?
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

[naviserver-devel] SSL support

[Mike <nee...@gm...>]

All,

I am unable to find SSL support in the code.  There is a #define
NS_DRIVER_SSL , however it does not appear to be providing the TLS/SSL
capabilities, just changing the default parameter values for the
driver.  Am I looking in the wrong place?

Re: [naviserver-devel] Why was NaviServer forked?

[Bernd E. <b.e...@ki...>]

Hi Mike,

> 1) Are you or the other developers committed to fixing bugs that do
> not affect your business?  Again, this is not hostile - but I want to
> know, since the dev. team is small, if I run into bugs, will I be told
> "we work on the parts we need, feel free to fix it and submit a patch"
> or can I expect more of a "this is an opensource project we run, so we
> will devote time to helpind you with this bug you found."

A bug does affect every single user of the server. As was already said, there 
are more than 3 people relying on Naviserver.

Please take a look here and change the status to "any":
http://sourceforge.net/tracker/?group_id=130646&atid=719006

This should answer your question.

>   Finally, I would like to ask that you post either the above verbatum
> or a modified version of your response somewhere on the NS wiki.
> Perhaps short bios of you, Vlad, and Stephen and when you care about
> developing NS would be quite helpful.  Right now the only information
> available is the "Key Facts" page, and it's quite hard to guage where
> this project comes from or why simply by reading that page.

I updated the wiki :-)

Bernd.

Re: [naviserver-devel] Why was NaviServer forked?

[Zoran V. <zv...@ar...>]

Am 12.07.2006 um 18:13 schrieb Mike:

> On 7/12/06, Zoran Vasiljevic <zv...@ar...> wrote:
>> Our fork (15.02.2005) was motivated with similar reasons. All of us
>> depend (to a high extent) to the server code which means that we'd  
>> like
>> to have certain "influence" about the direction of the develompent.
>> This proved to be a very time consuming and difficult task in the AS
>> project as things have been questioned, re-questioned, endlessly
>> flamed etc.
> ...
>> So. I hope I did not open more questions than I attempted to answer!
>> If yes, hit me with your rhythm stick :-)
>
> Zoran,
>   Thank you for your complete answer.  It does raise a few questions,
> but they are simple...
>
> 1) Are you or the other developers committed to fixing bugs that do
> not affect your business?  Again, this is not hostile - but I want to
> know, since the dev. team is small, if I run into bugs, will I be told
> "we work on the parts we need, feel free to fix it and submit a patch"
> or can I expect more of a "this is an opensource project we run, so we
> will devote time to helpind you with this bug you found."

I can't speak of others but if you have a problem with the server code
itself and if I see that I can help I will do that. I'm already  
maintaining
the Tcl threading extension so I know what you are asking about...
I believe the others will do the same.


>
> 2) Was anything from AS 4.5 merged into NS, if not - is there a  
> plan for this?
>

This is good question! I do not know by heart. I will have to go
and double-check against the 4.5 list. As the matter of fact, we
added quite a lot of core-code which is not in the 4.5. The only
source of such info at this moment is the ChangeLog. This contains
everything but is not that easy to read.

>   Finally, I would like to ask that you post either the above verbatum
> or a modified version of your response somewhere on the NS wiki.
> Perhaps short bios of you, Vlad, and Stephen and when you care about
> developing NS would be quite helpful.  Right now the only information
> available is the "Key Facts" page, and it's quite hard to guage where
> this project comes from or why simply by reading that page.

Well... I will have to see with other collegues... You know how it goes:
we are all programmers and it's easier for us to write code than to
write marketing materials! But I understand that this sort of info is
needed.

Cheers
Zoran

Re: [naviserver-devel] Why was NaviServer forked?

[Vlad S. <vl...@cr...>]

Yes, all bugs will be fixed, regardless of how Naviserver is used in our 
  businesses, the whole project is intended to be bug-free and stable 
and usable for everybody.

Some parts were merged from AS 4.5 recently, you can check ChangeLog 
from the CVS HEAD, more may be added but this is not the highest 
priority to keep-up with AS development.

Mike wrote:
> On 7/12/06, Zoran Vasiljevic <zv...@ar...> wrote:
>> Our fork (15.02.2005) was motivated with similar reasons. All of us
>> depend (to a high extent) to the server code which means that we'd like
>> to have certain "influence" about the direction of the develompent.
>> This proved to be a very time consuming and difficult task in the AS
>> project as things have been questioned, re-questioned, endlessly
>> flamed etc.
> ...
>> So. I hope I did not open more questions than I attempted to answer!
>> If yes, hit me with your rhythm stick :-)
> 
> Zoran,
>   Thank you for your complete answer.  It does raise a few questions,
> but they are simple...
> 
> 1) Are you or the other developers committed to fixing bugs that do
> not affect your business?  Again, this is not hostile - but I want to
> know, since the dev. team is small, if I run into bugs, will I be told
> "we work on the parts we need, feel free to fix it and submit a patch"
> or can I expect more of a "this is an opensource project we run, so we
> will devote time to helpind you with this bug you found."
> 
> 2) Was anything from AS 4.5 merged into NS, if not - is there a plan for this?
> 
>   Finally, I would like to ask that you post either the above verbatum
> or a modified version of your response somewhere on the NS wiki.
> Perhaps short bios of you, Vlad, and Stephen and when you care about
> developing NS would be quite helpful.  Right now the only information
> available is the "Key Facts" page, and it's quite hard to guage where
> this project comes from or why simply by reading that page.
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] Encoding

[Bernd E. <eid...@we...>]

Am Mittwoch, 12. Juli 2006 15:23 schrieb Bernd Eidenschink:
> it looks like the mechanism of changing the encoding works as expected, but
> the computation of the correct contentlength not.

ok. I think this is going on:

1. NsTclReturnObjCmd
If ns_return is called _without_ -binary switch, Ns_ConnReturnCharData is 
called. The length is computed from the UTF-8 representation.

2. Ns_ConnReturnCharData
calls

3. ReturnCharData
Knows what output encoding to use. If not UTF-8, it will call
Ns_WriteCharConn

...BUT BEFORE...

4.
Ns_ConnSetRequiredHeaders
calls Ns_ConnSetLengthHeader
(Set the Content-Length output header)

for

5.
Ns_ConnQueueHeaders
calls NS_ConnConstructHeaders
where
"Update the response length value directly from the header to be sent, i.e., 
don't trust programmers"

So the content-length is set.

NOW:

7. Ns_WriteCharConn is called, calls
Ns_ConnWriteChars
that with the help of Ns_ConnWriteVChars encodes
with the help of "Tcl_UtfToExternal" to the final encoding.

I can be fundamentally wrong, but it seems to me like the 
whatever-it-will-become final encoding length should be computed earlier...

What do you think?

Bernd.

Re: [naviserver-devel] Why was NaviServer forked?

[Mike <nee...@gm...>]

On 7/12/06, Zoran Vasiljevic <zv...@ar...> wrote:
> Our fork (15.02.2005) was motivated with similar reasons. All of us
> depend (to a high extent) to the server code which means that we'd like
> to have certain "influence" about the direction of the develompent.
> This proved to be a very time consuming and difficult task in the AS
> project as things have been questioned, re-questioned, endlessly
> flamed etc.
...
> So. I hope I did not open more questions than I attempted to answer!
> If yes, hit me with your rhythm stick :-)

Zoran,
  Thank you for your complete answer.  It does raise a few questions,
but they are simple...

1) Are you or the other developers committed to fixing bugs that do
not affect your business?  Again, this is not hostile - but I want to
know, since the dev. team is small, if I run into bugs, will I be told
"we work on the parts we need, feel free to fix it and submit a patch"
or can I expect more of a "this is an opensource project we run, so we
will devote time to helpind you with this bug you found."

2) Was anything from AS 4.5 merged into NS, if not - is there a plan for this?

  Finally, I would like to ask that you post either the above verbatum
or a modified version of your response somewhere on the NS wiki.
Perhaps short bios of you, Vlad, and Stephen and when you care about
developing NS would be quite helpful.  Right now the only information
available is the "Key Facts" page, and it's quite hard to guage where
this project comes from or why simply by reading that page.

Re: [naviserver-devel] Why was NaviServer forked?

[Zoran V. <zv...@ar...>]

Am 12.07.2006 um 16:22 schrieb Andrew Piskorski:

> On Wed, Jul 12, 2006 at 11:19:12AM +0200, Zoran Vasiljevic wrote:
>
>> of that, some were fiercely opposed. Well, what would you do in such
>> case?
>
> I would have started actively and frequently committing code to the
> AOLserver CVS, as Dossy explicitly asked/challenged you to do, rather
> than merely talking about doing so.

Ah, this goes again... I did that, was publicly exposed as one who  
"broke"
some internally used AOL API which (almost) resulted in all my changes
reverted back and a lot of strain on myself to fix that in the matter of
*hours* in the CVS HEAD!!!
What I did: I commited a "innocent" update to get keylists objectified
as in one of the RFE's! And the bitter pill: I even did not need that  
personally!
I commited just because somebody from the communitiy needed it! Net  
result
is: I had to work later, under stress for some hours to get the fix.  
And the
code was in the volatile CVS HEAD. Now, AOL used CVS HEAD for  
production.
No wonder that they let nobody change a bit there!

Now, you did not have that experience but I did. And not everything was
going over the public list. I realized that AS project was more an AOL
project with a large debugging community then community driven  
development.
And I was not the only one with that impression...

I do not blame anybody for that. I'm greatly thankful to people who
open sourced the code and still have the highest possible respect
to them. It is just that we (I'm speaking about my company) need some
more than what's available and we do not have the time to spend
persuading everybody if that is needed or not and why and why not.


>
> Look, Zoran, I know that's not your personal style, but as far as I
> could tell you basically won your last public argument with Dossy, and
> he must have known it.  He more or less threw up his hands and said,
> 'Ok, just start committing code, we'll work it out from there.'  And
> then you walked away!

Well, this is a very good observation. The reason was simple: I did not
have the *time* to try again. My company is relying on the code  
*more* than
AOL does. After all, the server is the core of our product and several
people live from that. So, having this in mind and having Vlad  
throwing the
towel in resignation after the who-knows-what flame about multiprotocol
support, we just joined the forces. Stephen was in the same position...
So, the critical mass was there.

>
> If I recall correctly, Zoran had (and still has) CVS commit to
> AOLserver, but used it only rarely.  Vlad had it too.  Stephen Deasy
> never even asked for commit, which is a huge shame, as from his
> explanations on the AOLserver list, he seems to be a real AOLserver
> expert with a deep understanding of the code.
>
> Ideally, I would like to see Zoran, Stephen, and Vlad all actively
> collaborating with Jim Davidson and the other folks working on
> AOLserver, on a single codebase.  Failing that, I would have liked to
> see the three of them SOUNDLY and IRREFUTABLY demonstrate that a fork
> was in fact necessary.

Other folks? Who is "other folks"? I see only Nathan commiting  
anything new.
I have very good relations to Nathan and Jim (I have a *great*  
respect to this
guys) but they are both "biased" as they are both from AOL.
Where is the community?
You can say: you are the community (meaning myself). And I say, yes I  
was
the community as I tried but did not succeed.

>
> IMNSHO that was not done, and thus the NaviServer fork was
> unjustified.  Oh well, I guess it's all largely irrelevant now...

Well, I would not like to comment on everyting (there are other people
here as well) but I will comment on "unjustified".

 From our perspective (I mean myself and my company) and I'm certain  
that
other people think the same: it was a great relief! Suddenly whole  
lot of
possibilities opened and time could be used in more productive way!

But you know all that already, as you were (and are) present all that  
time.

Cheers
Zoran

>
> -- 
> Andrew Piskorski <at...@pi...>
> http://www.piskorski.com/
>
>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel

Re: [naviserver-devel] Why was NaviServer forked?

[Andrew P. <at...@pi...>]

On Wed, Jul 12, 2006 at 11:19:12AM +0200, Zoran Vasiljevic wrote:

> of that, some were fiercely opposed. Well, what would you do in such  
> case?

I would have started actively and frequently committing code to the
AOLserver CVS, as Dossy explicitly asked/challenged you to do, rather
than merely talking about doing so.

Look, Zoran, I know that's not your personal style, but as far as I
could tell you basically won your last public argument with Dossy, and
he must have known it.  He more or less threw up his hands and said,
'Ok, just start committing code, we'll work it out from there.'  And
then you walked away!

If I recall correctly, Zoran had (and still has) CVS commit to
AOLserver, but used it only rarely.  Vlad had it too.  Stephen Deasy
never even asked for commit, which is a huge shame, as from his
explanations on the AOLserver list, he seems to be a real AOLserver
expert with a deep understanding of the code.

Ideally, I would like to see Zoran, Stephen, and Vlad all actively
collaborating with Jim Davidson and the other folks working on
AOLserver, on a single codebase.  Failing that, I would have liked to
see the three of them SOUNDLY and IRREFUTABLY demonstrate that a fork
was in fact necessary.

IMNSHO that was not done, and thus the NaviServer fork was
unjustified.  Oh well, I guess it's all largely irrelevant now...

-- 
Andrew Piskorski <at...@pi...>
http://www.piskorski.com/

Re: [naviserver-devel] Encoding

[Bernd E. <eid...@we...>]

Hi,

it looks like the mechanism of changing the encoding works as expected, but 
the computation of the correct contentlength not.

I downloaded the same page with 'ns_return', one time UTF-8 and one time 
ISO-8859-15. The download of the latter stops at the correct number of bytes 
but the Content-Length-Header is larger (the "UTF-8" size), so wget barfs, 
Opera reloads page etc.
(The "recoding" is then only a test of the encoding. And see, recoding the 
UTF-8 page to ISO results in the same byte length where the wget download 
stops aka. "breaks", the "ISO-content-length").

Maybe this came with adding the -binary switch?

tclresp.c:

    if (binary) {
        data = (char *) Tcl_GetByteArrayFromObj(dataObj, &len);
        result = Ns_ConnReturnData(conn, status, data, len, type);
    } else {
        data = Tcl_GetStringFromObj(dataObj, &len);
        result = Ns_ConnReturnCharData(conn, status, data, len, type);
    }

Re: [naviserver-devel] Why was NaviServer forked?

[Zoran V. <zv...@ar...>]

Am 12.07.2006 um 10:15 schrieb Mike:

> Hello,
>   Please do not interpret this as any sort of hostile request - I have
> read and searched all I could, but I am unable to figure out why
> NaviServer was forked from AOLserver and what the goals of the current
> development team are (why are they/you working on NaviServer)?  I am
> very excited to see the new strides toward better documentatioin and
> code cleanup and such - but I am a bit scared to sign on to use
> NaviServer without knowing why it exists or what the roadmap is.
> Please, Zoran, Vlad, or someone - tell me your motivations and goals
> both for the fork and for the future of NaviServer.  Maybe there's a
> pointer to a mailing list thread where this decision was made?  An IRC
> channel log?
>   Thanks!

Hi!

If you recall, AS forked already some years ago under the OpenNsd
name. Mostly by OACS people who were not very pleased with the way
the community issues have been handled by some "gatekeeper" persons
assigned from AOL. This was I believe in 2001 or 2002.

Our fork (15.02.2005) was motivated with similar reasons. All of us
depend (to a high extent) to the server code which means that we'd like
to have certain "influence" about the direction of the develompent.
This proved to be a very time consuming and difficult task in the AS
project as things have been questioned, re-questioned, endlessly =20
flamed etc.
It was no big fun and it was contra-productive for us to remain active
there, so we went out to build our own sandbox and try out new things
by ourselves.
Unlike AOL typical usage (a web-server) many of us (when =10I say many
I mean mainly myself and Vlad) use the server as a general-purpose
multithreading Tcl/C engine. Of course, this kind of usage-pattern is
not something of the AOL's highest-priority, consequently the changes
we need to add to the core server were not taken seriously and on top
of that, some were fiercely opposed. Well, what would you do in such =20
case?

So, as of now we have a pretty stable, in many respects improved code
and will continue to improve it as the time goes by. We will "borrow"
good ideas from the AOLserver project because there are (still) =20
excellent
people working there, and we will add our own salt when we think it's
needed. We will try to maintain compatibility to the AOLserver if =20
possible
yet this is not a high-priority issue.

Well (others may also comment), generally speaking, you will find
much less "obstruction" to your ideas if you want to participate in
the server development. There is no "Mr. Big" arround here and we solve
most of the issues in the democratic way: i.e. by voting. This is not
always easy, but the alternative is much worse (I personally know that
from the experience).

So. I hope I did not open more questions than I attempted to answer!
If yes, hit me with your rhythm stick :-)

Cheers,
Zoran

[naviserver-devel] Encoding

[Bernd E. <eid...@we...>]

Hi,

maybe you can give me a hint whats going on...

Background: My app uses a registered filter and proc to handle requests to 
files. ADP files (UTF-8) are ns_adp_parse'd (including data from database, 
UTF-8) and ns_returned (w/o -binary). I don't do any mapping of ADP files via 
config, additionally I have ns_unregister_adp calls on GET, HEAD, POST.

My default config says:
--------------------------------------------
section ns/encodings
.adp: utf-8
.html: utf-8

section ns/mimetypes
.adp: text/html; charset=utf-8
.html: text/html; charset=utf-8

section ns/parameters
outputcharset: utf-8
urlcharset: utf-8
preferredcharsets: utf-8
--------------------------------------------

This szenario works.

Now, if I want to automatically change the
output encoding from utf-8 to let's say iso-8859-15,
I try to do it like this:

--------------------------------------------
section ns/parameters
outputcharset: iso8859-15

section ns/mimetypes
.adp: text/html; charset=iso-8859-15
.html: text/html; charset=iso-8859-15
--------------------------------------------

In my test case, 'string length' on the parsed adp string
gives me 7109 bytes, 'string bytelength' 7147 bytes,
in the Header 'Content-Length' is 7147 and wget stops
after byte 7109 (e.g. Opera requests the page twice, haha,
I lost one day to figure out why):
string length: 7109 bytes = bytes returned
string bytelength: 7147 = Content-Length header

If I now 'ns_return -binary' those 7109-Tcl-ByteArray-Bytes
and request them (7109bytes + Content-length 7109) via wget 
(that works now, and Opera is happy again), 
I can 
recode iso-8859-15..utf-8 testpage.html
("hey, recode, assume it's iso, transform it to utf-8")
but not 
recode utf8..iso-8859-15 testpage.html.
(the testpage.html is created from the parsed adp).

I agree, I'm confused. Where to look for the bug? In my
configuration/app or in the server? Of course, using -binary switch is not the 
proper solution.

Thanks!
Bernd.

[naviserver-devel] Why was NaviServer forked?

[Mike <nee...@gm...>]

Hello,
  Please do not interpret this as any sort of hostile request - I have
read and searched all I could, but I am unable to figure out why
NaviServer was forked from AOLserver and what the goals of the current
development team are (why are they/you working on NaviServer)?  I am
very excited to see the new strides toward better documentatioin and
code cleanup and such - but I am a bit scared to sign on to use
NaviServer without knowing why it exists or what the roadmap is.
Please, Zoran, Vlad, or someone - tell me your motivations and goals
both for the fork and for the future of NaviServer.  Maybe there's a
pointer to a mailing list thread where this decision was made?  An IRC
channel log?
  Thanks!

Re: [naviserver-devel] ns_cache_forever

[Zoran V. <zv...@ar...>]

Am 11.07.2006 um 22:13 schrieb Stephen Deasey:

> The API works for me...
>

OK. Perhaps I was not clear enough:
We still have "ns_cache_create -ttl" syntax
The new code does not support this (-ttl) option.
I will now change our code to adjust to current
Tcl API and make new release. I will be not very
happy if after that the option gets renamed, dropped
or otherwise changed w/o very good (whatever)
reason.

So, the "API works for me" is:

   ... and I do not plan to change it in near future

or

   ... but I plan to add/drop X because of Y


??

Cheers,
Zoran

Re: [naviserver-devel] ns_cache_forever

[Stephen D. <sd...@gm...>]

On 7/11/06, Zoran Vasiljevic <zv...@ar...> wrote:
>
> Am 11.07.2006 um 21:44 schrieb Stephen Deasey:
>
> > Works for me.
> >
>
> Works for me as well. The question is: can I rely
> on the Tcl API and go change our Tcl code or should
> I wait?
>

The API works for me...

Re: [naviserver-devel] ns_cache_forever

[Vlad S. <vl...@cr...>]

i am not going to ask to change existing API

Zoran Vasiljevic wrote:
> Am 11.07.2006 um 21:44 schrieb Stephen Deasey:
> 
>> Works for me.
>>
> 
> Works for me as well. The question is: can I rely
> on the Tcl API and go change our Tcl code or should
> I wait?
> 
> Cheers
> Zoran
> 
> 
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> naviserver-devel mailing list
> nav...@li...
> https://lists.sourceforge.net/lists/listinfo/naviserver-devel
> 

-- 
Vlad Seryakov
571 262-8608 office
vl...@cr...
http://www.crystalballinc.com/vlad/

Re: [naviserver-devel] ns_cache_forever

[Zoran V. <zv...@ar...>]

Am 11.07.2006 um 21:44 schrieb Stephen Deasey:

> Works for me.
>

Works for me as well. The question is: can I rely
on the Tcl API and go change our Tcl code or should
I wait?

Cheers
Zoran